Apache :: upload large file error using mod_fcgi

Doctor · Joined: 19 Jun 2012 Posts: 10 Location: Taiwan, Taipei

Hi, I have a problem in using mod_fcgi 2.3.6 or 2.3.7 with apache 2.2.x, php 5.3.x in Windows 2k3 or 2k8. When I try to upload a large file bigger than 200M, it may fail with partial upload only. But the program and php environment works fine for apache module with thread safe mode.

Any help is welcome.

Steffen

Maybe you have to set larger: http://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html#fcgidmaxrequestlen

Steffen

Doctor · Joined: 19 Jun 2012 Posts: 10 Location: Taiwan, Taipei

fcgidmaxrequestlen has been set to 512M or 1GB. But still have problem. Current setting as follows:

<IfModule mod_fcgid.c>
AddHandler fcgid-script .fcgi .php
FcgidInitialEnv PHPRC "d:/Apache/php"
FcgidInitialEnv PATH "D:/Apache/php;C:/WINDOWS/System32;C:/WINDOWS;C:/WINDOWS/System32/Wbem;"
FcgidInitialEnv SystemRoot "C:/Windows"
FcgidInitialEnv SystemDrive "C:"
FcgidInitialEnv windir "C:/WINDOWS"
FcgidInitialEnv TEMP "C:/WINDOWS/Temp"
FcgidInitialEnv TMP "C:/WINDOWS/Temp"
FcgidInitialEnv PHP_FCGI_MAX_REQUESTS 1000
FcgidMaxRequestsPerProcess 1000
FcgidMaxProcesses 10
FcgidIdleScanInterval 120
FcgidIdleTimeout 900
FcgidIOTimeout 1200
FcgidBusyTimeout 300
FcgidConnectTimeout 60
FcgidMaxRequestLen 1073741824
FcgidWrapper "d:/Apache/php/php-cgi.exe" .php
AddType application/x-httpd-php .php
FcgidProcessLifeTime 3600
FcgidMinProcessesPerClass 0
FcgidFixPathinfo 1
</IfModule>

James Blond

What about this values in your php.ini ?

post_max_size
upload_max_filesize
max_input_time
max_execution_time
max_input_time
memory_limit

Are all values high enough?

If it is a design question than uploading so large files shouldn't be done with PHP.

Doctor · Joined: 19 Jun 2012 Posts: 10 Location: Taiwan, Taipei

post_max_size = 512M
upload_max_filesize = 512M
max_input_time = 300
max_execution_time = 3600
memory_limit = 768M

These values are high enough. And The same php.ini and php applications work fine under apache module with thread safe mode. Therefore, I think there may be some problem with the mode_fcgi.

Since I need to use the ZEND Guard to protect the php code, and the ZEND Guard supports only fast cgi. Otherwise, I would like to keep using the apache module which works fine and faster.

Steffen

You say above: ..it may fail with partial upload only...

Does this mean it works now and then, not always partial.

Any indication in the php and/or apache error.log (Set LogLevel Info) ?

Steffen

Doctor · Joined: 19 Jun 2012 Posts: 10 Location: Taiwan, Taipei

We can upload small files. If the file is large, it always upload partially.
I set the php.ini upload_tmp_dir to somewhere so that I can monitor it.

I see upload tmp file created with smaller file size in that folder. And then after several seconds the tmp file deleted automatically.

From the php application side, $_FILE['error'] gives us partial upload error code.

I set log level to info. Nothing special happens in error log.

[Wed Jun 20 18:37:32 2012] [notice] Parent: child process exited with status 2147483655 -- Restarting.
[Wed Jun 20 18:37:33 2012] [notice] Apache/2.2.22 (Win32) mod_fcgid/2.3.7 configured -- resuming normal operations
[Wed Jun 20 18:37:33 2012] [notice] Server built: Apr 20 2012 18:42:30
[Wed Jun 20 18:37:33 2012] [notice] Parent: Created child process 5324
[Wed Jun 20 18:37:33 2012] [notice] Child 5324: Child process is running
[Wed Jun 20 18:37:33 2012] [info] Parent: Duplicating socket 328 and sending it to child process 5324
[Wed Jun 20 18:37:33 2012] [notice] Child 5324: Acquired the start mutex.
[Wed Jun 20 18:37:33 2012] [notice] Child 5324: Starting 256 worker threads.
[Wed Jun 20 18:37:33 2012] [notice] Child 5324: Starting thread to listen on port 80.

Steffen

You said that with php as module (handler) all fine, looks must be some with fastcgi.

Have a look at all settings at http://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html
maybe there is one you have to set.

You wrote: $_FILE['error'] gives us partial upload error code
What is that error ?

Steffen

Doctor · Joined: 19 Jun 2012 Posts: 10 Location: Taiwan, Taipei

Yes. I do study the fastcgi setting and try a lot of combinations but I cannot find out what is the key parameter that lead to the error.

http://www.php.net/manual/en/features.file-upload.errors.php

UPLOAD_ERR_PARTIAL
Value: 3; The uploaded file was only partially uploaded.

Doctor · Joined: 19 Jun 2012 Posts: 10 Location: Taiwan, Taipei

One more thing to mention, from the task manager, we can see two httpd running normally. When we upload a large file fails, one of the httpd stop running and may disappear from the task manager. We need to restart httpd service.

Doctor · Joined: 19 Jun 2012 Posts: 10 Location: Taiwan, Taipei

After adding the mod_security, I can see something wrong here. Below is the audit log from mod_security.

--f0360000-H--
Message: Access denied with code 44 (phase 2). Match of "eq 0" against "MULTIPART_UNMATCHED_BOUNDARY" required. [file "D:/Apache/Apache/modsecurity/etc/modsecurity.conf"] [line "79"] [id "200003"] [msg "Multipart parser detected a possible unmatched boundary."]
Action: Intercepted (phase 2)
Apache-Handler: fcgid-script
Stopwatch: 1340438160381125 13750000 (- - -)
Stopwatch2: 1340438160381125 13750000; combined=0, p1=0, p2=0, p3=0, p4=0, p5=0, sr=0, sw=0, l=0, gc=0
Response-Body-Transformed: Dechunked
Producer: ModSecurity for Apache/2.7.0-rc2 (http://www.modsecurity.org/).
Server: Apache/2.2.22 (Win32)
Engine-Mode: "ENABLED"

--f0360000-J--
2,251453318,"Z:\\test-240M.zip","<Unknown ContentType>"
Total,251453318

--f0360000-Z--

Doctor · Joined: 19 Jun 2012 Posts: 10 Location: Taiwan, Taipei

I try to modify the SecRule MULTIPART_UNMATCHED_BOUNDARY to remove the 'deny' and I can upload a file with 240Mbyte.

Then I increase the file to 330MB. The upload fails again and the apache restart with the following log.
Actually the file is uploaded completely into SecTmpDir of ModSecurity.

[Sat Jun 23 17:11:43 2012] [info] mod_fcgid: server 192.168.0.43:d:/Apache/php/php-cgi.exe(3888) started
[Sat Jun 23 17:12:37 2012] [error] [client 127.0.0.1] ModSecurity: Warning. Match of "eq 0" against "MULTIPART_UNMATCHED_BOUNDARY" required. [file "D:/Apache/Apache/modsecurity/etc/modsecurity.conf"] [line "79"] [id "200003"] [msg "Multipart parser detected a possible unmatched boundary."] [hostname "localhost"] [uri "/test2/psmpodup.php"] [unique_id "T@WIXMCoACsAAASMAnAAAAAB"]
[Sat Jun 23 17:14:10 2012] [notice] Parent: child process exited with status 2147483655 -- Restarting.

James Blond

What does happen if you disable / do not load mod security?

Doctor · Joined: 19 Jun 2012 Posts: 10 Location: Taiwan, Taipei

Without mod_security, it fails to upload file > 200MB.

Adding mod_security is originally for the purpose to monitor more logs that what's wrong with Apache and mod_fcgi while uploading large file.

mrcullers · Joined: 16 Jun 2012 Posts: 22 Location: USA, Bradenton FL

I am not sure if this will help you any, but I still to this day stick with perl for any uploading simply because I can't stand php's strict uploading environment. If you use perl to upload the server simply can only stop you by file size limit within an account space.

I wrote the program, The Dynamic FTP System which was all written in 2005 in Perl. Any app I have written since I still switch to Perl for uploading and come back to PHP to continue.

Just my experience.... If it helps any.

Doctor · Joined: 19 Jun 2012 Posts: 10 Location: Taiwan, Taipei

Thank you for your advise. I don't have Perl experience. I can't modify like that because the upload involving other dynamic metadata fields.