[Page Previous 1, 2, 3 Next]

Topic: FFMPEG Windows Apache 64bit Limiting CPU Consumption

[2]

Topic: FFMPEG Windows Apache 64bit Limiting CPU Consumption

[C0nw0nk]

[jimski]

[C0nw0nk]

My log and process que is in the extension www.hwdmediashare.co.uk what just uses preg_match and writes outputs to the database.

Also if you want to know what process ID / video is currently converting a good way to know is to uses windows syslog and have php echo the currently converting media id name etc to there so you know.



You know if a conversion has failed via the fact you should mark number of attempts at converting a media item and +1 it each time it tries if it is looping you know you have a problem also using preg_match and retriving the ffmpeg output you know if a conversion has failed because ffmpeg will tell you.

How good is your knowledge on ffmpeg ? Perhaps you should understand the command line because your next hurdle will be making video outputs HTML5 friendly and it is not a walk in the park.

You need to make sure your mp4 outputs are compatible with all devices you also need to move metadata from the end of the file to the start another major problem is you can only use the aac audio encoded in mp4's because if you use mp3 or others IOS users do not have audio and can only watch the video with no audio then you have the issue of not every OS is compatible with MP4 files so you need to generate Both MP4 and Webm and FLV for those who do not use HTML5 and are still in the stone age of Flash.

Even using the settings youtube sets will cause issues you need to be custom. https://support.google.com/youtube/answer/1722171

Once you know ffmpeg's outputs you can just put preg_match in your php code to know if the output will fail or was a success if it fails you can tell php to delete or what ever you want it to do.

[jimski]

[C0nw0nk]

You have a massive task ahead of you my advice would be just buy or get a exsisting extension that already does all this work for you and you can customise it to your needs like i do.

here is a few i am aware of that you will proborly love and make your life easy aswell as save time and money.

http://www.hwdmediashare.co.uk
http://www.adultvideoscript.com
http://clip-bucket.com/

Clip bucket is the only free one there i used to use it but i prefer hwdmediashare since it suits my joomla needs it allows image, music, video, and regular file uploads so hwdmediashare was perfect for me since it did everything compared to the others.

I also use process lasso to restrict cpu on windows via assigning ffmpeg to specific cpu affinities. I leave 1-2 CPU cores free and assign maybe php or nginx to them and give ffmpeg the rest.

It depends on what your needs are i needed a CMS/Blog/Forum/Media component so i went with Joomla and HWDMediaShare it depends on what you want to create.

I will also add i dont know how good the others are on windows or with multiple conversion servers via network sharing i customised Hwdmediashare to do this task for me.

[jimski]

[C0nw0nk]

If you do encounter errors with your custom build and dont understand ask at http://superuser.com/ although i recon they will tell you what i told you that you should use a exsisting component and customise it since the job might be to big for you and take you to long.

Here is a part of my command line

[jimski]

[C0nw0nk]

No i use no third party software i use just what windows gives us in the first place.

In your PHP.INI set it to be syslog.

[jimski]

After playing with syslog I come to conclusion that I need to implement a job queue. Log parsing is just a substitute for a job queues and is not a robust solution for task processing.

And until PCNTL extension or equivalent is implemented on windows all job controls will be nothing else but a bunch of hacks.

[C0nw0nk]

[jimski]

[C0nw0nk]

Well my method works with no core hacks i have no clue on what way you are trying to do things in your app it is as simple as this.

$id This would be the id of the media item you are currently converting or grabbed with your script or cronjob to start your conversion in the first place.

[jimski]

[Jan-E]

With a little intelligence you can know if an instance of ffmpeg is hanging. You can use Sysinternal's PsList to find out how long ffmpeg is running ('Elapsed Time'). Depending on the length of your videos any ffmpeg that is running longer than, say, one hour is stuck.

http://technet.microsoft.com/en-us/sysinternals/bb896682

Just run 'pslist ffmpeg'. You don't even need the detailed view.

If you know the name of the source file, 'handle' will tell you which pid has that file open.

http://technet.microsoft.com/en-us/sysinternals/bb896655

[jimski]

Thanks Jan-E, this is just the silver bullet for ffmpeg vampire that I needed.

[C0nw0nk]

You dont even need that using preg_match from php ffmpeg outputs all its data to a console if you have launched it via php's exec, You can know if ffmpeg has stopped responding via the fact that it stops outputting data to the console i have never encounterd this though i have encounterd corrupt uploads that have caused some strange behavior but never ffmpeg crashes.

Do you know how to read outputs of external programs with PHP you should look into it. If it stops outputting data then you know it is frozen most likely i have never had this issue and i accept uploads of 2gb in size some videos even 10hours long that take more than 8hours to convert just for a single video.

If you are logging your outputs to mysql you need to increase a few mysql timeout values since mysql will close a connection after 8 hours causing problems and mysql also has a max packet size. You cant run on default values of configs everything needs changing up. Increase your PHP execution times too it may help and if your using Nginx set a limit to download speeds of files.

Like i said mysetup is a non dependant on third party app's as possible so non dependant all i need is to install ffmpeg imagick php and nginx and the site can run on any machine.

I think with that program you will end up having to read the outputs of that program with PHP instead of FFMPEG so why not just read ffmpeg's outputs first and see if it stops on a certain frame for longer than a certain period of time. Are you using the latest version of ffmpeg ?

[Jan-E]

[C0nw0nk]

My PHP Timeouts are actualy default because when you run PHP via cronjobs (cli) like i do the timeout value automaticly becomes 0 or -1 meaning it will only close the php process once the conversion is complete etc. (So when ffmpeg closes or stops running.)

http://php.net/manual/en/info.configuration.php#ini.max-execution-time

I only convert data via the cli.

As of mysql i did have to go overkill in this area because mysql closes connections after 8hours. And with 10hour long videos and more they can take a few days in some HD especialy WebM qualities to convert.
(Mysql timeout of current connection raised from default 8 hour to 2 days.)

[Jan-E]