#-----------------------------------------------------------------------------#
|                             UNATTENDED FFMPEG ENCODER                       |
|	        	https://github.com/supermasita/ufe             	      |
#-----------------------------------------------------------------------------#

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
(!) WORD OF WARNING: This proyect is in a development stage and not even the 
README is complete. Use it at your own risk, and if you want, join us :)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

1. INTRODUCTION
1.1 What is UFE ?
1.2 What is not UFE ?
1.3 Features
1.4 To-do
1.5 Credits

2. INSTALLING
2.1 Dependencies
2.2 Creating directories
2.3 Creating database
2.4 Creating FTP
2.5 Configuration
2.6 Installing crons

3. USING UFE
3.1 Encoding workflow
3.1.1 Checker
3.1.2 Encoder
3.1.3 Ftp

4. GETTING HELP
4.1 Troubleshooting
4.2 Known issues

-------------------------------------------------------------------------------

1. INTRODUCTION


1.1 What is UFE?

	UFE started as a bunch scripts used to encode and upload video in a 
completely unattended way. The goal is to make the encoding and uploading easy 
and foolproof, using presets to "normalize" all your videos, providing encoding 
best practices and specific quality for each device (broadband, mobile, etc.).
	
	When thinking UFE we had in mind the webmaster (or journalist, etc.) 
that has to encode several videos in different qualities for various websites, 
and upload them; UFE will make things easier.

	You can use UFE as a centralize solution for encoding videos, leaving
the encoding profiles to the Admin and not to the users, disabling them from
FTP access and not hogging their computers with CPU intensive tasks.

	UFE could be thought as a simple alternative to other encoding software 
such as: Handbrake, Sorenson Squeeze, Windows Movie Maker, etc. 



1.2 What is not UFE?

	UFE is not a fully featured video edition and encoding software. We 
focus on simple and reliable operation. 

	We plan to extend UFE with other proyects built around it but the "core" 
will always be as simple as possible.


1.3 Features
	
	* Multisite in mind. 
	* Video presets: you can create your own, enable/disable them (globally 
 	  or per site).
	* Simple MySQL structure (no auto increments, no joins).
	* Multiserver in mind (many encoders and master-master DB).
	* Uploads to FTP server using year, month, date folder structure.
	* Maximum number of FFMPEG processes may be configured for each encoder
	  (default: 2).
	* Non-blocking queues won't let a huge video consume all your processes
	  all leave all other waiting.
	* Logging in syslog.
	* Per video FFMPEG log.
	* Video hinting.

1.4 To-do
	
	* Create video thumbnail and store it as a blob.
	* Weighted queues.
	* Handle exceptions.
	* Create install script.
	* "Pythonize" scripts.
	* Create python daemon and avoid crons.
	* Monitor file last acces with inotify.
	* Improve logging.
	* Upload to folder as alternative to FTP.

1.5 Credits

	* Alejandro Ferrari - Sysadmin and Wowza expert 
	* Supermasita - Python coder wannabe 


-------------------------------------------------------------------------------

2. INSTALLING

(!) Don't hesitate to search for our help at Github if you have any trouble or
doubt about installation.

2.1 Dependencies

	* "FFMPEG" (http://ffmpeg.org/) (if using RHEL best install it from 
	http://atrpms.net/)
	* "Medianfo CLI" (http://mediainfo.sourceforge.net/)
	* "Pymediainfo" (https://github.com/paltman/pymediainfo)
	* "qt-faststart" (https://github.com/danielgtaylor/qtfaststart)


2.2 Creating directories
	
	1- Create a directory for the scripts (ex: /var/www/html/ufe/), where you will
	do the "git clone" or untar.
	2- Inside this directory create the following ones :
		* "video_origin" : were UFE retrieves videos for encoding.
		* "video_original" : where UFE temporaly moves videos retrieved 
		  from "video_origin".
		* "video_encoded" : where UFE temporaly stores encoded videos.
	3- Inside "video_origin" directory, create another directory called 
	   "default". Here you could create the origin directories for as many 
	   sites as you want to define.


2.3 Creating database
	
	1- Create a database called "ufe" (or whatever you like).
	2- Create a user ("ufe"?) and grants to use the database.
	3- Use the dump located in "lib" to create the tables, like this
	   (from command line):
   	   
	   [you@you ufe]$ mysql -u ufe -D ufe -p < lib/database_structure.sql
	
	(!) USE THE DB, USER AND PASS THAT YOU CREATED IN STEP 1 AND 2


2.4 Creating FTP 
	
	You will need at least one FTP account to upload your videos. Installing
and configuring an FTP server exceeds the reach of this document, please refer
to any online sources (http://www.cyberciti.biz/ offers great guide). 


2.5 Configuration
	
(!) This is the basic configuration to get you running with the "default" site.
Many sites can be configured in the same fashion. We will extend this README.
	
	1- Connect to DB and you would have to update the following fields in the
	   "sites" table with your credentials:
		* ftp_user
		* ftp_pass
		* ftp_host

	2- Modify the file "lib/config.py", changing the following variables
	   with your data (examples are provided in the file):
		* core_root 
		* incoming
		* original
		* encoded
		* tmppath
		* db_host
		* db_user
		* db_pass
		* db_database
		* ffmpeg_bin


2.6 Installing crons

	You will need the following crons running (change accordingly to your
	needs and setup):

	# Main scripts
	* * * * * python /var/www/html/ufe/ufe-checker.py > /var/tmp/ufe-checker.log 2>&1
	* * * * * python /var/www/html/ufe/ufe-encode.py > /var/tmp/ufe-encode.log 2>&1
	* * * * * python /var/www/html/ufe/ufe-ftp.py > /var/tmp/ufe-ftp.log 2>&1
	# Recycling (cleaning) of UFE
	* * * * * python /var/www/html/ufe/ufe-recycle_encoded.py > /var/tmp/ufe-recycle_encoded.log 2>&1
	* * * * * python /var/www/html/ufe/ufe-recycle_original.py > /var/tmp/ufe-recycle_original.log 2>&1
	* * * * * python /var/www/html/ufe/ufe-remove_old_registers.py >/var/tmp/ufe-remove_old_registers.log 2>&1
	# Cleans .xml and .err files created by "pymediainfo"
	* * * * * find /tmp/*.xml -mmin +5 -ls -delete > /var/tmp/ufe-mediainfo_expire_xml.log
	* * * * * find /tmp/*.err -mmin +5 -ls -delete > /var/tmp/ufe-mediainfo_expire_err.log

	(!) Contact us if you want to use inotify to monitor last access of
	    files with INCRON: 
	    (http://inotify.aiken.cz/?section=incron&page=why&lang=en)

-------------------------------------------------------------------------------

3. USING UFE

3.1 Encoding workflow
	
	* We assume that a video "test.flv" has been copied inside the 
	directory "video_original/default".

	* "ufe-checker" will scan the directory and if it finds that the file
	has not been modified in a given time period (defined in "config.py") 
	and continue.

	* "Mediainfo" will be used to read the video metadata. The file will be
	move to "video_original" folder and a register in "video_original" 
	table will be created.

	* Registers will be created in "video_encoded" table for each 
	"video_profile" defined in "video_profile" table that is globally 
	enabled, matches the video aspect ratio and is enabled for the site.

	* "ufe-encode" is spawned and it will encode any pending video.

	* After each encoding, "ufe-ftp" is spawned and any pending upload
	video is uploaded. 

	* "ufe-recycle_encoded" will erase encoded videos that have already 
	been successfully uploaded, in other to free space.

	* "ufe-recycle_original" will erase original videos, whose encoded
	videos have been already recycled.

	* "ufe-recyle_old_registers" will erase registers for those videos
	whose encoded and original videos have been recycled more than 
	48 hours ago.
 


3.1.1 Checker (to do)

3.1.2 Encoder (to do)

3.1.3 Ftp (to do)

3.1.4 States (to do)
	
	0 : disabled
	1 : pending
	2 : running
	3 : finished
	4 : error

-------------------------------------------------------------------------------

4. GETTING HELP

(!) To be done! Please contact us at Github!

4.1 Troubleshooting
4.2 Known issues