in my opinion there is no “optimal way” for every purpose. let me try to explain why (i have to go into detail for that). first of all some definitions, since “codec” (short for Coder-Decoder) can be used synonymously for different things:
- the encoded video (and of course audio) itself
- the software which does the encoding/decoding (which is oftenly not the same software)
first some video encoding basics
encoding a video to a certain format basically serves the purpose of saving bandwith/memory. usually codecs are optimized to meet certain goals - the most important are:
- en/decoding speed/latency - especially when it comes to streaming)
- encoding quality
- resource minimization (memory and cpu)
codecs usually can meet only 2 of these 3 criteria and for playback of video in v4 quality is usually top priority. the choice between resource minimization vs. speed is dependend on your application (i’ll come back to that later).
most high quality codecs nowadays aim for quality and resource minimization. most widely used is MPEG-4 Part 10 (aka AVC or H.264) which is a standard that defines a range of methods to encode videos in high quality. the thing is that you don’t have to use all the methods to encode your video (and the other way round: not every H.264 decoder is capable of decoding every H.264 video). the same is true for MPEG-2 video by the way.
H.264 is very efficient compressionwise in a frame by itself (using methods that can roughly be described as jpeg-like methods) and in comparing adjacent frames for changes in the image. simply said it for one frame it stores full image information (we call this an I-Frame) and for the following images we just store the differences between the current frame and some frame we are referencing (P and B frames). lots of memory is gained by encoding just the differences but this also means that to be able to decode a P or B frame we also need the frames they are referencing.
this is important for random access of the video (“jumping around”). many decoders just allow you to jump to positions where full image information is available (an I-Frame). in some videos two I-Frames might be over 200 frames apart which is a lot even when considering a framerate of 50 fps.
luckily good H.264 encoders let you tweak many of the encoding parameters, like how many frames apart a new I-Frame should be set (you can even set every frame to bd an I-Frame, of course at the expense of memory and (maybe) decoding speed).
now there is lots of other codecs that store full image information in every frame (like Motion-JPEG) and are therefore good for random access. some people here report good results with the picvideo mjpeg codec (which i never tried).
when you play back a video you need a software to do that. windows supplies a set of routines to decode the video an show it via directshow and the windows media foundation (in win7). microsoft itself supplies a set of “filters” to decode a few video types. these filters can be extended by installing additional filters which enables windows to decode other formats via directshow. as you can guess FileStream (DShow9) uses these capabilites to decode videos. i found the following aspects:
- pros: lots of different filters available - especially when it comes to standards like H.264. there is tons of decoders and many of them have been optimized to utilize the graphic card to do some of the decoding work. this way i’ve been able to play full-hd even on a nettop computer flawlessly.
- cons: it’s easy to lose track which of the installed filters now is actually doing the decoding work. you have to get used to use tools like graphedit or win7dsfiltertweaker to be absolutely sure your computer does what it is supposed to do.
there are a few programs that do not rely on the windows functions to decode video. VLC player is one of them and the FileStream (EX9,Texture VLC) uses its routines to decode video.
- pros: vlc can do far more than just play video (which it does really good by the way).
- cons: not sure about stability and graphics hardware accelleration (other might know more about this).
one more option is the new Player node which allows you to load plain images directly and show them consecutively like a video. this was possible before also eg. with a filetexture and quickly changing the filenames but you would need a quick computer and very quick disk (SSD) to cope with the data. many people prefered images in the DDS format as this can be decompressed on the graphic card directly. what’s cool about the player node is that it can load multiple images at the same time in different threads which increases the probability of a smooth playback.
one other thing might be very cool about this player (but i have not tried this out yet): variable playback speed. just to be clear: this is possible with the filestreams as well but from my experience not smoothly. i’ll have to make some experiments to confirm this.
i have no idea about CPU usage using this method but i guess it is higher than using a FileStream. Also a quick disk is obviously nescessary as these single images are probably much larger than videos using temporal compression.
from my experience you have to try out a lot of different options to get to the method that is “optimal” for your application as different demands require different solutions. i myself made good experiences using MPEG-2 and H.264, even with the plain windows decoders (on win7pro). also ffdshow tryouts as a codec/filter pack worked great for me many times (and also has some nice post-processing features). its codebase relies on libav of the ffmpeg project, on which also VLC depends a lot. i’m not sure if it is still being developed and the new rising star aiming to replace ffdshow seems to be LAV Filter but i did not yet try out this one.
for encoding i use ffmpeg a lot on the commandline as it allows me to tweak most of the parameters. it has a steep learning curve though since you have to know what you are doing. as mentioned before, Handbrake is a nice gui relying heavily on ffmpeg/x264 and allows you also to batch convert videos.
depending on the application i used MPEG-2,H.264 and DDS-sequences with success.
wow, this has become a long one. i’m not sure if it helped you but at least i hope it gave you clearer insights why this topic is not so trivial as one would wish.