Tag Archives: Hardware

HD-PVR Post-production

As already posted, last year I bought the Hauppauge HD-PVR. Since I’m a geek (am I?!) and I like doing things my way, using all the software provided with the device wasn’t an option I was going to choose. Also, since I always want maximum picture quality I always set the HD-PVR to record at full bitrate (13.5Mpbs CBR).

As you can imagine recording at 13.5Mpbs will produce some quite big files, but thats all good as I have plenty of disk space to waste (temporarily). For example, a 1 1/2 hour recording will take approx. 9.7GB. That’s big, even with lots of disk space to waste. So, I convert the recording to at least an MKV (AVC/AAC) as an archive/computer playable version and sometimes a DVD (to watch on the TV set, which only has a DVD player, for now).

Now since I’m a command line freak (did I say I was a geek?!), here’s how I do it. In order to do the same you will need at least the following software (all free):

1. The first step is is to load the .TS (or M2TS) file in DGAVCIndex:

DGAVCDec

And save the whole stream into a DGA project (File -> Save Project or F4). This will also demux the audio and create a [filename] PID 1100 DELAY [delay]ms.aac file.

2. Then I create an Avisynth script to load the DGA project in VirtualDub to make a stream selection (strip the beginning/end and commercials). The script looks like this:

LoadPlugin("DGAVCDecode.dll")
V=AVCSource("Project.dga")
A=DirectShowSource("Project PID 1100 DELAY -180ms.aac", video=false, seek=false)
AudioDub(V, A)
DelayAudio(-0.180)

[UPDATE 1/3/2010: You may want to use seekzero=true instead of seek=false as some seeking may be required depending on the context.]

Make sure to match the DelayAudio() with the delay from the AAC file then load the AVS into VirtualDub. This part can be time consuming as this is where I seek through the whole stream to find commercials and other parts I want to strip. For every part to strip I write down the first frame and last frame numbers to insert into the AVS. Once this is done you simply trim all the frames you want to keep like this:

Trim(29, 13187)+Trim(19031, 57183)+Trim(60138, 62091)

This strips the following frame ranges: 0-28, 13188-19030, 57184-60137 and 62092 to the end. Basically, Trim tells Avisynth to keep the specified frame range.

3. Now is the time to start encoding. First you will need to calculate your bitrate to make sure the newly encoded stream will fit your target media. Open the Bitrate Calculator and input your stream length and audio bitrate (I use 128Kpbs) and select your target media. Keep in mind the target bitrate is an average bitrate therefore you may want to bring it down by 1-2% to make sure you won’t have to re-encode the stream a second time.

For encoding the video, I use the following 2-pass encode:

x264.exe --profile high --pass 1 --bitrate 16000 --threads auto --thread-input --output Project.ts.avs.264 Project.ts.avs
x264.exe --profile high --pass 2 --bitrate 16000 --threads auto --thread-input --output Project.ts.avs.264 Project.ts.avs

For the audio, I use the following 1-pass encode:

wavi Project.ts.avs - | faac.exe -b 128 -o Project.ts.avs.aac -

Then I end up with 2 files, Project.ts.avs.264 and Project.ts.avs.aac which I multiplex in MKVToolnix to produce an MKV file. Make sure to select your video framerate in mmg.exe as .264 files do not store that information (you should get a warning after adding the .264 into mmg). Optionally, you can also select the language for both streams. That’s it for the high quality archive copy!

4. To create a DVD compliant structure, I use mencoder (lavcodec) for encoding both the video and audio. Here’s the command line I use for a high quality encode:

mencoder.exe -mc 0 -noskip -ovc lavc -nosound -lavcopts threads=2:vcodec=mpeg2video:vrc_buf_size=1835:vrc_maxrate=9800:vbitrate=9668:keyint=15:vpass=1:trell:mbd=2:precmp=2:subcmp=2:cmp=2:dia=-10:predia=-10:cbp:mv0:vqmin=1:lmin=1:dc=10:vstrict=0:aspect=16/9 Project.ts.avs -o Project.ts.avs.mpg -ofps 30000/1001 -of mpeg -mpegopts format=dvd:tsaf
mencoder.exe -mc 0 -noskip -ovc lavc -nosound -lavcopts threads=2:vcodec=mpeg2video:vrc_buf_size=1835:vrc_maxrate=9800:vbitrate=9668:keyint=15:vpass=3:trell:mbd=2:precmp=2:subcmp=2:cmp=2:dia=-10:predia=-10:cbp:mv0:vqmin=1:lmin=1:dc=10:vstrict=0:aspect=16/9 Project.ts.avs -o Project.ts.avs.mpg -ofps 30000/1001 -of mpeg -mpegopts format=dvd:tsaf
mencoder.exe -mc 0 -noskip -ovc lavc -oac lavc -lavcopts acodec=ac3:abitrate=128:threads=2:vcodec=mpeg2video:vrc_buf_size=1835:vrc_maxrate=9800:vbitrate=9668:keyint=15:vpass=3:trell:mbd=2:precmp=2:subcmp=2:cmp=2:dia=-10:predia=-10:cbp:mv0:vqmin=1:lmin=1:dc=10:vstrict=0:aspect=16/9 -srate 48000 -af lavcresample=48000 Project.ts.avs -o Project.ts.avs.mpg -ofps 30000/1001 -of mpeg -mpegopts format=dvd:tsaf

Here you could tweak the first and 2nd pass for better performance (remove CPU intensive switches). Also, vpass=3 isn’t a mistake for pass no. 2 (you can read mencoder’s manpage for more details). Now at this point if everything went fine with the encode you should end up with a Project.ts.avs.mpg file.

5. Time to create the DVD structure with DVDAuthor. This is as simple as this:

dvdauthor.exe -t -o PROJECT Project.ts.avs.mpg
dvdauthor.exe -T -o PROJECT

Here, PROJECT is an output folder containing the DVD structure (AUDIO_TS and VIDEO_TS folders). The -t switch creates the titleset and the -T switch creates the table of content (which contains only the single titleset created from -t).

6. The last step to create the DVD is to burn the disc. Just fire up ImgBurn and select Create image file from files/folders and under the Output menu select Device to burn the result to disc.

Et voilà. Of course there is always a thousand ways of re-encoding a stream so keep in mind this is only one way of doing it… :-)

Binary Prefix Origins

How about having fun facts about the binary prefix mess? But why, you might wonder. Well it all started when I wanted to convert 1 megabyte into bytes. Easy you might say, simply multiply by 1000000 and you’re done! Yes, it should be that easy however its not. Why? Ahh the big question. Before answering that question, we need some background info.

Find 3 problems with this appellation: 1.44MB 3,5 inch floppy disk.

  1. The capacity, originally described as 1 440 kB (kilobytes) before being “translated” to 1,44 MB, is in fact a little over 2 % inaccurate because of the double misuse of a decimal prefix
  2. The physical size is 90 mm, not 3,5 inches
  3. The word floppy no longer really applies as it did to the 5,25 inch predecessor

Considering how perfectionist computer scientists should be its quite amazing to realize how they can screw up that bad in such a short statement. Be reassured, the blame is to be shared by many other parties.

So, now that we know how bad things are, how can we convert 1 megabyte into bytes? Well, it all depends (heard that answer before? If not, then ask that question to your nearest geek friend).

According to the SI standard everything is 10 base so 1MB is 1000000 bytes. If you want things to be 2 base then you should follow the IEC 60027-2 standard and say 1MiB for 1048576 bytes.  The capital “B” is not part of the standard, its referred to as bytes while lower case “b” is referred to as bits (8 bits for 1 byte). Easy enough?

Fore more info, check out Wikipedia.

(HD-)PVR happy [part 2]

So following my first post on this matter now I got an up & running system with plenty of power to stream HDTV. The next logical step was achieving the ability to play the stream flawlessly in real-time. Going down the road I might end up with a Linux media center but for now I’m focusing on playing/recording. There are many really good media players for Linux however I choose to go with MPlayer since my background experience with MPlayer is quite advanced (for CLI fans like me).

Once I had the HD-PVR driver loaded, recording was a simple matter to dumping /dev/video0 (or whatever your HD-PVR ends up on) into a TS file. I love Linux for that, no fuss. The driver supports some basic options like adjusting the bitrate and also boosting the input audio (since the device tends to loose some audio strenght when encoding). I’ve configured the driver to load with boost_audio=1 and adjusted the bitrate to the max (13.5Mbps) for maximum picture quality (I don’t mind the time required to re-encode later to fit a dual layer DVD or other media, plus I have plenty of storage space). So loading/configuring the driver and saving the stream is as simple as this:

# sudo modprobe -v hdpvr boost_audio=1
# sudo v4l2-ctl --device=/dev/video0 --set-ctrl=video_bitrate=13500000
# cat /dev/video0 > stream.ts

Just hit Ctrl+C to stop saving the stream. Finding the right options for playing the stream took a bit more time & tweaking since the HD-PVR stream is interlaced. I ended up with a command line that looks like this:

# cat /dev/video0 | mplayer - -correct-pts -vc ffh264vdpau -vo vdpau:deint=0 -mc 0 -delay 2.154 -cache 8192

This -correct-pts is required to play the stream live but can be omitted when playing a recorded TS file. The -mc 0 and -delay 2.154 are required only for playing the live stream as well (the delay might defer for you). You may not need the caching either when you’re playing a saved TS file. Since the box has quad power I can also use multi-threading decoding:

# cat /dev/video0 | mplayer - -correct-pts -vc ffh264 -vo vdpau -mc 0 -delay 2.154 -lavdopts threads=4 -cache 8192 -vf field=0

With this CPU the multi-threading playback is as smooth as using the GPU VDPAU capabilities. Spreading the decoding over several threads is one option of using multi-core CPU. The other is to use a multi-threading aware version of MPlayer/ffmpeg (still experimental and has issues). If I want to play the live stream and record at the same time I simply add a pipe and use tee (did I say I love Linux?):

# cat /dev/video0 | tee stream.ts | mplayer - -correct-pts ...

This is all very useful but a lot of manual intervention is required every time I want to record/pause the live stream. The plan is to come up with some interface to cover all the basic functionality of a fully functional PVR:  play/pause/stop/record. I haven’t looked around for such an interface, however I’m thinking this could be a sweet project as it seems fairly easy to build (projects, projects, projects… :-P ). This could also make an excellent future post!

(HD-)PVR happy [part 1]

Last year I bought the Hauppauge HD-PVR model 1212 since I wanted to record HD TV at home (HDCP being an issue I had to go analog). My goal was to setup a home PVR with spare parts lying around and configure the device on that box, running Linux. Well, it didn’t happen… until last week!

Support for HD-PVR running at 1080p just didn’t work back then as the Linux HD-PVR driver was very alpha and MPlayer had a hard time feeding on the HD-PVR stream (which can be H.264/AAC with the analog audio input or H.264/AC3 with the optical audio input).

Now that the HD-PVR Linux driver has entered mainline since v2.6.30 I thought it was a good time to try the setup again. At the same time I updated the hardware used in the PVR (I bought a Q9550 and 4GB of DDR2).

I installed Ubuntu Jaunty 9.04 on a pair of ST3300831AS drives so I had to setup software RAID as I’ve set drives in stripping (RAID 0) inside the MediaShield BIOS (the box is using an EVGA 630i/7150 planar). The system has also seen a video card upgrade to a 8800GT (which comes from my main station, which I upgraded to a GTX280). The onboard 7150 GPU may not have been enough for running full 1080p and besides, I had that 8800GT collecting dust.

Since the box’s primary role is to provide realtime video streaming I thought I’d be running some of the latest software available to ensure smooth playback. Ubuntu is great for running cutting edge software with the support of all the PPAs on Launchpad so I added the xorg crack pushers to my sources.list.d. It provides the latest X server along with an updated MESA. I also added the NVIDIA VDPAU Team PPA to get the latest NVIDIA driver v190 series. I guess maybe now you see where this is going since the HD-PVR feeds H.264 and the box has an NVIDIA GPU (hint). I know, I know, the box has a Q9550 which can handle full 1080p even without VDPAU… but its there and its supported, so why not use it? The extra power from the Quad could be used wisely.

As the latest kernel in Jaunty is v2.6.28 (as of this writing) I had to upgrade to at least 2.6.30 to get the HD-PVR driver from the mainline or compile it myself. Well, since I’m lucky and I chose to run cutting-edge packages from the xorg crack pushers PPA I also have access to an updated kernel package v2.6.30! Awesome.

To be continued…