Less than a year ago, I figured out how to export TV from MythTV to write to an SD card for taking TV on the go. I did eventually get things to work but even after all my effort there, I wasn’t getting the video compressed at all. Also, it took a long time just to write the video to the SD card. I wanted to improve the overall process.
First, the hardware. My SD card reader was in the front of my main MythTV computer which lives inside an “nMediaPC” case. The card reader, therefore, came with the case and had a cable for me to plug into the motherboard, an ASUS P5N7A-VM. That means the card reader is capable of speeds as fast as I-have-no-idea. It was November of 2010 that I first got MythTV into a case and the motherboard is USB 2.0 – cutting edge for then. But a lot has happened to SD cards in 7 years. 7 years ago, we were excited about SD cards that held megabytes. Now, my latest SD card holds 256 gigabytes. And of course, transfer speeds have increased now too on the latest hardware.
I knew I wasn’t going to upgrade the motherboard or the case at this time, so I just wanted to find a way to get the latest transfer speeds with minimal changes. I added a USB 3.0 PCIe card and into that new card I plugged a USB 3.0 card reader. I tested everything interactively and confirmed the two devices worked. Plug and play. No drivers to deal with. The only thing that was a bit disappointing is that in the /dev/disk/by-id folder, the links for the new gizmo showed “usb-Generic_STORAGE_DEVICE” rather than including the thing that was being used like the older one which included the text “usb-Generic_USB_SD_Reader”. I did briefly search for drivers to see if I could improve the listing there – i.e., make it more precise. But since it worked and since I didn’t easily find any drivers, I moved on.
The final hardware issue was getting the new SD card mounted. (Okay, I guess sort of software?) The new SD card came formatted with exFAT. That seems like the smart choice for SD cards and yet Linux doesn’t support it natively. (Add this to my list of rants against Linux and why Linux can’t be a real OS for regular people yet.) Fortunately, installing exFAT compatibility is relatively easy. Just install the exFAT stuff with: <code>sudo apt-get install exfat-fuse exfat-utils</code> and you can use “mount” to mount the SD card.
Next was the software. One of the problems with my script was that it didn’t compress the files at all. I figured what I lost in terms of the number of things I could fit, I would gain in how quickly I could copy them to the SD card. But a secondary problem turned out to be that the native format of the recordings from MythTV didn’t play well on my devices – the video was sort of tearing and full of lines (perhaps an interlacing issue). I discovered that I did need to transcode the files to H.264 to get them to play well, anyway. And I had modified my script to include the transcoding some time in the past year. The files played well on most of my devices but on my wife’s iPad, there was no audio. (For info about how the iPad can see these files on an SD card, I’ll write another post.)
Now with an even bigger SD card capacity, smaller files were even less of an issue but I still wanted to transcode them. I also needed to fix the audio issue for my wife’s iPad. But I didn’t want the transcoding to take very long because when it comes time to go somewhere, the last thing I want to do is plan the TV I want to take with me three days in advance. Way easier if I could just load up TV in the last hour or two before I leave.
I started by trying to find more efficient parameters for ffmpeg. I eventually shifted from profile of “slow” to profile of “ultrafast”. While definitely being “ultrafast” compared to “slow”, it still takes about half of the real time of the recording to transcode. I think that is probably an increase of about 5 fold over what it had been with “slow”. So instead of a 1-hour recording taking 2.5 hours, it’d only take a half hour to transcode. An improvement but not the 5 minutes I was hoping for.
I spent a long time fiddling with HandBrake to try that as an alternative to ffmpeg. I had used the HandBrake interface ages ago to rip a DVD but eventually decided to rip DVDs through a script that uses mencoder and mplayer instead. But this time I wanted to try the HandBrakeCLI. I encountered many problems. First, it failed with the audio codec default. What kind of software can’t run with the default codec? I switched to force it to “ffaac” (the two “f” version, not the one “f” which also didn’t work) and I got past the audio encoder only to fail with the video encoder. Googling seemed to indicate that HandBrake is a mess when it comes to deployment on Ubuntu and that I should be using a “stebbins” build instead. So I added the PPA and tried to use that version instead. But I could never satisfy all the dependencies. After what seemed like a few dozen dependencies that I tried to install, I eventually got to one where one part of the code was dependent on one version of a library and the other part was dependent on a different version – no way to resolve it short of recompiling. I’m out. I unwound the “stebbins” stuff and said goodbye to HandBrake. Again.
Okay, so with my CPU, it looks like half real time is as good as it is going to get. Too bad, but I’ll have to deal with it for now. Next, I needed to update my script to work with the new card reader. It’s a little hacky but I want to be able to work with either card reader, so I have it try for the new one and if it fails there, it moves on to the older one and tries there. While hacky it works for what I need for now.
But it got stuck on trying to mount the SD card. I am using “pmount” in my script so I don’t need to deal with permissions to mount the SD card. (Another thing to add to my Linux rant.) But apparently pmount doesn’t know exFAT. Oh crap, there’s a bug with pmount. So I’ve fixed Linux to work with a file system it should have already known and then I fixed it so any user could insert an SD card without needing root privs. But the two fixes didn’t work together. Stupid Linux. Okay, I guess I did rant a bit about it here.
Fortunately, the bug report included a patch. Unfortunately, that meant that I would need to compile it myself. I figured I’d give it a try. I got the download for my version of Ubuntu. I had a few dependencies I had to get, but not too bad and then the <code>configure</code> ran through cleanly. Nice. Now to patch. Nope, the patch failed. Ugh. Okay, what’s in the patch. One change to one file. That looks pretty easy. So I changed the C source file with a manual edit, ran the <code>make</code> and <code>make install</code> and it worked without any more hitches. Except I still got an error when mounting the SD card with pmount. I rebooted and finally the error went away. Now I was finally able to run my script without any errors.
How about the performance results? The transcoding continued to be about half the duration of the video to be transcoded which was about 5 times as fast as before. And the copying of a file to SD after transcoding ended up 5 times as fast as it was with the old SD card and the old card reader. I guess I had thought I had been using up more time with the copy and less with the transcoding. Now knowing how long the transcoding was taking and that I was able to speed that up considerably from the “slow” it had been, the whole process is way faster. But my new hardware only contributes slightly to the big improvement.
Here’s my script – as always, no warranty, no support. export-to-SD.sh