I’ve got that new tablet that I’ve been writing about, the Teclast X16, and I want to be able to “watch TV” on it. No, not live TV (except for the occasional truly live event, I have little use for live TV) but shows that have been recorded on MythTV. The shows are on the server and I could stream them from there using a MythTV Android client. And that does work. But if I can avoid needing a network connection, I’ll be better off and most importantly, then my TV can travel with me.
The “tPad” does communicate through Bluetooth and I could use a BT file transfer to load files to it. But I really don’t want to struggle with those kinds of connections. I could also do a USB transfer which skips the flakey BT connection but then I still have to worry about filling up the “drive”. I’d rather use the SD card slot on the tPad to store the stuff to watch. (Technically, it is a Micro SD and it is labeled as TF. Since I’ll be using an SD adapter to write to it from the MythTV server, for the rest of this post I’ll just refer to it as an SD card.) And rather than stuff in an empty card and then transfer stuff to the tPad and store it on the SD card, way easier to just put the SD card into my MythTV server and drop the files on the card there. Sure, there’s an element of sneaker-net involved. But this isn’t something I’m going to do daily and it really isn’t too much trouble to stuff a card into the front panel of the MythTV server. Besides, this way, if the SD card fills up, I know I have enough TV to last me a good long while and don’t have to worry about breaking the tPad functionality by accidentally filling up the main “drive”. The only real hard part was setting up MythTV to automate getting the recordings to the SD card.
The mission was to create a “User Job” that would write directly to the SD card and then I could use the front end to send a show to the card. MythTV stores files using somewhat cryptic filenames. They are the internal channel number followed by the date/time stamp. They are unique but they are not easy to browse through. So it isn’t like I could look through the pile of numbers and see a recording that I want to save to SD. Using the front end to select shows is way easier.
The first step was to create a script to be called as the User Job. The fundamentals of the script are pretty straight forward: given a file as input, write that file to the SD card using a new filename that is easy to browse. I should comment that I played around with both reducing the size of the file and stripping out commercials. (As a starting point, I used the info in this post from mythtv-users.) The shrinking just ended up taking longer than it is worth. Unlike say an iPhone, the tPad has a decent screen size and a resolution of 1920×1080 so I wouldn’t want to reduce the resolution too much. Which means while I might be able to shrink the file somewhat, it’d be a lot of processing delay for little payoff. As for the commercials, well, I’d be happy to strip them out but I had trouble with the documentation for the tools (lots of info about old ways, not so much about new ways). And unfortunately, since the commercials aren’t always detected with complete accuracy, I might find myself on a plane with a recording file that jumps over a significant part of the show. I may revisit the idea later (and use a script like this one), but for now, my script is going to just copy the existing recording. The script is included below.
The second step was to call the script from MythTV. I had done a user job before but it got lost somewhere ages ago after rebuilding the server so I was basically starting from scratch. It took some digging to remember where the User Jobs are defined – in the backend, not the frontend. And on the General setup section, near the end. After I did this, I was feeling pretty good. Back in the frontend, I selected a recording and fired the User Job from the Menu on the recording. And… nothing. Debugging this turned into a 2-day long affair. And that’s why I thought it deserved a post here.
The first problem was that I didn’t enable User Job 1 after defining it. I found that setting back in the General backend setup but it was off the side of the screen on the VNC window I was using so I had to scroll to find it.
After enabling the job, I could see the job turn red in the Information -> System Status -> Job Queue. I could select the job and tell it to requeue but I’d get the same result. Looking in the log file (/var/log/mythtv/mythbackend.log), I could see that the job attempted to run and that it failed quickly. But I couldn’t tell why. And none of the debug statements I was putting in my script were bubbling up into the backend log nor were any of the script actions leaving any evidence that the script had even started. I tried a number of things to make sure I had the right permissions (I did; already 755) and that the file was in the right place (including trying to put the script in the /home/mythtv/.mythtv directory) but still nothing. In desperation, I changed the User Job to call the script with a fully qualified path and that worked. The one thing I thought I remembered about creating User Job scripts – that they run from your .mythtv directory – turned out to be incorrect.
With the script now running, I can see evidence that it did start – finally progress. But it appears to fail right away. This was also frustrating to debug. I opened a second terminal session and switched to the mythtv user (using sudo su) and ran the command that was coming up in the backend log file as failing. In other words, my script with the parameters that were passed. This allowed me to see the error. The problem was the %STARTTIME% is returning the ISO version, not the plain one. It appears to be a bug in MythTV that STARTTIME and STARTTIMEISO are both returning the ISO version of the start time. So I needed to work around that by having my script clean out the punctuation characters from the STARTTIMEISO value.
After that fix, I could see a different error. This time it was complaining that it didn’t have permission to access the SD card to create any directory or file. And I was able to confirm that in my mythtv user terminal session. The problem is that the SD card was inserted while I was logged in as me and therefore Ubuntu had mounted it as a device for my user when I clicked on it in Thunar. Isn’t that nice? Well usually. But in this case, I wanted to have MythTV be able to write to the device on my behalf. To resolve this, I created the directory “SD” in /media and updated the fstab file to add the following two lines (source):
# SD card mounted as any user access
/dev/sdd1 /media/SD vfat rw,nofail,umask=000 0 0
This line is actually a bad idea. Despite the fact that the device looks like an SD card with “sdd1”, it is actually just the usual linux “sd” for any “drive” to be mounted followed by the next letter in line. And since I have three disks already mounted, the fourth one gets a “d”. It would be way better to use a UUID here instead of the device identifier. But what if I use different SD cards? I’d need to add an entry here for each UUID of each card I use. I suspect the next time I stuff in a USB drive, I’m going to get an error that it isn’t “vfat”. When I do, I’ll try to figure out a more clever way of ensuring that every mount of the SD card gets general access. For now, I’m cheating.
I tried running the script with the commands from the log file again in the mythtv user terminal session and got further. This time, it said it couldn’t preserve ownership because the FAT card doesn’t support permissions. In my script, I had wanted to preserve the date that the file was created so my copy command included the “-p” flag. But -p actually preserves timestamps, permissions, and ownership. So I changed it to --preserve=timestamps.
The script worked for the SD card I had been using which was a small test one – only 512 Mb and a full size SD card from an old camera. But now I needed to test the “real” one, a 64 Gb MicroSD card. As expected, the formfactor didn’t matter at all. But the storage capacity of the card did. Unlike my test card, I couldn’t format the 64 Gb one to FAT32 (known in Linux as “vfat”) since it has a 32 Gb size limit. I started by trying to format the new card on the tPad and that failed. I didn’t get to tell Android what format to write it as (I’ll save my ranting about the ridiculous limits of Android for a later post) and it didn’t tell me why it failed – all I know is after three tries, it seemed pretty clear I’d be formatting the card somewhere else. I tried formatting it as exFat on Windows but that wasn’t visible to Linux. The other choice for formatting on Windows is NTFS so I tried that and it worked on the MythTV linux server and the tPad could read it just fine. So I finally had the card formatted. (Who knew this would be a challenge!?)
Okay, time to put the new card into the server (using the SD adapter). Now I have mounting problems. First of all, the fstab change made above made it so removing the card didn’t automatically unmount the card so I don’t like that. Secondly, even after getting the prior card fully unmounted, putting in the new one didn’t mount that one correctly. And I tried to force it with a mount -a, I got the error I had been anticipating where it said that the filesystem was the wrong type. I tried just changing the line in fstab to be ntfs instead of vfat. Then when I ran the script, I got an “Operation not permitted” error. Apparently the kernel usage of NTFS is limited to read only access? Either that or the timestamp preservation was the problem. I switched the fstab line to ntfs-3g instead and that did the trick.
But again mounting and unmounting turned out to be a pain. Because now the ntfs-3g package was returning an error due to the drive being included in the fstab. And I still have the problem that my fstab is marking any drive I insert (including USB) as being the SD card with ntfs-3g. I could switch to UUID instead of device specific, but then I am limited to using only one SD card and still have mounting limitations where I’d need to sudo to do the mounting. I could add the “user” option to the fstab line to get around needing to sudo. But at this point I’m pretty well fed up with the use of fstab to meet my needs. So a few paragraphs above where I wrote that I was going to cheat “for now” turned out to be very short-lived.
I tried to get the Ubuntu automounting to work. Ubuntu would automount the card as the user that was logged in. But I was back to the permissions issue again. And stupid Ubuntu doesn’t have any way to automount any removable device as a drive available for all users. The more I play around with Linux, the more it is obvious why it can’t be a primetime consumer level OS. (Again, a rant for another day and another post.)
I eventually settled on the idea that I was going to need my script to do the mounting itself. That’s the only way I could ensure that the device is mounted and mounted as the right user with the ability to write to it. I thought about trying to get the script running as root and add the script to sudo access and all the other nonsense. But I’ve done that before and it’s messy. Besides, there ought to be a way to do this!! If Ubuntu can automount for the user, why can’t I do it from the command line? Well, I didn’t quite get an answer to that question but I did find the pmount package which gives me the same effective functionality (source). And with this functionality, I was able to update my script to include the ability to mount the SD card during script operation. And the script is clever enough to attempt to determine which mounted device is the SD card so it doesn’t always assume it will be sdd1 as my prior solutions did. (I used the by-id directory with links to find the right one by name and then used readlink to get the original device. lsblk, mount, blkid, and fdisk didn’t have the data I needed to get the device type.)
One final problem came up. While my script would run as my own user, it wouldn’t run as mythtv user. It was a similar permission problem. But I didn’t panic because I remembered reading in the pmount doc that for users to use pmount, they would need to be in the plugdev group. My normal user already was but the mythtv user was not. I added it the mythtv user but it still didn’t work in my test session. Out of time and out of patience, this was just about the end of the whole idea. And eventually I tried logging out of the mythtv user in the terminal session and logging back in as mythtv user again. That fixed it. Okay, so group changes apparently need to be made before login. Good to know. Fortunately, this isn’t an issue for actual implementation since each session run as a User Job from MythTV will be effectively a new session.
Finally, my script ran and did what I wanted it to do! A word of caution, however: it takes forever for Ubuntu to write to an SD card. It turned into about one half hour TV show would take about an hour to write – twice as long to copy a file as to watch the show in the first place. I had queued up 12 shows to write (varying in length from 30 minutes to 2 hours) and went to bed; when I woke up in the morning, it was only half way through. Aigh! Therefore, if you are heading off on a trip like I was, be sure to start copying stuff to the SD card days before – don’t wait for the night before to load the SD card. There are two things to pursue, therefore. One is potentially finding a way to speed up writing to an SD card form Ubuntu. I found some ideas about maybe changing the way the IO scheduler works with mounted drives but that might mean I need to go back to dealing with the fstab again so I may need to keep digging for better ideas. The other thing to pursue is to shrink the size of the file I am writing to the card. As I wrote above, I did give that some thought but ended up deciding storage space was cheap enough, it wasn’t worth the trouble; now, with the realization that it takes so long to write to the SD card, maybe it would be worth the trouble after all.
In summary, here’s the setup that is the net of all the efforts above.
- Add User Job 1 description and command to Myth backend setup. Description is
Write To MicroSDand UserJob1 is/home/<username>/.mythtv/write-to-microSD.sh %DIR%/%FILE% "%TITLE%" "%SUBTITLE%" "%STARTTIMEISO%" - Enable User Job 1 in backend setup
- Create script write-to-microSD.sh in the
/home/<username>/.mythtv directory(download below) - Set execute permissions on the script:
chmod 755 write-to-microSD.sh - Create directory to serve as mountpoint for SD:
sudo mkdir /media/SDand change permissions for it:sudo chmod 777 /media/SD - install pmount:
sudo apt-get install pmount - add mythtv user to plugdev group:
sudo adduser mythtv plugdev
And here’s the script I am using now. I’ll update the script content here if I change my script.
write-to-microSD.sh