Since posting about the week of 1983 TV Guide viewing, I've had questions from some people wondering about the storage and other hardware and software we use for our media library. It's really not very complicated to do, though I do have preferences and recommendations.

So here's what we've got.

Motivation:

Mainly I don't like the level of control streaming companies have. That they monitor everything we do, and that stuff comes and goes from services like Netflix and Amazon Prime on their timeline, not mine. I don't like the concept of paying for things like Spotify so that I can rent access to music I already own.

I realized like 15 years ago that while we often spent $200/$300 per week on CDs earlier in our marriage, Natalie and I were drifting away from actually listening to it much, because who wants to dig around for a CD to hear one song, then move to another CD. Ultimately, the same applies to movies, we have lots of DVDs, and I don't want to have to dig through booklets just to watch a couple of James Bond movies.

It's super easy to maintain, and we like being able to watch Saturday morning cartoons, "Nick-at-Nite" or throw on music videos while we play arcade games and eat pizza. Once up and running, it's all pretty much push-button access to all the media we like.

Media:

- 2000-2500 CDs (Maybe 200GB of music)

- Couple hundred movies, really probably not as many as most people.

- Lots of TV shows. Space-wise, this is where it adds up fast when you're ripping a box-set of 10 seasons of some show.

- Commercials, mainly from the '80s and '90s, but I'll grab anything fun that strikes us.

- Music videos. We have an overall collection of around 2000, and a subgroup of about 700 which represent "'80s arcade or pizza place" music. That's music that was just ubiquitous when we were growing up in the '80s and early '90s, and you heard it all the time whether you liked it or not. I've since come to appreciate these songs and bands in a way I didn't when I was a dickhead punk kid.

So all told, there's about a 5TB library of stuff, mainly TV shows, but also a decent music library that needs to get maintained and served.

Hardware:

- Ripping machines - Mainly, all I need is the maximum number of DVD trays I can get my hands on. There's nothing special here. My tools work on Mac or Linux so I can work wherever. We have one main Mac Pro that has 2x 8TB drives mirrored which hold the master copy of the media collection.

- NAS - Seagate GoFlex Home from like 10 years ago. I think I originally bought this with a 1TB drive, and have since upgraded it twice, which is kind of a massive pain. Now it's got an 8TB drive which has a copy of the media library from our main machine. I'll get into the pros and cons of this thing below.

- Raspberry Pi - I have a multi-use RaspberryPi which does various tasks to make things convenient and optimizing TV viewing. There are a handful of scripts which create random playlists every night for various categories of music videos, TV shows (Sitcoms, 'BritBox', 'Nick-at-Nite'), etc. It also runs mt-daapd, which I'll get into below.

- Amazon Fire Sticks - We have a couple of them. I'm not super impressed with their 8GB storage limit, but I'm definitely happy enough for the money they cost. They're cheap, around $20 now, and they do what they say on the box. Play video. I have side-loaded Kodi 17.x, but they seem not to quite have the resources for 18.x, though I'm really not sure why not. It's just slower.

- The Shitphone Army - I've got obsolete phones (Samsung Galaxy S4-ish) around the house and decent speakers set up so we can have music playing while doing the dishes for example.

Software:

- Kodi - I mentioned Kodi, which is just an excellent Free Software media library manager. Kodi gets /such/ a bad rap because of all the malware infected pirate boxes for sale, but you never see much from people who actually use it to manage a locally stored library of media they own. Can't recommend it enough. Get familiar with customizing menus in Kodi and making home-screen buttons linking directly to playlists. It's worth it and makes it look nice and easy to use.

- mt-daapd - I'm running out of patience with music streaming, though everything does work right now. MT-Daapd just basically serves up a library of music using the DAAP protocol, which used to be used by iTunes

- DAAP (Android app) - This could be great, but it seems to be completely un-maintained, and somewhat recently moved from being open source to closed, so unless I have an off-line copy of the source, there go my dreams of updating it. But it works well on the Shitphone Army and on the road so we can basically stream from anywhere. Other DAAP players for Android are pretty much all paid applications, and none of them seem to work better particularly than DAAP.

- Scripts A handful of poorly written scripts for ripping DVDs and maintenance of the library (below)

Recommendations:

Players - While the Fire Sticks work great, they're really very dependent on having constant access to Amazon. Were I installing mainly a Kodi machine, it would be much better to use a Raspberry Pi either with a direct-connected drive or mounting a network share. It's super easy to set up with ready-to-go disk images which boot straight into Kodi.

Playlists - Create lots of playlists. Playlists and randomizing things are two things that Kodi is terrible at, so I don't try to make it do it. These scripts run nightly on the Raspberry Pi and make .M3Us for us.

Filenames - Have a good naming convention. All my playlists are M3Us of just lists of files. That means that you don't get Kodi's metadata database with the pretty titles and descriptions, and so the files must be named descriptively enough that you can tell what episode you're looking at from the list of filenames. My template is "Name of the Show - S02E25 - Title of the Episode". Kodi's scrapers work well with that format and it makes it easy enough to fire up the Nick-at-Nite playlist and decide where to jump in.

At various times, I've considered parsing a copy of the Kodi database to suck out the metadata and add it in before the file location. In an M3U, that looks like this:

#EXTINF:185,Ian Dury & The Blockheads - There Ain't Half Been Some Clever Bastards
/mnt/eSata/filestore/CDs/Ian Dury & The Blockheads/Ian Dury And The Blockheads The Best Of Sex & Drugs & Rock & Roll/17 There Ain't Half Been Some Clever Bastards.mp3

It seems like having all that sqlite stuff happening would add a lot of overhead to generating playlists, and having well-named files saves me from having to worry about it, so I haven't bothered.

Storage - Though I use a "Home NAS" product that overall I've been pretty happy with, it does irritate me. Consumer market stuff is /so/ proprietary that it's quite hard to just get to the Linux system beneath and customize it the way you see fit. Specifically in the case of the GoFlex, "rooting" it even involved replacing Seagate's customized version of SSH with a vanilla one. Screw that up and you brick the device. I also run into network bottleneck issues with that thing. While you can enable jumbo frames, for instance, when syncing new content the CPU gets pegged, I believe I'm running out of network or disk buffer, which is kind of unacceptable in a NAS device.

Building it today, I'd just use a Raspberry Pi 3 with a USB drive enclosure. For the time being, my growth curve is still (barely) pacing along with the largest "reasonably priced" drives on the market. My ceiling is about $200 per drive when I do upgrades, because I am a very cheap man.

I have no opinion on consumer RAID arrays. I can only imagine consumer RAID based NASs come with all the shit I hate about the GoFlex. Yes, I'm biased against consumer grade garbage tech and that's probably not going to change. I'll have to buy one someday I'm sure, but for now it's all being kept simple.

Backups Keep backups. While I have multiple copies of everything, it does make me somewhat nervous that the only part of the media library currently being backed up off-site is the MP3 collection. That's got to change, and rsync is your friend. Ultimately I'll probably end up upgrading my home Internet from 20Mb/2Mb to something which will allow me to sync over a VPN tunnel to somewhere off-site (friend's house, work...).

Sample Scripts:

Here are some samples of the shitty bash scripts that run this whole nonsense. I know the better ways to write these, but the fastest possible way to hammer these out worked well enough and there's no way I'm going to bother going back and fixing them to be honest.

Rip CDs

I use an application called MAX on the Mac to rip CDs. I think its usefulness might be coming to an end, and I'm not sure what to do about that. It uses (used?) MusicBrainz database to automatically fingerprint and tag discs, but the last CD I ripped it seemed to have problems. You can run iTunes side by side with Max and drag the metadata over from there, so maybe that works well enough?

Anyway, I use that because I rip to both 320k CBR MP3 and FLAC. I have a shitload of stuff that really should be re-ripped since they're 128k and no FLAC, but I've so far been unmotivated to do so.

I wrote a bunch of stuff to move all the output files around and update iTunes libraries. Honestly I don't rip a whole lot of new music, which is a shame and which I should really fix.

Rip DVDs

DVD ripping is a lot more fragile than it should be. Good software like Handbrake are bullied into removing the ability to rip protected DVDs, and things are being pushed toward the commercial. I use mencoder in the script below.

DVD titles are sketchy at best, and as far as I know, you can't really fingerprint a DVD and scrape titles in the way you can with CDs. So I do what I can. I take whatever title the DVD presents and make an output directory based on that name plus a timestamp. That way if you're doing a whole box set and all the DVD titles are the same they're at least writing out to separate directories and not overwriting each other.

As far as file-naming, unfortuantely we don't live in the future yet and that's all down to manually renaming each output file. I use the information from TVDB, not IMDB, since that's the default library used by Kodi's scrapers. Sometimes the order of things is different between that and IMDB (production order vs airing order vs DVD order issues plague this whole enterprise).

#! /bin/bash

timestamp=`date +%m%d%Y%H%M`
pid="$$"
caffeinate -w $pid

id=$(drutil status |grep -m1 -o '/dev/disk[0-9]*')
if [ -z "$id" ]; then
echo "No Media Inserted"
else
name=`df | grep "$id" |grep -o /Volumes.* | awk -F "Volumes\/" '{print $2}' | sed 's/ /_/g'`

fi
name=`df | grep "$id" |grep -o /Volumes.* | awk -F "Volumes\/" '{print $2}' | sed 's/ /_/g'`
echo $name
dir="$name-$timestamp"
mkdir /Volumes/Filestore/dvdrip-output/$dir

echo $dir

for title in {1..100}
do
/Applications/mencoder dvd://$title -alang en -ovc lavc -lavcopts vcodec=mpeg4:vhq:vbitrate="1200" -vf scale -zoom -xy 640 -oac mp3lame -lameopts br=128 -o /Volumes/Filestore/dvdrip-output/$dir/$title.avi
done
chmod -R 775 /Volumes/Filestore/dvdrip-output/$dir

Playlist Script

The simplest Music Videos one below just looks at one directory of videos and one directory of TV commercials and randomizes all the content into an M3U. The more complicated ones have dozens of directories, and I'm sure I'm doing this array-building the wrong way. I'm sure I could have a text file with the un-escaped directory names I want and read that to build the array, either way, it really doesn't matter because if I want to add a TV series, I still have to edit a file, so this works fine. I've also thought about having a file in each directory like ".tags" that I search for terms in, like "comedy,nickatnite,british" and build the array from that, I dunno, sounds like work.

#! /bin/bash

array=`find ./ -type f;
find ../../Commercials -type f`

printf '%s\n' "${array[@]}" | sort -R | grep -v dvd_extras | grep -v "./$" | grep -v "\.m3u" | grep -v -i ds_store | grep -v ".nzb" | grep -v ".srt" > full-collection-random.m3u

- rsync the TV library. I have several of these, one for TV shows, one for movies, music videos, mp3s etc. It's just somewhat faster to only sync the thing I'm actually adding content to, rather than have to stat the entire library every time I rip a single DVD. The TV show sync tool also deals with the playlists, which are actually created on the NAS drive, so they have to be copied local before syncing or else they'll just get destroyed every day.

This checks to see if the NAS volume is mounted, if not it will mount it and re-run the script.

#! /bin/bash

mounted=`cat /Users/xrayspx/xrayspx-fs01/.touchfile`

if [ "$mounted" == "1" ]
then

cp ~/xrayspx-fs01/Common/TV\ Shows/1\ -\ Playlists/* /Volumes/Filestore/Common/TV\ Shows/1\ -\ Playlists/

rsync --progress -a --delete /Volumes/Filestore/Common/TV\ Shows/ ~/xrayspx-fs01/Common/TV\ Shows/

~/bin/umounter.sh
exit 1
else
mount -t smbfs //192.168.0.2/filestore ~/xrayspx-fs01/
~/bin/synctv
fi

WHAT HAPPENS NEXT? Stay tuned and wait. for. it.

Ok, so I had just gotten off of 128, and I wasn't going a thousand miles an hour, and there was no Stop 'n Shop, but there was a Wegman's. I threw RoadRunner on, because it's the very best song to finally break the 300,000 mile barrier on this rustbucket. Aaaaand, it didn't do it. Why yes, I am a sysadmin, why do you ask?

Just a bit further toward home though, I found a real Plymouth RoadRunner Superbird in a parking lot:

Natalie and I think it's this one right here.

So yeah apparently the odometer thing's A Thing with Toyotas from the mid-2000s. And they don't see it as a "design flaw" that every inspected car becomes illegal at 299,999 miles. It's like $500 to fix, apparently, which means it's still worth it to keep it on the road, but it is the second-costliest and absolutely the most pervasive and 100% predictable non-recall inspection-failing repair in this car's 14 year history.

A couple of years ago, Natalie and I canceled cable since we found it had literally been a year since we watched anything live on TV. I've built a pretty good "lattice of convenience" to store a media library of "Crap we like" and conveniently stream it pretty much anywhere.

Over the years, we've collected maybe 3000 CDs and a several hundred DVDs, including many box sets of TV series we like. I feel like we spent a TON on CDs when we were younger that most people didn't.

For a couple years now, I've been telling Natalie that if I had a couple bucks and an inclination to build a thing or interact with people, I'd Do It Right. I'd make fresh donuts daily. If I worked at Red Arrow, I'd make the case that the absurd Milford, NH third shift should dedicate themselves to making bread for the rest of the day. Instead, they recently discontinued that shift.

So in that vein... I have a lot of music videos. To go with them, I've downloaded a bunch of '80s commercials for Nostalgia's Sake. Things like cereal, BMX bikes, Underoos, Schoolhouse Rock, and since you can guess I'm from the Boston market, Spags and Dunkin' Donuts.

At Spags, they'll save you money:

These old commercials, pre and post Fred, all tout the freshness of their product. It's the "Freshest you can buy". Then they stopped making donuts on site. There is no longer a fryer in the building at this point. I guess they nuke or salamander their croissant, bagel or muffin sandwich and formed eggs, I dunno they do something to make them hot, but none of this stuff could be mistaken for "fresh".

Sadly, I imagine that by and large, they were right. They /are/ the freshest donuts most people can get. Their 1980, Pre-Fred accusations that "most super markets donuts are made by machine" actually came about in my store in around 1994 or 1995. The day the woman* who fried and filled and decorated the donuts every day moved to Heath and Beauty Aids, which was coincidentally the day before /I/ learned to "make" donuts and muffins.

Muffins were from mix, + any other stuff like frozen blueberries, cinnamon apple chunks or cranberries or whatever else we'd mix in. Muffins were easy, but donuts were even easier. They came in frozen now, so all we did was heat 'em up, glaze them and put them out. We still glazed, filled, sugared, dipped them by hand, and the "baking" process wasn't that bad, they tasted fine, i guess. I never really ate them much. You tend not to want to eat muffins if you're covered in muffin mix for 5 hours every day. Now that I think of it, even though we were frying donuts daily, there's no way that wasn't just frozen dough that we bought in. Don't get the impression these were scratch-made, just rolled and fried on site. When I stopped making donuts and muffins, we had transitioned away form even making the muffins from mix, they now came in as frozen batter in muffin cups and plastic trays. We'd just move them to metal trays, bake 'em and serve 'em up. That was probably '95 or so at the latest.

The couple of times I've gone into Dunkin's (as it's now trying to rebrand itself officially) in the last 2 decades, I've come away with the impression that they don't even do that much anymore. I don't see that there's even room for a table to glaze and decorate anymore. Do donuts just come in all pre-filled and room temperature ready to put out? Like the Krispy Kremes at the convenience store?

There is still a good donut place in our town, but I imagine even they aren't actually scratch-making anything. It's just fresher than Dunkin's. Meanwhile I'm pretty sure the baker my parents went to in the '70s and '80s hand-made everything in his shop from scratch from donuts to birthday cakes.

* - Hey Millennials, here's some trivia: I remember the woman who made the donuts was about 23 or 24, and that she and her husband, who drove forklift as I recall, and with whom she owned a house, had managed to save enough in their water jug full of change to go on vacation in the Caribbean. Just sayin'.

Natalie and I are on a kind of meta-road trip. We're not actually going to see The Thing, but we're seeing the roadside attractions which have sprung up around The Thing to amuse and draw in visitors.

Today we went to Roadside America, which is a massive O gauge model railroad layout. 6000 feet, assembled over 60 years of one Laurence Gieringer's life, from when he was 9 until he died.

Natalie took tons of photos, but I put up 3 short videos covering about 15% or so, along one short edge:

Mine, mountains, farms:

The zoo:

Midcentury Downtown: