Hacks

xrayspx's picture

Sharp Boombox Repair

Music: 

Tom Petty - Freefallin'

RDPLauncher

TL;DR: Here's the Link:
RDPLauncher

I use RDP a lot and had some scripts to let me launch lots of RDP sessions without having to enter my random-generated passwords over and over. I wasn't happy with how I was handling those passwords so I've made it more secure using gpg and KeePassXC. Last night I made it compatible with Windows and MSTSC which will be uploaded here shortly once it's cleaned up a bit.

Basically I'll click a shortcut for whatever host, which runs my launcher. I get prompted for my GPG passphrase, which reads from an encrypted file containing my KeePassXC passphrase, which is then used to retrieve the user password for launching the RDP session.

Gpg-agent uses a cache-TTL to "hold the door open" for 10 minutes by default, so I can launch a bunch of sessions and only type my passphrase once.

Requirements:

- gpg client and running gpg-agent (gpg4win, etc) with a private key set up, etc.
- cygwin if you're running Windows
- KeePassXC (or some other key-store that has a command-line interface
to query the database. In the beginning I was just using the gpg file
with user/password pairs, so that works too)

The tool has a few neat features:

- If run from the command line with no arguments, it will prompt for user/pass/host/domain, good for one-off sessions to machines I won't log into much. That's great since I spend all my time in terminal windows and this stops me having to go back and forth to the mouse and keyboard while entering credentials.

- If launched with -b, it prompts you for information for a one-off connection, but will also build a new shortcut launcher from a template. So like for the first connection to a machine you know you're going to use a lot. (Linux/Mac only)

- Automatically tunnel sessions over ssh. This means I can launch RDP sessions on my Mac and they'll seamlessly proxy through my work laptop to the VPN.

For tunneling, I am taking an arbitrary range of 200 ports and incrementing them based on what's currently listening. If there's already a process listening on port 6201, then try 6202 etc until there's an open one. So I can easily open 20-30 ssh tunneled sessions each with its own ssh process which will close down when the RDP window closes. 200 is "probably overkill", which means it might just be barely enough in the real world.

The launcher shortcut mechanics are a bit different on my Linux and Mac machines so I split the -b script builder piece out based on OS. On Linux, I use KDE/Plasma, and so I generate these as KDE desktop files which look like this:

#!/usr/bin/env xdg-open
[Desktop Entry]
Comment[en_US]=
Comment=
Exec=/home/xrayspx/bin/rdplauncher.sh -h it-host.xrayspx.com -d xdomainx -u xrayspx
GenericName[en_US]=
GenericName=host.xrayspx.com
Icon=remmina
MimeType=
Name[en_US]=
Name=host.xrayspx.com
Path=
StartupNotify=true
Terminal=false
TerminalOptions=
Type=Application
X-DBUS-ServiceName=host.xrayspx.com
X-DBUS-StartupType=
X-KDE-SubstituteUID=false
X-KDE-Username=

On the Mac side, I use shell scripts with the extension .rdp (which conflicts with Microsoft's client, but I don't care since I never use their client anyway). Those just launch using Terminal, so it does pop a terminal for a fraction of a second, but I really don't have a problem with that. To get the Terminal window to close (and I do associate these files with Terminal.app specifically rather than iTerm2), open Terminal.app, go to the Terminal menu -> Settings -> Profiles (tab) -> "Basic" or whatever profile is your default -> Shell (tab). Choose what action to take when the shell exits. I have it set to "Close if the shell exited cleanly" and "ask before closing" set to "only if there are processes other than the login shell..."

The launcher for that looks like:

#! /bin/bash
rdplauncher.sh -h host.xrayspx.com -d xdomainx -u xrayspx &

I generate those from the KDE .desktop files with a command like this:

for host in $(ls | grep "\.desktop$" | awk -F ".desktop" '{print $1}'); do cmd=$(grep Exec $host.desktop | awk -F "xrayspx/bin/" '{print $2}'); echo "\!#/bin/zsh" >> $host.rdp; echo "$cmd &" >> $host.rdp; done

That creates .rdp files in the same directory as the .desktop files, so now they can be moved around, have chmod set, etc.

If I call it with AppleScript or Automator instead of a bash script as above, none of the password retrieval process works. I think it short circuits and sends the output back to the AppleScript rather than the bash script which ran the command. If I can get that working that would be ideal.

The mechanics on Windows are similar to the Mac method. a .bat file which launches the bash script via Cygwin:

C:\cygwin64\bin\mintty.exe -w hide -e /bin/bash -l -c '/home/user/bin/rdplauncher.sh -h host -u username -d domain'

On Windows at least the Cygwin window it creates is hidden from the user, so that's nice.

xrayspx's picture

Lots of RDP

Music: 

Annie Lennox - Why?

Do you do lots of RDP? Like lots and lots? I do, and even with password management it's annoying. I tend to use generated passwords for all my normal user, Domain Admin user and obviously Administrator accounts. That means lots of workarounds to deal with those passwords while doing bulk RDP sessions.

A typical use case for me is to RDP to 20 machines at a time, run a thing, wait, and log out. I've always scripted this, but not always in strictly the safest way. Plaintext passwords stored in a script, or read off disk. The philosophy is "if someone can read this script, I've already lost the game anyway", but still it's ugly and sick, and so I fixed it. In my defense, the Red Team never did pop my laptop...

I already use gpg-agent to facilitate unpacking of log files. On my syslog servers I roll logs over hourly, gzip them and then gpg encrypt them to my key. Then I can download a bunch of them, run my logunpack script, enter my passphrase once and since gpg-agent caches that credential for a period of time, decrypt all my files in one go.

What I wanted here was basically a way to have keepassxc.cli "hold the door open" and cache the passphrase like gpg-agent does. So what I've done is to use gpg-agent itself for that purpose. I have a GPG encrypted file containing my KeePass-XC passphrase, and I open it using gpg-agent, so it can be reused until gpg-cache-ttl expires.

I've also always had slightly different copies of this script for use cases of "Fullscreen on my laptop" and "fullscreen on larger displays", so I have a switch here for "resolution" as well. "fs" for fullscreen or "fsbm" for "big monitors". Since I'll never go to my office again, that's pretty much never going to get used. The default for the $res value will remain 1280x960. Reasonable enough.

I also added prompts so that it'll ask for host, domain, user and password if you run the script with no prompts from a shell. So /that/ will be super useful to me when I have to do a one-off connection to some remote host but don't need a whole launcher for it. While I'm at it, I put in the -b switch so that you can have it generate a launcher based on that input. That saves me hand editing a template when I add a new RDP host.

I use Linux, but this should work with minimal-if-any changes on Mac and Windows/Cygwin, both of which can run xfreerdp and gpg-agent. I have a good automated ssh-tunneled RDP setup for my Mac, so I might try using that with this so I can use a 4k display for those "busy RDP days".

Being that I do run Linux, here's how I launch this. KDE desktop files like this:


xrayspx@dummyhost:~/rdps$ cat windowsmachine
#!/usr/bin/env xdg-open
[Desktop Entry]
Comment[en_US]=
Comment=
Exec=/home/xrayspx/bin/rdplauncher.sh -h windowsmachine -d domain -u xrayspx
GenericName[en_US]=
GenericName=windowsmachine
Icon=remmina
MimeType=
Name[en_US]=
Name=windowsmachine
Path=
StartupNotify=true
Terminal=false
TerminalOptions=
Type=Application
X-DBUS-ServiceName=windowsmachine
X-DBUS-StartupType=
X-KDE-SubstituteUID=false
X-KDE-Username=

So anyway, here's the thing: RDPLauncher

xrayspx's picture

Rippin' DVDs

Music: 

Dana Carvey - Choppin' Broccoli

Today in Lattice of Convenience news, here's how to rip DVDs.

I barely understand the mencoder command that is the backbone of this thing, and there are many better ways to do lots of the stuff in this script, in fact I know several of those better ways, and looking at it fresh, I see some redundant stuff that cancels out other stuff. But it runs, and I use it, so here goes.

Ripping DVDs isn't fun, the disk labels are iffy at best, even within a single box set you might go from the Gold Standard "TV Show - S1D1" to "DVD_VIDEO" as a disk label. So it can get kind of ugly. To mitigate that I create an output folder based on the DVD disk label + a timestamp. If you get a run of disks with the same name, at least they're not overwriting each others files because the timestamp will shift. I currently have a dvdrip-output directory with the following DVDs in it:

...
DVD_VIDEO-090720202337
DVD_VIDEO-090820201025
DVD_VIDEO-090820201027
DVD_VIDEO-090820201142
I_LOVE_LUCY_S2_D1-090520202354
I_LOVE_LUCY_S2_D3-090620201047
LUCY_S1D1-090520201043
LUCY_S1D2-090520201043
LUCY_S1D3-090520201359
...

Those are all from the same box set. So that's 3 naming conventions from one series. To be fair I think that while it's the same company producing them they probably came as separate "season" boxes rather than one big set. Still. Come on. Jesus.

Another big gotcha I've hit, again mainly with TV series box sets, a single show might exist on the disk as many as THREE times. Once as a "standalone episode", once as "episode with commentary track" and once as part of a massive concatenated file of all the episodes on that disk. In the case of the commentary track, that audio seems to be separate, so the actual episode rips to exactly the same filesize, the commentary track seems not to be something I have access to, so you just get two identical files at the end.

So as you're ripping, that's going to triple the rip time.

The way I'm trying to fix that is to rip the first 30 seconds of every Title on the disk, then do a SHA sum on those ripped sample files. As a Title rips, when it's done I'll drop its clip checksum into a "rippedchecksums" file. The next TItle starts the first thing it does is check to see if its checksum has already been ripped. If it has, skip it. It seems to catch 100% of repeated Titles, and probably 70% of the "Big Concatenated File" cases will match the sum for Title 1. Saves a shitload of time.

In this case, Title 1 is a standalone episode, and Title 21 is the Big Concatenated File of all the episodes on the disk. Title 21 will be skipped. Since I get about 70 or 80 FPS on my Mac Pro, that probably saved 90 minutes of rip time or so with 3 hours of video on the disk:

763b6035c4bf239b4425fb8f484018387574baca /Volumes/Filestore/dvdrip-output/DVD_VIDEO-090720202337/1-sample.avi
59cca1b18759647e13e3e1b6a4facace0520fc06 /Volumes/Filestore/dvdrip-output/DVD_VIDEO-090720202337/10-sample.avi
125add4181b9dc6eee57c32c07568765b8e4483b /Volumes/Filestore/dvdrip-output/DVD_VIDEO-090720202337/11-sample.avi
4daae35d014032964fe57e70e2cc3450f7dac4e5 /Volumes/Filestore/dvdrip-output/DVD_VIDEO-090720202337/12-sample.avi
a942f31a9ee42c5839772f733b2c666195397ad5 /Volumes/Filestore/dvdrip-output/DVD_VIDEO-090720202337/13-sample.avi
8c9473a940a9bc685d84e0ac29c66f53efa6667d /Volumes/Filestore/dvdrip-output/DVD_VIDEO-090720202337/14-sample.avi
29d2200d8c46ac11417119b4b7179e4b526d99cf /Volumes/Filestore/dvdrip-output/DVD_VIDEO-090720202337/15-sample.avi
466860b79bba6d132fcc97d6dc7c0c3a20dd771c /Volumes/Filestore/dvdrip-output/DVD_VIDEO-090720202337/16-sample.avi
f4ae11cca0752956c4d6025a8760a260a59fe79b /Volumes/Filestore/dvdrip-output/DVD_VIDEO-090720202337/17-sample.avi
00753d529f4bbf4081f647056cf44db7c630c198 /Volumes/Filestore/dvdrip-output/DVD_VIDEO-090720202337/18-sample.avi
b7f9c9087fed6b00d22de5033c153f9ffb3cd3b1 /Volumes/Filestore/dvdrip-output/DVD_VIDEO-090720202337/19-sample.avi
14efcb6164f1424b894cc28200ab621ec805ecd0 /Volumes/Filestore/dvdrip-output/DVD_VIDEO-090720202337/2-sample.avi
6c411c8869f1e6bc9a6ec298ba9b6a5c9eefc9ae /Volumes/Filestore/dvdrip-output/DVD_VIDEO-090720202337/20-sample.avi
763b6035c4bf239b4425fb8f484018387574baca /Volumes/Filestore/dvdrip-output/DVD_VIDEO-090720202337/21-sample.avi

At the end of it, I still end up with just a directory full of files labeled 1 through whatever.avi. I have to take a few seconds per file to get it to "TV Show - S01E01.avi". But from there FileBot can mass-rename them with episode titles.

So here's the full ugliness. You'll want to adjust all the paths. I should have made variables, but I don't care, I maybe have 3 or 4 ripping trays running at a time on various machines, so I don't mind just changing the paths for each host. Works on OSX and Linux, and probably Windows with Cygwin, but I don't care about Windows so I'm not going to test it.


#! /bin/bash

timestamp=`date +%m%d%Y%H%M`

id=$(drutil status |grep -m1 -o '/dev/disk[0-9]*')

if [ -z "$id" ]; then
echo "No Media Inserted"
else
name=`df | grep "$id" |grep -o /Volumes.* | awk -F "Volumes\/" '{print $2}' | sed 's/ /_/g'`

fi
name=`df | grep "$id" |grep -o /Volumes.* | awk -F "Volumes\/" '{print $2}' | sed 's/ /_/g'`
echo $name
dir="$name-$timestamp"
mkdir /Volumes/Filestore/dvdrip-output/$dir

maxtitle=`/Applications/mencoder dvd://100 -o bob | grep "titles on this DVD" | awk '{print $3}'`

for title in {1..100}
do
if [ $title -le $maxtitle ]
then
/Applications/mencoder dvd://$title -alang en -ovc lavc -lavcopts vcodec=mpeg4:vhq:vbitrate="1200" -vf scale -zoom -xy 720 -oac mp3lame -lameopts br=128 -endpos 30 -o /Volumes/Filestore/dvdrip-output/$dir/$title-sample.avi
shasum /Volumes/Filestore/dvdrip-output/$dir/$title-sample.avi > /Volumes/Filestore/dvdrip-output/$dir/$title-checksum
touch /Volumes/Filestore/dvdrip-output/$dir/rippedchecksums.txt
fi
done

cat /Volumes/Filestore/dvdrip-output/$dir/*checksum >> /Volumes/Filestore/dvdrip-output/$dir/allchecksums.txt

for title in {1..100}
do
if [ $title -gt $maxtitle ]
then
chmod -R 775 /Volumes/Filestore/dvdrip-output/$dir
sleep 3
drutil tray eject
exit 0
fi
sum=`cat /Volumes/Filestore/dvdrip-output/$dir/$title-checksum | awk '{print $1}'`
match=`grep $sum /Volumes/Filestore/dvdrip-output/$dir/rippedchecksums.txt`
if [ -z $match ]
then
echo "CURRENTLY RIPPING TITLE #$title"
/Applications/mencoder dvd://$title -alang en -ovc lavc -lavcopts vcodec=mpeg4:vhq:vbitrate="1200" -vf scale -zoom -xy 720 -oac mp3lame -lameopts br=128 -o /Volumes/Filestore/dvdrip-output/$dir/$title.avi
echo $sum >> /Volumes/Filestore/dvdrip-output/$dir/rippedchecksums.txt
rm /Volumes/Filestore/dvdrip-output/$dir/$title-checksum
rm /Volumes/Filestore/dvdrip-output/$dir/$title-sample.avi
fi
done
chmod -R 775 /Volumes/Filestore/dvdrip-output/$dir

xrayspx's picture

Bouncing from Kodi to EmulationStation, and back

Music: 

Ninety-Nine And A Half (Won't Do) - Wilson PIckett

Update:

----
As pointed out on the RetroPie forum, just add the loop in autostart.sh, duh: I searched for a while before writing this thing and if I'd seen anyone mention that I'd have just done that instead.

while :
do
kodi
emulationstation
done

I also think it makes a more sensible default for RetroPie to implement. That's all I actually wanted at the start.

However...

Now I've added Features. I can hijack my loop and add one-off commands.

So now there's a Desktop button in my Kodi main menu that will touch a file to cause the loop to gracefully exit Kodi and send me to a desktop session. When I leave the desktop session, it takes me back to Kodi. So that's pretty goddamn convenient.

-----

Because if there's one thing I love, it's having to sysadmin my TV.

Like most reasonable people I use a Kodi mediacenter to run my TV. Lately this has been on a Raspberry Pi 4 running RetroPie. Generally people boot RetroPie into EmulationStation and use it as an emulator, such as on an arcade cabinet. I'm also one of those people.

But in this case I primarily use the TV to watch TV shows and movies, but also want to run console games, so I upgraded to a better RPi and migrated from LibreElec to RetroPie.

RetroPie lets you choose whether to boot into EmulationStation or Kodi, which is fine, and the idea is that if you quit Kodi, it loads ES so you can play games. That works fine. Once. The trouble is in going the other way. If you quit EmulationStation, you exit to a shell. If you run Kodi from within the Ports menu in EmulationStation, well, now you're running both ES and Kodi. This also changes the behavior the next time you quit Kodi to play a game. You end up back in the Ports menu with Kodi highlighted, because ES never quit.

So, that's what I fixed.

The way the RetroPie tool works is they create a script at /opt/retropie/configs/all/autostart.sh. If you have Kodi booting first, it will have two lines:

kodi-standalone
emulationstation.

That script gets run at login time for the pi user. Basically it runs Kodi, and autostart.sh is still running. When Kodi exits, it runs ES and autostart.sh exits. If you wanted to you could just put 1000 lines of:

kodi-standalone
emulationstation
kodi-standalone
emulationstation
kodi-standalone
...

However that's ugly, so I kind of daemon-fied it with a bash script of my own that I wanged together in like 10 minutes, and then I launch that through their autostart.sh. I didn't want to replace their script with mine because the RetroPie one could get regenerated with an upgrade or if I hit something in RetroPie-config. It's safer to have their script call mine.

So what I do is I start with whichever application is passed to me in the command line:

autolaunch.sh -f kodi

Then I start an infinite loop and, based on what application the script is called with, it will start the first application. When that app exits, I change the value of the variable so that the next time it loops, it runs the other one:


#! /bin/bash

while getopts f: name
do
  case $name in
    f) fval="$OPTARG";;
    ?) printf "Usage %s: [-f application to start] args\n" $0
    exit 2;;
  esac
done

while :
do
  if [ $fval = kodi ]
  then
    kodi-standalone
    fval="emulationstation"
  elif [ $fval = emulationstation ]
  then
     emulationstation
    fval="kodi"
  fi
done

Downsides and ToDo's:

Obvious downside is that this makes it difficult to get a shell at the console of the machine. However, I can count on one hand the number of times I've had to do that in the last 6 years or so of running my TV from a Raspberry Pi, so I really don't care.

A definite ToDo is to add some level of process control and general safety so I don't somehow end up running a bunch of instances of Kodi and ES. I did test with "Restart Emulationstation", so it would pick up new games, and it seemed to work as expected. It didn't launch another instance of Kodi or anything.

My main ToDo is to have the ability to use more launchers. Basically right now I have a "Games" menu item in my Kodi main menu, I hit it, it just runs the Kodi "Quit" command, which causes ES to start. Same thing in ES, though I'm just quitting it using the context menu at the moment.

I'd like to be able to add a "Desktop Session" button to quit Kodi or ES and launch a desktop with a browser for those very rare times I want a browser on my TV. This would also solve the "can't get a local shell" problem, at least mostly. I could add a "quit to shell" in this way obviously as well. I think the best way to do this is to stop the script as I exit Kodi and restart it with a new starting value, like -f startx. Kind of like if it were a real system daemon.

However I think in my case, since I'm not a very good programmer, I'm going to just bang this out with a file in /var/tmp or somewhere which carries the "Next Command", so rather than update $fval as I am now, I'd check that file and have it read in each loop to set fval. That would allow me to hijack it from outside the loop.

So I'm in Kodi, if I quit, it's going to set $fval to "emulationstation" and load ES. However, if I run a shell script, and /then/ quit or killall kodi-standalone, that shell script can populate /var/tmp/nextcommand or whatever with "startx".

Then, when Kodi quits, it sets $fval to ES, the next loop comes, but instead of just launching ES, we check to see if there's a value in nextcommand. If there is, set $fval to that and run it instead.

Then you'll start an X session, and when that quits, it should take me back to Kodi.

I seem to recall Kodi's internal tools are pretty good, and I can combine "run this external command" with "run this internal 'quit' command" and assign that to a menu "Action". Just need to remember where all that stuff is.

xrayspx's picture

Hello Woodgrain My Old Friend

Music: 

A couple of weeks ago Natalie bought an Atari 2600 for me as a present. It came with 60 games, two joysticks and paddles, so it was a pretty good deal. She did ask the seller if it worked, and she said it did.

When Natalie got it home, I took the 2600 apart to make sure nothing had leaked all over the place or was otherwise obviously broken or loose and it was SUPER clean, which was encouraging. Just some minor nicotine film everywhere, but it was even dust-free inside. So I hacked together a super crappy cable to convert from the Atari's RCA RF cable to coax to get it hooked up to the TV. Unfortunately it didn't seem to work after all. Natalie was pretty bummed out, but I decided that she'd tried to get me a present, and instead got us a project, and so I convinced her not to bother the seller or leave negative feedback or whatever. After all, as far as the seller knew, the last time she saw it hooked up, it probably did work.




As it was, it did the same thing whether there was a cartridge in or not, just various interference patterns:

The 2600 is like the VW Bug of electronics. There's not that much in there to break, what can break is relatively easy to repair even for rank amateurs like us, and they sold 30 million of them, so there are loads of parts. We'd make it work, and we'd learn A Thing. And if it didn't work at the end, just dip in and grab another console and try again.

To that end, we bought a $5 RCA -> Coax converter, a 2600 re-cap kit, and went way overkill on a swell Hakko solder station to replace the garbage iron I had and hated using. I also grabbed some junk electronics to show Natalie what we're trying to do. We're gonna heat this stuff up, get these components out, get all the old solder out and put new components in. I thought of doing a composite video conversion at the same time, but I kind of wanted to take it one step at a time.

There are only like 5 caps in here to replace, and they also sent a replace voltage regulator. I had previously tested the voltage regulator and it was fine, (near enough) 12V on the input and 5V output, so I opted not to replace it. We split the re-cap duties so both of us could get in some practice on the new soldering iron. It worked great, 0 -> 600 degrees in seconds. We pulled the 3 socketed chips out and hosed the sockets with contact cleaner along with all the switches and the cartridge port.

After that work, I hooked it back up again and....Definite Progress:

A staticky display, but definitely playable. Joysticks work, paddles work (after a hosing out with the brake cleaner). I decided to stop touching anything until the RCA->coax converter showed up, since I was 90% sure the problem was probably that janky as fuck cable I gooped together. Natalie had tracking on it, and they said it was delivered "Today" to our same street address, but 3 towns away. It turns out they'd sent the wrong tracking number, however the correct tracking number showed it as having been delivered a week ago. A quick harried rummage through a week's worth of opened shipping envelopes and we found it. Hooked it up and, yeah, I'd say this is pretty much handled:

I see no point in doing a composite conversion. This is 100x better than any image you'd ever get off of one of these when they were new. If the RF modulator starts to fail, it's always an option later on.

So now the 2600 is in its place along side the arcade cabinet. Of course all of these roms are available in emulation, and I've got an archive of nearly all 2600 games. If we really want to play some serious Fishing Derby, we can do so there. But it's really nice to have a functional 2600 with original games. Natalie is totally enamored with this whole thing and is already stacking up her favorite games. It's totally likely that we'll flip the whole game-time layout and put the arcade cabinet on music video duty while we play Atari on the main TV.

There are still games that don't work, and we often have to switch the console off and on once or twice to get a game to start. Some are also pretty susceptible to any nudge to the console, so we'll probably get it back apart, clean the cartridge port properly, make sure all the contacts are good and re-tin them if they aren't, and see if that helps. But really, this instability is par for the course for 8 year old me. It's part of the charm


--


I have a very close long-time friend who's been trying to mentor me into doing electronics projects since...1988 or so? I never got any good at it. I only ever had one of those garbage un-regulated plug-it-in-and-wait-10-minutes soldering irons that never really got hot enough. I couldn't be bothered at the end of the day. I was too busy playing games to care. I think he was pretty happy to see me tackle even such a simple project and dip our toes in.

At the end of it, his only comment was "Working with electronics takes on a different dimension if you have a scope. Fun to see what is happening in a circuit.".

So I guess we gotta get project-ing.

Fixed Tags:
xrayspx's picture

Running the Lattice of Convenience

Music: 

New Order - 5 8 6

Since posting about the week of 1983 TV Guide viewing, I've had questions from some people wondering about the storage and other hardware and software we use for our media library. It's really not very complicated to do, though I do have preferences and recommendations.

So here's what we've got.

Motivation:

Mainly I don't like the level of control streaming companies have. That they monitor everything we do, and that stuff comes and goes from services like Netflix and Amazon Prime on their timeline, not mine. I don't like the concept of paying for things like Spotify so that I can rent access to music I already own.

I realized like 15 years ago that while we often spent $200/$300 per week on CDs earlier in our marriage, Natalie and I were drifting away from actually listening to it much, because who wants to dig around for a CD to hear one song, then move to another CD. Ultimately, the same applies to movies, we have lots of DVDs, and I don't want to have to dig through booklets just to watch a couple of James Bond movies.

It's super easy to maintain, and we like being able to watch Saturday morning cartoons, "Nick-at-Nite" or throw on music videos while we play arcade games and eat pizza. Once up and running, it's all pretty much push-button access to all the media we like.

Media:

- 2000-2500 CDs (Maybe 200GB of music)

- Couple hundred movies, really probably not as many as most people.

- Lots of TV shows. Space-wise, this is where it adds up fast when you're ripping a box-set of 10 seasons of some show.

- Commercials, mainly from the '80s and '90s, but I'll grab anything fun that strikes us.

- Music videos. We have an overall collection of around 2000, and a subgroup of about 700 which represent "'80s arcade or pizza place" music. That's music that was just ubiquitous when we were growing up in the '80s and early '90s, and you heard it all the time whether you liked it or not. I've since come to appreciate these songs and bands in a way I didn't when I was a dickhead punk kid.

So all told, there's about a 5TB library of stuff, mainly TV shows, but also a decent music library that needs to get maintained and served.

Hardware:

- Ripping machines - Mainly, all I need is the maximum number of DVD trays I can get my hands on. There's nothing special here. My tools work on Mac or Linux so I can work wherever. We have one main Mac Pro that has 2x 8TB drives mirrored which hold the master copy of the media collection.

- NAS - Seagate GoFlex Home from like 10 years ago. I think I originally bought this with a 1TB drive, and have since upgraded it twice, which is kind of a massive pain. Now it's got an 8TB drive which has a copy of the media library from our main machine. I'll get into the pros and cons of this thing below.

- Raspberry Pi - I have a multi-use RaspberryPi which does various tasks to make things convenient and optimizing TV viewing. There are a handful of scripts which create random playlists every night for various categories of music videos, TV shows (Sitcoms, 'BritBox', 'Nick-at-Nite'), etc. It also runs mt-daapd, which I'll get into below.

- Amazon Fire Sticks - We have a couple of them. I'm not super impressed with their 8GB storage limit, but I'm definitely happy enough for the money they cost. They're cheap, around $20 now, and they do what they say on the box. Play video. I have side-loaded Kodi 17.x, but they seem not to quite have the resources for 18.x, though I'm really not sure why not. It's just slower.

- The Shitphone Army - I've got obsolete phones (Samsung Galaxy S4-ish) around the house and decent speakers set up so we can have music playing while doing the dishes for example.

Software:

- Kodi - I mentioned Kodi, which is just an excellent Free Software media library manager. Kodi gets /such/ a bad rap because of all the malware infected pirate boxes for sale, but you never see much from people who actually use it to manage a locally stored library of media they own. Can't recommend it enough. Get familiar with customizing menus in Kodi and making home-screen buttons linking directly to playlists. It's worth it and makes it look nice and easy to use.

- mt-daapd - I'm running out of patience with music streaming, though everything does work right now. MT-Daapd just basically serves up a library of music using the DAAP protocol, which used to be used by iTunes

- DAAP (Android app) - This could be great, but it seems to be completely un-maintained, and somewhat recently moved from being open source to closed, so unless I have an off-line copy of the source, there go my dreams of updating it. But it works well on the Shitphone Army and on the road so we can basically stream from anywhere. Other DAAP players for Android are pretty much all paid applications, and none of them seem to work better particularly than DAAP.

- Scripts A handful of poorly written scripts for ripping DVDs and maintenance of the library (below)

Recommendations:

Players - While the Fire Sticks work great, they're really very dependent on having constant access to Amazon. Were I installing mainly a Kodi machine, it would be much better to use a Raspberry Pi either with a direct-connected drive or mounting a network share. It's super easy to set up with ready-to-go disk images which boot straight into Kodi.

Playlists - Create lots of playlists. Playlists and randomizing things are two things that Kodi is terrible at, so I don't try to make it do it. These scripts run nightly on the Raspberry Pi and make .M3Us for us.

Filenames - Have a good naming convention. All my playlists are M3Us of just lists of files. That means that you don't get Kodi's metadata database with the pretty titles and descriptions, and so the files must be named descriptively enough that you can tell what episode you're looking at from the list of filenames. My template is "Name of the Show - S02E25 - Title of the Episode". Kodi's scrapers work well with that format and it makes it easy enough to fire up the Nick-at-Nite playlist and decide where to jump in.

At various times, I've considered parsing a copy of the Kodi database to suck out the metadata and add it in before the file location. In an M3U, that looks like this:

#EXTINF:185,Ian Dury & The Blockheads - There Ain't Half Been Some Clever Bastards
/mnt/eSata/filestore/CDs/Ian Dury & The Blockheads/Ian Dury And The Blockheads The Best Of Sex & Drugs & Rock & Roll/17 There Ain't Half Been Some Clever Bastards.mp3

It seems like having all that sqlite stuff happening would add a lot of overhead to generating playlists, and having well-named files saves me from having to worry about it, so I haven't bothered.

Storage - Though I use a "Home NAS" product that overall I've been pretty happy with, it does irritate me. Consumer market stuff is /so/ proprietary that it's quite hard to just get to the Linux system beneath and customize it the way you see fit. Specifically in the case of the GoFlex, "rooting" it even involved replacing Seagate's customized version of SSH with a vanilla one. Screw that up and you brick the device. I also run into network bottleneck issues with that thing. While you can enable jumbo frames, for instance, when syncing new content the CPU gets pegged, I believe I'm running out of network or disk buffer, which is kind of unacceptable in a NAS device.

Building it today, I'd just use a Raspberry Pi 3 with a USB drive enclosure. For the time being, my growth curve is still (barely) pacing along with the largest "reasonably priced" drives on the market. My ceiling is about $200 per drive when I do upgrades, because I am a very cheap man.

I have no opinion on consumer RAID arrays. I can only imagine consumer RAID based NASs come with all the shit I hate about the GoFlex. Yes, I'm biased against consumer grade garbage tech and that's probably not going to change. I'll have to buy one someday I'm sure, but for now it's all being kept simple.

Backups Keep backups. While I have multiple copies of everything, it does make me somewhat nervous that the only part of the media library currently being backed up off-site is the MP3 collection. That's got to change, and rsync is your friend. Ultimately I'll probably end up upgrading my home Internet from 20Mb/2Mb to something which will allow me to sync over a VPN tunnel to somewhere off-site (friend's house, work...).

Sample Scripts:

Here are some samples of the shitty bash scripts that run this whole nonsense. I know the better ways to write these, but the fastest possible way to hammer these out worked well enough and there's no way I'm going to bother going back and fixing them to be honest.

Rip CDs

I use an application called MAX on the Mac to rip CDs. I think its usefulness might be coming to an end, and I'm not sure what to do about that. It uses (used?) MusicBrainz database to automatically fingerprint and tag discs, but the last CD I ripped it seemed to have problems. You can run iTunes side by side with Max and drag the metadata over from there, so maybe that works well enough?

Anyway, I use that because I rip to both 320k CBR MP3 and FLAC. I have a shitload of stuff that really should be re-ripped since they're 128k and no FLAC, but I've so far been unmotivated to do so.

I wrote a bunch of stuff to move all the output files around and update iTunes libraries. Honestly I don't rip a whole lot of new music, which is a shame and which I should really fix.

Rip DVDs

DVD ripping is a lot more fragile than it should be. Good software like Handbrake are bullied into removing the ability to rip protected DVDs, and things are being pushed toward the commercial. I use mencoder in the script below.

DVD titles are sketchy at best, and as far as I know, you can't really fingerprint a DVD and scrape titles in the way you can with CDs. So I do what I can. I take whatever title the DVD presents and make an output directory based on that name plus a timestamp. That way if you're doing a whole box set and all the DVD titles are the same they're at least writing out to separate directories and not overwriting each other.

As far as file-naming, unfortuantely we don't live in the future yet and that's all down to manually renaming each output file. I use the information from TVDB, not IMDB, since that's the default library used by Kodi's scrapers. Sometimes the order of things is different between that and IMDB (production order vs airing order vs DVD order issues plague this whole enterprise).

#! /bin/bash

timestamp=`date +%m%d%Y%H%M`
pid="$$"
caffeinate -w $pid

id=$(drutil status |grep -m1 -o '/dev/disk[0-9]*')
if [ -z "$id" ]; then
echo "No Media Inserted"
else
name=`df | grep "$id" |grep -o /Volumes.* | awk -F "Volumes\/" '{print $2}' | sed 's/ /_/g'`

fi
name=`df | grep "$id" |grep -o /Volumes.* | awk -F "Volumes\/" '{print $2}' | sed 's/ /_/g'`
echo $name
dir="$name-$timestamp"
mkdir /Volumes/Filestore/dvdrip-output/$dir

echo $dir

for title in {1..100}
do
/Applications/mencoder dvd://$title -alang en -ovc lavc -lavcopts vcodec=mpeg4:vhq:vbitrate="1200" -vf scale -zoom -xy 640 -oac mp3lame -lameopts br=128 -o /Volumes/Filestore/dvdrip-output/$dir/$title.avi
done
chmod -R 775 /Volumes/Filestore/dvdrip-output/$dir

Playlist Script

The simplest Music Videos one below just looks at one directory of videos and one directory of TV commercials and randomizes all the content into an M3U. The more complicated ones have dozens of directories, and I'm sure I'm doing this array-building the wrong way. I'm sure I could have a text file with the un-escaped directory names I want and read that to build the array, either way, it really doesn't matter because if I want to add a TV series, I still have to edit a file, so this works fine. I've also thought about having a file in each directory like ".tags" that I search for terms in, like "comedy,nickatnite,british" and build the array from that, I dunno, sounds like work.

#! /bin/bash

array=`find ./ -type f;
find ../../Commercials -type f`

printf '%s\n' "${array[@]}" | sort -R | grep -v dvd_extras | grep -v "./$" | grep -v "\.m3u" | grep -v -i ds_store | grep -v ".nzb" | grep -v ".srt" > full-collection-random.m3u

- rsync the TV library. I have several of these, one for TV shows, one for movies, music videos, mp3s etc. It's just somewhat faster to only sync the thing I'm actually adding content to, rather than have to stat the entire library every time I rip a single DVD. The TV show sync tool also deals with the playlists, which are actually created on the NAS drive, so they have to be copied local before syncing or else they'll just get destroyed every day.

This checks to see if the NAS volume is mounted, if not it will mount it and re-run the script.

#! /bin/bash

mounted=`cat /Users/xrayspx/xrayspx-fs01/.touchfile`

if [ "$mounted" == "1" ]
then

cp ~/xrayspx-fs01/Common/TV\ Shows/1\ -\ Playlists/* /Volumes/Filestore/Common/TV\ Shows/1\ -\ Playlists/

rsync --progress -a --delete /Volumes/Filestore/Common/TV\ Shows/ ~/xrayspx-fs01/Common/TV\ Shows/

~/bin/umounter.sh
exit 1
else
mount -t smbfs //192.168.0.2/filestore ~/xrayspx-fs01/
~/bin/synctv
fi

xrayspx's picture

This week in Rad Helicopter News

Music: 

Kenny Loggins - Danger Zone

Witness the power of this fully operational Lattice of Convenience.

As some people know, I've become relatively intense when it comes to hoarding archiving media of all kinds, and recently that means '70s and '80s TV shows, cleaning out all the box sets I can find from Goodwill and antique stores and ripping them. For some time Natalie and I have been toying with the idea of taking a day from a random '80s TV Guide and watching that day in TV shows.

In the last few weeks, we've also started to listen to Ken Reid's excellent TV Guidance Councillor podcast, and have decided there's no time like the present. And in fact, driven by Reid's format, we decided to do a whole week. We pretty much pre-planned the whole week, though in some cases I wasn't able to get the exact episode of a show, so we had to make due with something else from the series. We also decided to stick to the 3 major networks, since that's all either of us knew, living in the sticks and all.

We started with Saturday, January 7, 1984, with TJ Hooker, The Love Boat with Charo, and Fantasy Island with Vic Tayback, Katherine Helmand and Richard Hatch. Natalie decided she likes TJ Hooker, and so we'll hunt down more of those even though I killed the show because I tend to immediately ID "The Guy". Whoops. We just chose a Charo episode of Love Boat because I couldn't get the one from the day, and Charo is delightful at all times. We need more Charo.

On Sunday, we stuck largely with sitcoms, since we've seen things like the Knight Rider episode from that day. So it was Ripley's Believe it or Not, Alice, One Day at a time, and the Jeffersons. Ripley's and Alice were both harder to find than I would have thought. We had to settle for like the second episode of Alice, and just any Ripley's I could find.

Monday was a mix, starting with That's Incredible, then Newhart and Emerald Point N.A.S. Emerald Point /should/ have been an awesome show. Within the first 3 minutes you've got: MacGyver O'Neill. Crashing an F-14. Into a Cuban. How can that fail? By making it a soap opera, that's how.

Tuesday NBC took it all. A-Team, Riptide, Remington Steele. Natalie does like a good crime-fighting buddy-show, so there really wasn't any contest. I couldn't get excited about a late-season Three's Company or Happy Days.

Wednesday got us to hit ABC for The Fall Guy, then back to default NBC for a Very Special Facts of Life, Night Court and St. Elsewhere. As I remember, all Facts of Life were very special.

Thursday night was another NBC sweep. Gimme a Break!, Family Ties, Cheers, Buffalo Bill and Hill Street Blues.

Friday, ABC rounded it out with Benson and Blue Thunder, which we kind of loved. It was our second 'Copter-Based show after Riptide, and had me yelling at the TV most of the time.

We opted for Episode 1 of Blue Thunder, since the schedule had us watching Ep 2, and figured we'd start at the top. A maniac with a pre-Vietnam era light observation aircraft was flying over We-Can't-Call-This-LA shooting down police helicopters and even strafing the funeral of one of the pilots he killed. Obviously the answer to this isn't "send up two fighter jets immediately after the first incident to knock him out", it's "Regular LAPD cops need a stealth helicopter with a goddamn gatling gun mounted on it to fly silently around and light up bad guys". There was some confusion over organizational affiliation, since everything points to these guys being regular cops (right up to the annoyed Captain yelling "CHHHHEeeeneyyyyy!"), except Butkis or Bubba Smith were all "Freeze, Federal Agents". On the other hand, who cares, all I know is Darryl Gates would have given his right eye for one of these things in 1992.

So far, likes and dislikes:

As I mentioned, Natalie loves her some buddy-shows and dumb action and so she definitely wants more TJ Hooker, Hardcastle and McCormick and Remington Steele. Natalie also remembered how much she liked The Jeffersons and Newhart, though she was kind of "meh" on One Day at a Time. For that one, we watched both the "current" episode, and the first episode of the series, since this was the last season, and wasn't really "representative' in my mind. I think I'd get more Alice if we can. For as popular a show as it was, it seems pretty hard to find. Similarly, Benson and Webster, long running show, stupidly hard to get.

On the other hand though, Emerald Point Naval Air Station. Wow. Just. Jeez. I think Richard Dean Anderson had to do MacGyver and Stargate SG-1 just as penance for the existence of this show. I can't say for sure that the debt has been fully repaid.

Ripley's Believe it or Not and That's Incredible are kind of hard to find, and that's a shame, since we live in The Future and so it was fun to go look up the people on those shows for a "Where are they Now" running commentary between us.

This kid had a pretty decent career at one point.

This week of TV really does show how much we just wanted to be entertained in the '80s, versus everything having to be Real and Gritty as it is now. Sure there was Hill Street Blues, but there was also Magnum, Riptide and Hardcastle and McCormick. We weren't weighed down with SVU type bummers every week. I saw an analyst who chalks this up to Vietnam. In the same way that Magnum, coming back from Vietnam didn't want to process it, and chose instead to live as a large child for a little while, so the US chose to amuse ourselves with dumb escapism.

I can't say I disagree with that theory, since I'm basically doing just that right now. I'll take my Cylons toaster-headed and shelve the HD Edward James Olmos version until happier times.

Tags:
xrayspx's picture

The Lattice of Convenience

Music: 

Def Leppard - Bringin' on the Heartbreak

A couple of years ago, Natalie and I canceled cable since we found it had literally been a year since we watched anything live on TV. I've built a pretty good "lattice of convenience" to store a media library of "Crap we like" and conveniently stream it pretty much anywhere.

Over the years, we've collected maybe 3000 CDs and a several hundred DVDs, including many box sets of TV series we like. I feel like we spent a TON on CDs when we were younger that most people didn't.

xrayspx's picture

Stereo Slide Viewer Hack Proof of Concept

Music: 

Peaches - The Inch

Wherein there's some history, and a major pet project.

Pages

Subscribe to RSS - Hacks