Playlists

I had a request to share some playlist management stuff so I thought I should explain myself. I've got a significant CD collection, and a somewhat-significant collection of TV shows. This is fine on its own, but lots of media is pretty worthless without well curated playlists that you really don't have to think about. So I built Spotify, MTV and Syndicated TV.

* NOTE: If you have a better way to do any of this let me know and I'll fix it. I particularly have the sense, which is not backed up by my testing, that "sort -R" isn't great.

Music's easier so we'll start there. I use Strawberry to manage my music. This was all running under Clementine and aside from some DB schema changes, the scripts are portable between them.

Until relatively recently I was never a big fan of "star" or "heart" ratings, but Clementine/Strawberry will store this metadata in the MP3 itself so I should be able to quickly recover if I lose my music database. In the app I have a few Smart Playlists like 3-Stars, 3 Stars + (This is 3, 4 and 5 star tracks), 4-Stars, 4-Star + and 5 Stars. To use 4 Star as an example, the rules look like this:

Match every search term (AND)
Rating - Greater than - 3.5 Stars
Rathing - Less than - 5 Stars
Ratin - Not Equals - 5 Stars
Length - Greater Than - 8 Seconds

That results in a playlist of 8423 songs with ratings between 4 and 4.99 stars. There was a bug in Clementine which I got fixed where ratings could exceed 5, so I'm a little careful to deal with weirdo cases, but it's pretty simple. I also have a bunch of manually selected playlists, so like an '80s one, '90s, and "Barn Radio". Barn Radio is our catch-all for the ubiquitous music we heard from the late '70s through late '80s. For Natalie that was largely with her dad in the dairy barn, for me it was the music of my 2 hours on the bus every day.

Anyway, I have all these .m3us stored in a folder along with my MP3s called "playlists_base". These are used by a nightly playlist generator that pulls ~200 tracks and makes daily playlists running 8 or 10 hours each. The reason for this is that streaming software such as Airsonic-Advanced kind of chokes on massive playlists. It could be Airsonic itself, it could be populating the mobile client, I don't really know or care, other than to say it works great with list sizes under about 1000 tracks or so, so I keep them shorter.

The x-Star playlists are all built from the database like this 4 Star + playlist below. You can see it do a couple of different Star Rating DB queries, dump out the tracks to $playlist_tmp.m3u, then cat that file and do a random sort to generate the final version. It's pretty easy to adjust the mix based on ratings, so if I wanted to weight high-rated tracks I could do that by adjusting how many tracks of the 200 are returned by each search:


#!/bin/bash

rm /Volumes/Filestore/CDs/playlists/4\ Stars\ +.m3u

i=1

while [ $i -le 100 ]
do

### Switching from Clementine to Strawberry ###
#       file=$(sqlite3 /var/tmp/clementine.db "select filename from songs where rating > "0.9" order by random() limit 1;" | awk -F "file://" '{print $2}')
        file=$(sqlite3 /var/tmp/strawberry.db "select url from songs where rating > "0.9" order by random() limit 1;" | awk -F "file://" '{print $2}')

        ### Clementine data encodes special characters and accent marks and stuff so I'm using
        ### Joel Parker Henderson's urldecode.sh to undo that: https://gist.github.com/cdown/1163649

        data=$(/home/xrayspx/bin/urldecode.sh "$file")
        if [ -f "$data" ]
        then
                ### Have to escape leading brackets because grep treated it as a range and would allow duplicates ###
                ### Can't do that in "data" because \[ isn't in the filename so they'll fail ###

                escaped=$(echo "$data" | sed 's/\[/\\[/g')
                #echo "$escaped"

                ### Avoid duplicates
                match=$(grep -i "$escaped" /var/tmp/4-star-tmp.m3u)
                if [ -z "$match" ]
                then
                        echo "$data" >> /var/tmp/4-star-tmp.m3u
                        ((i++))
                fi
        fi
done

i=1

while [ $i -le 100 ]
do
### Switching from Clementine to Strawberry ###
#        file=$(sqlite3 /var/tmp/clementine.db "select filename from songs where rating >= "0.8" and rating          file=$(sqlite3 /var/tmp/strawberry.db "select url from songs where rating >= "0.8" and rating 

        ### Clementine data encodes special characters and accent marks and stuff so I'm using
        ### Joel Parker Henderson's urldecode.sh to undo that: https://gist.github.com/cdown/1163649

        data=$(/home/xrayspx/bin/urldecode.sh "$file")
        if [ -f "$data" ]
        then
                ### Have to escape leading brackets because grep treated it as a range and would allow duplicates ###
                ### Can't do that in "data" because \[ isn't in the filename so they'll fail ###

                escaped=$(echo "$data" | sed 's/\[/\\[/g')
                #echo "$escaped"

                ### Avoid duplicates
                match=$(grep -i "$escaped" /var/tmp/4-star-tmp.m3u)
                if [ -z "$match" ]
                then
                        echo "$data" >> /var/tmp/4-star-tmp.m3u
                        ((i++))
                fi
        fi
done

cat /var/tmp/4-star-tmp.m3u | sort -R > /Volumes/Filestore/CDs/playlists/4\ Stars\ +.m3u

rm /var/tmp/4-star-tmp.m3u

Those Star Rating lists are called at the beginning of my overall static playlist script, but the Barn playlist and other manually selected ones are built from the "playlists_base" directory. I basically just edit those .m3us in place with Strawberry as we add CDs. They just the files, do a random sort and pull the top 200. This will use any .m3u in .../playlists_base/ and make a daily file from it:


#!/bin/bash

#scp xrayspx@pro:~/.config/Clementine/clementine.db /var/tmp/

### Switching between Clementine and Strawberry ###
#cp /Volumes/Filestore/CDs/playlists_base/clementine.db /var/tmp/

cp /Volumes/Filestore/CDs/playlists_base/strawberry.db /var/tmp/

/home/xrayspx/bin/3-star-playlist.sh
/home/xrayspx/bin/4-star-playlist.sh
/home/xrayspx/bin/5-star-playlist.sh
/home/xrayspx/bin/get-the-led-out.sh

ls /Volumes/Filestore/CDs/playlists_base/*.m3u > /Volumes/Filestore/CDs/playlists_base/m3us.txt

while IFS= read -r file
do

        filename=$(echo $file | awk -F "/Volumes/Filestore/CDs/playlists_base/" '{print $2}')

        echo Filename: $file

        rm "$file.full"
        rm "$file.scratch"
        rm "/Volumes/Filestore/CDs/playlists/$filename"

        ###Testing a change since Strawberry creates playlists without EXTINF lines ###
#        array=`grep EXTINF "$file" | sort | uniq`
        array=`grep -v EXTINF "$file" | sort | uniq`

        printf '%s\n' "${array[@]}" | sort -R > "$file.full"
        head -n 200 "$file.full" > "/Volumes/Filestore/CDs/playlists_base/$filename.scratch"

        n=0
        while IFS= read -r extinfo
        do
#       echo $extinfo
                term=`echo $extinfo` # | cut -d "," -f 2-`
#       echo $term

 ###Testing a change since Strawberry creates playlists without EXTINF lines ###
 # grep -A 1 -m 1 "$term" "$file" >> "/Volumes/Filestore/CDs/playlists/$filename"

        grep -m 1 "$term" "$file" >> "/Volumes/Filestore/CDs/playlists/$filename"
        done 

        rm "$file.full"
        rm "$file.scratch"

done 

rm /var/tmp/clementine.db
rm /var/tmp/strawberry.db

For TV shows it's a bit more complicated. I've got individual scripts for things like Sitcoms, Saturday Morning Cartoons, Buddy-Cop shows, Nick-at-Nite, etc. Each script uses a text file which just lists the relative path to the directories I want to randomize. I just read in that text file then scan each directory and build an array that again I sort -R and dump in an m3u. You'll see a couple of my conventions here, like the "dvd_extras" folders I use for any extras that I want to keep but don't want to have show up in the mix, as well as a bunch of other crap I grep out.

This script references "./.sitcoms.txt", which looks like this:


./Archer (2009)
./30 Rock
./Absolutely Fabulous
./Alexei Sayle's Stuff



#! /bin/bash

array=$(
while read line
do
        find "$line" -type f;
done < .sitcoms.txt
)

printf '%s\n' "${array[@]}" | sort -R | grep -v -w "batch" | grep -v dvd_extras | grep -v "./$" | grep -v "\.m3u" | grep -v -i ds_store |
 grep -v "\.nzb" | grep -v "\.nfo" | grep -v "\.sub" | grep -v "\.sfv" | grep -v "\.srt" | grep -v -i "\.ifo" | grep -v -i "\.idx" |
 sed 's/^/..\//' > ./1\ -\ Playlists/Sitcoms.m3u

This dumps out to a folder called "1 - Playlists" inside my TV Shows directory, just so it shows up first. There's a folder in there for Blocks as well, in which I create blocks of 10 random episodes of a bunch of shows. This is built to replicate like TBS/TNT/USA in the evening where you just sit and watch a block of whatever is on. In practice I do this wrong and tend to be too picky about these and just watch blocks until I've worked my way through a whole series and wind up tired of it forever.

One thing I do for things like Nick at Nite and overall Sitcom lists and stuff is that I mix in commercials. I don't do this very well though, I just treat my directory of commercials like any other TV show. I'd rather do "pull a TV show, toss in two commercials, repeat", but I'm not there yet I guess.

The last type of lists I build are for music videos. I break this into a few different playlists, one overall catchall that pulls in all videos, a playlist for MTV 120 Minutes, and one for "Arcade / Pizzeria" music. Basically the ubiquitous music you'd hear in a pizza shop or arcade in the '80s or '90s. I do the same commercial thing here as well.

Example:


#! /bin/bash

array=`find ../120\ Minutes -type f;
find ../../../Commercials -type f`

printf '%s\n' "${array[@]}" | sort -R | grep -v dvd_extras | grep -v "./$" | grep -v "ERRORS$" | grep -v "\.sh" | grep -v "\.m3u" |
 grep -v -i ds_store | grep -v ".nzb" | grep -v ".srt" > 120\ Minutes.m3u