Volunteering With My Programming Skills For A Non-Commercial Radio Station
During the Crimbo Limbo period between Christmas and New Years, I volunteered with my programming skills. That’s how it came to be that I built a tool to create a browsable music library for the non-commercial radio station Great American Songbook. This article discusses why and how.
How I came to volunteer
For me, programming is like an itch. If I haven’t programmed in a while I start to feel fidgety. Furthermore, I am deeply passionate about automation and eliminating bullshit jobs. This was actually how I started programming in the first place - back in my time when I was a linguist there had been dozens of manual tasks multiple people (including me) performed over and over. I got so annoyed by it that I taught myself how to program in order to get rid of these.
Volunteering was something I had considered already for a long time. I have approached multiple organisations over the past year, i.e. local hospitals at the height of coronavirus pressures. Most of the time, I either never heard back or have been told there was no need for help. It is surprising through how many hoops one has to jump just to work for free!
The Great American Songbook is a radio station I have been listening to for many years now. Swing music and dance are big passions for me. I donated some money to that radio station and that’s how I thought again about volunteering: couldn’t I donate my time and skills additionally to money? That’s how I contacted the radio station and almost instantly received an answer.
During a zoom call, the radio producer walked me through the website’s setup and how the radio stream operates. In my personal circle, I don’t have many people of age and no one who could let alone operate a tablet. Therefore, I was incredibly impressed by the radio producer who is a pensioner and runs the radio station completely out of their own pocket. They have a setup of multiple servers, a website with SSH certificate, templates from GitHub, site updates using FTP, … - and they browsed through their media library using FreeCommander like a champ!
Our initial conversation immediately lay bare automation needs: On the website, there is a section which lists the complete music library. That way, listeners know what songs are available (and which aren’t) so that they can request songs to be played more frequently or ask for new songs to be added to the rotation. Creating this PDF was a constant long-lasting pain to the radio producer, they only did it a few times a year because it took so long. This is where my tool comes in!
Building the tool
The task at hand was to crawl through the entire music collection of the radio producer and to generate a sorted list of all the tracks. This list should be sorted either by the artist or title. It should also be possible to exclude subdirectories from the generated catalogue.
It was clear to me that I wanted to have the catalogue displayed in an interactive table using HTML so that listeners can filter and sort easily. I also decided to add CSV export functionality to my tool so that the radio producer could still convert the list into a PDF if needed.
I wanted to build a tool which could be executed by the radio producer without any of my input - that’s what automation is about after all: empowering the user. This is why I decided to use the Go programming language. Although I’m an expert in JavaScript/TypeScript, it is not a great solution for tools that should be executed by people/machines outside your control. I wouldn’t want to give the radio producer a list of things they need to install and commands they need to execute! With Go, I can compile the tool into one single executable file which the radio producer can start with a simple double click as they would do with any other program.
Once I knew all these requirements, I immediately started programming. I created a small command-line tool using the prompter library. I decided against any command-line flags and arguments as the tool was supposed to be used by non-techy people as well with just a double click. Therefore, I think it is way better to use a solution like prompter which asks the user for their input. That way, you can also give the user hints and examples. Unlike with avoidable --help messages, these prompts will definitely be seen by the user, there is no way around them. The prompter library is also incredibly easy to use:
func promptForDirectory() string {
directory := prompter.Prompt(
"Please provide us with the path of the " +
"directory where all your music lies. I.e. C:\\Music",
""
)
if directory == "" {
log.Fatal("You need to enter a valid path")
}
return directory
}
One of the trickiest bits for me was to recursively iterate over all directories and subdirectories. I use the filepath.Walk() function which already works recursively out of the box! Due to my familiarity with JavaScript, I hadn’t expected this though and assumed subdirectories were ignored. This is why I initially created unnecessary duplication where I would recursively call filepath.Walk again for every subdirectory - madness!
Golang is a highly readable programming language if done well - it requires the user to abstract things out a bit though. Consider the below code where I I try to determine whether a file path (path) starts with any item of the ignoredDirectories list. The code itself is quite readable but I have it placed in the midst of another function - I should have created a helper function called Find() instead. We live to make mistakes 🤷.
// This should be its own function, something like `Find()`
var shouldBeIgnored bool
for _, ignoredDirectory := range ignoredDirectories {
if ignoredDirectory != "" {
if strings.HasPrefix(strings.ToLower(path), strings.ToLower(ignoredDirectory)) {
shouldBeIgnored = true
break
}
}
Reading out artist name etc. from MP3 files was fairly straight-forward thanks to the tag library. The same goes for creating CSV files which works with the built-in library encoding/csv.
More interesting - and challenging - was creating the catalogue as HTML table. Some people surely wonder whether I went with some grand solution like a serverless database and API calls to fetch items - way too complicated! Remember our goals: replacing the PDF file and empowering the radio producer. We wouldn’t want to teach them about database access, requiring them to host things, or me hosting things for them which might not work any longer if I don’t maintain it. Instead, I went for a simple and proven solution: Bake all the songs as table rows into a static HTML file.
The main idea of this tool was to have one single executable to give the radio producer. Therefore, having HTML mustache templates was out of the question as well. This is why I ended up hard-coding an actual string into my .go file which represents the base HTML skeleton. This skeleton is then enriched with all the table rows for each song where each item is converted into a <tr> element.
I initially created my own basic HTML table including custom filter functions but I was shocked by the atrocious performance of my naive scripts. The Great American Songbook music library contains 16.000 items - due to that size there is a need for smart filters. This is how I stumbled over the DataTables project which provides a fantastic out of the box solution for displaying tables, including pagination and filtering. Even on tables with 16.000 rows the search and sorting functionality works tremendously fast and stable.
While the filtering is fast, I had to do some trickery in regards to page loads. Because all of the <tr> elements are in the HTML page, the final page is 1.48MB large. This is a good decrease in size compared to the PDF (2.5MB) but it’s still quite large to load for browsers. It turns out that if there are more than 16.000 elements on the page, your browser has to do quite a lot of work to process all of these items. This is why I had a look around and used the solution from patdavid.net where I first hide the table on page load. Instead, the site is showing a loading spinner. Once all elements are processed by the browser and in the DOM, I hide the loading spinner and display the table. I am using the jQuery event method $(document).ready() for that.
The end result works pretty well and the radio producer was very happy about it - what previously took hours became fully automated and finished in around 60 seconds!
Implementing it into the webpage
The radio producer told me that they would be able to implement the solution as iframe on the website - which was why I went with a separate HTML file to begin with. After a few hours, they contacted me though to ask whether I could help out integrating it. Of course I could!
Implementing the iframe into the existing website was quite tricky. The main problem was that I had to wrap my head around the website’s framework which the radio producer purchased from another developer. While greatamericansongbook.info is a single-page application, there aren’t any frameworks being used, it’s all vanilla JavaScript: everything in jQuery, all scripts somewhat minified (highly shortened variable and function names but at least line breaks and indentation left intact). An extra pain point was that the website doesn’t use any version control - it’s just a bunch of files on an FTP server.
Another problem presented itself in having to have the iframe only loaded when the user requires to see it. As I discussed earlier, the HTML file for the table is very large. If you have an iframe element in your HTML file with a src attribute pointing to another page your browser will also load that other page though - even if the iframe isn’t immediately visible to users. This is why the page loads were suddenly quite slow. I solved it by removing the src attribute from the iframe element in the initial HTML file. I managed to find in the source files the function which is being called whenever the URL changes. This is how I added another jQuery listener which only fires when the URL contains #!/page_LIBRARY_TABLE - in that case, I am adding to the iframe the src attribute leading to our big HTML table. Thus, the browser only loads our big table once it’s necessary.
The final product looks something like this (might not be on the website yet, development version):
The iframe embedded into the Great American Songbook page
Final thoughts
This was a fun and surprisingly challenging project which was right up my alley. I love that programming allows me to empower people: the radio producer doesn’t have to spend hours anymore having to create a PDF library and the listeners can browse the entire catalogue with ease, allowing them to become more engaged.
Volunteering was a lot of fun and is something which I will want to do again. As outlined above, it isn’t always that easy to find suitable projects. I think I got really lucky with how trusting and open the radio producer was. Even more so, they immediately grasped what I meant when I said “I do automation” and they had a fitting task for me in mind right away. Thank you very much!