Events have me thinking about this a lot lately.
Is it always possible to reprogram the simulator? We shall see. I hope to elaborate more later. In the meantime, enjoy one of my favorite scenes in all of Star Trek.
“I never took the Kobayashi Maru test until now. What do you think of my solution?”
― Spock
This presentation caught my eye the other day.
I worked on a fairly large Angular project a couple of years ago and find myself in general agreement with the points made about Angular. I’ve also been playing with Reagent recently.
“You know being relevant or coming up with something interesting, funny to say about what’s current is just as hard as it might ever be depending on the serendipity of it all.”
― Lily Tomlin
My apologies to Jean Shepherd.
As I’ve gone through the process of writing this over Thanksgiving weekend, the title has changed three times. I believe I’ve finally landed on one that best descibes the content. It a rambling story about data and discovery. Hmmm, that gives me an idea for a book, Clojure: One Story at a Time. Or some such nonsense. There’s nothing earth shattering here. Unless you’re inclined to believe that earth shattering things start with simple thoughts, questions, and explorations.
“Great things are done by a series of small things brought together.”
― Vincent Van Gogh
Lately, I’ve had a number of project ideas that all have something to do with my iTunes library.
What do I know? I know that the information is stored in an XML file,
typically located at ~/Music/iTunes/iTunes Music Library.xml, but I didn’t really know the format.
I do have some experience with NeXT/Mac OS X/iOS development and figured it was
a plist file.
Sure enough, taking a peek inside showed that was the case.
However, knowing it is a plist file conveys about the same information as saying it’s a JSON file.
I need to know what data is under what keys and how is it all organized.
This seemed like a good job for starting a Clojure REPL and doing some exploring. So, a REPLing we will go …
There are a number of good sources for running a Clojure REPL (clojure.org, braveclojure.com). I really like Cursive Clojure so I typically set up a Leiningen project and get a local REPL going with these instructions.
So, with a quick
$ lein new itq
$ cd itq
I am in the itq (iTunes query) project.
You can set up a REPL in your editor/environment of choice or just run
$ lein repl
REPL driven development (RDD) has been talked about a lot (see Google). This is good but I feel like I should nuance it just a little and, instead of using the phrase REPL Driven Development, I want to suggest the phrase REPL Enabled Thinking (RET).
For me, thinking undergirds development. And thinking continues during and remains after development. I need to think about the problem, the domain, the data, and anything else that comes to mind. The Clojure language and its REPL are just tools that allow my mind to explore the space of the problem that I’m interested in. RDD seems to me to put the emphasis on doing development. With this mindset, the REPL environment then provides a way to get the code developed. But RET puts the emphasis on the thinking about your problem and the REPL is a means to that end. It’s a way to enable the thinking.
This is not keyboard driven writing. I would humbly submit it is keyboard enabled storytelling.
One of the heroes of my technical formative years is Richard A. O’Keefe. In the introduction of his book The Craft of Prolog he says,
“If your Prolog code is ugly, the chances are that you either don’t understand your problem or you don’t understand your programming language, and in neither case does your code stand much a chance of being efficient.”
― Richard A. O’Keefe, The Craft of Prolog
This points out that the key to success involves understanding. We are in the business of thought work. But here’s the rub, no two people have the same mind. I think the hardest thing in the world is for one mind to understand another mind. I’m probably naive in this but I often think that many of the debates within the software world could be less volatile if we kept in focus the notion that different minds think differently.
And … are you ready? … that’s ok.
It helps some people to write their tests first because something clicked in their thinking about ordering things this way and they resonate with this way of working. But for others, there are situations where this would be a roadblock to them understanding their problem. Whenever the debate volume increases I want to whisper, “It’s ok, you can do what works for you.” But that’s another story.
But this idea, that understanding and thinking is paramount, relates to my REPL story because REPL Enabled Thinking most accurately describes what I’m setting out to do. As I start to explore this problem and its data, I gain understanding by directly manipulating parts of the data, enabled by the tools I have in the REPL. And I can build on these tools by adding more functions, possibly saving some to use as part of the final solution. Along the way, I can write tests, documentation, or whatever, to capture how I want things to work as I figure them out. All in the same environment. It’s one fluid workflow where I can do all of these things at the time that best fits with how I’m thinking about the problem. I don’t focus on doing as much as learning to understand my problem space. It’s a subtle shift in thinking but I believe a meaningful one. It’s also the difference between assuming you know everything about your problem, banging out some code with passing tests, and then thinking you’re finished. Unlikely.
After a quick search I found the clj-plist library.
That was helpful. No need to write a plist parser. Just add the dependency to my project.clj file and I’m good to go.
(defproject itq "0.1.0-SNAPSHOT"
:dependencies [[com.github.bdesham/clj-plist "0.9.1"]])
In this case I know I’m going to want to save some of what I do in the REPL.
So, I start by creating a file to hold a namespace of things that I’ll create while exploring the iTunes data.
Then I can define things in the file editor and evaluate them in the REPL.
A glance at the clj-plist docs and I know how to turn a plist file into Clojure data structures.
So, after copying the iTunes library I want to use into the project,
I start with these forms in a file that I evaluate in the REPL.
(ns itq.parse
(:require [com.github.bdesham.clj-plist :as pl])
(:import [java.io File]))
(def itl
(pl/parse-plist
(File. "<path-to-project>/itq/iTunes Music Library.xml")))
Then, place the REPL in the itq.parse namespace.
(in-ns 'itq.parse)
In theory, itl has my iTunes library as a Clojure data structure.
Finally (well, it really hasn’t been that long), I can look at it. So, I type in the REPL
itl
But before hitting return I start thinking … I have a pretty big iTunes library … I’ve been here before … the REPL is going to take a while as it prints out my library data … and I’m not really going to learn much of anything … I better check what it is and how big it is …
(class itl)
=> clojure.lang.PersistentHashMap
(count itl)
=> 10
Hmmm, only 10 keys, what are they?
(keys itl)
=>
("Major Version"
"Show Content Ratings"
"Playlists"
"Music Folder"
"Minor Version"
"Date"
"Tracks"
"Application Version"
"Library Persistent ID"
"Features")
"Tracks" sounds promising but there’s some large collections in there somewhere.
Easy enough to see what I’m dealing with …
(into {}
(map
(fn [[k v]] [k [(class v) (if (coll? v) (str (count v) " items") v)]])
itl))
=>
{"Major Version" [java.lang.Long 1],
"Show Content Ratings" [java.lang.Boolean true],
"Playlists" [clojure.lang.PersistentVector "195 items"],
"Music Folder" [java.lang.String "file:///Volumes/mediaHD/iTunes/iTunes%20Media/"],
"Minor Version" [java.lang.Long 1],
"Date" [org.joda.time.DateTime #object[org.joda.time.DateTime 0x3d37e74a "2015-10-27T19:44:01.000-05:00"]],
"Tracks" [clojure.lang.PersistentHashMap "23177 items"],
"Application Version" [java.lang.String "12.3.1.23"],
"Library Persistent ID" [java.lang.String "88ABD0BA83F503C5"],
"Features" [java.lang.Long 5]}
Yep, a lot of tracks. I should say that what I’m eventually going to want to do is read in my artist,
album, and track information into a Datomic database so I can play with the data even more.
To do that I want to know how the data is organized in this file. It looks like it’s all in that large "Tracks" map.
I could continue this process of cautiously peeking into the structures to get an idea of their size before fully looking at or a part of it.
(def tracks (itl "Tracks"))
=> #'itq.parse/tracks
(let [[[tk tv]] (take 1 tracks)]
[(class tk) (class tv)])
=> [java.lang.String clojure.lang.PersistentHashMap]
(let [[[tk tv]] (take 1 tracks)]
[(class tk) (count tv)])
=> [java.lang.String 27]
Ok, 27 keys is not too bad but I don’t know what the values are and this is getting a little tedious. I should be able to write some functions to help speed this up. After a little playing (right in the REPL) I come up with this.
(declare explore-map)
(declare explore-vector)
(defn explore
"Safely explore possibly large map/vector structures in the repl.
mv - map, vector
threshold - if count is under threshold, recursively explore
peekn - if over threshold, recursivley explore peekn entries
and summarize the rest"
[mv threshold peekn]
(cond
(map? mv) (explore-map mv threshold peekn)
(vector? mv) (explore-vector mv threshold peekn)
:else mv))
(declare big-map)
(defn explore-map
[m t p]
(if (< (count m) t)
(into {} (map (fn [[key val]] [key (explore val t p)]) m))
(big-map m t p)))
(defn big-map
[m t p]
(let [peeked (take p m)
rest (drop p m)
more (str (count rest) " more entries")]
(assoc (into {}
(map (fn [[key val]] [key (explore val t p)]) peeked))
:more more)))
(declare big-vector)
(defn explore-vector
[v t p]
(if (< (count v) t)
(into [] (map (fn [e] (explore e t p)) v))
(big-vector v t p)))
(defn big-vector
[v t p]
(let [peeked (take p v)
rest (drop p v)
more (str (count rest) " more entries")]
(conj (mapv (fn [e] (explore e t p)) peeked)
more)))
I’ll let the appropriate function look inside of the structure and if the size is under a threshold I’ll show it all, otherwise I’ll only show a part of it.
(explore tracks 10 1)
=> {"30645" {"Library Folder Count" 1, :more "26 more entries"}, :more "23176 more entries"}
Oh yeah, I already know there are 27 keys in that map.
(explore tracks 28 1)
=>
{"30645" {"Library Folder Count" 1,
"Disc Number" 1,
"Total Time" 332066,
"Disc Count" 1,
"Equalizer" "Rock",
"Persistent ID" "5B219C7E2973CDCE",
"Artist" "Joe Louis Walker",
"Album" "Live At Slim's Volume 1",
"Play Date" 3428123742,
"Location" "file:///Volumes/mediaHD/iTunes/iTunes%20Media/Music/Joe%20Louis%20Walker/Live%20At%20Slim's%20Volume%201/03%20Don't%20Play%20Games.m4a",
"Track ID" 30645,
"Track Count" 10,
"Track Number" 3,
"Date Modified" #object[org.joda.time.DateTime 0x592ec0f3 "2011-07-03T17:03:17.000-05:00"],
"Bit Rate" 675,
"Size" 28068234,
"Date Added" #object[org.joda.time.DateTime 0x79eae4d "2011-07-03T17:02:51.000-05:00"],
"Play Count" 1,
"Year" 2008,
"File Type" 1295270176,
"Track Type" "File",
"Name" "Don't Play Games",
"Sample Rate" 44100,
"Genre" "Blues",
"Play Date UTC" #object[org.joda.time.DateTime 0x5b14aca "2012-08-18T08:35:42.000-05:00"],
"File Folder Count" 5,
"Kind" "Apple Lossless audio file"},
:more "23176 more entries"}
(explore tracks 28 3)
=>
{"30645" {"Library Folder Count" 1,
"Disc Number" 1,
"Total Time" 332066,
"Disc Count" 1,
"Equalizer" "Rock",
"Persistent ID" "5B219C7E2973CDCE",
"Artist" "Joe Louis Walker",
"Album" "Live At Slim's Volume 1",
"Play Date" 3428123742,
"Location" "file:///Volumes/mediaHD/iTunes/iTunes%20Media/Music/Joe%20Louis%20Walker/Live%20At%20Slim's%20Volume%201/03%20Don't%20Play%20Games.m4a",
"Track ID" 30645,
"Track Count" 10,
"Track Number" 3,
"Date Modified" #object[org.joda.time.DateTime 0x592ec0f3 "2011-07-03T17:03:17.000-05:00"],
"Bit Rate" 675,
"Size" 28068234,
"Date Added" #object[org.joda.time.DateTime 0x79eae4d "2011-07-03T17:02:51.000-05:00"],
"Play Count" 1,
"Year" 2008,
"File Type" 1295270176,
"Track Type" "File",
"Name" "Don't Play Games",
"Sample Rate" 44100,
"Genre" "Blues",
"Play Date UTC" #object[org.joda.time.DateTime 0x5b14aca "2012-08-18T08:35:42.000-05:00"],
"File Folder Count" 5,
"Kind" "Apple Lossless audio file"},
"43515" {"Library Folder Count" 1,
"Total Time" 3539200,
"Persistent ID" "4208E480BB2E44CC",
"Comments" "Long before being recognized as an outstanding interviewer and talk show host, Dick Cavett started his career as a performer by doing magic. On this episode of The Spirit of Magic podcast, Mr. Cavett talks about his lifelong passion for magic, his apprec",
"Artist" "Dodd Vickers",
"Album" "The Magic Newswire",
"Play Date" 3475728561,
"Location" "file:///Volumes/mediaHD/iTunes/iTunes%20Media/Podcasts/The%20Magic%20Newswire/MNW%20%23195%20__%20DICK%20CAVETT.mp3",
"Composer" "Wizard & The Vegas Showgirl",
"Track ID" 43515,
"Date Modified" #object[org.joda.time.DateTime 0x6de8524d "2013-09-16T09:39:16.000-05:00"],
"Sort Album" "Magic Newswire",
"Bit Rate" 128,
"Size" 56898510,
"Date Added" #object[org.joda.time.DateTime 0x68dc8150 "2013-09-16T09:39:16.000-05:00"],
"Artwork Count" 1,
"Release Date" #object[org.joda.time.DateTime 0x7d16387d "2010-06-22T11:27:40.000-05:00"],
"Play Count" 1,
"Year" 2010,
"Track Type" "File",
"Name" "MNW #195 :: DICK CAVETT",
"Sample Rate" 44100,
"Genre" "Podcast",
"Play Date UTC" #object[org.joda.time.DateTime 0x213534ce "2014-02-20T08:09:21.000-06:00"],
"File Folder Count" 4,
"Podcast" true,
"Kind" "MPEG audio file"},
"9333" {"Library Folder Count" 1,
"Disc Number" 1,
"Total Time" 307400,
"Disc Count" 1,
"Persistent ID" "941D2F3691078CC6",
"Artist" "Israel Kamakawiwo'ole",
"Play Date" 3279050530,
"Location" "file:///Volumes/mediaHD/iTunes/iTunes%20Media/Music/Israel%20Kamakawiwo'ole/Unknown%20Album/01%20Somewhere%20Over%20The%20Rainbow%20-%20What%20a%20Wonderful%20World.m4a",
"Track ID" 9333,
"Track Count" 1,
"Track Number" 1,
"Date Modified" #object[org.joda.time.DateTime 0x5c4f6611 "2007-11-27T22:14:19.000-06:00"],
"Bit Rate" 248,
"Size" 9619241,
"Date Added" #object[org.joda.time.DateTime 0x56117e25 "2007-11-27T22:13:27.000-06:00"],
"Play Count" 1,
"File Type" 1295270176,
"Track Type" "File",
"Name" "Somewhere Over The Rainbow - What a Wonderful World",
"Sample Rate" 44100,
"Play Date UTC" #object[org.joda.time.DateTime 0x4ef3ca4c "2007-11-27T23:22:10.000-06:00"],
"File Folder Count" 5,
"Kind" "AAC audio file"},
:more "23174 more entries"}
It’s track info keyed on the "Track ID". I’ll probably just want the tracks in a flat collection.
Also, It looks like there’s some variation between track map keys. This kind of thing makes me curious (well, more curious).
I wonder what is the maximum set of track keys across all of the tracks?
Also, what is the full set of keys ever used? Should just be a function away. Actually, let’s make it a few functions and
flatten down the track info while we’re at it.
(def tracks (into [] (vals (itl "Tracks"))))
=> #'itq.parse/tracks
(defn track-keys-count
[tracks]
(reduce (fn [ks m]
(into ks
(map #(vector % (inc (get ks % 0)))
(keys m))))
{}
tracks))
=> #'itq.parse/track-keys-count
(sort (track-keys-count tracks))
=>
(["Album" 22680]
["Album Artist" 2287]
["Album Rating" 45]
["Album Rating Computed" 34]
["Artist" 22192]
["Artwork Count" 9654]
["BPM" 115]
["Bit Rate" 19606]
["Clean" 100]
["Comments" 2761]
["Compilation" 646]
["Composer" 11805]
["Content Rating" 546]
["Date Added" 23177]
["Date Modified" 20892]
["Disabled" 1]
["Disc Count" 13776]
["Disc Number" 14445]
["Episode" 312]
["Episode Order" 167]
["Equalizer" 14859]
["Explicit" 69]
["File Folder Count" 19635]
["File Type" 14792]
["Genre" 22710]
["Grouping" 542]
["Has Video" 1864]
["Kind" 21354]
["Library Folder Count" 19635]
["Location" 22632]
["Loved" 2]
["Movie" 1509]
["Music Video" 3]
["Name" 23177]
["Part Of Gapless Album" 195]
["Persistent ID" 23177]
["Play Count" 11997]
["Play Date" 11191]
["Play Date UTC" 11191]
["Podcast" 3131]
["Protected" 633]
["Purchased" 1287]
["Rating" 45]
["Rating Computed" 41]
["Release Date" 6625]
["Sample Rate" 18498]
["Season" 61]
["Series" 318]
["Size" 21973]
["Skip Count" 1677]
["Skip Date" 1638]
["Sort Album" 3142]
["Sort Album Artist" 182]
["Sort Artist" 2062]
["Sort Composer" 75]
["Sort Name" 1411]
["Sort Series" 21]
["Stop Time" 1]
["TV Show" 306]
["Total Time" 21323]
["Track Count" 15762]
["Track ID" 23177]
["Track Number" 16968]
["Track Type" 23177]
["Unplayed" 3287]
["Volume Adjustment" 5]
["Year" 18734]
["iTunesU" 2535])
That’s the full set of keys and how many times each is used. I would like to know the type of values behind each key, and make sure we’re done with colletions.
(defn track-keys-class
[tracks]
(reduce (fn [ks m]
(into ks
(map #(vector % (class (get m %)))
(keys m))))
{}
tracks))
=> #'itq.parse/track-keys-class
(sort (track-keys-class tracks))
=>
(["Album" java.lang.String]
["Album Artist" java.lang.String]
["Album Rating" java.lang.Long]
["Album Rating Computed" java.lang.Boolean]
["Artist" java.lang.String]
["Artwork Count" java.lang.Long]
["BPM" java.lang.Long]
["Bit Rate" java.lang.Long]
["Clean" java.lang.Boolean]
["Comments" java.lang.String]
["Compilation" java.lang.Boolean]
["Composer" java.lang.String]
["Content Rating" java.lang.String]
["Date Added" org.joda.time.DateTime]
["Date Modified" org.joda.time.DateTime]
["Disabled" java.lang.Boolean]
["Disc Count" java.lang.Long]
["Disc Number" java.lang.Long]
["Episode" java.lang.String]
["Episode Order" java.lang.Long]
["Equalizer" java.lang.String]
["Explicit" java.lang.Boolean]
["File Folder Count" java.lang.Long]
["File Type" java.lang.Long]
["Genre" java.lang.String]
["Grouping" java.lang.String]
["Has Video" java.lang.Boolean]
["Kind" java.lang.String]
["Library Folder Count" java.lang.Long]
["Location" java.lang.String]
["Loved" java.lang.Boolean]
["Movie" java.lang.Boolean]
["Music Video" java.lang.Boolean]
["Name" java.lang.String]
["Part Of Gapless Album" java.lang.Boolean]
["Persistent ID" java.lang.String]
["Play Count" java.lang.Long]
["Play Date" java.lang.Long]
["Play Date UTC" org.joda.time.DateTime]
["Podcast" java.lang.Boolean]
["Protected" java.lang.Boolean]
["Purchased" java.lang.Boolean]
["Rating" java.lang.Long]
["Rating Computed" java.lang.Boolean]
["Release Date" org.joda.time.DateTime]
["Sample Rate" java.lang.Long]
["Season" java.lang.Long]
["Series" java.lang.String]
["Size" java.lang.Long]
["Skip Count" java.lang.Long]
["Skip Date" org.joda.time.DateTime]
["Sort Album" java.lang.String]
["Sort Album Artist" java.lang.String]
["Sort Artist" java.lang.String]
["Sort Composer" java.lang.String]
["Sort Name" java.lang.String]
["Sort Series" java.lang.String]
["Stop Time" java.lang.Long]
["TV Show" java.lang.Boolean]
["Total Time" java.lang.Long]
["Track Count" java.lang.Long]
["Track ID" java.lang.Long]
["Track Number" java.lang.Long]
["Track Type" java.lang.String]
["Unplayed" java.lang.Boolean]
["Volume Adjustment" java.lang.Long]
["Year" java.lang.Long]
["iTunesU" java.lang.Boolean])
(defn track-keys-info
[tracks]
(let [counts (track-keys-count tracks)
classes (track-keys-class tracks)]
(merge-with vector classes counts)))
=> #'itq.parse/track-keys-info
(sort (track-keys-info tracks))
=>
(["Album" [java.lang.String 22680]]
["Album Artist" [java.lang.String 2287]]
["Album Rating" [java.lang.Long 45]]
["Album Rating Computed" [java.lang.Boolean 34]]
["Artist" [java.lang.String 22192]]
["Artwork Count" [java.lang.Long 9654]]
["BPM" [java.lang.Long 115]]
["Bit Rate" [java.lang.Long 19606]]
["Clean" [java.lang.Boolean 100]]
["Comments" [java.lang.String 2761]]
["Compilation" [java.lang.Boolean 646]]
["Composer" [java.lang.String 11805]]
["Content Rating" [java.lang.String 546]]
["Date Added" [org.joda.time.DateTime 23177]]
["Date Modified" [org.joda.time.DateTime 20892]]
["Disabled" [java.lang.Boolean 1]]
["Disc Count" [java.lang.Long 13776]]
["Disc Number" [java.lang.Long 14445]]
["Episode" [java.lang.String 312]]
["Episode Order" [java.lang.Long 167]]
["Equalizer" [java.lang.String 14859]]
["Explicit" [java.lang.Boolean 69]]
["File Folder Count" [java.lang.Long 19635]]
["File Type" [java.lang.Long 14792]]
["Genre" [java.lang.String 22710]]
["Grouping" [java.lang.String 542]]
["Has Video" [java.lang.Boolean 1864]]
["Kind" [java.lang.String 21354]]
["Library Folder Count" [java.lang.Long 19635]]
["Location" [java.lang.String 22632]]
["Loved" [java.lang.Boolean 2]]
["Movie" [java.lang.Boolean 1509]]
["Music Video" [java.lang.Boolean 3]]
["Name" [java.lang.String 23177]]
["Part Of Gapless Album" [java.lang.Boolean 195]]
["Persistent ID" [java.lang.String 23177]]
["Play Count" [java.lang.Long 11997]]
["Play Date" [java.lang.Long 11191]]
["Play Date UTC" [org.joda.time.DateTime 11191]]
["Podcast" [java.lang.Boolean 3131]]
["Protected" [java.lang.Boolean 633]]
["Purchased" [java.lang.Boolean 1287]]
["Rating" [java.lang.Long 45]]
["Rating Computed" [java.lang.Boolean 41]]
["Release Date" [org.joda.time.DateTime 6625]]
["Sample Rate" [java.lang.Long 18498]]
["Season" [java.lang.Long 61]]
["Series" [java.lang.String 318]]
["Size" [java.lang.Long 21973]]
["Skip Count" [java.lang.Long 1677]]
["Skip Date" [org.joda.time.DateTime 1638]]
["Sort Album" [java.lang.String 3142]]
["Sort Album Artist" [java.lang.String 182]]
["Sort Artist" [java.lang.String 2062]]
["Sort Composer" [java.lang.String 75]]
["Sort Name" [java.lang.String 1411]]
["Sort Series" [java.lang.String 21]]
["Stop Time" [java.lang.Long 1]]
["TV Show" [java.lang.Boolean 306]]
["Total Time" [java.lang.Long 21323]]
["Track Count" [java.lang.Long 15762]]
["Track ID" [java.lang.Long 23177]]
["Track Number" [java.lang.Long 16968]]
["Track Type" [java.lang.String 23177]]
["Unplayed" [java.lang.Boolean 3287]]
["Volume Adjustment" [java.lang.Long 5]]
["Year" [java.lang.Long 18734]]
["iTunesU" [java.lang.Boolean 2535]])
Well, I’m happy with this for now. I have a good understanding of the data. I remember something about a "Playlists"
key so I better make sure that’s nothing I care about. I’ll just explore from the top.
(explore itl 11 1)
=>
{"Major Version" 1,
"Show Content Ratings" true,
"Playlists" [{"Playlist Persistent ID" "7DFD01EA07FCC290",
"All Items" true,
"Visible" false,
"Master" true,
"Playlist Items" [{"Track ID" 6085} "20311 more entries"],
"Playlist ID" 82971,
"Name" "####!####"}
"194 more entries"],
"Music Folder" "file:///Volumes/mediaHD/iTunes/iTunes%20Media/",
"Minor Version" 1,
"Date" #object[org.joda.time.DateTime 0x1b471e14 "2015-10-27T19:44:01.000-05:00"],
"Tracks" {"30645" {"Library Folder Count" 1, :more "26 more entries"}, :more "23176 more entries"},
"Application Version" "12.3.1.23",
"Library Persistent ID" "88ABD0BA83F503C5",
"Features" 5}
(explore itl 11 3)
=>
{"Major Version" 1,
"Show Content Ratings" true,
"Playlists" [{"Playlist Persistent ID" "7DFD01EA07FCC290",
"All Items" true,
"Visible" false,
"Master" true,
"Playlist Items" [{"Track ID" 6085} {"Track ID" 6083} {"Track ID" 6091} "20309 more entries"],
"Playlist ID" 82971,
"Name" "####!####"}
{"Playlist Persistent ID" "73F3C095D7F3E012",
"Music" true,
"All Items" true,
"Smart Info" #object["[B" 0x72669460 "[B@72669460"],
"Distinguished Kind" 4,
"Smart Criteria" #object["[B" 0x23eee98c "[B@23eee98c"],
"Playlist Items" [{"Track ID" 8083} {"Track ID" 7031} {"Track ID" 5705} "15605 more entries"],
"Playlist ID" 103282,
"Name" "Music"}
{"Playlist Persistent ID" "8BFA8F6E5F0B5084",
"All Items" true,
"Smart Info" #object["[B" 0x7a605286 "[B@7a605286"],
"Distinguished Kind" 47,
"Smart Criteria" #object["[B" 0x5f15adc4 "[B@5f15adc4"],
"Playlist Items" [{"Track ID" 6697} {"Track ID" 6695} {"Track ID" 8293}],
"Playlist ID" 118893,
"Name" "Music Videos"}
"192 more entries"],
"Music Folder" "file:///Volumes/mediaHD/iTunes/iTunes%20Media/",
"Minor Version" 1,
"Date" #object[org.joda.time.DateTime 0x1b471e14 "2015-10-27T19:44:01.000-05:00"],
"Tracks" {"30645" {"Library Folder Count" 1, "Disc Number" 1, "Total Time" 332066, :more "24 more entries"},
"43515" {"Library Folder Count" 1,
"Total Time" 3539200,
"Persistent ID" "4208E480BB2E44CC",
:more "24 more entries"},
"9333" {"Library Folder Count" 1, "Disc Number" 1, "Total Time" 307400, :more "20 more entries"},
:more "23174 more entries"},
"Application Version" "12.3.1.23",
"Library Persistent ID" "88ABD0BA83F503C5",
"Features" 5}
A playlist looks like some info accompanying a list of track ids. None of which I care about right now.
I’m certain that I will someday do something similar with some other data so I want to save that explore function to
use later. I gathered up the definitions from my REPL history, added code for sets, sequences, and some tests.
The project is on github at https://github.com/hby/repl.
Just pull the code down and follow the README.md instructions.
It’s not very hard to begin to put together a bunch of functions that you can use for exploration in any REPL.
If you’re new to Clojure, or functional programming, or languages that have a REPL, I hope this encourages you to crack open a REPL and start to think about your own story. It’s a means to whatever end you have in mind.
“I knew when I met you an adventure was going to happen.”
― A.A. Milne
clojure.core defines the vars *print-length* and *print-level* that a REPL can use to limit its output.
You can set a var in the REPL by
(set! *print-length* 25)
I have found that not all REPL environments honor this. Could be something I’ve done wrong, though.
[UPDATE: I must have done something wrong, as I suspected. I went back to try this out and these setting are being honored. I’ve only tried them out in Cursive but have no reason to believe they don’t work everywhere.]
But even if they do, I wanted more flexibility than they provide. However, they are good to know about and setting values for them does make a good safety measure in case you do stumble across some data that is larger than you thought.
I’ll never see my dad grow old.
Sorry, I didn’t mean to shock you or make you feel uncomfortable. But that is a fact that I’ve pondered many times recently.
My dad died unexpectedly on October 14, 1997. He was only 67 and still active and enjoying life. My memory of him was always as a low-key yet active guy. For me, that last snapshot of him from 1997 will be how I think of him for all time as there is no new data coming in.
It was the farthest thing from my mind when he died, but lately, through my wife and others, I’ve seen the emotions that come with watching a father become frail. And the thought has come to me that, for better or worse, I’ll never experience that.
For some reason I find it a kind of heavy reality that this, quite normal, experience that most people will have, I will never have. That’s just the fact of it.
Pondering never has always been weighty for me.
(BTW, this is not the long inaugural post I mentioned before. I guess it didn’t have to come next.)
“When I discover who I am, I’ll be free.”
― Ralph Ellison, Invisible Man
This is a subject that has come to my mind a number of times. Probably more so lately because I’ve been interviewing. But it also comes up occasionally when talking to younger programmers. What do we (whatever we are) call ourselves? What do we think, to ourselves, that we do?
I work in computer software as a programmer. My degree is in Electrical Engineering. When I started out I called myself a software engineer. But that was mainly because I actually was an engineer by degree and I guess, in some way, I wanted credit for that even though I never really worked as an engineer (at least not in the sense that my degree would indicate). Turns out, companies don’t mind giving out that title.
I no longer think that way. At least the part about needing credit. I was young.
Now, I do think that engineering, in its most general defintion, does include the kinds of things a programmer does. But I see enough use of the term ‘software engineer’ that could cummunicate more simply with the term ‘programmer’. In my old age I just want to be simple and direct so that is what I’ve now been responding with when someone asks me what I do.
I’m a programmer.
I hear a lot of people say they are a ‘developer’. That isn’t bad if the context is understood. But I want to say one thing no matter who I’m talking to. Saying I’m a programmer saves me from saying I’m a software developer. But I confess, all this was to lead up to what I really want to talk about.
Nearly my entire career I have noticed that a lot of programmers refer to themselves as a pick-a-language programmer. During college I used APL, Prolog, and Lisp more than anything else. But I never would have thought to identify myself as a Lisp programmer, for instance. After leaving college I used C++ for nearly 5 years, then Objective-C for 2+ years, and then mostly Java for more years than I can count. Regardless of the time, I’ve always thought of myself simply as programmer. Now that I’m trying to do more Clojure, am I no longer a Java or iOS programmer? I know there are certain niche markets where you need to identify yourself more narrowly. In fact, I just remembered that I did used to call myself a NeXT programmer. I would still tell people that I am a programmer and if needed I would say I can do iOS programming.
Part of the reason for thinking this way is probably due to how I came into programming in college. Learning a computer language was always anchored in solving problems. I learned APL as a means to write CPU design simulation. I learned Lisp as a means to write image processing and understanding programs. I learned Prolog as a means to write expert systems. Later I studied some of these languages for themselves but that wasn’t the original motivation. I think this stuck with me.
I’m somewhat sensitive to this because of the specific (narrow?) focus I usually see in job postings. Especially within the DoD world that I’m sometimes connected to. I get this. I get why this is. I’m not really arguing against it. But I have seen this result in people thinking of themselves (without much internal reflection) in very narrow ways. Specialization is cool. But picking language X and becoming and X programmer for life is not a good long term strategy these days. And to be honest it makes me sad. And to be even more selfishly honest I’m sad because the more a programmer views themself this way, the more it reinforces the narrow focus of most job listings. Which, I suppose, is what I really wanted to talk about.
Maybe there’s some arogance buried there on my part. Could be, but it’s an accurate description of where I am now. One thing I know, I don’t want to fall into an unrefelctive, narrow life. Which is why I think about these things. I’m not one to look down on other poeple that are content in their own place that they have chosen. But it is also true that some young programmers won’t necessarily know the larger world of programming without someone encouraging them to look beyond what they currently know and what they are currently doing. I’ve found that employers vary widely in how much they encourage their people to look beyond the job they are doing. And I’m not even going to get into the current college situation.
This is all getting better with the rise of the term polyglot programmer. While, to me, that’s still just programing, I won’t quibble (in general) if it leads to more job openings that emphasize programming per se. And as for the the so-called full-stack, hybrid, etc. developers, I’m not going there either. They are not at the heart of my concern.
So, I’m curious. What do you call yourself? I’m not talking about titles. How do you tell other people who you are to you? Was this natural or did you come to it over time? And how much should we care to talk about this with younger programmers? Or am I thinking too much about nothing?
“Who are you?”
“No one of consequence.”
“I must know.”
“Get used to disappointment.”
― William Goldman, The Princess Bride