Will the real Big Data please stand up?

With time on my hands just recently I decided to get to work on a pet project I’d been thinking about doing for quite a while. The project was to go through my e-music collection and have a look-see what I’d accumulated over the years, at least what remained after numerous house moves and relocations along the way, and catalogue it. So, using state of the art methods (Data management) and technology, ok so it was only Excel tables, charts and pies, I fixed a simple twofold objective:

Wordle2-> Put some order into a hodgepodge of music and answer a question: What sort of music did I really like
-> Do it in such a way it would make the self-respecting Database Administrator in me proud of a job well done, who knows if I could do it with a personal database today, why not something along the lines of Amazon’s AWS tomorrow!

Getting started I knew for a fact that two tendencies would get confirmed, 1) a lot of the music would have been from the 1970’s and 2) most of it would be Rock, in one form or another, Led Zeppelin, ELP, Genesis, Pink Floyd, YES and the Rolling Stones being major influences of mine, then as now. What actually came out of the number crunching was in fact quite instructive and revealing, my guilty little, musical, secrets would finally be exposed.Decades_of_music

What I was also looking for in the cataloging, and got, were the indicators and the culminating peaks pin pointing memorable milestones over the years, milestones immortalized by the likes of Pink Floyd with “Dark Side of the Moon” in 1970s, The Alan Parson’s Project in the early 1980s and Slipknot in the early 2000s. The head-scratcher though was explaining the build-ups, the apexes and then the dips. The statistics only confirmed the fact not the reason, e.g. Was the music from then or from an earlier period and I just happened to be listening to it at the time!

… and then, predictably, if the statistics detailed the pinnacles they also brought to light the depressions of almost Mariana proportions corresponding with those barren, desolate, less happier times everyone goes through in their lives, times when not even the Leonard Cohens, the Stings, the Coldplays or the James Blunts of the time didn’t do much to help, worse, they risked getting themselves labeled as reminders of things best left forgotten.

The devil is in the detail!

Whoever coined the phrase “The Devil is in the detail” got it right. Initially the plan was to see if I could organize the data in such a way that I could create some jolly charts and pies, feel smug about it, then show everyone what a good DBA I was by posting some of the stats here. What actually happened was a long drawn out battle of will and logic with my Laptop and the different media systems it housed.Genre

I began by categorizing the data by Year*, Genre, Album and Artist the way I wanted it to be, only to find out that my laptop had other plans, re-categorizing a file according to metadata and tags it accessed but I couldn’t. Frustrated but determined I remembered that metadata could be modified and if the PC wouldn’t let me do it manually I’d go and find one of the many freebie tools available on the web… mind over matter!
* Categorizing by the year the tune was initially released, i.e. “As time Goes By” a tune interpreted by Satchmo in 1943!

It wasn’t over yet. Both the PC maker’s media app and the Windows OS media system being connected to the web both automatically updated the metadata again according to the data available online in their own specific media libraries. The result was confusing to say the least and that’s when I decided to download Freebie #2. Rock

But I still wasn’t out of the woods. After finally fixing the metadata, on the PC and again on my Androids (Androids really are something else) I started on the statistics, remembering one tip from the past: Keep it simple and use macros and formulas as much as possible otherwise you’ll be spending hours checking and cross checking figures. That saved me a lot of time but apparently not enough, the gentle voice of reason sitting next to me still wanted to know why I should be spending so much time on something so straightforward!

Finally the cataloging started showing results, for example, barring one George Harrison song and a couple of Wings’ songs, the Beatles and specifically John Lennon were missing from the listing, as were certain other icons and this raised several questions: What was the real purpose of the “e-collection” ? Why didn’t it contain much of the several hundred Vinyls and CDs I’d managed to hang onto over the years? Why wasn’t the best album ever or Lennon’s “Imagine” listed there? Where was this project going?

On looking at the near zero activity the statistics were showing me, namely for periods late 1980s and early 1990s, I drew the following conclusions:

-> These corresponded with periods I was either offline, so to speak, or embarked on some adventure or other
-> I’d purposefully blanked out some of all my all time favourites, afraid they be tainted by some event
-> That at the time, musically speaking, there simply wasn’t anything worth listening to on the radios where I was.
-> There’s still quite a bit that needs cataloging… so I’d better get on with it!

Over_the_Years

Kurt who?

It was only later, with a change of situation, the coming of internet and the start of the 2000s that I was able to objectively go back over the 90s looking for anything worth retrieving (a minutes’ silence for the late and regretted Napster and Kazaa culture of the early 2000s). Curious, I spent weekends surfing the web, sifting through the myriad of listings of 1990s music only to conclude that, barring one to two exceptions, there really wasn’t anything really worth retrieving, not the Britpop, not the boys bands and certainly not the Dance culture.
As for the Grunge, well that was over before I knew it existed and it was only after listening to some of the Punk Rock, Nu Metal and Indie music my son was listening to at the time (i.e. the Foo Fighters, Slipknot, System of a Down, Blink 182, Green Day, Taking Back Sunday, in the late 90s and then Kasabian, My Chemical Romance and Biffy Clyro in the 2000s) that I found anything worth listening to which accounts for the rise in the statistics the chart shows in the early 2000s, and if you’re wondering why the dip towards the end of the 2000s, call it the “Deezer effect”.

Where to now?

It’s possibly Time ticking away but some of the music around today still isn’t worth listening to but then perhaps it isn’t meant for me either, which I can live with. There’s so much music around, legit of course, that if you don’t like what you hearing then you simply surf for something else. There’s everything possible out there on the web fighting for your attention. The commercial radios of the 70s and 80s gave way to the Napsters and the Kazaas of the 2000s which in turn gave way to the Shazams, the Deezers and the Spotifys of the 2010s. 

If this is the shape of things to come then bring it on, in the meantime, I’ll just carry on with my e-cataloging.

Advertisements

About nickrichards38

http://www.amazon.com/NJ-Richards/e/B0094FXXEW
This entry was posted in Everyday life, IT & Computing and tagged , , , , , , . Bookmark the permalink.