A Blog About Making Culture
My Library Is Dead
January 10, 2007
My iTunes library, of about 12,000 songs consisting of 60GB of data, got corrupted today. The MP3 files themselves are okay, but all of the metadata is blown away -- playlists, play counts, ratings, and all my little tweaks to song and album titles.
I'm surprised by how much grief this causes me. As it turns out, my experience of these songs is determined by the record of how I've lived with them -- without the information about how I've listened to them, and how often, and what I thought of them, they're just not my songs. It makes sense, at some level; Art without curation or creation without witness leaves a work mute. But as geeks, a lot of us wouldn't even necessarily see this as data loss -- the original files, after all, is still there.
There's a lesson here for the prophets of abundance, and for all of us who see formerly valuable things becoming commodities. We might be able to replace the raw materials with their abundant, or even free, counterparts. But if our emotional experiences are lost along the way, the perfect digital copies are worthless, too.
Previously: Blogging For A Living
Next: Places to Go
Curating.info
...and now I'm off to curate my coffee table: The terms "curator" and "curating" are being slung around in a wide variety of contexts these days, mostly to do with the curator-as-filter. It is intriguing to see a term that is usually used in a fine arts context to be used in other contexts (in the th read more »Jeff Harrell
I believe iTunes keeps a running backup of your library in the file iTunes Music Library.xml. Have you tried removing the iTunes Library file and letting iTunes rebuild it from the backup?
No idea if that’ll work, but I suppose it’s worth a try.
Ian
Anil the lesson is once again: back up often. Sounds like an opportunity to use your Blog to discuss back up scenarios - not just locally on your home work or play computer but also for all the online web-blogs and sites out there. Ian
Mark
I sympathize. The metadata you add by interacting with your songs en-masse is valuable. You’ve explained it better than I ever could, and goodness knows I tried.
dvg
Ouch. That is a disaster of epic proportions. I’ve been tweaking and updating all my iTunes metadata for years. If I lost all the information I’ve meticulously added to my 10k song library over the last several years I don’t know what I would do. I had never thought to add my iTunes metadata to my backup routine until now. Any tips? Do you think it’s safe to simply archive the xml and itl files?
Jurgen
I feel for you—been very afraid of this very thing happening for a while now. I keep nightly updates of my WinAmp media library. Sure, the playcounts and ratings aren’t the music, but by recording our personal history with the music they do contain lots of valuable information.
On the other hand, you get to start over, and perhaps that’s exciting too? A clean slate and all. Perhaps you’ll develop a whole new set of tastes.
ramanan
It is pretty amazing how much less interesting iTunes becomes once you remove all that information from it. It’s also interesting that for you, and many people I know, the music they own is more than just the sounds stored in an MP3. My friend gets depressed everytime iTunes munges up his iTunes Library.
Also, is the fate of all iTunes libraries their ultimate corruption? It certainly seems that way. I’m surprised Apple doesn’t do more to protect this data when it operates on the library file.
David
That’s interesting. I ditched my metadata and reconstructed my iTunes library last fall when replacing my hard drive, and I felt no loss at all. I had my music, all ID3’d and classified, and so long as it played, I was happy.
Once upon a time, the record of how we lived with music was a visceral one: I know I listened to Living Colour’s “Time’s Up” countless times in my Nissan Maxima, on my old metal Sony Discman with the wires running up the dash. One did not need a computer to track play counts and add stars.
I suppose this is why I have yet to digitize the majority of my music (despite the thousands of tracks on my iPod I have a good thousand or so CDs I’ve never ripped), and why year-end blog posts stating “here are the bands I played the most” somehow strike me as a little thin compared to true “what I love” items. For me, still, the best music is felt, not logged.
Also: dude! No backup?
Jeremy Zawodny
That reminds me of the time Steve Jobs Ruined My Thanksgiving: http://jeremy.zawodny.com/blog/archives/005732.html
Jeremy
BTW, your OpenID “Sign in” button leads me to a page that generates an “Internal Server Error” :-(
Sumit Chachra
These are the kind of stories that have prompted me to start thinclientblog.com and prompted my move away from Apple products.
Hopefully you had something like the last.fm itunes plugin (audioscrobbler) installed, so your music preferences might be (sort of) backed up with them. Even with the new iPhone, if I can access all my data (and listen to music through last.fm or pandora) why do I need iTunes and iSync?
Roland Dobbins
You know, I read a posting every couple of months from one or another of the ‘digital cognoscenti’ about having some sort of data disaster, and how it really sucks, and often (not in your case) berating Microsoft, or Apple, or whomever for their woes.
And the common unifying theme is apparently this - that the digital cognoscenti apparently don’t know how to/can’t be bothered to make backups.
If you had a backup, you could just restore your iTunes Library, and voila! you’d have all your metadata back, at least as of the last time you backed up (which should be nightly, right?).
Besides your metadata, is what you do for your day-job not important enough that you should really back up your data? If not, maybe you’re overpaid.
;>
Evan Martin 
Backups? I have multiple backups of my music archive because I’ve nearly lost it once or twice and the pre-grief I experienced was enough to scare me into it.
gwen
Maybe too little too late, but I regularly back up the iTunes Library and iTunes Music Library.xml files in addition to the music. Okay, I’m not so great about remembering to back up the music, but at least I’ll have a list to keep me warm. I’ve actually had to futz around with those and/or trash/replace them a couple times over the years, sometimes you can open in a text editor and salvage much of the content. Also, I think when you alter song names, comments etc. it changes the ID tags on the actually music file in addition to the metadata file, though I’m not absolutely positive, so hopefully you still have your fixed names and things even if you’ve lost your song lists. At the very least you’ve still got your last.fm account…good luck.
p.s. This is all on a Mac over here so ymmv.
gwen
p.s. you listen to way too much Prince ;P
jhn
Yeah, the phear of losing my 300 gigs of music inspired me to just run a mirror of my entire 500 gb internal HD.
As it stands, iTunes keeps both an XML and a binary version of your library information. You can use the XML to recover, if it’s still around. Your iTunes 7 directory should have * a Music folder * an “Album Artwork” folder (for downloaded artwork and caches, most artwork is still going to be inside the music itself) * iTunes Library, a binary database * iTunes Library copy, a backup, * iTunes Music Library.xml * iTunes Music Library copy.xml
You’ve lost all 4 of those files? (Or both, I’m not sure what generates the copies.) If you still have the original XML, just start iTunes without the binary but with the XML present and it will rebuild the binary from the XML.
Prasanth
You could have saved all that data by just copying the xml file in the itunes folder on windowsxp. I am not sure how that works on mac. Thats why just stick to the mp3 tags. Don’t waste time doing all the other stuff.
jhn
Oh yeah, you wouldn’t have lost any metadata that gets popped into the ID3 tags. So your song titles and album titles and so forth are not lost. The iTunes library only stores playcounts, ratings, and playlists. I live and die by my playlists so it would suck to lose them but it’s not as bad as having to re-tag your music.
Meep
I’ve had my iTunes library messed up twice already, but I don’t wax depressive about it. I don’t forget music that I like, and in fact sometimes I rediscover music this way. Of course, the iPod in my head has a tendency to play strange things that I actually dislike…
Karen
I have the opposite, yet similar problem: Right after I backed up my ailing computer, it died. I restored the backed-up data onto my shiny new computer, and though the songs were there, the metadata was not (or if it is there, I cannot find it).
Now I feel like my iPod is trapped in time. I can’t add anything new because the minute I synch I will lose all the playlists I spent so much time creating.
Eric Eggertson
This is exactly the issue Thomas Hawk has been talking about with online photo services. A lot of them let you export your photos, but not the metadata. On a site like Flickr, the metadata can represent thousands of hours of your time.
Isaac
That’s a crushing blow for sure. It’s crazy how comfortable the playlist gets when you’ve got that metadata built up — like having your old, comfortable boots stolen. A new pair will never feel the same way, no matter what.
The idea of “owning our music” is a dubious one, as wel. Who really owns it anyway? A while ago, Napster started promoting the idea “own nothing - listen to everything” for its all-you-can-eat subscription plan. I can’t imagine our ownership society will go for that very quickly, but it’s a peaceful idea to be sure - to let the music go and just experience what you can, when you can.
Jason
As a digital music/iTunes/last.fm obsessive, this totally freaks me out.
steve clayton
spooky…I had this exact conversation with Hugh MacLeod tonight who said he has £300 worth of someone elses tunes…sorta
Asbjørn Ulsberg
It is total insanity that iTunes (or any other music management application) doesn’t write the metadata to the files themselves. I can understand having the binary library file for performance issues, but not writing the metadata (at least that which fits inside the ID3 tag, like images, artist information, etc) into the MP3 files themselves is unfathomable.
Not only does it sucks if iTunes decides to attach C4 to its database and pull the trigger, but it sucks if you want to use the files somewhere else too, for example on another computer or in another music player. Had the information been copyrighted to Apple, I’d understand the decision to lock it in, but considering that it’s iTunes’ users that have used hours and hours writing this information into their own songs themselves, it doesn’t make any sense at all.
Kevin Spencer
The “how I’ve listened to them” and “how often” can also be solved by using last.fm.
David Jacobs
It is total insanity that iTunes (or any other music management application) doesn’t write the metadata to the files themselves.
Over the “millions and millions” of reads and writes previously discussed, inevitably an mp3 would get corrupted or destroyed, and this would much be worse than occasionally having to restore your backed up iTunes library.
Phillip Winn
Asbjørn, much of the metadata is stored within the files themselves, in the form of ID3 tags. (So Anil can relax — he hasn’t lost title tweaks as long as he has the song files.) However, Apple — and other applications — often come up with bits of metadata that aren’t part of the ID3 spec. What then? Extend (and therefore splinter) the ID3 spec? That’s hardly playing nice with the community, and would earn Apple — or any other company — a lot of deserved grief. No, metadata outside the ID3 spec does need to be stored outside the MP3 files.
There are multi-user issues as well. While titles and album art and lyrics and such are intrinsically part of the song, a play count or a rating is unique to the combination of the song and the user, as are playlists. Some of my MP3 files are in both my library and my wife’s, and we definitely have different ideas about the rating for certain songs!
Mark
No, metadata outside the ID3 spec does need to be stored outside the MP3 files.
OK, but it’s still a long leap to the conclusion that such metadata “needs” to be stored in undocumented vendor-specific binary formats. (And no, the XML “backup” files that iTunes creates do not contain all the metadata that Anil cares about. In fact, they are lacking specifically the metadata that Anil cares about most.)
And once again, as has happened so many times in the past, it is amazing how quickly the discussion has shifted from “Apple destroyed my data” to “let’s all talk about backup strategies.” Yes, backing up my data is important. Hard drives fail. I do stupid things on the command line. My software destroys its one and only complete copy of my data. Hmm… one of these things is entirely preventable and its repeated occurrence is entirely the software vendor’s fault; see if you can figure out which one.
Asbjørn Ulsberg
That iTunes writes ID3 — is that a new feature? Because I’ve used iTunes since early version 6 and used the exact same MP3 that iTunes have managed in other applications and all the changes I’ve ever made in iTunes has never shown up in either of the other applications. Not artist name, not song title, not anything. Which is why I’ve drawn the conclusion that iTunes infact doesn’t “do” ID3 at all.
But if it does, I hereby withdraw my negative comment. And keeping non-standard-ID3 stuff out of the ID3 tag is of course the right thing to do. But Apple could extend ID3 openly as well, as long as they do it in a backwards compatible way and lets everybody know how and why they are doing it.
Post a comment