Topic: Sorting a Music Collection

[Bill Roggensack]

Has anyone had a positive experience with software that has been designed to serve as a quick indexer and database for recorded music? I have been curious about one called Music Collector, and I seem to recall both Rivers and Slack were using software designed for this purpose. My needs are simple - keep track of what I've got, avoid duplication, find things quickly (ha!), and (hopefully never) satisfy the curiosity of an obnoxious insurance adjuster. I'm currently relying on hi-res photos taken one shelf at a time, and use TreeSize to capture music directories from my lossless and MP3 libraries. The best thing I've got at present is a Squeezebox server - once I have music ripped to disk, it's easy to find, and I never have to put it away when I'm finished listening.

[Richard]

What's iTunes 

[Bill Roggensack]

What's iTunes 

Software that mostly makes me nervous when it's not aggravating me.

[dj]

Has anyone had a positive experience with software that has been designed to serve as a quick indexer and database for recorded music?

After the last two posts, I'm almost afraid to say it, but yes, I use such software, and the software I use is iTunes.

iTunes is not without it's problems.  Foremost among them is that it was never designed for a really large collection.  The library descriptor is just an xml file.  If your library gets really big, the file gets really big, and performance slows a lot as the OS is constantly paging parts of the file into and out of memory.  (Yes, I know, getting a 64 bit machine with a lot of memory will help this - it's on its way). 

The big problem with iTunes is the same problem with any database software: no matter how well it's designed, it's only as good as the information in the database.  What you get if you just rip a CD and accept the information from Gracenote is garbage.  CD information is in so many formats that it's all but useless.  If you want the database functions to work well, you have to spend time to rationalize the data whenever you rip a CD (i.e make sure the album titles of multi-volume sets are consistent, change the artist names when Gracenote just gives you "Various Artists", correct the Year field so it always reflects what you want, and add the appropriate info to Grouping, Genre, Composer, Comments, etc).  If you take the time to do all this, iTunes gives you a pretty darned good database, with some decent tools to search and order based on the various data fields.

If I could change one thing about iTunes to make it more useful, it would be to allow multiple entries in the Artist field, so that for example, I could enter Roosevelt Sykes and Dobby Bragg.  I realize that one can just put all the names in one field and do a search with "contains", but this just seems so darned inelegant.     

[Rivers]

I could have written something similar to dj's post but he said it all. I use iTunes but it's a love / hate relationship at best. I'd drop it like a ton of bricks if I could find something with a better data model behind it. It has improved a lot over time but sophisticated it ain't. The data model is iTunes big weakness if you're at all serious about cataloging music. Otherwise, it sort of does the job.

Al Young has built a data model that impressed me with its possibilities when I saw it yonks ago, and may want to chime in here.

[Stuart]

I use iTunes, but mostly from habit--and I use an older version. As DJ  points out, the information for imported CDs comes from the Gracenote db, which works if the info is accurate and complete, but misspellings, inconsistencies and other incorrect info abound, so it can be a real chore to go through and correct the mistakes. My daughter prefers WinAmp over iTunes, but although I've looked at it, I've never done any kind of comparison.

[alyoung]

Al Young has built a data model that impressed me with its possibilities when I saw it yonks ago, and may want to chime in here.

Hey, Mark, the things that stick in your mind eh? What I have is a relational database that enables me to find tracks and artists in my collection by using  a variety of keywords. The good news is that it works fairly well; the bad news is that it's very labor-intensive -- can't pick anything up from iTunes; it all has to be manually entered (and me a two-finger typist).

It's done on MS Access. I have about five tables, but the three main ones contain details of all the media in my collection (78s, 45s, LPs, cassettes, CDs),  the tracks (including the artist, the recording date, the original issue and the issues on which I have them) and the personnel (includes all the names, plus where the tracks were recorded and any "session notes"). So far, I've got details entered for about 40,000 tracks.

The main keys used to link the tables are the recording date, the artist name and the issue on which I have the track, but the wonders of Access enable me to assemble information by any of the criteria covered in the fields. Put it all together and I can replicate a Dixon & Godrich entry ... if only I could get the Access reports to work as I think they should. (Reports are easily the weakest aspect of Access, however if you're the only one using the database, you seldom need them.)

Wonderful ... but as I say, it's hard work -- so much so that only the gospel component of my collection is fully entered, and just keeping that up to date is time-consuming enough. (The focus on gospel is because that's where most of my disco etc  research is done.) So I've started incorporating a lite version that will keep track of tracks without going into all the detail of the full version (no personnels, no detailed original issues are the main difference).

Overall .. it works for me, but it might not be for everyone, and if I'd known how much work it was going to be when I started, I might have had second thoughts.       

[Rivers]

Interesting perspective. Since I do this sort of stuff for a living (data modelling) I should have a stab at creating the ultimate (according to me anyway) data model, which would include grabbing external data from Gracenote. Or maybe I won't get around to it, which is much more likely. iTunes is good enough for what it is (i.e. free, lets you find stuff) but it's seriously limited if you want to do research or otherwise slice and dice.

This also touches on another thread going on how do you classify your music collection. With great difficulty, if you only have one genre you can assign to something, which is clearly good enough for 99% of the iTunes user base.

[Stuart]

What we need is the equivalent of the db that libraries use, but specifically for our purposes. They purchase them from a company/developer and turn on the items that are in their holdings. Makes infinitely more sense than having each and every library start from scratch and do it themselves. When the libraries were initially going from the old card catalog system to the on-line system, the books would be bar coded one at a time, and the code matched with the on-line info, often at circulation when they were checked out. That's the way they did it at Rutgers when I was there.

If a db was available for music with all the necessary fields, cross referencing and hyper-linking in place--in other words, the basic structure or framework--then one could import specific modules, and add additional info where necessary. Perhaps it could be like the Weeniepedia effort or the lyrics repository that we have here, where we all do our bit. We could clean up and refine the info from Gracenote and other sources, as well as poach info from Stefan's site (with his permission, of course) for the LPs. Obviously it will take a lot of work, coordination and cooperation. But it's something to think about. It might even be doable, given the will.

[Rivers]

Visionary stuff Stuart. This is the way it should go, for sure.

I imagine the extended weenie (sorry) corpus has probably oh, some highish percentage of all relevant stuff issued on CD and LP within their collective iTunes installations. We've already discussed the limitations of iTunes but you've got to start somewhere. A good start might be getting all that metadata into one place, purging out the less well-documented dupes and otherwise cleaning it up.

Then there's the maintenance thing, keeping it up to date with new releases. Re. adding external links, info and other extensions, good points. It's worth talking about even if we never come close to doing it since at the very least it tends to put the finger on what needs to evolve and what's wrong with the current situation.

[pete1951]

Most of my collection I would call blues, but I would feel uncomfortable to file it using ethnicity (misterjones `one has to be African-American to qualify...`)
Where would BookerT and the MGs go?
Jessie Edwin Davies?
The Memphis Horns?
Italian opera my sound best song performed in Italian but you don`t have to be from Italy ,
Classic blues could be played by anyone ,(but most of my favorite singers were born in the Delta,)
I am neither African nor American but I am a Blues Guitarist
Pete T

[Lyle Lofgren]

Given the fact that you're not collecting mainstream material, the choice is either to use someone else's data base and have worthless information, or make your own, which is labor-intensive. I chose the latter, and use MS Access. I use it for CDs only (I put my reel-to-reel tapes on MS Word years ago, scanning the original typewritten tapelists and correcting them by hand).

I don't try to physically separate the CDs. I assign each one a sequential number without regard to content. I then enter that on a database table called "CD List." It contains 5 columns: CD#, Title, Who, What, and CD ID. The "Who" is the person's name associated with the CD. The "What" is either performer or editor (for anthologies), and "CD ID" is publisher and CD number (i.e., Yazoo 2047).

On a second database table called "CD Contents," I enter the title of each track. This DB also has 5 columns: CD#, Track, Song Title, Artist, and Comments.

Here's an example from items 4 & 5 of "CD List." You'll have to imagine the columns:

CD #   TITLE   WHO   WHAT   CD ID
4   STAY AWHILE   Jody Stecher & Kate Brislin   Performer   Rounder CD 0334
5   TIMES AIN'T LIKE THEY USED TO BE, VOL. 1   Richard Nevins   Editor   Yazoo 2028

Here's an example of the first 3 tracks of CD#5. I had no comments, so that column is blank.

CD #   TRACK   SONG TITLE   ARTIST
5   1.00   BLUES IN A BOTTLE   Prince Albert Hunt
5   2.00   DOLLAR BILL BLUES   Charlie Jordan
5   3.00   LOST JOHN DEAN   Bascom Lamar Lunsford

Track numbering took some thought. For multiple CDs, I use track 101 for the first track of the first CD in the box,  201 for the first track of the second CD, etc. For a medley on (say) track 6 of the first CD in a boxed set, I'd use 106.1, 106.2, etc.

It works great, because with Access, I can find either a title or an artist, and since I have the CDs filed sequentially, I can find the CD right away. The downside is that the only time I enter any data is on long winter nights when there's nothing else to do. I have indexed about 200 CDs so far (less than 1/3 of my collection). The CD Contents DB has 4800 entries. If I were going to finish this before I die of old age, I'd have to hire some unemployed English major to do it.

Lyle

[dj]

This also touches on another thread going on how do you classify your music collection. With great difficulty, if you only have one genre you can assign to something, which is clearly good enough for 99% of the iTunes user base.

You can do multiple entries in any field in iTunes and use the "contains" operator to search on it.  Thus, for one song I might have the Genre Blues CountryBlues JugBand.  (I collapsed the names together because at the time I started I didn't know that the iTunes operators would work on strings with embedded blanks - I might do that differently now).  The problem is that most people entering data see something like "Genre" and just assume that it can only be one thing, so only one thing gets entered.  And, of course, that one thing can be more or less accurate depending on who's doing the entering.  My favorite was one of Document's Bumble Bee Slim disks, for which someone had entered the genre as "Unclassifiable".

I love Stuart's idea of a dedicated country blues database (as long as I don't have to do the work!).         

[Rivers]

I'd better dig deeper into the edge functionality within iTunes, obviously. I'd kind of written it off as a serious data model but that's interesting. Was it you dj who was logging recording dates, locations, labels etc into it?

Part of what puts me off is I've had to rebuild the darn library too many times after moving disks, failed cpus, and so on. All the edge stuff has disappeared each time, as well as locally-scanned artwork. I just don't trust it any more though for sure it's more stable than it was.

[Stuart]

Part of what puts me off is I've had to rebuild the darn library too many times after moving disks, failed cpus, and so on. All the edge stuff has disappeared each time, as well as locally-scanned artwork. I just don't trust it any more though for sure it's more stable than it was.

One of the tricks is to find out where everything is, write it all down on a piece of paper that hopefully won't get lost and back everything up. I use an older version and inadvertently gorked the iTunes Music Database file and the iTunes Music library file (which is different from iTunes Music folder) that were in the iTunes folder (which is different from the one that usually gets installed in the Program Files folder by default) in the My Music file that was located in the My Documents folder in XP Pro. What a headache! Lost all my playlists and a lot of other info specific to the way I had things set up. Perhaps iTunes has changed so that things aren't scattered all over the place, but I'd check, just to be sure. BTW, Who's on first?