Topic: Discographies for the mad collector

[DerZauberer]

As the year is winding down a little approaching the holiday period, I have a little time in the evenings to listen to some good music, rediscover some favorites, listen to some stuff I hardly ever listened to ... and then go down the rabbit hole of "Do I have everything by King Solomon Hill, hm, what versions are out there, on which compilations do I have them, what's the quality like, damn, why can't I figure out which version is which" ... at which point I sometimes start adding tags to my FLAC/MP3 files where I then put in recording dates, matrix numbers, etc.

I have the great Agram Blues Books discographies, I have an older copy of the American Premium Record Guide 1900-1965, I have various blues books, liner notes, etc ... but as much as I like digging and that's part of the fun, at some point I would really REALLY like to be able to access a modern website/database that just has all the info. Discogs is incomplete and has some errors (and rarely lists any unreleased/alternative takes), the "Blues & Gospel Records" almanach is really expesnive and hard to get, Stefan Wirz has a a lot but again it's not searchable...

So: Wouldn't the (blues) world be ready to have a nice, clean, searchable database (maybe even collaborative or Wikipedia-like so people can edit information from numerous sources) on the web? Are there "evil forces" (ie collectors that like their secrets) preventing this? Why do we still choose to publish these discographies as books and not as online reference databases?

Alternatively - if this is all to controversial - do you know of other sources/sites I can use when I'm listening to the next weird DOCD with rare recordings of little-known artists and I ask myself "this sounds good, where did he come from, is there more, ..."?

[CF]

Yeah it seems like this kind of question is being asked more and more. Howard Rye who co-compiled B&GR is still at it, updating the 1994 edition with (I think) Chris Smith assisting (Chris is one of the authors of the Penguin's Guide to the Blues). Folks have queried whether there is a chronological discography . . . can you imagine how interesting that would be?
It may come down to a group of fans doing this work in conjunction with the the original discographers and with many sets of eyes scanning for accuracy.
Anyway, no it doesn't exist yet but it sounds like more and more folks are interested in an exhaustive online source

[Rivers]

I did a lot of work on a data model for such a thing a while back. Searchability / sortability requires a fully relational database, much like the one this forum is built on. Then you have to come up with the user interface, and code it.

Count me in if you'd like to consider doing a project, even if it's just to explore feasibility, capture requirements, and define a project.

You probably realize that to make a solid job of it would be, er, a lot of work and it would take, um, quite a while.

Maybe there's an open source package out there already that would fit and could be modified. The wikimedia package database would not, IMO, fit, it's a different animal. The easiest way to do it is not necessarily the best way to do it, a lot depends on where you want to host it. Should it be a stand alone website, or integrated with forum etc software?

There is a powerful aspect of this website one could use to host it. Talk about fully integrated systems, that would be a gas, for us anyway. I'm game if you want to discuss further.

[TonyGilroy]

Brian Rusts's pre war Jazz Discography is on line free.

https://78records.files.wordpress.com/2017/06/rust_jazz-records_free-edition-6.pdf

[Rivers]

Nice, searchable text, thanks.

[DerZauberer]

Great feedback, thanks. The free Jazz "book" I think might show the way forward ... if all the major discographies were available at least in an electronic format, that would be a good starting point! Of course ideally not only as a fully laid out cumbersome PDF, but as the "raw" file(s) behind it. As a first step, having the books electronically would enable everyone to look up things quickly, and maybe implement a cross-volume search engine that pulls the relevant portions into a search results page.

The challenge, of course, is finding a compelling data model that fits "everything", and then finding the time (and/or group of people) to put together an excellent "first shot at it", that would be so compelling to the community that everyone would want to migrate to it.

One thing to find out I guess would also be the amounts of money being made by compiling and selling these discographies. As much as some of them are called "labor of love", at the same time these are being sold as physical books for a decent price and not just put online as a free-to-use PDF.

Ah well ... might spend some time thinking about this during the holiday period...

[TonyGilroy]

The Brian Rust discography has been put on line by Mainspring Press.

Over the years they have published a vast number of discographies all heavily priced which, for me, is fully justified given the amount of research involved but inevitably I've bought very few.

They are selling off remaining stock and going fully on line and they may be worth contacting as it's possible that they are contemplating something akin to what you're suggesting. They also have links to other discography sites.

There's a very good Modern jazz discography site at

https://www.jazzdisco.org/

and Blues & Rhythm magazine has a free on line post war cajun music discography.

Country is covered by Praguefrank who has a massive range of artist discographies available.
http://countrydiscography.blogspot.com/

There's a lot out there but it would be a brave soul with enormous free time who could contemplate integrating all these and no doubt others I don't know about.

[DerZauberer]

There's a lot out there but it would be a brave soul with enormous free time who could contemplate integrating all these and no doubt others I don't know about.

Yeah that's a core point. In my "mind palace", I have the idea of first using a combination of a good data format and a bit of human calibration of an AI thing to do a rough sort of existing source material, auto-check for duplicates and cross-populate all kinds of info ... to have a very big database of "well it's better than nothing, at least it's all there".

The challenge would then be to have a group of dedicated people (this can't be one person alone) to tackle different sections of the catalog and review, merge, add, etc. the data. Alongside, we'd have to come up with naming and sorting conventions (e.g. artist(s) shown on the record vs. actual artist names, is it Bukka White or Booker White or Booker T. Washington White, oder "Bukka" White).

Ok, I'll stop now. It's a huge undertaking and I'll REALLY sit down and think now. My firm belief otherwise is that this can only be a non-for-profit labor of love, without an online community to work on it there's no way this can be anywhere near complete. On the other hand, developments in computer AI give me hope that we are getting closer to a stage where we can "...just dump data in there and the machine will sort it out pretty good in no time".

[Rivers]

I don't think it would be all that hard actually. The requirements capture and data model are the thing. For example, artists with alternative names; you could handle that in a couple of different ways at the database level.

I do not share your optimism for developments in AI. AI is just a buzzword for "a lot of software that never quite automates what you're trying to do". You need coders that know the requirements, working with a database that fits the task.

Initial loading and ongoing administration of data is part of any project of this nature, it has to be coded. Since you can't type all this stuff in obviously it would usually go through a couple of processes that can be run, improved and rerun until clean.

Defining scope would have to be rigorous. For example some people would want to cram details of every recording, from every genre, ever made into such a thing. I guess if the data model was right you probably could. But would you want to? Not me.

[Stuart]

I've been away from database stuff for decades now, but as Rivers says, putting together the architecture and structure would not be difficult. Some random thoughts...

Given what it would cover, I would suggest having it located at or sponsored by an institution, probably at an university like UCSB and/or UI that have strong music departments. I would also recommend open source software because with anything proprietary there is always uncertainty about what may happen in the future, according to people I know who have been there and done that.

The case doesn't have to be made for its viability as a resource that serves those who are custodians of culture and history, specifically music, and all its sub-categories, genres, etc. That's a given. Getting the funding to hire the people to do the work is another matter. Having it under the umbrella of an institution gives it a certain legitimacy and would perhaps make it easier to get financial support, maybe from NEH, etc..

I take the long view on this. A sponsoring institution(s) so hopefully it isn't dependent on a small group of private individuals. A relational database that covers all the genres, etc., and one that can also be compartmentalized. Don't make the stated focus too narrow. The ability of volunteers to contribute so as to flesh out the content. And administrators to keep an eye on things to make sure that everything runs smoothly.

My late college roommate worked in database development since he was an undergrad in the late 60s. He spent a couple of decades as a software engineer at Terradata and at AT&T and NCR before that. We used to discuss this stuff from time to time. It can be done, but organizing the people, finding a sponsoring institution, and securing the ongoing resources to support it without giving up control is where the challenges lie, IMHO.

[Rivers]

Good post there Stuart.

A relational database that covers all the genres, etc., and one that can also be compartmentalized. Don't make the stated focus too narrow.

Absolutely. Any data model would need to be fully comprehensive and scaleable. You would design it to handle every case, even if you didn't envisage ever loading chronic ephemera, for example, last week's pop stuff. Somebody would though.

Compartmentalization is an interesting discussion. Often the downfall of such data models is the failure to realize that a title can be filed under multiple genres, which is the same relationship problem as artists with multiple names. Getting those aspects right would be key to capturing a real world view that reflects how we actually think about and use the data. Get them wrong and you end up with another failed discography nobody wants to use gathering dust on the web.