My concern with this approach is that you will need like minded participants. At the very least open minded.
Several of us are rather eclectic in our tastes and if narrowed down to one artist, 75% of the participants are likely to tune out to an artist they do not like or choose not to understand further than they already know.
very good point.
but that could be counteracted by having somebody else rank your chosen artist's songs.
say, for example, you (JML) pick the Stranglers - ok, so you rank your top 20 or 25, or whatever the decided on number is, and submit that as the master list for them.
now, say there are 30 participants - there could be a draft held where we would get to choose who we would like to rank our artist - so, in this case, when your draft pick comes up you choose simey to rank the Stranglers songs - you would send her those 20/25 songs alphabetically, as to not influence - but the ranking master list is held, and would be revealed as we went along (e.g., #20 ranked by simey is "Walk on By", which was ranked #11 by JML) ... makes sense?
or instead of a draft to assign, it could all be drawn by random - only catch is a participant cannot wind up ranking their own submitted artist.
this would necessitate listening to the batch of songs by the artist assigned you, as the ranking is required - and by having them sent alphabetically, as i mentioned before, it removes at least some bias.
dunno if you lot have tried this before? as i pop in and outta here quite a bit, and have missed a ton of these drafts/rankings/etc