Oh boy! Well you've come to the right place, as he is one of my favorite directors.
Absolute must-sees: Ghost Dog: The Way of the Samurai; Down By Law; Paterson. I absolutely love each of these movies, with Ghost Dog being one of my top movies ever. I've watched each of the first two multiple times, but need to watch Paterson again; I fell in love with both the main characters.
Not for everyone, but probably for you: Stranger Than Paradise; Dead Man. You won't be scared away by their being (1) in black-and-white and (2) slow-moving. They're both gorgeous. Of the two, I'm a bigger fan of Stranger Than Paradise, which many consider his masterpiece.
Worth seeing if you liked the others: Night on Earth; Broken Flowers; Coffee and Cigarettes. The last one is hit-or-miss, but when it hits it really hits. As a music lover you might enjoy it more than some other people do.
Don't bother with the others.