There's a great compilation if you wanna start, credited to the Late Great Daniel Johnston, called Discovered Covered.
2 disks, with his work on one, and covers on the other. So that if you don't like him or a specific period in his work, you can appreciate the covers (done by Tom Waits, M. Ward or Bright Eyes to name the ones I dug).
His site regularly provides mp3 to get to know him a bit better - and you'll have the visuals too.
http://www.rejectedunknown.com/multi...udio/audio.htm
My favorite is Rejected Unknown, and is quite recent.
And Some Things Last A Long Time on 1990 is really moving.