Just walk up and down Berwick Street (off Oxford Street) - Sister Ray is down there, plus two reasonable second hand places. Also nearby (as in they're on roads that branch off Berwick Street) are Sounds of the Universe (Soul Jazz label's shop) and Phonica.
In Notting Hill you've got a good secondhand place near the tube station, plus Rough Trade just off Portobello Road.
There's a bigger branch of Rough Trade in East London.
And there's Kristina (which only does vinyl) in Dalston, not all that far from Cafe Oto.
www.sisterray.co.uk
www.soundsoftheuniverse.com
www.phonicarecords.com/
www.roughtrade.com/
http://kristinarecords.com/
www.mgeshops.com/