Banqueting House, view from Westminster Bridge, St Margaret's Church, Jewel House, St John Smith Square, view from Lambeth Bridge, Museum of Garden History.
The walk is best done in daylight. However even in the dark the views are great as major buildings such as Big Ben are well lit.
The traffic in London is dense. If you can find pedestrian crossings, I suggest you use them even if it means walking a bit further. Guidance is given where possible.