Hoe off or hand weed seedlings when small. Better still, try to remove dead flower heads regularly to prevent seed dispersal. Other garden plants that can become prolific self-seeders include Anemanthele lessoniana, camassia, chives, fennel (Foeniculum vulgare), Nectaroscordum siculum, sisyrinchium and Verbena bonariensis.
Digging out unwanted plants may work for a while, but is only likely to be a temporary solution. Suppression under black plastic or weed membranes is an alternative but again could take several growing seasons to be effective.
Beware putting invasive plants and their seedheads on the compost heap, as this is unlikely to reach a high enough temperature to kill off seeds, tough roots or underground stems (it is all right if they have already been killed off with a weedkiller). Instead, place them in the municipal green waste, as this is composted on an industrial scale, where tough weeds should be killed off. Burning may also be appropriate, but check your local Council guidelines.
For herbaceous weeds, try a programme of spraying using a systemic herbicide containing glyphosate* – Roundup and Tumbleweed are common brand names of such products. For woodier plants, choose triclopyr (SBK Brushwood Killer) or glyphosate* formulated for stump killing (e.g. Deep Root Ultra Tree Stump & Weedkiller).
*A note on glyphosate May 2016: After reviewing glyphosate, the European Parliament has given the go ahead to relicense it but proposes disallowing certain uses such as public open space and restricting it to professional use only. The final resolution will be decided by national authorities this summer which may mean the withdrawal of glyphosate-based weedkillers for home gardeners in the UK. Check the RHS website for further updates.
Weedkillers for gardeners (Adobe Acrobat pdf document outlining weedkillers available to gardeners; see sections 1a and 4)
Chemicals: using spot and broad-scale weedkillers
Chemicals: using a sprayer
Chemicals: using safely and effectively
Weeds: non-chemical control