Mandatory wikipedia links:

• Tax havens (short version: low tax)
  
• Offshore Financial Centres (short version: low regulation, volume out of touch with economy).

Of course many centers are both, in a small place even a low fee on bank registration is more than enough to cover government needs and so no tax.

On the mechanism at work, these places sell one commodity: obscurity. There are two non exclusive ways to reach it:

• Legislate it (banking secrecy laws, eg Swiss)
  
• Have the executive/judiciary branch not look at possible crimes

Let's not forget the second way is as efficient as the first and is used by the UK, as seen with Carousel VAT fraud:

Figures from Eurocanet, a European Commission sponsored project, released in September 2006, appear to show that the United Kingdom is the main victim of this fraud, having lost an estimated €12.6 billion during 2005-6, followed by Spain and Italy which each lost over €2 billion during this period.

UK loses the most because it doesn't monitor that its businesses follow the law in its simplest form. See the article for a detailed description of the scheme.