Of these, 817 were released from prison more than five years ago, UK Border Agency statistics reveal.
Some have been deported, only to be returned to Britain after tricking officials by claiming they were not from the country to which they were sent.
Other cases remain swamped by legal red tape, with offenders using laws which put protection of their children above the need to deport.
One criminal frustrated attempts to deport him nine times before he was sent back earlier this year.
Rob Whiteman, the agency’s chief executive, acknowledged deportation took too long. He told a committee of MPs: ‘I don’t think I can guarantee that it will come down rapidly.’
The Border Agency, also under pressure over long queues for immigration checks at Heathrow, released the figures showing 3,900 foreign criminals. were living in the community.
Many were freed from prison by courts because there was little chance of them being deported swiftly.
Unless they have permission to work, they will be living on benefits with access to the NHS and schools.
Tory MP Michael Ellis told Mr Whiteman: ‘People are getting fed up with this. It’s taking too long.’