Many are sub-Saharan Africans suspected of being mercenaries hired by the Gaddafi regime.
The UN said the new Libyan government had responded positively when pressed to deal with the issue.
The BBC's Barbara Plett in New York says this was the first UN assessment of the situation in Libya since the end of the eight-month civil war.
The report, by the UN Secretary General Ban Ki-moon, estimates that 7,000 prisoners in Libya are currently held in prisons and makeshift detention centres, most under the control of revolutionary brigades.
Human rights "While the (National Transitional Council) has taken some steps toward transferring responsibility for the detainees from brigades to proper state authorities, much remains to be done to regularize detention, prevent abuse and bring about the release of those whose detention should not be prolonged," the report says.
Mr Ban said: "I believe that the leaders of the new Libya are indeed committed to building a society based on the respect for human rights."
"Achieving this requires the earliest possible action, however difficult the circumstances, to end arbitrary detention and prevent abuses and discrimination, against third country nationals as well as against any group of Libya's own citizens," he added.
The UN's Libya envoy Ian Martin welcomed last week's appointment of an interim government in Tripoli.
"It is indicative of the difference from the attitudes of past regime that there is no denial that human rights are being violated and in most cases international organisations are granted access to detainees," Mr Martin told the BBC.
"The new minister of the interior told me he welcomed public criticism as strengthening his hand in tackling the issues," Mr Martin added.
But whatever the attitudes of new government members, Mr Martin told the Security Council the interim government faced enormous challenges:
- disarming and integrating revolutionary fighters who have now taken over law and order functions in the absence of a police force
- securing weapons stockpiles and stopping the proliferation of arms
- building from scratch an electoral system able to hold elections by June.