One Stop shop for Daily Digest - News, Views and analysis of the political developments of the Horn of Africa. Now you can follow by email alerts situated at the bottom. Please feel free to forward any item of interest - it is your blog too (Make it your Home Page)
Unesco says its destruction would be "an enormous loss to humanity", but no damage has been reported there yet.
IS now control the nearby airport, prison and intelligence HQ, the Syrian Observatory for Human Rights says.
The militants have previously demolished ancient sites in Iraq that pre-date Islam.
The BBC's Jim Muir in Beirut says acute international concern over Palmyra might actually spur the jihadists on to make destroying it a priority, since they delight in challenging and horrifying world opinion.
The ancient ruins are situated in a strategically important area on the road between the capital, Damascus, and the contested eastern city of Deir al-Zour.
Palmyra is also close to oil and gas fields which the Syrian government uses to generate electricity for areas under its control in the west of the country.
Dating back to the 1st and 2nd Century, when the region was under Roman rule, Palmyra is dominated by a grand, colonnaded street.
Syrian state media said pro-government forces had been pulled out of Tadmur, the modern settlement next to Palmyra, after "assuring the evacuation" of most of its inhabitants.
The town's population would normally number around 70,000, but it has recently been swollen by an influx of people displaced from other combat areas.
The Syrian Observatory reported that more than 100 pro-government troops were killed in overnight clashes around Palmyra.
A researcher from the monitoring group also told the BBC on Friday that IS now controls more than half of Syrian territory.
However, the BBC's Arab Affairs Editor Sebastian Usher says this figure may give a false impression because there are large areas to the east under IS control that are not very significant strategically.
Analysis: Jim Muir, BBC News, Beirut
Many questions will now be asked in Damascus and Baghdad - and above all in Washington - about how the militants have managed to score major advances in both Iraq and Syria this week despite all the efforts to stop them.
IS was supposed to be on the defensive in Iraq, where the prime minister announced weeks ago the launching of a campaign to drive the militants out of Anbar province. Now he's lost its capital, Ramadi, just days before they took Palmyra in Syria.
The western coalition's bombing campaign has clearly hurt IS where it could. But it could never compensate for ground forces which are not competent, equipped or motivated enough to stand firm and hit back.
Only the Kurds in the north of both countries (most recently in north-eastern Syria) have proven able to do that.
Unesco's director-general, Irina Bokova, said any destruction to Palmyra would be "not just a war crime but...an enormous loss to humanity".
"It's the birthplace of human civilisation. It belongs to the whole of humanity and I think everyone today should be worried about what is happening," she said in a video statement.
Syria's head of antiquities, Maamoun Abdul Karim, said on Wednesday that hundreds of Palmyra's statues have been moved to safety but that large monuments could not be transferred.
"This is the entire world's battle," Mr Abdul Karim warned.
He called on the US-led military coalition against IS to prevent the group destroying the ancient site. However, the coalition says it does not co-ordinate its actions with the Syrian government.
The founder of the Association for the Protection of Syrian Archaeology, Cheikmous Ali, said the operation to move the objects began two months ago, but it picked up speed earlier this week.
"Some objects are still there, it's not 100% empty," he told the BBC.
But "considerable damage" had already been done by Syrian air force bombing and soldiers digging trenches at the site, he continued.
Are you in Palmyra? Do you have friends and family there? We're also looking for your photographs of the Syrian city should you have ever visited it. You can email email@example.com with your experiences firstname.lastname@example.org with your images.