Osama Bin Laden House Demolished
The al-Qaeda leader was shot dead at the compound in the north-western city near the capital Islamabad in May 2011.
Security forces brought in bulldozers and began destroying the tall outer walls of the compound after sunset, reports say.
They set up floodlights to carry out the work.
Residents have told the BBC that an unannounced curfew has been placed on people in the area, and they have been asked not to leave their homes, reports the BBC's Ilyas Khan in Islamabad.
But many are reporting seeing and hearing bulldozers and diggers at Osama Bin Laden's former compound.
The demolition started early in the evening and is still said to be continuing.
The site is a large compound with several high walls built around the actual house.
An official of the Abbottabad district administration told the BBC the operation was expected to be completed by about 03:30-04:00 (22:30-23:00 GMT).
Officials say the compound was handed over by the military to the civil authorities before the operation started.
Officials say it was decided to demolish the compound soon after the 2 May 2011 raid, but it was put off when the government constituted a judicial commission to investigate the raid.
"Since the commission has almost completed its work and did not need the compound for any further investigation, it was decided it should be razed," an official said.
He said the reason for the demolition was the visitors the place continued to attract, which posed a security threat to the area, located in the midst of an important garrison town.
Osama Bin Laden is said to have lived in the compound with his family for several years before he was killed.
On that day, a team of US special forces flew from Afghanistan to Bin Laden's hiding place during the night in stealth helicopters on a secret operation.
They swept through the buildings within the high-walled enclosure and shot dead a total of five people, including Bin Laden.
Some 40 minutes later, they left, taking with them Bin Laden's body and a hoard of computer data devices and other information containing intelligence about al-Qaeda and Bin Laden's activities.
The compound has been a painful reminder for Pakistan, which was embarrassed by the unilateral US operation that killed Bin Laden, correspondents say.