Thousands of civilians remain trapped.
Once Sirte falls, Libya's leaders say they will declare liberation, even if Col Gaddafi remains on the run.
"There is a very vicious battle now in Sirte," said National Transitional Council (NTC) chairman Mustafa Abdel Jalil in the capital Tripoli.
"Today our fighters are dealing with the snipers that are taking positions and hiding in the city of Sirte."
AnalysisWhat looked yesterday like a rapid sweep through Col Gaddafi's hometown has slowed down to a grinding battle to take each street and each building.
Civilians have continued to leave Sirte when they can, in cars overloaded with people and their possessions. They're stopped and searched thoroughly for any evidence of pro-Gaddafi sympathies before they can move on, although many say they have no other place to go.
They looked terrified. Some said they still loved Gaddafi and would have stayed and fought, but for the welfare of their families.
They were convinced, they said, that they would be killed when they came out. In an effort to ease those fears, government fighters handed out bottles of water and flags of the new Libya that these dogged supporters of the old regime have so fiercely resisted.
Pro-Gaddafi snipers fired from the rooftops of the Ouagadougou conference centre, the university and a complex of flats.
However, the NTC side won control of a key boulevard which connects the Ouagadougou centre to central Sirte.
NTC fighter Faraj Leshersh told Reuters the Gaddafi loyalists were experts at operating unseen, using trenches or burning tyres to give themselves cover to move between buildings.
"They took advantage of the dust and they advanced a little. There is 500m (yards) between us and them," he said.
Civilians continued to leave Sirte, on foot and by car. They were stopped and searched by NTC forces at checkpoints.
The NTC gave civilians the opportunity to leave before the assault began.
However, thousands remained in the city, unable to get out or fearful after warnings from pro-Gaddafi fighters that they would be attacked by the interim forces if they surrendered.
Efforts to negotiate with loyalist commanders have also failed. On Thursday, Col Gaddafi delivered an audio message urging Libyans to take to the streets "in their millions" to resist the interim leaders.
Pro-Gaddafi forces also control the desert enclave of Bani Walid, but it is seen as less significant as it does not lead to any exit routes from the country.