Gillard and opposition leader Tony Abbott were stranded in The Lobby restaurant as they were pursued by dozens of demonstrators from a nearby protest of Australia Day, which marks the arrival of British settlers in 1788.
The two leaders were dramatically escorted through the crowd by security agents and riot police brandishing shields, and the visibly rattled Gillard tripped and fell during the rush.
The demonstrators had reportedly pounded the building's glass walls, shouting "shame" and "racist".
They had been attending so-called "Invasion Day" commemorations at the nearby Aboriginal tent embassy, a permanent camp of indigenous activists celebrating its 40th anniversary this year.
Michael Anderson, founder of the tent embassy, said the group had been angered by remarks made by Abbott earlier in the day about the embassy no longer being relevant.
"He said the Aboriginal embassy had to go, we heard it on a radio broadcast," Anderson told the Australian Associated Press news agency.
"We thought no way, so we circled around the building."
"What (Abbott) said amounts to inciting racial riots," he added.
Aborigines, whose cultures stretch back tens of thousands of years, are believed to have numbered around one million at the time of British settlement, but there are now just 470,000 out of a total population of 22 million.
They are Australia's most disadvantaged minority, with shorter life expectancy and much higher rates of imprisonment and disease than their non-Aboriginal counterparts.