Hooper was warned by Australia's chef de mission Nick Green not to wear the shirt again after sporting it at his opening fight on Monday.
Green said Hooper had apologised and assured him he would not happen again given it was a long-standing rule of the Olympics that only flags of competing nations can be displayed at Games events.
But the National Congress of Australia's First Peoples, which advocates for the rights of Aborigines and Torres Strait Islanders, said the incident was "bureaucratic insanity".
"Boxer Damien Hooper has a right to show his identity as an Aboriginal person in the Australian team," it said in a statement issued late Tuesday.
"He has nothing to apologise for. Our peoples should be able to show that they are both Aboriginal and Australian."
Anthony Mundine, a former world boxing champion, told The Sydney Morning Herald that Hooper "did the right thing" by breaking team rules to wear the black, red and yellow flag on his chest in London.
"I take my hat off to him for that stance," Mundine was quoted as saying.
"It takes a person with big balls to make a big stance like that.
"We want to be proud of a flag that we fly and the current Australian flag just doesn't sit well because of its dark history."
The Aboriginal flag has been the cause of controversy before, with indigenous runner Cathy Freeman sparking debate in 1994 by celebrating her Commonwealth Games 200m gold by doing a victory lap with it.
Hooper, who also has the Aboriginal flag tattooed on his body, had earlier defended his wearing of the shirt ahead of Monday's fight in which the hugely-talented 20-year-old light heavyweight beat US fighter Marcus Browne.
"I'm an Aborigine representing my culture and my people here at the Olympic Games," he said.