Senior federal government Minister Mathias Cormann is standing by his comments that people who attended were being "reckless" and "irresponsible" at a time of a pandemic.
Twenty thousand people marched in Sydney and crowds rallied in Melbourne, Brisbane, Adelaide and some regional cities and towns on Saturday despite public health warnings.
The Australian protests in solidarity with African-American George Floyd who died while being arrested in Minneapolis, also showed support for the Aboriginal community to highlight high levels of Indigenous incarceration and deaths in custody.
But Senator Cormann, the federal finance minister, thought the protesters were "quite irresponsible".
"I think it is incredibly selfish, it is incredibly self-indulgent, and yes it does impose unnecessary and unacceptable risks on to the community," Senator Cormann told Sky News' Sunday Agenda Program.
Labor's Indigenous Australians spokeswoman Linda Burney described Senator Cormann's comments as "tone deaf" and "politically expedient".
Protesters took a knee at Sydney's Belmore Park and held a minute silence for Indigenous Australians who have died in custody. Source: AAP
"Mathias Cormann should know better than to describe these protests yesterday, this cry from the heart of many thousands of people across the world and in Australia, as self-indulgent and reckless," she told reporters in Sydney.
While Deputy Opposition Leader Richard Marles was "uncomfortable" about people turning up for these marches, he said he understood why they did it.
"I don't feel like I'm in a position to say to indigenous Australians ... that this is a selfish and indulgent act," he told ABC TV's Insiders program.
"I'm not about to engage in that kind of judgement of those who did it."
Senator Cormann later told reporters in Perth while people had the right to protest over legitimate issues, the country was in a pandemic environment.
"Many, many Australians have lost their jobs as we impose restrictions on the economy in order to save lives, to suppress the spread of the virus, people have been unable to attend funerals of their loved ones to help stop the spread of the virus," Senator Cormann said.
"But we are going to have a mass gathering of tens of thousands of people in complete breach of the rules that apply to everyone else, it is absolutely reckless and irresponsible and it shouldn't be happening."
Deputy Chief Medical Officer Paul Kelly says while he can understand why people joined the protests, health authorities now have to be extra alert.
More than 5,000 people packed Victoria Square in Adelaide, where police praised the organisers. Source: AAP
"We don't know if anyone in those mass gatherings were infected or infectious, and so it is a wait-and-see approach," Professor Kelly told reporters in Canberra on Sunday.
"I was encouraged to see the face masks yesterday ... those people we're doing the right thing."
Health Minister Greg Hunt said he hoped the rallies would not lead to a new wave of infections.
"We don't know whether people will be infected," Mr Hunt told ABC radio on Sunday.
"But if there is someone who is infectious in the midst of a crowd like that, that can have a catastrophic impact."
People in Australia must stay at least 1.5 metres away from others. Check your state’s restrictions on gathering limits. Testing for coronavirus is now widely available across Australia.
If you are experiencing cold or flu symptoms, arrange a test by calling your doctor or contact the Coronavirus Health Information Hotline on 1800 020 080. The federal government's coronavirus tracing app COVIDSafe is available for download from your phone's app store.
SBS is committed to informing Australia’s diverse communities about the latest COVID-19 developments. News and information is available in 63 languages at .
