Labor frontbencher Chris Bowen said the fact refugees on Manus Island and Nauru detention centres were accepted into the US after extreme vetting under a people-swap deal proved their claims were genuine.
"These people on whose fashion sense he was commenting, who were going to America - by virtue of [the] definition of the fact they're going to the United States, had been classified by his government as refugees, genuine refugees by definition," Mr Bowen told reporters in Sydney on Friday.
"They are refugees, they've been found by him [Immigration Minister Peter Dutton] and his government to be genuine refugees under the [UN 1951 Refugee] Convention - and he might choose to point that out rather than being the commentator on the efficacy or otherwise of their clothing.
"Peter Dutton should spend less time being the fashion police and more time trying to get genuine third-party resettlement for those who are [at offshore processing centres] on Manus Island and Nauru."
His comments came after Immigration Minister Peter Dutton doubled down on his "Armani refugees" comments.
Mr Dutton on Thursday took aim at the first group of refugees to leave Australia's offshore detention centres for resettlement in the United States.
Sydney radio host Ray Hadley put to the minister during a regular interview on Thursday that a photograph of the group published by News Corp this week looked like a fashion show on a catwalk in Paris or New York.
"Somebody once said to me the world's biggest collection of Armani jeans and handbags was up on Nauru waiting for people to collect it when they depart," Mr Dutton told 2GB radio.
Speaking in London, Mr Dutton promised to continue to make sure Australia wouldn't take in economic refugees and backed his comments about some of the 54 who had left the processing centre on Manus Island for the US.
"No," Mr Dutton bluntly said when asked if he would back down from his comments on Thursday about so-called economic refugees, who he said had "got on a boat, (and) paid a people smuggler a lot of money", in an interview on 2GB radio in Sydney.
"I think you can look very closely to see some people, who paid people smugglers up to $20,000, these aren't people who are the most impoverished in the world.
"These are people that have displaced the most impoverished or those that are facing certain persecution and sought to come to our country.
"We have been very clear about the fact that they will not settle in our country, so we are critical of some of those people, have been consistently critical and I am not going to deviate from that."
More than 50 refugees this week left offshore detention on Manus Island and Nauru for a new life in the US.
'Enjoying themselves'
Mr Dutton, asked about an image of those preparing to depart Port Moresby, said a lot of people who ended up in the island camps had not come from war-ravaged areas but were instead economic refugees.
They'd received "an enormous amount of support" from Australian taxpayers for a long time.
"We have been taken for a ride, I believe, by a lot of the advocates and people within Labor and the Greens who want you to believe this is a terrible existence," Mr Dutton said.
"These photos demonstrate otherwise. People have seen other photos in recent weeks of those up on Manus out enjoying themselves outside this centre, by the beach and all the rest of it."
Mr Dutton said he had long predicted once people were off Manus Island and Nauru "they'll start to tell a very different story about how it wasn't that bad".
"There is a very different scenario up on Nauru and Manus than people want you to believe," he said.
'Lack of understanding'
Amnesty International labelled the comments extremely irresponsible.
"They also show a complete lack of understanding of the refugee convention," refugee co-ordinator Graham Thom said in a statement.
He suggested Mr Dutton is putting at risk the opportunity for vulnerable and traumatised refugees to be safely resettled in the US.
"It is absolutely despicable that Peter Dutton would risk that by downplaying the acute vulnerability of these refugees at a time when the US is looking to cut its humanitarian program to its lowest level in over a decade," he said.
The refugees arrived in Australian waters years ago and were transferred to offshore detention under a strict government policy to block anyone who arrived by boat from entering the country.
They were recently cleared by US authorities for resettlement under a deal struck between the former Obama administration and the Turnbull government.
Up to 1,250 refugees are expected to be resettled in the US.
