Senator Nigel Scullion says no one has approached him about changing the date of Australia Day.
"It never comes up as an issue," he told ABC radio on Friday.
Instead, Indigenous Australians are focused on their children getting a good education and access to health care.
"If you want to divide the nation this is how we go down that line," Senator Scullion said.
Indigenous Health Minister Ken Wyatt was surprised to hear his colleague hadn't been approached about a date change by some indigenous Australians.
"Certainly I've had individuals who have that view talk to me, I've had them text me, even as late as yesterday," he told ABC TV.
"He may not have had that approach, but I certainly have, and I know that my other indigenous colleagues have been part of that discussion as well."
Their comments come after Tony Abbott described British settlement as "a very good thing" and attacked Bill Shorten for having a bet each way on the date of Australia Day.
The former Prime Minister said British settlement was something all Australians "on balance" could celebrate, and the country would not be improved by "wallowing in a kind of endless carping self-criticism".
"British settlement was a very good thing, it wasn't good immediately for everyone but the country, the modern Australia that emerged from British settlement stemming from the 26th of January 1788, is something that all of us on balance can and should be proud of," Mr Abbott told 2GB radio.
Mr Shorten on Thursday said he supported Australia Day staying on January 26, but would not sneer at Indigenous Australians wanting to discuss changing the date.
Mr Abbott said the Labor leader's ambiguity on the debate over the public holiday showed he was not ready to run the country.
