It depends on the public holiday.
Supermarkets and convenience stores are open on most public holidays. However, on Christmas Day, Easter Friday and Easter Sunday you should expect to see supermarkets closed across the country.
ANZAC Day is another public holiday where there are trading restrictions in place before 1 PM — so expect to see supermarkets closed then too.