Mr Dutton said the jobs and skills summit, scheduled to take place over two days in September, would be nothing but a talkfest.
The coalition has previously suggested an increase of the threshold to allow pensioners and veterans to work to get more people into the labour market.
The opposition leader had previously described the event as a union stunt.
"Given the list of demands from the ACTU ... a talkfest with the unions wasn't a good way to spend my time when I'd already put forward a pretty reasonable suggestion," he told ABC Radio on Monday,
"(The government) roundly rejected it, they've now picked it up and saying that it will be part of the job summit, as it should, but we'll have good policies in good time before the next election that will address all of this."
Mr Dutton said the slowdown in migrant workers coming through Australia had also compounded the labour shortage issues, which had affected multiple industries.
"You are seeing businesses that can't open for dinner if they've opened for lunch because they don't have the staff in their restaurant," he said.
"You are seeing aged care facilities where people aren't getting the requisite care that they require."
While Mr Dutton turned down an invitation for the summit, Nationals leader David Littleproud will attend, as will Greens leader Adam Bandt.
The summit will bring together about 100 people representing unions, businesses, civil society and government.
Ahead of the summit, there had been calls to lift Australia's annual migration cap to allow more skilled workers.
The current cap sits at 160,000 a year, but the government is weighing up whether to increase it to 180,000 to 200,000.
Business Council of Australia chief executive Jennifer Westacott said an overhaul to the migration system was one way of helping to solve the skills crisis.
"We need to move from a short-term, ad hoc system to long-term planned migration with a focus on four-year visas, pathways to permanent migration, and future planning of our population growth," she said.
"We want to see a catch-up boost to the permanent migration program, with at least two-thirds of places for skilled workers."
Ms Westacott said the backlog of current visa approvals also needed to be addressed urgently.
Along with a boost to migration, the Business Council outlined multiple areas that needed to be addressed at the summit, including the development of future industries, the skilling up of Australians and tackling disadvantage.
Ms Westacott said the summit needed to also focus on the advancement of women and a win-win workplace relations system.
"The summit must agree the reset is to restore the role of collective bargaining as the centrepiece of a modern IR system because it delivers better outcomes for both workers and employers," she said.
"But it must be designed to deliver outcomes for both workers and employers and be accessible to different types of employers. It also has to be vastly simpler and easier to navigate."
