Ties have been deteriorating since a Canadian company sent around 100 shipping containers that included rotting rubbish wrongly labelled as recyclables to Philippine ports in 2013 and 2014.
Manila set a May 15 deadline for Canada to take the rotting trash back, after President Rodrigo Duterte berated Ottawa over the issue last month.
Canada has since said it is working to arrange for the containers' return, but has not said when exactly that might happen.
Philippine Foreign Secretary Teodoro Locsin said letters recalling the ambassador and consuls to Canada have been sent and the diplomats would be in Manila "in a day or so".
"Canada missed the May 15 deadline. And we shall maintain a diminished diplomatic presence in Canada until its garbage is ship-bound there," Locsin wrote on Twitter.
Canada's foreign ministry did not immediately respond to a request for comment.
The garbage has strained ties between the two nations, which were already tested after Canadian Prime Minister Justin Trudeau's questioned Duterte's deadly drug crackdown.
Duterte bristles at any international criticism of his signature policy, which has seen police kill thousands of alleged addicts and pushers since 2016.
Last year Duterte cancelled the Philippine military's $235 million contract to buy 16 military helicopters from a Canada-based manufacturer after Ottawa put the deal under review because of the president's human rights record.
During a speech in April, Duterte threatened to unilaterally ship the garbage back to Canada, saying "let's fight Canada. I will declare war against them."
Duterte frequently uses coarse language and hyperbole in speeches about opponents.
Following the comments, Canada offered to repatriate the waste and the Philippines said Ottawa would shoulder the expense of disposal.
Manila's Bureau of Customs said last week the Philippines was ready to send back the waste but Canada needed several more weeks to prepare documentation.
Some 69 shipping containers of trash remain after 34 others have already been disposed of in the Philippines, the finance ministry said.