Denmark's Fuglsang was expected to challenge for overall victory after winning the chief pre-Tour warm-up event, the Criterium du Dauphine in June.
And he was sitting fifth overall before a crash on Wednesday in which he suffered two fractures to his arm.
He struggled badly on Thursday's Pyrenean stage, falling out of overall contention and when he was dropped early on Friday by the peloton, he decided to call it quits.
"It's a big loss for me and the whole team," admitted Aru, who had already seen another team-mate in Dario Cataldo crash out on Wednesday.
"We'll try to keep going and recover well," added the Italian, who leads reigning champion Chris Froome by just six seconds.