Several current and former heads of state are attending the event
Mnangagwa, until recently one of Mugabe's closest allies, will take the oath of office at the national sports stadium on the outskirts of Harare before thousands of supporters, dignitaries and foreign diplomats.
Snipers took up positions around the stadium amid tight security as jubilant Mnangagwa supporters streamed in, with many dancing as music played.
"We are excited and expecting a lot from Mnangagwa. We have been under a dictatorship for a very long time," 23-year-old Sharon Mauyakufa said, referring to Mugabe.
"Mugabe is very old -- we do not expect that he will be punished for his crimes. How do you punish a 93-year-old? But his wife and others must be charged if they committed crimes."
The former president, who ruled the southern African country for 37 autocratic years, was ousted from office when the military intervened after he had sacked Mnangagwa as vice president.
"We thank you our soldier," said one banner in the sports ground.
Mugabe is in increasingly frail health and had been positioning his wife Grace as his successor, but the army chiefs acted to halt the plan and usher in Mnangagwa.
State-run media had earlier claimed that Mugabe may even attend his successor's swearing-in -- but later suggested that after he and Mnangagwa talked about the inauguration, he agreed he "needed time to rest".
Mnangagwa also "assured him and his family maximum security and welfare" for their future as private citizens and "appraised him of preparations for (Friday's) inauguration," the state-run Herald news site also reported.
Buses brought well-wishers to the 60,000-capacity stadium from the early hours of Friday.
"Come and be an eyewitness of history being made, the historic ushering in (of) a new era and better country," said a statement from the ruling ZANU-PF party calling on people to attend the inauguration.
Mnangagwa, 75, said this week that Zimbabweans were witnessing "a new and unfolding full democracy", though critics say he is a ZANU-PF hardliner who gained power in a de facto military coup.
He is known as "The Crocodile" for his ruthlessness and is accused of overseeing ethnic massacres by the army in the 1980s and the 2008 election violence when Mugabe was at risk of losing the vote.
Ahead of the inauguration, the army warned that criminals had been impersonating soldiers since the crisis to extort money from the public and called on Zimbabweans to obey the law.
Britain, the former colonial power, said it was sending Africa Minister Rory Stewart to the ceremony.
Regional heavyweight South Africa said President Jacob Zuma would not be present as he was hosting a visit by Angola's new head of state.
Zuma praised Mugabe, noting "his contribution to the liberation of the Southern African region and the decolonisation of the continent".
Mugabe had ruled since Zimbabwean independence in 1980, exercising almost total authority to crush any sign of dissent.
The majority of Zimbabweans have only known life under Mugabe -- the world's oldest head of state -- during a reign defined by brutality, rigged elections and international isolation.
His iron grip on power ended on Tuesday when his resignation letter was delivered to parliament, where MPs had convened to impeach him.
Mugabe was last seen in public on Friday and gave a defiant televised address on Sunday.
Neither he nor his wife Grace has been seen since, though they are expected to be given protection by the government.
The main opposition group, the Movement for Democratic Change, said it was "cautiously optimistic" that Mnangagwa would not be as "evil, corrupt, decadent" as Mugabe.
In the week before Mugabe resigned, military vehicles rolled down Harare's streets, army generals made a TV address in the early hours and tens of thousands of Zimbabweans demonstrated against the veteran leader.
Zimbabwe's once-promising economy collapsed under Mugabe's rule, and many hope Mnangagwa will push through reforms to bring in investment.
Unemployment is over 90 percent, and in his first speech after being announced as the next president he promised "jobs, jobs, jobs!"