The continent was "saddened by the death of a revolutionary," said Democratic Republic of Congo’s Felix Tshisekedi.
Mr Magufuli died last week following heart complications at the age of 61.
His successor President Samia Suluhu Hassan remembered him as a champion of the poor and a religious man.
"He wasn’t just our leader but also a guardian and parent to many... and an honest man," President Samia said.
Nicknamed the bulldozer, Mr Magufuli was popular with many Tanzanians who approved of his no-nonsense governance style.
Critics, however, accused him of being an autocrat and of clamping down on dissent.
He also downplayed the effects of coronavirus and stopped the publication of the country’s case numbers and deaths. Opposition politicians say that Mr Magufuli died from Covid-19, but this has not been confirmed.
Tanzanian leaders attending the funeral and the majority of the thousands of people at the stadium in Dodoma did not wear face masks or observe social distancing - health measures that the late president often mocked.
However, visiting leaders and other delegations did wear masks.
Over the weekend tens of thousands of people in the country’s largest city, Dar es Salaam, flooded roads to pay their respects.
On Sunday there was a stampede at a stadium where Mr Magufuli’s coffin was on display - the number of casualties is not yet clear.
He will be buried in his north-western hometown of Chato on Friday.
What did the African leaders say?
Mozambique President Filipe Nyusi said the former Tanzanian president "will stay in the hearts" of many.
In his tribute, Malawi’s President Lazarus Chakwera referred to Mr Magufuli as "Africa’s finest son" whose "life of service" would be remembered.
South Africa’s President Cyril Ramaphosa described him as a "true pan-Africanist" who was unapologetic about being an African.
Mr Ramaphosa remembered the former president for being a "warrior" against corruption, and who worked for his people.
He also said the Tanzanian president was a champion for African culture and traditions, especially the use of Swahili, East Africa’s lingua franca, throughout the continent.
"Swahili has been introduced in South African schools as a honour to the late President John Magufuli who insisted on its use," Mr Ramaphosa said.
This was echoed by Botswana’s President Mokgweetsi Masese, who said that Mr Magufuli was a "great teacher", like Tanzania’s founding President Julius Nyerere.
"Even in Botswana he expected us to speak Swahili… We too have introduced Swahili in our curricula."
In his tribute, Kenya’s President Uhuru Kenyatta also addressed Mr Magufuli’s successor, saying: "To you my sister and now my counterpart, the road has been shown to you by our brother President Magufuli."
"The Democratic Republic of Congo and the African Union together are saddened by the death of the revolutionary Dr John Magufuli. We’re praying for Tanzanians. The DR Congo is with you in this difficult period," said President Tshisekedi, who is the current chairman of the African Union.