BBC has reported that Karemangingo was shot about 50m (164ft) from his residence in the Liberdade neighbourhood on Monday afternoon.
He was reportedly confronted by gunmen on three vehicles who intercepted his car before they fired a hail of bullets at him.
The killers are so far not known, much less the reasons for the crime that shocked the neighbourhood and the Rwandan community in Mozambique.
The police were at the scene for forensic work and the body was taken to a provincial hospital.
The Association of Rwandan Refugees in Mozambique said Karemangingo was a refugee doing pharmaceutical business.
In 2019, another Rwandan, Louis Baziga who served as the chairman of Rwanda’s diaspora in Mozambique was shot dead in a similar style in a Maputo suburb after gunmen intercepted his car.
At the time, Karemangingo who is an ex-soldier of the former government (Ex-FAR) was identified among three suspects behind the murder of Baziga.
The remaining suspects include Diomède Tuganeyezu, a pastor, businessman and ex-soldier as well as businessman Benjamin Ndagijimana.
In July 2016, they were taken to court on the accusation of attempts to kill Baziga.
The prosecutor Octávio Zilo accused the three men of attempting to kill Baziga to become proprietors of Pentecostal Church in Revival Mozambique which they founded along with the deceased.
In March 2018, another Rwandan called Vital Hitimana running a business in Villa Olempique town was shot dead with rapid-fire and admitted to hospital over sustained injuries.
Mozambique accommodates approximately 3000 Rwandans, majority of whom living in Maputo City.
It is said that some of them work with rebel groups hostile to the Government of Rwanda including RNC and hunt their colleagues accusing them of spying for Rwanda.
It is also reported that Karemangingo had alerted security officials of suspected plans to assassinate him.