The PM also pledged to help initiate reforms in the United Nations Security Council.
Japan has pledged Sh1 trillion ($10 billion) for infrastructure development in Africa.
Japanese Prime Minister Shinzo Abe said the fund will be used to build infrastructure such as hospitals, roads and energy generation facilities.
When combined with the private sector’s investments, the amount could rise to $30 billion, he said.
“Japan is a country that hopes to resolve the problems affecting Africa together with Africa and will not let up,” he said.
Mr Abe said another $500 million will be channelled through international agencies to support health initiatives in the continent and combat epidemics.
The PM also pledged to initiate reforms in the United Nations Security Council and have an African country get a permanent seat in the body by 2023.
“UN Security Council is the most powerful body of the UN but there is no African permanent member in it...Japan is also eyeing a similar seat,” he said.
Mr Abe was speaking at the opening ceremony of the Tokyo International Conference on African Development (Ticad) in Nairobi on Saturday.
Speaking at the same function, President Uhuru Kenyatta urged leaders to push for fair global trade.
He said there is a worrying trend for the developed world to turn inwards, even after benefiting from open trade before.
“Indeed, if we look back, the wealthiest countries today with very few exceptions, got rich by trading with others. The critical ingredient of prosperity in the last century has been free and fair trade, infrastructure integration of regions, educated citizens who enjoy economic liberty and responsible governments.
“Yet there is now a turn among many countries, to turn towards more isolationist or grossly unfair positions on trade. I believe Ticad is a positive response to this trend,” he said.
The Ticad meeting, the sixth since 1993 but the first to be held in Africa, is meant to discuss development issues and the support of Japan for Africa.