I think you're mixing up the causal relationship between infant mortality, population growth, and economic growth. High population growth does not necessitate low economic growth. In fact, it's economic growth that tends to determine the rate of population growth, not the other way around.
It depends. If capital accumulation is an important limiting factor in your country's growth, as it is in most of Africa, then adding more people will tend to increase the overall GDP but decrease the per capita GDP. If you're at the productivity frontier, however, and you're more limited by your ability to figure out how to use more capital (innovation). In this case there is still a small marginal return on having more capital per worker, but every worker is also a source of more innovation so its really hard to say whether adding more people (up to a certain point) will increase or decrease the per capita GDP.
Generally speaking, when a country develops to a certain point it gets what's called a demographic dividend[1]. The population has been growing rapidly in the past, so there aren't that many old people to take care of. With wealth people start having less babies, though, so there are also a relatively small number of children to take care of relative to the number of working adults. Countries in this position can, if they manage to avoid messing up, grow very rapidly.
High population growth causes slow economic growth /per capita/ (because the available income is distributed among more people).
In the book "Poor Economics" the authors show how poor parents tent to invest most resources they have in only child's education. All the other children get much less education and therefore only low paying jobs, thus remain poor. Less population growth would have a direct positive economic effect on the society's average level of education.
High population growth is also very expensive in terms of public services, you need to build more schools, hospitals, police, etc. All that investment in /quantity/ prevents you from investing it in /quality/.
Anyway once a non-industrialized population reaches a population saturation point such that more people cannot create more agricultural produce (like here in Bangladesh) there's a squeeze on per capita economic output of the country. Sure the extra labor can now be used to industrialize, in theory. But the problem is without capital to industrialize with (as is the case here) a cycle of poverty can form which gets worse.
But isn't the important question whether population growth is remaining below economic growth? And in that sense, it's a perfectly valid concern that an increase in population without an accompanying increase in economic growth is a serious problem. I live in South Africa where unemployment rates are already very significant; an increased population growth rate seems almost certain to make this situation worse.
