I think he probably means GDP growth as a proxy for aggregate wealth growth (assuming the ratio of GDP to aggregate wealth is roughly constant in time).

What he really is concerned with is the amount of productive capital and the value of infrastructure and land.

Part of the point is that productivity increases lead to a decrease in the demand for labour unless the amount of capital chasing after labour increases at least at the same rate.

A society committed to the notion that government is always bad will have bad government. And it doesn't have to be that way. — Paul Krugman