I don't have a problem with it per se; I have a problem with what it does to storytelling.

As for birthrates falling... wealth is acting as a kind of pesticide (ignore the insult implied there). Pesticides ultimately cause resistance. Wealth, over a long enough period, will select for people who use that wealth to reproduce more. This selection will be both cultural (think Amish) and genetic (think multiple births, earlier onset of menarch, and predisposition to behaviors that lead to pregnancy.)