Not to mention the fact that most of the shots where the supposedly-small characters were next to a normal-sized (or larger) person, they did use actual small people for the shots (see where Bilbo's getting on the ship at Gray Havens in the next-to-last epilog of RotK for an example).

I read somewhere where John Rhys-Davies's small double had more screen time than he did.

Not that the doubles actually got dialog or face shots or anything, but still.