Was it I Will Fear No Evil where the protagonist stopped the action to complain about restaurant chairs, and explain to the reader that if you kicked up a fuss about every single thing people would see how right you were? I read it as a young teen, and that, and not any of the squirmy sex stuff, was the end for Heinlein and me. Even at the time I could tell that this was somebody who thought that his food tasted funny without the familiar tang of waiter spittle.