Many birds are drab, at least to UV-deficient vision; they use songs or bowers to attract mates. And you might better ask why humans are so drab: chimps, gorillas, monkeys, mice, cats, bears. Of course, some of these do have sex-specific colors, but not most. Scent, nocturnality, not getting eaten.

We know hair/fur can do red and yellow. Blue fur exists (blue fox, russian blue house cat) though they're blue-grey; I don't know if any bright blue exists, or green (short of sloths and polar bears letting algae grow on them). Don't know if there's any hard limit. Red's better for getting attention and brown's usually better for camouflage.