I'm not much of a small animal person but my mom has always kept rabbits (She had 30 baby rabbits at one point) and they seem to get more skiddish then my sisters pet guinea pig.

I'd so go with what your heart wants. It sounds like a guinea pig would fit the Bill for you.