I would think since they live in the mountains in South America, as long as he has shelter, he'd probably be okay. I know a couple people who keep outdoor cavy and I don't remember them having issues.
It would really depend on how cold it gets there with you though.