Mice are social and so one mouse would not be as happy as two. You will need an escape proof cage, bedding (aspen seems to be the favorite, I use pine for my rats and it works too) a good food mix (store mixes tend to suck, I make my own),A water bottle, something for them to chew (their teeth always grow so buy them something suitable for pet rodents to chew, they make all kinds of things) SOmeplace to hide (I use paper towel tubes) and treats as desired.
I have to say that mice stink.....very badly. I used to have to clean mice cages at my old job and everything was a sticky, grimey, stinky mess....DAILY!
If you can afford the space and extra couple bucks, rats are far better to breed both for their less stink like qualities, and they seem to breed better compared to mice in appropriate conditons.