You may try NOT sitting there staring at him the whole time the mouse is in there. Could be more shy than you know. It is ok to let a mouse in the enclosure for 5 - 10 minutes, then check up on things.

I, myself, wouldn't worry about it too much though. Right now they go on and off feed, just how it is during this time. Mine will take one week and not the next then refuse for a few weeks then take a meal for two weeks straight, blah, blah, blah lol I do fee live rats but I am breeding them so it makes it easier when they refuse, I just put them back in the grow out tub.