I'm so sorry for your loss, it sounds like you went above and beyond for the little guy.
As for the mouse, if it might be comforting to take care of them I'd say keep them around and get them a little habitat, but if it would just add stress during a difficult time maybe someone else would take them in from craigslist or something?