Sounds like an abscess to me honestly. It could very well rupture on its own and drain out, you could medicate with neosporin and keep the area clean if the rat can't reach to clean itself. I would start a course of antibiotics now to help. Without a vet, no way to treat specifically for one type of bacteria or another, but a full round of antibiotics wouldn't hurt in my opinion.

I am in your boat. I love my rats too, but can't justify taking any (except one) to the vet as they are meant to be feeders.