Ok, did a lot of googling and found a few things. When you guys said "fertilizer" I immediately started thinking of the little white, and irregularly shaped pelleted ... things... that is found in most soil. (Google tells me this is "Perlite") And obviously, white isn't yellow so I thought you all were insane.
But I did find, on two forums, members asking about insect "eggs" in their soil. They got responses saying that they are NOT eggs, but are "time released fertilizer pellets". One article stated that gardeners are usually surprised to find them because they "show up on their own time frame". Which explains why I didn't see them when I first potted, and now here they are.
So, I guess the final verdict is that they are fertilizer pellets.
Thanks for making me feel like a moron, guys.
At least I can rest assured that I won't have a hundred baby lizards running around my apartment.