Yup, that's a tooth. It will be okay. it's better for the tooth to come out than to get jammed up in the gum which can become infected. If in a week or so he's gaping or there is any swelling get to a vet, but the chances of complications from this are very low.