I don't have a crestie, but I have other lizards. I would say that as cautious and nervous as you are...you aren't going to hurt him. Handle him for small amounts of time so you can get used to each other. Keepers that are as interested and careful as you are end up being great reptile parents. It just feels 'fragile' now because it's new. You probably felt the same way about your first snake! He'll settle down, you'll settle down, and soon ya'll will feel like old pals!
Take your time, slowly up your handling. You'll do great!