It'll definitely get better with time, don't get discouraged! The important thing is to give him consistently positive experiences. It'll take time and lots of patience from both of you, but eventually he'll put it together that being handled doesn't end with him getting hurt and he'll come to recognize you as being safe. Just don't expect miracles overnight.