My cat doesnt really like me, why?

well i saved my cat when he was a baby, his mom left him. and well that was 2 years ago, he still is very skiddish and he does come up and let me hold him and pet him and stuff but then all of a sudden he gets weird and scratches me when i pet him and runs away from me and he usually hides all day sleeping, i never hit him and he is usually a very good cat. but my question is. why does he scratch me when i pet him at times and why do you think he hides away all day?