The thing that hurts the most is seeing him happy with another girl and wishing that it could have been you instead.
If a girl starts out all casual with a guy and she doesn't tell him that she wants a relationship, it will never become a relationship. If you give the guy the impression that casual is okay with you, that's all he'll ever want. Be straight with him from the start. If he gets scared and runs away, he wasn't right for you.