Don't know if it's true, but I have heard that cats don't like being looked at.Oh but they always seem to go to the people that don't like them!
So people who like them look at them and people who don't try to avoid looking at them, so the cat goes to sit on the people who don't like them !