Explain what Holden means when he says people are always ruining things for you

Catcher in the rye

I think this is the perfect way to summarize Chapter 12 ... Holden's timing is never on target. He tells Lillian,"people are always ruining things for you," when he has to leave the bar and can't join her. She is described as a girl with a "great" body ... but she's at the bar with a naval officer, so why would Holden join her. He's being sarcastic with Lillian and speaking to himself all at the same time.