Divock Origi scored for the fourth consecutive match after Darren Randolph's blunder to save a point for frail Liverpool, who sit third, six points behind leaders Chelsea.
Liverpool goalkeeper Loris Karius last week gifted Bournemouth a winner and Klopp's side -- so good going forward and top of the table last month -- are being regularly undermined by their defensive failings.
Karius' positioning on Sunday for West Ham midfielder Dimitri Payet's equaliser, which cancelled out Adam Lallana's opener, was questionable.
Michail Antonio then capitalised on a mistake by Joel Matip to put the visitors 2-1 up before Origi's fortuitous 48th-minute leveller.
Klopp was hardly convincing in his defence of the German stopper Karius, 23, saying merely that he had "no idea" about whether his summer signing was at fault for the Payet free-kick strike because he had only seen it once.
But the Liverpool boss was clearly not happy with the way Liverpool were opened up by the Hammers at Anfield.
"Nearly everybody was involved in our offensive situations and we had two centre-backs and sometimes Hendo (Jordan Henderson) behind the ball, and nearly everybody else the other side," he said.
"We had a problem with second balls but not too often. The goals (we conceded) weren't in situations like that but we let the game go a little bit away in these moments, we were not completely in control like we could have been."
Klopp acknowledged that on current form it will be difficult to catch red-hot Chelsea, who have won their last nine Premier League matches and now look the team to beat.
"It?s difficult to stay in the race with Chelsea when they win everything. This is the moment to collect as many points as possible," he said, calling for his side to remain calmer in future.
"The way to get a real top-flight team is for them to make the right decisions, and stay cool."
West Ham manager Slaven Bilic has problems of his own.
He was pleased with the point, which saw his side climb out of the relegation zone, but they have not won in six matches and the pressure is on.
"After we lost our last game 5-1 against Arsenal and after a start here that couldn?t be worse, we expect the confidence to drop," he said, reflecting on Lallana's fifth-minute opener.
"But we showed a great reaction after the goal. Particularly in the first half, I was pleased. I was pleased with the character over the 90 minutes, but in the first half, we showed we could hurt them."