Levein's side dominated possession for long spells but couldn't break the deadlock before Ross Millen's 73rd-minute winner for the visitors at Tynecastle.
Hearts have now failed to score in their last four matches in all competitions and will be knocked off top spot - a position they have held since the opening day of the campaign - if Celtic collect at least a point at Livingston on Sunday.
Levein, who is without strikers Steven Naismith and Uche Ikpeazu through injury, admitted his team did not show enough quality in the final third.
"There are loads of ways of losing a match but that's the most frustrating, that you have loads of possession and dominate the play," the Hearts manager said.
"We should have been better in the final third but to lose that way makes me feel really frustrated.
"We've created chances - the players who we have out injured, I think it would be easier to score goals if we had them in the team - everybody knows that.
"At the same time, we need people to step up and at times like this, put the ball in the back of the net."
In the day's other games, St Johnstone thrashed Hamilton 4-0 and Dundee drew 1-1 against St Mirren.