Antonio Conte's side had been knocked out of first place by Arsenal on Saturday, but they reopened a three point lead over the Gunners thanks to Costa's superb finish in the closing stages at Stamford Bridge.
Costa's 12th league goal in his last 15 appearances gave Chelsea a ninth successive league win for the first time since 2007.
The west Londoners have scored 23 goals and conceded just two in that blistering streak, raising hopes of a Premier League title in Conte's first season in charge.
While this was far from Chelsea's best effort under the Italian, their refusal to be denied by a stubborn opponent suggests they have the required character and class to last the pace in the title race.
"I'm very happy. We faced a tough, physical team. They're difficult to break down and we showed great patience to find the right solution," Conte said.
"It wasn't easy but my players showed me great character and motivation to win."
Conte made one change from last weekend's impressive win against title rivals Manchester City as Serbia midfielder Nemanja Matic replaced Cesc Fabregas despite the Spaniard's influential performance at Eastlands.
It was 38 years since Albion had left the Bridge with three points, but Tony Pulis's side arrived in confident mood after winning three of their last four matches.
While Chelsea monopolised possession in the early stages, they couldn't break down Albion's well-drilled rearguard and it was the visitors who threatened first when Chris Brunt won a tackle with Marcos Alonso before drilling a powerful strike wide.
Despite his reputation for costly errors, David Luiz has been the model of consistency since returning to Chelsea this season, but the Brazil defender reverted to his old bad habits when he allowed Salomon Rondon to brush him off the ball, presenting the burly Albion striker with a clear path into the penalty area.
Fortunately for Luiz, Rondon was unable to hit the target and the centre-back was quick to hold up his hand in contrition.
Caught in Albion's defensive web, it was almost half an hour before Chelsea untangled themselves long enough to manage their first shot when N'Golo Kante's scuffed effort deflected off team-mate Pedro before drifting just wide.
It didn't help Chelsea's cause that Eden Hazard took a bruising challenge from Gareth McAuley that left the Belgium midfielder nursing his wound before gingerly returning to fray.
Albion goalkeeper Ben Foster was finally called into action in the 41st minute when he plunged to his right to keep out Luiz's free-kick.
With Chelsea still labouring to make the breakthrough after half-time, Luiz's free-kicks were the only source of menace for Foster, who saved another of the Brazilian's efforts after it deflected off Hazard.
Whether he was affected by the injury or not, Hazard was well below his best, leaving Chelsea short of invention in the final third and hindered by their insistence on playing sideways rather than penetrating behind the Albion defence.
Conte sent on Willian and Fabregas in a bid to spark Chelsea into life and the changes paid dividends in the 76th minute, with a helping hand from Albion defender McAuley.
When Fabregas played a hopeful ball down the right flank, McAuley was in position to snuff out any danger, but he was too casual and Costa showed why Conte dubbed him a "real warrior" this week as he took full advantage.
Muscling McAuley off the ball, Costa was suddenly clean through but there was still work to be down and the Spaniard was up to the task as he curled a fine finish into the far corner to send Chelsea back to the top.