The decision followed two days of talks between Grant, 53, and the club's owner and chief executive Peter Kenyon.
The Israeli coach's position had been subject to intense speculation after Chelsea finished runners-up in the Premier League and Champions League.
Chelsea said in a statement on their website that the search for a new coach would now begin and that the club will make no further comment.
The statement said: "Chelsea Football Club can confirm that Avram Grant has had his contract as manager terminated today. This follows meetings over the last two days.
"Everybody at Chelsea FC would like to thank Avram for his contribution since taking over as manager last September.
"We will now be concentrating all our efforts on identifying a new manager for Chelsea and there will be no further comment until that appointment is made."
Grant, a close friend of Chelsea owner Roman Abramovich, took over on 20 September 2007 after Mourinho was sacked.
Despite losing just two Premier League games while in charge, Grant attracted criticism from fans who claimed his team lacked style.
He was also criticised for losing February's League Cup final against Tottenham and the Champions League to Manchester United on penalties.
Grant joined Chelsea last summer from Portsmouth as director of football, a decision which apparently did not please Mourinho.
Mourinho was later dismissed after their 1-1 draw with Rosenborg at Stamford Bridge in their opening Champions League game.
Grant, who had never managed a club in England, was asked to take over and on 13 December 2007 signed a four-year contract.
Both Kenyon and chairman Bruce Buck paved the way for change by insisting that Chelsea's performance this season was simply "not good enough".
Buck said: "We have very high expectations at Chelsea and a couple of second place finishes is just not good enough for us. "So although we never would have thought in September when Jose Mourinho left that we would be able to make it into a Champions League final as we did - and that is fantastic - Chelsea are here to win trophies.
"So although it was an excellent season, we are still disappointed."