Leicester had the ball in the Palace net after 27 minutes but Jamie Vardy's effort was disallowed.
Jordan Ayew, Schlupp feature as Crystal Palace pip Leicester City
Leicester were made to pay as Milivojevic put Palace into the lead after 39 minutes with a stunning strike.
Recommended articles
The former England striker made the most of a poor touch from Crystal Palace goalkeeper Vicente Guaita on his Premier League debut and slotted the ball into the back of the net but he was adjudged to have fouled the keeper.
The Serbian midfielder picked up the ball 25 metres from goal and fired a powerful shot which flew past Kasper Schmeichel into the top left-hand corner.
Jordan Ayew played 79 minutes of the game after he was replaced by Alexander Sorloth whiles Jeffrey Schlupp started the match from the bench, replacing Cheikhou Kouyate in the 65th minute.
JOIN OUR PULSE COMMUNITY!
Eyewitness? Submit your stories now via social or:
Email: eyewitness@pulse.com.gh