This is the first time holders Bayern have been knocked out of the cup by a non-Bundesliga club since losing to then fourth-tier Magdeburg in 2000/01.
After Bayern midfielder Marc Roca was the first player to miss with his team's sixth penalty, Kiel's former Bremen forward Fin Bartels converted the winning spot-kick to spark wild celebrations.
The upset means this is the first time European champions Bayern have been eliminated from a cup competition since head coach Hansi Flick took charge in November 2019.
Bayern were 2-1 up thanks to goals by Serge Gnabry and Leroy Sane, either side of a superb Bartels strike, before a last-gasp header by Kiel captain Hauke Wahl on 95 minutes took the game to extra time, then penalties.
The German FA gave Bayern permission to postpone their cup tie against Kiel from late December.
Bayern fielded a near-full strength side with only top-scoring striker Robert Lewandowski and centre-back Jerome Boateng rested.
Lewandowski's stand-in Gnabry gave Bayern the lead after only 14 minutes.
When Kiel goalkeeper Ioannis Gelios parried a Thomas Mueller header, it fell straight to the feet of Germany winger Gnabry who tapped into an empty net.
Having started 2021 top of the second division after five straight wins, Kiel caused nervous looks in Bayern's ranks when Bartels, 33, put them level.
The former Bremen veteran sprinted onto a long ball, avoided the challenge of Bayern defender Bouna Sarr and fired past Manuel Neuer eight minutes from half time.
However, Sane restored the lead just three minutes after the break.
Sane expertly curled his free-kick over the Kiel wall and out of reach of Gelios' glove.
As heavy snow started to fall, Kiel kept plugging away and were rewarded with the equaliser five minutes into added time.
Sloppy defending at a corner when Wahl was left free in the area and the centre-back flicked his header into the far corner of the Bayern net to force extra time.
The European champions laid siege to the Kiel goal in a bid to avoid penalties but Gelios pulled off a string of saves.
In the last 16, Kiel will host fellow second-division side Darmstadt on February 2.
Bayer Leverkusen, who beat Eintracht Frankfurt 4-1 on Tuesday in their postponed second round tie, are at Rot-Weiss Essen, the only fourth-tier club left in the competition.