The Canada-born former Germany international has been on an extended sabbatical from ice hockey, spending five years in English football's top flight with Southampton after joining the club in 2014.
However, the 59-year-old was relieved of his duties at Southampton in April following a management shake-up. That freed him to return to the NHL, where he last coached the Edmonton Oilers from 2012-2013.
"Throughout his career, Ralph has shown the ability to adapt to a variety of high-pressure environments while leading some of the world's elite players," Sabres general manager Jason Botterill said in a statement.
"His strong communication skills, leadership and diverse background make him a uniquely qualified candidate to lead our team going forward."
Krueger, who also Switzerland's national ice hockey team at the 2010 Winter Olympics, will be handed the task of attempting to snap nearly a decade of failure in Buffalo.
The Sabres have not made the playoffs since 2011, and have not reached the final of the Stanley Cup since their lone appearance at the end of the 1998-1999 season.