The Germans were the reigning champions when they crashed out in the first round four years ago in Russia and they were eliminated in repeat fashion in Qatar despite beating Costa Rica 4-2.
Germany took the lead after Serge Gnabry’s opener and despite two goals from Kai Havertz and a late strike from Niclas Fuellkrug, the four-time champions finished third in Group E behind Spain on goal difference.
“It is unbelievably bitter for us because our result would have been enough,” forward Thomas Mueller said.
“It’s a feeling of powerlessness.”
Japan surprisingly advanced in first place with six points after coming from behind to topple Spain.
Alvaro Morata headed Spain in front early on, but Japan dramatically hit back at the start of the second half.
Ritsu Doan and Ao Tanaka’s goals saw them mount a sensational comeback, just as they did in a shock opening win over Germany.
Tanaka’s goal was highly controversial after the officials relied on VAR to decide Kaoru Mitoma had kept the ball in play by the finest of margins to set him up.
Japan will play Croatia, the beaten finalists in 2018, in the next round while Spain go on to face Morocco.
It is the fourth time Japan have reached the last 16 but they have never progressed beyond that stage.
“The players are showing us a different and new view of being able to fight on the world stage,” said Japan coach Hajime Moriyasu.
“I would like to set a new record of the last eight or better.”
Germany’s top-selling daily Bild offered a scathing assessment of the team’s latest failure.
The Germans were also knocked out in the last 16 at Euro 2020. “How embarrassing! We are out,” read its headline. “Three tournaments in a row in which we didn’t get anything done.
“The football world used to tremble before us. We were praised as a ‘tournament team’. Now Germany is just a football dwarf.”
Belgium bundled out
Second-ranked Belgium were also sent packing after a goalless draw with Croatia.
Belgium’s exit was painful, with their players slumping to the turf after substitute Romelu Lukaku failed to capitalise on a string of gilt-edged chances.
Instead it was Luka Modric’s Croatia who celebrated, qualifying for the knockout stage as the second-placed team in Group F, with five points.
Morocco beat Canada 2-1 to top the group with seven points, reaching the last 16 for the first time in 36 years.
The Belgians, knowing victory would take them through, poured forward in the second half in frantic search of a winner but fell agonisingly short.
Lukaku smashed the ball against the post around the hour mark and squandered other chances to the disbelief of Belgium’s fans.
With just moments remaining he seemed certain to score from close range but could only chest the ball into the arms of Croatia goalkeeper Dominik Livakovic.
The 2018 semi-finalists head home after an embarrassing campaign in which they scored just one goal.
Belgium have had a miserable stay in Doha with rumours of in-fighting and a number of the country’s “Golden Generation” including Kevin De Bruyne, Eden Hazard and Thibaut Courtois may have played on the biggest stage for the final time.
Martinez steps down
In the aftermath of their exit, Belgium coach Roberto Martinez announced he was leaving his job.
“That was my last game with the national team, I can’t carry on,” Martinez said. “It’s the time for me to accept that this is the last game.”
Morocco made it through to the knockout stage for the first time since 1986, thanks to first-half goals from Chelsea star Hakim Ziyech and Youssef En-Nesyri.
Canada, already eliminated before the final round of group games, cut the deficit through an own goal but Morocco held on to win.
“It wasn’t easy, but when you have an opportunity to make history, you remember it,” said coach Walid Regragui.
“We’ve made lots of people happy this evening, but let’s not stop here.”