His emphatic performance was Rosberg's first pole after 110 previous attempts, his lap of 1min 35.121sec well clear of Hamilton's 1:35.626. The Briton will be seventh on the grid after a penalty for changing his gearbox.
That means the Mercedes cars will line up on the front of the grid together -- the first all-Mercedes front row since the 1955 Italian Grand Prix at Monza.
There was a major shock with Sebastian Vettel, the two-time reigning world champion, failing to make the third qualifying stage with his surprisingly lacklustre time of 1:36.031.
"It went really well and it's a very special feeling," said Rosberg, 26, who has never been on pole and never won a race since making his Formula One debut in 2006.
"It's fantastic, it all came together. It's perfect till now so I am very happy. It shows that we are moving forward.
"We are not quite where we want to be in the races, but we are progressing.
"The first two races of the season have not gone too well. We've not been as good in races as qualifying lately."
The German was seventh on the grid in the first two races of the season, in Australia and Malaysia, before sliding down the field to end the two races 12th and 13th.
Hamilton, who has said he was relishing starting further back as a result of his five-place grid penalty, admitted he would need to do something special to catch the two Mercedes at a dry but cool Shanghai International Circuit.
"We are in a good position," said the Briton and 2008 world champion, who took pole position in Melbourne and Sepang and was left bitterly disappointed when he ended both grands prix in third.
"The car feels good, but it was tough, for sure," Hamilton added, congratulating his friend Rosberg, whom Hamilton said he had "grown up with" on the racing circuit.
Schumacher, who came out of retirement in 2010, was full of praise for his younger team-mate.
"I said 'well done' and it's a bit of a surprise," said the seven-time world champion. "But for me, there's no reason to be unhappy. I'm happy for him to get his first pole. Congratulations.
"We will try to keep our positions tomorrow."
The Sauber of Kamui Kobayashi was fourth fastest and Lotus's Kimi Raikkonen was fifth-quickest, followed by Hamilton's team-mate Jenson Button and the Red Bull of Mark Webber.
Felipe Massa was the other notable casualty of the second qualifying round in his clunky Ferrari, while team-mate and current championship leader Fernando Alonso was only ninth-fastest.