The 14-page manuscript was written in 1892 by Pierre de Coubertin, a French aristocrat and athletics enthusiast known as the "father" of the modern Olympics.
It achieved nearly nine times its pre-sale estimate of $1 million following 12 minutes of intense bidding at a sale in New York, Sotheby's said in a statement.
"Today's record result stands as a testament to Pierre de Coubertin's vision of more than a century ago, and the reverence with which the Olympic games are still held," said auctioneer Selby Kiffer.
Coubertin outlined his vision for reviving the ancient Olympic Games as a new international competition in the manifesto.
He argued that athletic endeavor had benefits for individuals and society.
The manuscript is the only known copy of the speech which Coubertin delivered in Paris in 1892.
Coubertin founded the International Olympic Committee two years later and the first modern Games took place in 1896 in Athens.
The previous world auction record for sports memorabilia was the $5.6 million paid in June for a jersey worn by baseball legend Babe Ruth, Sotheby's said.