Six-time champion Federer was beaten 7-5, 7-5 in the evening match on the opening day after Djokovic had swatted aside debutant Matteo Berrettini in the earlier Group Bjorn Borg contest in London.
Federer and Djokovic will meet later in the week in a repeat of this year's epic Wimbledon final, with two players advancing from the four-man group.
The Swiss 20-time Grand Slam winner recovered quickly from being broken in the first game of the match but the Austrian fifth seed repeated the dose in the 11th game and took the opening set 7-5.
Federer, uncharacteristically wayward, squandered an early chance to break in the second set and the games went with serve until the 11th game again, when Thiem broke to love after the third seed dumped a forehand into the net.
The Austrian stuttered while serving for the match, forced to save two break points, but he held his nerve to take the set 7-5 and seal a deserved victory on his second match point.
"I thought Dominic saved himself real well when he had to," said Federer. "So I thought he played a tough first-round performance today, and my game was probably just not good enough, and the start didn't help."
"It's a normal tournament from here on forward," added the 38-year-old, who may now have to beat Djokovic to progress.
"Not allowed to lose anymore for me. That's how it is every week of the year for the last 20 years, so from that standpoint there is nothing new there."
Thiem came into the match with a 4-2 winning record against his illustrious opponent but the result is still a surprise. The only time Federer has failed to progress to the knockout stages was in 2008.
"We had a good and close match, and it's always special to beat Roger, who is a legend," said Thiem, who impressed with his clean hitting.
"It's amazing to play against probably the best player of all time, but luckily I have a pretty good record against him."
Earlier, Djokovic began his quest for a record-equalling sixth ATP Finals title and sixth year-end number one spot a 6-2, 6-1 romp against Berrettini.
The Serbian was last week nudged off the top position in the rankings by Rafael Nadal, who is also in the eight-man field in London.
- Djokovic eyes No. 1 -
Djokovic must reach the final at the O2 Arena to stand any chance of dislodging Nadal but there are major questions over the Spaniard's fitness.
If the Serbian does claim the year-end number one spot, he would match the record of Pete Sampras and victory at the end-of-season event would equal Federer's mark.
The second seed saw off the challenge of Berrettini in just 64 minutes, converting five of seven break points and at one stage going on an eight-game streak.
"It feels great to be back," said the Wimbledon champion. "Earlier this year I had a great stay in London, and over the years I've played well in this arena."
Nadal, who has never won the event, is bidding to clinch the year-end number one spot in the ATP rankings for a fifth time.
The tournament, featuring the year's best eight players, is contested in a round-robin format with the best four players and doubles teams reaching the knockout semi-finals stage.
Nadal, in Group Andre Agassi, begins his campaign on Monday against defending champion Alexander Zverev.