The former cruiserweight world champion, who dismantled the holder in London last year, faced an improved Joshua but outboxed him once again in just his fourth heavyweight bout.
Usyk, now unbeaten at 20-0, immediately called out Fury, who announced his latest retirement earlier this month but has signalled his willingness to resume his career.
Victory over the 6ft 9ins (2.06m) Fury would unite all four heavyweight belts and constitute a crowning glory for the sublime Ukrainian, a former outstanding amateur and Olympic heavyweight champion.
“I’m sure that Tyson Fury is not retired yet,” Usyk told the crowd. “I’m convinced he wants to fight me. I want to fight him. If I’m not fighting Tyson Fury I’m not fighting at all.”
The fight was watched on free-to-air TV by millions of Ukrainians living under the Russian invasion. Usyk signed up to fight for his country before accepting the rematch.
“I give this victory to my country, to my family, to my team and to all the military who are defending the country,” he told the 12,000-seat King Abdullah Sports City Arena in Jeddah.
An angered Joshua hurled down Usyk’s belts and peppered his post-fight speech with expletives despite the presence of Crown Prince Mohammed bin Salman, de facto ruler of the deeply religious country.
“I just spoke from my heart. It has been so tough,” Joshua said afterwards, adding: “It takes real strength and tonight there’s a little crack in the armour.”
The judges scored it 113-115, 115-113 and 116-112 for Usyk as Joshua narrowed the gap but was still found wanting.
“It was an incredible performance - he’s just too good,” said Joshua’s promoter Eddie Hearn.
“You saw the reaction from AJ, that was from a human who wanted to win so badly, with so much pressure on his shoulders. I think he just exploded because he lost and he was devastated.”
Joshua, the 6ft 6ins (1.98m) two-time world champion, is left staring at an impasse in his career after his third defeat in his 12th straight title fight left him with figures of 24-3-0.
Joshua, accused of hesitancy in their first fight in London, was the early aggressor by round two as he repeatedly landed his big right hand.
A heavy right hook stung Usyk in round three and Joshua was continuing to find the jab as the Ukrainian’s busy footwork opened up gaps.
A low blow from Joshua left Usyk wincing in the fifth and another backed him onto the ropes in the sixth but the tricky southpaw came firing back with some lightning combinations.
A flurry to the body put Usyk back on the ropes in the eighth and Joshua had the Ukrainian hanging on in the ninth after a right-left hook combination as the fight came alive.
But Usyk came storming back after the bell with a succession of clean shots as Joshua took some serious punishment despite landing a massive right of his own.
The mesmerising Usyk increased his control in the closing stages, slipping punches and landing quick combinations as Joshua searched in vain for the knock-down.
Earlier on the card, Somali-born Briton Ramla Ali knocked out the Dominican Republic’s Crystal Garcia Nova in round one in the very first women’s professional boxing match in Saudi Arabia.