But Ritchie De Laet halved the deficit in the 72nd minute with a flick from a Riyad Mahrez corner that the Goal Decision System showed had just crossed the line before Jamie Vardy turned in Danny Drinkwaterʼs low cross for the equaliser 10 minutes later.
With Villa hanging on, Mahrez floated a cross into the box and Dyer, a half-time replacement for Shinji Okazaki, showed huge courage to beat outrushing goalkeeper Brad Guzan to the ball and head home.
Also on Sunday, Ryan Mason scored and was then stretchered off as Tottenham Hotspur won 1-0 at Sunderland to record their first victory of the Premier League season.
Sunderland were on top for three-quarters of the match at the Stadium of Light, but it was Spurs who prevailed when substitute Erik Lamela teed up Mason for an 82nd-minute winner.
In Saturdayʼs games, Wes Hoolahan stole the spotlight as Norwich secured their second Premier League win of the season with a 3-1 success against Bournemouth.
Alex Neilʼs side took the lead when Hoolahan set up Cameron Jeromeʼs first half opener and Hoolahan added the second with an eye-catching individual effort before debutant Matt Jarvis netted the third at Carrow Road.
Olivier Giroud came off the bench to ensure Arsenal didnʼt pay for their profligacy in a 2-0 win over Stoke at the Emirates Stadium.
Theo Walcott had put Arsenal ahead in the first half, but the Gunnersʼ fans once again spent most of the afternoon frustrated at wasted opportunities before Giroud struck late on to secure their third victory of the season.
Kelechi Iheanacho opened his Manchester City account with a last-minute winner as his side maintained their perfect start to the Premier League campaign by winning 1-0 at Crystal Palace.
Palace, who had started the day in second place, looked set to claim a point before Iheanacho finished from close range shortly after appearing as an 89th-minute substitute to secure City's fifth successive win.
Saido Berahino returned to the West Brom squad after his transfer request was rejected last month but had little impact in a 0-0 draw at home to Southampton.
Berahino, who had vowed never to play for Baggies chairman Jeremy Peace again after the club blocked a move to Tottenham Hotspur, received a mixed reception at The Hawthorns after coming on as a 55th-minute substitute for Rickie Lambert.
Watfordʼs Odion Ighalo scored the only goal of the game against Swansea to earn them their first win at the fifth attempt since promotion to the English top flight.
The Nigeria international celebrated signing a new five-year contract when sweeping home in the 59th minute after keeper Heurelho Gomesʼ long punt up field with Troy Deeneyʼs headed knockdown setting up unmarked Ighalo to finish the move in style.
Everton substitute Steven Naismith's hat-trick secured a 3-1 win against Chelsea as the stuttering Premier League champions suffered their third defeat of the season.
Naismith, who came on early in the game, scored twice in the first half and capped a fine individual display with a third late on.
Nemanja Matic netted the pick of the gameʼs goals before the break with a spectacular long-range strike, but that will be of little consolation for Chelsea manager Jose Mourinho, whose side have collected just four points from their opening five matches.
Anthony Martial scored a memorable debut goal as Manchester United beat old enemies Liverpool 3-1 at Old Trafford.
The 19-year-old French striker was quick to make his mark after coming on as a substitute for Juan Mata.
Christian Benteke had just halved Unitedʼs two-goal lead with a sensational overhead bicycle kick when Martial picked the ball up wide on the left, tip-toed between three defenders and tucked a shot into the bottom-right corner in front of the Stretford End.
Second-half goals from Daley Blind and Ander Herrera had put United in control and Martial's strike provided the perfect finale to an evening that had begun with David de Gea returning to the United goal following the collapse of his move to Real Madrid.
Dimitri Payet ensured Slaven Bilic enjoyed another memorable evening at Steve McClaren's expense as West Ham cruised to a 2-0 win over struggling Newcastle on Monday.
Payet scored once in each half to lift Bilic's Hammers to fifth place and leave Newcastle rooted to the bottom of the table without a league win this season.
Don't forget you can still join in with The Phuket News' EPL predictions competition with a chance to win great monthly prizes and an overall prize worth B90,000. Simply register at: http://www.thephuketnews.com/login-game-premier-league-competition.php?nocache=1442299624