Italy dominated Berti Vogts' visitors throughout a one-sided encounter in Palermo on Friday but had to rely on a late header from Giorgio Chiellini to claim all three points from their second group match.
Indeed Chiellini's name was the only one on the scoresheet, the Juventus defender having opened the scoring on 44 minutes before putting into his own net from an Azerbaijan corner with 14 minutes left.
Faced with dropping two precious points, Conte later deflected criticism having seen strikers Simone Zaza and Ciro Immobile, midfielder Claudio Marchisio and defender Leonardo Bonucci all come close to scoring in a busy first half.
The former Juve handler insisted Italy, who were man-marked for long periods of the game, more than deserved the win.
"Overall, it was a bit of a scant victory but we more than deserved the win," said Conte, who succeeded Cesare Prandelli in the summer following the Azzurri's disastrous first round exit from the World Cup.
"They had almost 10 men man-marking. They scored without having any real shots on goal and we came back in fantastic fashion."
The Stadio Barbera had yet to see a win this season amid promoted Palermo's poor start in Serie A and Italy missed several great opportunities before breaking the deadlock on the cusp of half-time.
Juventus midfieler Andrea Pirlo, who made his return from a long injury lay-off in time for a 3-2 league win over Roma last week, swung in a corner that goalkeeper Karam Agayev misjudged, allowing Chiellini to power home a header from six yards.
Azerbaijan had lost 2-1 to Bulgaria last month and were content to defend en masse and strike on the counter whenever possible.
Still, Italy laboured to build on their lead. Early in the second half Sassuolo striker Simone Zaza notably went close with a stunning overhead kick from just inside the area and as legs tired Conte made a series of decisive changes.
Pirlo was replaced by Alberto Aquilani with 17 minutes remaining but disaster struck on 76 minutes when Chiellini turned the ball into his own net after Gianluigi Buffon failed to get his hands to a corner that caused mayhem in the box.
Further changes saw midfielder Matteo Darmian replaced by Antonio Candreva and Sebastian Giovinco come on in place of Alessandro Florenzi and minutes later Giovinco found the breakthrough when he delivered for Chiellini to flick a glancing header past the rooted Agayev.
Giovinco should have made it 3-1 but saw his looping drive from Marchisio's through ball smack off the crossbar.
Giovinco later indicated that his lack of starting appearances for Juventus in recent seasons meant he would be "considering" his options before his contract expires in June.
Conte, meanwhile, defended his decision to send the diminutive striker on late in the game: "Giovinco was supposed to replace Zaza, then they (Azerbaijan) scored.
"Sebastian knows how to beat players one-on-one, he adds something extra when he comes off the bench and he has a keen eye for goal.
"I don't gift appearances to anyone. Giovinco played because he deserved to."
Malta coach Pietro Ghedin, once part of Italy's managerial staff under Cesare Maldini, Dino Zoff and Giovanni Trapattoni, said last week he is brimming with condfidence ahead of Monday's clash at the Ta'Qali Stadium.
Malta on Friday suffered a 3-0 home defeat to Norway.