Mihajlovic was fired following Torino's 2-0 Italian Cup defeat to holders Juventus, allowing the experienced 56-year-old to take over the northern side.
“Torino Football Club announces that it has entrusted the role of coach of the first team to Mr Walter Mazzarri,” a statement on the club's website confirmed.
It said Mazzarri began work immediately and took charge of training on Thursday.
Torino had earlier confirmed Mihajlovic's departure after 18 months in charge, thanking the 48-year-old and his staff “for their commitment and passion”.
Torino slumped out of the Italian Cup to Juventus amid controversy in the Allianz Stadium midweek after goals from Douglas Costa and Mario Mandzukic.
Mihajlovic had been banished from the sidelines during the game as he protested Mandzukic's second-half goal which came after a Sami Khedira challenge on Ghana international Afriyie Acquah.
He paid for Torino's recent form which has left the club tenth in Serie A with just five wins in 19 games this season including six draws in their last eight league games.
Mazzarri arrives just in time for the January transfer window.
Torino play Bologna, just below them in Serie A on Saturday, in their last game until January 21, before a two-week break.
Mihajlovic joined Torino in the summer of 2016, having been sacked by AC Milan in April that year.
He previously managed Bologna, Catania, Fiorentina, Sampdoria and the Serbia national team.
Mazzarri was sacked by Watford last May after just one season in England despite ensuring the club's Premier League survival.
The cigar-loving Italian has plenty of experience in his home country having previously coached top flight clubs including Sampdoria, Napoli and Inter Milan.
He has reportedly signed a contract with Torino until 2020.