The 38-year-old Renzi trounced rivals Gianni Cuperlo, a party apparatchik, and Giuseppe "Pippo" Civati with around 68 percent of the vote.
He was given an ovation from supporters in Florence, where he welcomed the strong turn-out of around 2.5 million voters, saying Italians "have shown that they are worth more than their leadership class".
The election of Renzi, who has said he takes inspiration from Tony Blair and Barack Obama, marks a transformation for the Democratic Party (PD) given his youth and the fact that he did not rise through the ranks of what was once Europe's largest communist party.
Despite being a product of the more moderate Christian Democrats Renzi, who has vowed to overhaul the party, insisted that his election was "not the end of the left".
"We are changing the players but we are not going over to the other side of the pitch," he said.
The youthful Renzi has also long campaigned against the middle-aged leadership of his own party and pushed for a more centrist programme, although leftist critics accuse him of being thin on concrete proposals.
Organizers had expected turnout to be lower than for the PD's primary in 2009 when Pierluigi Bersani was elected leader, given Italians' growing disenchantment with politics.
Bersani had to step aside at the height of Italy's leadership crisis following inconclusive February elections, and the party elected union boss Guglielmo Epifani as its temporary leader in May.
Little known until he challenged Bersani in last year's primary, Renzi has been riding high in the popularity sweepstakes even on the right.
He has pushed for more cuts in spending on Italy's unwieldy bureaucracy - amid widespread anger over high salaries for public officials even during a painful recession - as well as a greater focus on education.
His supporters say that if Renzi had won against Bersani for the party leadership in 2012, the PD would have won handsomely in the February general elections.
Italian Prime Minister Enrico Letta, who heads a grand coalition that includes the Democratic Party (PD), said he looked forward to a "fruitful" working relationship with the new party leader.
Letta also praised the strong turnout, saying it meant the Democratic Party was a "bastion against rising populism".
Sunday's balloting was open to non-members of the PD and to overseas voters in a bid to boost turnout, and the voting age was lowered to 16.