"My experience of government finishes here," said a downcast Renzi after accepting a defeat of almost 60-40 percent over his constitutional reform bid.
The result, coming on the coattails of Brexit and Donald Trump's US election win, has plunged Italy into a period of political uncertainty and cast a shadow over the short- and long-term future of the eurozone's third-largest economy.
After a first meeting with President Sergio Mattarella to discuss his departure, Renzi posted a link on his Facebook page defending his record since he came to power in February 2014.
"1,000 difficult but wonderful days. Thanks to everyone. Viva l'Italia," he wrote.
Renzi, 41, was expected to formally tender his resignation to Mattarella after a final cabinet meeting set for 6.30 pm (1730 GMT).
Mattarella will then be charged with brokering the appointment of a new government or, if he is unable to do that, ordering early elections.
Traders not panicking yet
Initial market reaction was subdued. The euro briefly sank to a 20-month low as investors fretted that political instability could scupper efforts to resolve a long-running banking crisis, and over the possibility of an election that could see anti-EU parties challenge for power.
Italy's FTSE MIB stock index fell 2.0 percent at the opening before clawing back some ground, underperforming other European markets. Italian bond yields rose slightly, having already edged up prior to Sunday's vote.
Traders were reassured in part by the result of Europe's other crucial vote this weekend, which saw Austria reject a far-right candidate for president.
"The next steps are far from clear for Italy and traders are not panicking yet", said Craig Erlam, senior market analyst at Oanda.
Some analysts said the referendum could come to be seen as a landmark moment.
Holger Schmieding, at the Berenberg private bank, said the risk that Italy could choose to leave the euro, while still remote, had increased.
Capital Economics said: "Italy has taken the first step along a path that could lead it out of the eurozone."
'A people's victory'
Populists across Europe rejoiced at Renzi's downfall, with the founder of Italy's own anti-establishment Five Star movement Beppe Grillo calling for an election "within a week".
Grillo said a snap election should be held on the basis of a recently adopted electoral law designed to ensure the leading party has a majority in the Chamber of Deputies, despite having previously called for this to be changed.
Observers varied widely in their interpretation of the vote.
Britian's eurosceptic Nigel Farage, who spearheaded the "Brexit" campaign, said it looked "more about the euro than constitutional change".
But former Bank of Italy economist Lorenzo Codgno insisted: "The outcome of the referendum is much more complex and nuanced than 'just another wave of protest across the globe'."
Giovanni Orsina, Professor of Politics at Rome's Luiss university, said four out of five voters had cast their vote politically rather than on the merits of the reform.
"The vote has broad similarities with the Brexit and Trump phenomenona," he said. "The electorate voted against the establishment, against Brussels. They didn't get into the subtleties."
Early election unlikely
Poll data showed the No vote was strongest in areas with high unemployment, in the relatively poor south of the country and amongst youth, pointing to a correlation with levels of disaffection.
Most analysts see immediate elections as unlikely, partly because all the main political parties have already begun discussions on revising the country's electoral laws.
As things stand the two houses of parliament would be elected by two different systems, which is seen as a recipe for a post-vote stalemate nobody wants.
The most probable scenario is a caretaker administration dominated by Renzi's Democratic Party taking over to complete the electoral reform before an election that has to take place by March 2018.
The new administration's most pressing priority will be finalizing the country's 2017 budget, which the European Commission has threatened to reject as too expansionary given Italy's debt levels.
Finance Minister Pier Carlo Padoan is the favourite to succeed Renzi as prime minister and the outgoing leader may stay on as head of his party - which would leave him well-placed for a potential comeback to frontline politics at the next election, whenever it is.
Padoan cancelled a trip to Brussels on Monday.