The boat, believed to have been carrying over 600 migrants including women and children, ran into difficulty about 15 nautical miles off Libya and sent out a distress call, which was picked up by the coastguard in Sicily.
Two vessels - the Doctors Without Borders (MSF) ship Dignity One and Irish patrol vessel LE Niamh - were immediately dispatched to the scene, but the stricken fishing boat capsized after the migrants moved to one side in the hope of being rescued, the coastguard said.
Coastguard spokesman Filippo Marini said around 400 people had been rescued from the water while 25 bodies had been recovered.
"The boat overturned and sank quickly because it was made of metal," Federico Fossi, spokesman in Italy for the UNHCR, told AFP.
The agency's chief spokeswoman Melissa Fleming said on Twitter that "100 (migrants) were in the hull" of the fishing vessel when it capsized.
Seven ships and two helicopters were helping search for survivors.
More than 2,000 people have already died trying to cross the Mediterranean to Europe this year, the International Organization for Migration (IOM) said on Tuesday.
Wednesday's disaster could be the worst since 800 migrants were feared drowned off Libya in April.
Working flat out
Ireland's Defence Minister Simon Coveney confirmed that the Niamh was diverted to the scene at 7:00 am (0900 GMT) and warned "the loss of life is likely to be significant".
"The crew of the Niamh are working flat out with their counterparts to rescue as many as possible," he said in a statement.
Nawal Soufi, an Arabic-speaking Italian based in Sicily who is often contacted by migrants in distress, told AFP she had received a call earlier on Wednesday about a boat in trouble with more than 600 people on board.
She said she was called by someone onboard who said the fishing boat, carrying men, women and children, was in trouble because its engine room was flooded.
Marini confirmed it was probably the same boat.
The latest shipwreck was set to shift the focus in the immigration debate back to the struggles of those fleeing war zones, after weeks of complaints from France and Britain over the disruption caused by asylum seekers at the Channel Tunnel.
Also on Wednesday, 94 people on board a rubber boat were rescued in a separate operation, coastguards told AFP.
European Commission chief Jean-Claude Juncker told AFP on Wednesday that EU governments have a duty to help the flood of migrants arriving in Europe and not cave in to "populist" demands to turn them back.
He said he was disappointed that EU ministers had failed late last month to agree on how to distribute a total of 40,000 mostly Syrian and Eritrean migrants from overstretched Italy and Greece.
They agreed to start relocating a little over 32,000 of the arrivals among the EU's 28 members in October, falling around 8,000 short of the target agreed by EU leaders at a summit in June.
"If we don't get there on a voluntary basis, we will have to reconsider the Commission's proposals," the former premier of Luxembourg said.
Nearly all of the people crossing the Mediterranean so far this year, often in rickety boats and at the mercy of human traffickers, have landed either in Italy (97,000) or Greece (90,500), according to the IOM.
The so-called central Mediterranean route proved by far the deadliest, with just over 1,930 people dying as they tried to cross from Libya to Italy, while only about 60 died trying to reach Greece.
The eastern route is shorter and IOM said that traffickers taking people to Italy tended to use more unseaworthy vessels.