Rancadore, known as "The Professor", was arrested after evading Italian authorities for 20 years. The 65-year-old had been living quietly in the London suburbs.
Italian prosecutors accuse Rancadore of fleeing Italy as he faced trial over his alleged Cosa Nostra connections.
Senior District Judge Howard Riddle said he had based his decision on a ruling in a similar case by the Administrative Court, which has jurisdiction over lower courts in England.
The ruling represents a turnaround for the judge, who said his original decision was to extradite Rancadore because he was satisfied that the European arrest warrant was valid and that to send him to Italy would not breach his human rights.
However, Riddle changed his decision following a ruling last week by the Administrative Court, which binds lower courts in England, involving the court of Florence and a Somali man, Hayle Abdi Badre, who said his rights would be breached by prison conditions.
"The judgment of the Administrative Court is binding on me," Riddle said.
"The higher court accepted that a similar assurance given in that case was in good faith, but was not sufficient."
The judge discharged Rancadore at Westminster Magistrates' Court in London, but told him he would have to wear an electronic tag while Italian authorities lodge an appeal.
The court heard that Rancadore, who has a stent fitted and suffers from angina, had severe chest pains last week and was admitted to hospital.
During the extradition hearings, the court heard evidence from law professor Patrizio Gonnella that Italy has some of the most crowded prisons in Europe.
Speaking about how the health conditions of inmates are handled, Gonnella said: "Today the matter of safeguarding of health is the biggest, the most critical, issue in our prisons."
Rancadore was convicted in 1999 of Mafia association and extortion in Trabia, near Palermo. The Italian authorities want him to serve a seven-year jail term imposed in his absence in 1999.
In his judgment, Riddle said Rancadore "has been here for 20 years" and had a "long-standing marriage and children who are now adults".
Rancadore adopted the name Marc Skinner, using the maiden name of his wife's mother.