Attorney General Goolam Vahanvati told the court that the marines would instead be charged with unspecified offences under the country's criminal code, which could be murder.
"The honourable law minister has recorded his opinion that provisions of the SUA Act (maritime and anti-piracy act) are not attracted in this case," Vahanvati told the top court.
However the case will again be delayed while the court examines whether India's top National Investigation Agency, which usually handles matters of national security, was the right investigative body for the case.
The court hearing comes one day after Defence Minister A.K. Antony denied India was backing down over the case amid fury in Rome over delays in the prosecution and the planned use of the anti-piracy law.
"There will be no compromise. We are not going back in any way in the case. We are going ahead with the case as per Indian laws," Antony told reporters.
Italy last week recalled its ambassador to India and summoned the Indian ambassador to express concern at delays in court proceedings in a case that erupted in 2012.
Marines Massimiliano Latorre and Salvatore Girone are accused of shooting the fishermen off the coast of Kerala while they were serving as security guards on an Italian-flagged cargo ship.
The pair, who have been given bail and are staying at the Italian embassy in Delhi, say they mistook the fishing boat for a pirate vessel and only fired warning shots.
India had originally sought to prosecute the marines under the anti-piracy law, but it said the marines would not face the death penalty if convicted because prosecutors would not use that clause in the legislation.
Italy insists the pair should be tried on home soil since the shootings involved an Italian-flagged vessel in what Rome insists were international waters.
India asserts the killings took place in waters under its jurisdiction.