New Delhi's plans to try the men under the maritime security law had drawn strong protests from Rome which feared the move would result in violation of an earlier Indian government pledge not to seek execution of the two marines.
"The NIA (National Investigation Agency) will not be prosecuting the marines now," home ministry spokesman K.S. Dhatwalia said.
"The ministry had earlier given permission to the NIA to prosecute the marines. That decision has now been revised," he said.
India's next step in the prosecution of the marines was not immediately clear.
The marines were accused of murder over the shooting deaths of two fishermen off the coast of Kerala while serving as security guards on an Italian-flagged cargo ship in February 2012. The case sparked a diplomatic row between India and Italy.
Massimiliano Latorre and Salvatore Girone say they mistook the fishing boat for a pirate vessel and only fired warning shots.
India has dragged its feet in starting a trial, with legal experts attributing the delay to uncertainty over which law to use to prosecute the men.
The delay prompted the Italian marines last month to ask India's Supreme Court to drop murder charges against them and allow them to return home.
To speed up the process, the top court on Monday gave the Indian government a week to make a final decision on prosecution of the marines.
Italy insists the pair should be prosecuted on home soil as the shootings involved an Italian-flagged vessel in international waters. India says the killings took place in waters under its jurisdiction.
The marines have been given bail and are staying at the Italian embassy in New Delhi. They were allowed to go home to vote in elections and returned to India for trial in March last year. Rome initially refused to send them back to India, triggering a bitter diplomatic stand-off between the two countries.
India, which uses the death penalty in what it says are the "rarest of rare cases", assured Italy at the time of the men's return that they would not face execution if found guilty.
The return of the two marines to India caused huge controversy in Rome and prompted Italy's foreign minister to resign in protest.