Around 14 people were in the tower when it was knocked over, plunging some into the cold water and trapping others in a lift which toppled into the sea, media reports said.
One of the victims was reported to be Daniele Fratantonio, 30, who worked for the coast guard operations centre. The other two have yet to be identified.
Rescue workers dived into the inky waters around the port – the busiest in Italy – in a frantic search to find around six or seven people believed to be missing after the accident, which left four seriously wounded.
Others used dogs trained to find people in earthquake zones to see if survivors were trapped under the rubble around the tower.
The number of missing was not clear because the accident happened during a shift change at the vast metal tower, which bent over by 45 degrees before partially collapsing.
Three people were believed to have been trapped inside the lift, which divers were attempting to access and open under water, according to Il Secolo XIX daily in Genoa.
"I heard a terrible din and rushed out of my cabin," Roberto, the port's night watch, told La Repubblica newspaper. "It was an incredible sight: the control tower was leaning perilously."
At dawn, a mobile telephone which began to ring beneath the rubble strewn along the port raised the hopes of locating survivors, but it rang off before rescue workers could localise the sound.
An employee of the Messina Line company based in Genoa which owns the Jolly Nero confirmed that the ship had been involved in an accident when leaving the port but said the reason was not clear.
"The weather conditions were perfect, there was no wind, there were no other ships on the move. A ship of that size should not have been making that manoeuvre," Luigi Merlo, the head of Genoa's port authority, told reporters.
The crash spooked Italians still reeling from the Costa Concordia night-time shipwreck off Giglio island in January 2012 which left 32 people dead.
Indictment hearings against six suspects in the cruise liner disaster began in Italy on April 15th.
The main suspect is captain Francesco Schettino, who is accused of multiple manslaughter, causing a shipwreck, misinforming the coast guard after the crash and abandoning the ship during the rescue.
Around 3:30 am (0130 GMT), the Jolly Nero was moved away from the crash site, which was manned by dozens of firefighters according to an AFP photographer.
Genoa mayor Marco Doria said Italians were in mourning after this "very serious port accident which has struck an entire city".
The Italian container ship is almost 200 metres (655 feet) long, 30 metres (98 feet) wide, and has a gross tonnage of over 40,500. It was bound for Naples.
The ship's owner, Stefano Messina, who arrived at the port soon after the crash, choked back tears as he told journalists: "We are all utterly shocked. Nothing like this has ever happened before, we are desperate."
Prosecutors in the northwest Italian city opened an investigation while the Jolly Nero was sequestered by police, and the captain detained for questioning, reports said.
"Based on the few details which have emerged so far, it was an incomprehensible manoeuvre which could only be explained by a mechanical failure," said Il Secolo XIX newspaper, based in Genoa.
The captain was quoted as saying: "Two engines seem to have failed and we lost control of the ship."