A woman was killed in Casamicciola, in the north of the small tourist island, hit by debris that fell from a church, with the body of another spotted in the rubble of a collapsed house, local media reported.
Around 4am (0200 GMT) emergency workers rescued a seven-month-old baby, crying but alive, after hours of effort. They were still struggling to free the baby's two brothers, who were alive but trapped and communicating with emergency services through the rubble, according to Naples prefecture.
According to local media, the two children, aged 4 and 7, are trapped under a bed but are in touch with rescue workers by phone. It was their pregnant mother, healthy and safe, who raised the alarm. The father was rescued at 2:30am (0130 GMT).
"In Casamicciola, a building collapsed and three people were pulled alive from the rubble -- two women and a man," the head of the local department of civil protection, Angelo Borrelli, said at a press conference early Tuesday.
Two small communes, Casamicciola and neighbouring Lacco Ameno, had borne the brunt of the quake, he added.
The tremor hit the northwest of the island at 8:57 pm (1957 GMT), with a depth of some 10 kilometres.
Italian authorities first put Monday's quake at 3.6, but subsequently revised it upward to 4.0 magnitude. The main earthquake was followed by 14 smaller aftershocks, Borrelli said. Several buildings in the area collapsed and others had large, ominous cracks.
'A horrible experience'
The Rai News 24 television channel broadcast images of holidaymakers by their cars with their bags packed, with other people sitting in their gardens or outside their houses.
According to local media, many cars were waiting at Ischia port eager to head back to Naples as soon as possible. One ferry left at 2:30 am and another was scheduled for two hours later.
The quake struck just days ahead of the first anniversary of the 6.0 magnitude quake that killed nearly 300 people in and around Amatrice in central Italy. In October 2016 and January 2017 three other earthquakes hit
the same region.
"I was on the couch watching TV. Blackout, shaking, something fell on my head. I scream, my mother grabs me and we ran outside," one witness wrote on Twitter.
The quake response has benefited from the presence of emergency responders who happened to already be on the island to fight the forest fires that have plagued Italy this summer, local media said.
Teams of firefighters, including two units specialised in extracting people from the rubble, were quickly mobilised, said Bruno Frattasi, who oversees the fire department.
Ischia's only hospital was also hit and had to be partially evacuated. Five patients were evacuated to another medical facility by helicopter.
Restaurants were packed and many stores were still open when the shaking began, witnesses said on Twitter.
"A horrible experience, everything was shaking, plunged into darkness, houses were collapsing... a nightmare," a witness wrote on Twitter.
Electricity was restored after just a few minutes, but many people stayed outdoors after the quake, reluctant to go inside.
Ischia has been a frequent victim of earthquakes, with its worst dating back over a century. Estimated at a magnitude of 5.8, it killed over 2,000 people in July 1883.
Much of Italy's land mass and some of its surrounding waters are prone to seismic activity with the highest risk concentrated along its mountainous central spine.
Italy straddles the Eurasian and African tectonic plates, making it vulnerable to seismic activity when they move.