Prime Minister Matteo Renzi and President Francois Hollande, in Brussels to attend an informal EU summit, met Jewish leaders outside the museum and then bowed their heads in tribute to a rabbi's prayer.
The three Socialist leaders were joined by European Parliament speaker Martin Schulz, a German Socialist.
Belgium investigators are probing a possible terrorist link in the shooting by a lone gunman that has shocked the world.
The attack – carried out cooly by a "well-prepared" killer, according to police and video images – killed three people outright and left a young man brain-dead.
It was the first for more than 30 years in Belgium and has revived fears of a return of violent anti-Semitism to Europe.
The museum in the heart of Brussels has been closed as investigators continue their inquiry.