"The message is clear, terrorism cannot continue to attack everywhere in the name of Muslims. We want the whole world to listen," Abdellah Redouane, head of the Islamic Cultural Centre in Rome, said at the demonstration in the capital.
"The Koran is against violence", "Islam is peace" and "Solidarity with Paris victims" read banners held up by members of the Muslim community, who gathered in the rain in a square in central Rome under a heavy police guard.
The rally, under the slogan "Not in my name", began with a minute's silence for the victims of the attacks on Parisian nightspots last week which left 130 people dead.
In Milan, where security has been ramped up following fears of an attack on landmarks like the famous Scala opera house, people chanted "No to Isis", referring to the Islamic State group which claimed the France attacks.
"No to terrorism, yes to mosques," said Davide Piccardo from Caim, an umbrella organisation of Islamic societies, calling for a greater understanding and openness in Italy towards moderate Islam and "a crackdown on worrying Islamophobia".