Roberto Carbonero and Fabrizio Ricca removed the rugs from a Muslim prayer room that had been created especially for a conference on Islamic fashion.
The pair insisted that the decision to remove the rugs was not down to prejudice or racism, but was because the city hall is a secular building.
"If someone wants to pray, there are lots of religious centers they can go to," La Stampa reported the councillors as saying.
"It's not as if people build a chapel whenever there's a Christian conference."
Their actions were met with widespread criticism. Michele Paolino, the local leader of the Democratic Party, called it "an arbitrary and violent act," adding that "these actions offend people of all religions."