The moves follow similar decisions earlier this week by Britain, France and the United States regarding their embassies in Sanaa, the Yemeni capital.
The Italian foreign ministry said in a statement that "we hope mediation efforts led by the UN envoy Jamal Benomar will create security conditions allowing a return of diplomatic personnel to Yemen as soon as possible".
UN Secretary General Ban Ki-moon said Thursday that Yemen was "collapsing before our eyes" towards civil war.
A German foreign ministry spokeswoman said "the situation is anything but stable" and called the ouster of the government by Shiite militiamen known as Huthis "unacceptable, dangerous and with consequences for the region".
"We decided yesterday to temporarily close the embassy in Sanaa and the personnel departed the country early this morning," she said.