Investigative weekly Le Canard Enchaine revealed that the 41-year-old had been nominated before Christmas as a member of the 2014 jury for a French academy at the sumptuous Villa Medici, which every year selects a group of French-speaking artists for a 12 to 18-month residency.
Jury members are tasked with auditioning candidates for the prestigious fellowship among writers, filmmakers, photographers, stage designers, art historians and others.
But on Wednesday, the culture ministry said the actress - whose affair with Hollande was revealed last week by glossy magazine Closer, prompting his devastated partner to check into hospital for stress - would not be appointed as planned.
"Her name was put forward by Eric de Chassey, director of the Academy of France in Rome-Villa Medici, but the minister decided not to appoint her," the office of Culture Minister Aurelie Filippetti said.
"I do not want to give rise to any controversy," Filippetti later told journalists.
"She was completely competent to be a member of the jury... but it was not helpful to add another controversial issue."
Gayet had been due to be part of the jury alongside other personalities such as French prima ballerina Marie-Agnes Gillot and Italian composer Lucia Ronchetti.
She will now be replaced by screenwriter Emmanuele Bernheim, Filippetti said.
De Chassey, meanwhile, told iTele television that he picked Gayet for "professional reasons", ruling out any favouritism.
"She is a recognised cinema producer, a recognised actress, which has already enabled her to take part in juries, including important juries such as... in Cannes", he said.
"She also has a lot of interest in contemporary art, in literature, in music, which everyone in the art world knows."
Hollande, 59, has refused to comment on his affair with Gayet, and at a press conference on Tuesday defiantly deflected questions on the issue.
He admitted that he and his long-term girlfriend Valerie Trierweiler were going through "painful moments" and indicated that her status would be clarified before a scheduled trip to the United States next month.
Trierweiler, France's de facto First Lady, has been invited to accompany Hollande on an official visit that includes an overnight stay at the White House.
"Everyone in their personal lives can go through tough times. That is the case (for me). These are painful moments," he said.
"But I have one principle: these private affairs are dealt with in private. This is neither the time nor the place to do it so I will not be responding to any questions about my private life."
He later added that Trierweiler, 48, was resting, but refused to comment further on her state of health.
Gayet, meanwhile, has been laying low since the affair burst into the open on Friday, with Closer publishing a series of photos showing Hollande and the actress arriving separately for trysts at a flat close to the presidential Elysee Palace.
The Villa Medici, where French king Louis XIV established the Academy at the end of the 17th century, looks over the historic centre of Rome.