His one-day trip to Europe's migration hotspot culminated dramatically when three Syrian refugee families, all Muslim, returned with the pontiff to Rome
The pope told reporters that his gesture was "a drop in the ocean" but hoped that afterwards, "the ocean will never be the same again".
"All refugees are children of God," he said aboard the papal plane on its flight back to Rome.
The departure of the 12 refugees, who will be housed by the Vatican, was the climax of an emotional visit that saw migrants kneeling in tears before the 79-year-old Roman Catholic leader.
"You are not alone... do not lose hope," the pope, who was accompanied by Ecumenical Patriarch Bartholomew and Archbishop Ieronymos, the head of the Church of Greece, said at the Moria registration centre, where around 3,000 migrants are being held.
The vast majority have requested asylum but will likely be deported under a controversial agreement reached last month to tackle Europe's refugee crisis by sending all irregular migrants who land in Greece back to Turkey.
As the pope was escorted through Moria to meet a select number of migrants, one man broke into tears as he knelt at the pontiff's feet, requesting his blessing.
Another woman who slipped past security to approach the pontiff also broke down in tears as he paused to listen to her.
Other migrants gathered outside held hand-made signs that read "We want freedom", "Let my people go" and "Papa cherche a nous sauver" ("Pope, try to save us").
A group of small children presented the pope with a dozen drawings.
Refugees are not numbers, they are people who have faces, names, stories, and need to be treated as such.— Pope Francis (@Pontifex) April 16, 2016
Lesbos is one of the first ports of call in the EU for the hundreds of thousands of asylum-seekers who have fled war, poverty and persecution in the Middle East and Asia across the Aegean Sea via nearby Turkey in the past year.
The religious leaders said a prayer at Lesbos harbour in memory of the hundreds of migrants who have drowned during the voyage in overcrowded smuggler boats.
"Merciful God... though many of their graves bear no name, to you each one is known, loved and cherished," the pope said.
Earlier they signed a declaration calling on the international community to "respond with courage" to the humanitarian crisis and calling on religious communities to step up efforts to assist refugees.
The pope stressed that migrants were not numbers but people with "faces, names and individual stories" who were preyed on by "unscrupulous thugs" and called for "resolute" efforts to clamp down on arms trafficking.
"The world will be judged by the way it has treated you. And we will all be accountable for the way we respond to the crisis and conflict in the regions that you come from," Bartholomew said.
The refugee influx has sparked fierce disagreements between EU members and brought the bloc's system of open borders to the brink of collapse.
Lesbos has become the focus of criticism of the EU's deal with Turkey to take back migrants who travel to the Greek islands, in return for billions in EU cash.
All new arrivals on the island are being held at Moria while they wait to be processed to determine whether they can legitimately claim asylum or should be returned as "economic migrants". Rights groups have accused Greece of turning the centre into a detention camp.
Migrant arrivals in Greece have drastically fallen since the agreement took effect while the number of deaths among people making the perilous crossing has also dropped.
Another 125 arrived via the Aegean in the last 24 hours, including 46 on Lesbos, the Greek government said on Saturday.
The refugee families to be taken in by the Vatican, including six children, will be initially cared for by the community of Sant'Egidio in Rome, the Holy See said.
They come from Damascus and Deir Azzor, an area currently occupied by jihadists, and lost their homes to bombings, it said.
The pope said that all 12 had papers which were in order and that he had played no part in their selection.
"There were two Christian families but their paperwork wasn't ready. (Religion) was not grounds for exemption. All refugees are children of God."
Francis had framed his visit as an exercise in raising awareness of the "worst humanitarian disaster since the Second World War".
The former Jesuit priest has repeatedly said he does not accept the EU's distinction between those fleeing conflicts, like the war in Syria, and those fleeing poverty and starvation created by global economic inequalities.
His trip to Lesbos followed an equally symbolic journey to the Italian island of Lampedusa in 2013.
There, in the aftermath of several deadly sinkings of people smugglers' boats packed with migrants, he made one of the most famous speeches of his term in office, railing against the "globalisation of indifference" which has allowed thousands to perish at sea in their pursuit of a better life for themselves and their families.
Over one million people crossed clandestinely from Turkey to Greece in 2015 and some 150,000 have made the trip since the start of this year.