Pompeo, accompanied by U.S. Ambassador to the Vatican, Callista Gingrich, met Parolin and the Vatican's Foreign Minister, Archbishop Paul Gallagher.
The Vatican's two top diplomatic officials have accused him of trying to drag the Catholic Church into the U.S. presidential election by denouncing its relations with China.
The remarks came after Pompeo accused the Church in an article and a series of tweets this month of putting its "moral authority" at risk by renewing an agreement with China over the appointment of bishops.
Parolin and Gallagher said that Pope Francis had declined a request from Pompeo for an audience, as the pope avoids meeting politicians ahead of elections.