Soldiers who staged the takeover said on state television they had dissolved the West African nation's government and constitution and closed all land and air borders.
Pick-up trucks and military vehicles accompanied by motorcyclists and cheering onlookers.
One military source said the only bridge connecting the mainland to the Kaloum neighbourhood, where the palace and most government ministries are located, had been sealed off.
"The Guinean people are free," said Oumar Diallo as he celebrated on the streets.
Gunfire erupted and fighting broke out near the presidential palace in Conakry on Sunday morning.
Videos shared on social media on Sunday afternoon, which Reuters could not immediately authenticate, showed President Alpha Conde in a room surrounded by army special forces.
Conde, whose whereabouts were not immediately clear, won a third term in October after changing the constitution to allow him to stand again, despite violent protests from the opposition, raising concerns of further political upheavals in a region that has seen coups in Mali and Chad in recent months.
The Guinean government has drastically increased and multiplied taxes in recent weeks to replenish state coffers. The price of fuel has increased by 20%, causing frustration among many Guineans.
Guinea has seen sustained economic growth during Conde's decade in power thanks to its bauxite, iron ore, gold and diamond wealth, but few of its citizens have seen the benefits.