Video filmed by local IRT Poltava TV channel showed heavy police presence around a scene where a man had been detected at first.
The area was cordoned off and police cars were parked to limit access.
After negotiations, a man exchanged the police officer for a police colonel and drove off with him in a car provided by police.
It was the second hostage-taking in Ukraine in three days. On Tuesday (July 21), an armed man held 13 people captive on a bus for hours before releasing them.
Police on Thursday were trying to arrest a man suspected of hijacking a vehicle when he took out a grenade and threatened to kill one of the officers, Deputy Interior Minister Anton Gerashchenko said on Facebook.