Ms Stefanik made the comments during an appearance on Meet the Press earlier this month. The appearance marked the third anniversary of the Capitol riot.
The lawmaker said she had "concerns about the treatment of the Jan 6 hostages."
Donald Trump has also called individuals convicted for attacking Congress and the Capitol police during the riot "hostages."
She likened their convictions to the "weaponization of the federal government against not just President Trump, but we’re seeing it against conservatives.”
Mr Goldman said he did not intend to force a quick resolution of the censure — which, as a member of Congress, he has the right to do — but said he hoped the GOP would stand up and hold their own accountable.
“Our preference would be for Republicans to recognize that kind of rhetoric has to stop and find a spine and stand up to Donald Trump,” he said in an interview.
Mr Goldman said he would consider forcing a vote on the matter if Republicans refuse to address Ms Stefanik's comments on their own.
Ms Stefanik defended herself in a post on X/Twitter.
“Desperate corrupt Far Left House Democrats are attempting to force a vote to censure me for my unwavering support for the Constitution, the rule of law, and election integrity. Democrats are in desperate free fall because they know President Trump will defeat Joe Biden this November,” she wrote.
House Speaker Mike Johnson also criticised the move, calling it ‘absurd.’
“I just heard about Goldman’s censure resolution against Elise Stefanik. I think It’s patently absurd. She’s one of the best leaders and best communicators in Congress. She is doing an exceptional job and the the idea that he would use censure to attack a political opponent is just ridiculous,” he said, according to Puncbowl News’ Jake Sherman.
Democrats have largely criticised Ms Stefanik's use of the word "hostages" to describe the individuals convicted of participating in the Capitol riot.
House Minority Leader Hakeem Jeffries said she should be "ashamed of herself."
Congressman Jamie Raskin, who served on the House Select Committee that probed the Capitol riot, suggested she was trying to convince Mr Trump to select her as a running mate in the 2024 election.
Censuring has become more commonplace in the modern, highly divisive American political landscape. Last year, Republicans censured three Democratic lawmakers. Congressman Adam Schiff was censured for investigating Mr Trump, Congressman Jamaal Bowman was censured for setting off a fire alarm in an office building inside the House, and Congresswoman Rashida Tlaib was censured for comments she made regarding Israel's military action in Gaza.
Democrats have also censured other Republicans. In 2021, Congressman Paul Gosar was censured for posting an animated video that depicted him murdering Congresswoman Alexandria Ocasio-Cortez.
Prior to that, only one other member of the House has been censured in nearly 40 years.
While Democrats have largely attempted to shy away from issuing censures over speech issues, Mr Goldman argued his move was justified considering the context of the comments made by Ms Stefanik.
“What it comes down to is whether the speech by a member of Congress — does it go over the line where it promotes violence, some form of discrimination or bring serious disrepute on the institution,” he said.
He went on, stating that her comments were directly related to "the safety and security" of the Congress.
“If you provide comfort to those people who have been charged and convicted of violent attempts to overthrow our government, you are supporting people who attacked the Capitol and attacked this body," he said.