The case is due to be heard in public by a Cricket Discipline Commission panel at the start of next month.
Both Bresnan and Hoggard face charges from the England and Wales Cricket Board but became the latest witnesses to pull out on Friday.
Yorkshire County Cricket Club and a number of individuals have today been charged following an ECB investigation into racism and other allegations at the Club and its handling of those allegations.
Read more ⬇️
— England and Wales Cricket Board (@ECB_cricket) June 15, 2022
Ex-Yorkshire all-rounder Bresnan told The Times: “I am willing to release everything because I’m out of the process. But they just charged me. How is that possible without even speaking to me?
“It’s like being charged and tried without even being arrested. That’s how it feels.
“He’s (Rafiq) saying I did use that language (the P-word), along with others, but gave no example. There are no witnesses. I vehemently deny that. I grew up in a place where that’s not right.
“Two and a half years of non-stop articles, leaks, tweets, different stuff coming out without any right of reply. I couldn’t guarantee to witnesses that statements they wanted to make in private would not appear in the public domain.
“We’ve been ordered to keep quiet up until this point now because it might have affected the case. It just hasn’t felt right, the whole thing.”
Hoggard agreed with Bresnan’s sentiments and insisted the investigation had failed everyone.
“The process has failed everybody. Every party involved has a problem with the way this process has been dealt with,” he told BBC Sport.
“Azeem has a problem with it, all the respondents have, (former Yorkshire chairman) Lord Patel has, Yorkshire have. There has got to be a better way.
“I’m pulling out because I don’t think it’s a fair process.
“There are no winners in this. It is not an admission of guilt. The people who know the truth, know the truth. That is all that matters to me.”
Andrew Gale, another of the individuals charged, announced last year he was not willing to engage with the process, which he described as “tainted”.
The ECB announced last June it had charged a number of individuals with improper conduct and alleged breaches of its anti-discrimination code.
Yorkshire were also charged over their handling of the allegations.
Responding to the latest withdrawal, the ECB said: “Individuals are entitled to choose not to participate in the hearings if they wish, but the cases will still be heard in their absence and we are satisfied that the disciplinary process in this matter has been both rigorous and fair.
The process has failed everybody. Every party involved has a problem with the way this process has been dealt with
“The ECB’s investigation and disciplinary process has been overseen by an independent committee and specialist leading King’s Counsel (KC).
“As with any case before the Cricket Discipline Commission, defendants are entitled to a fair hearing by an independent and experienced CDC Panel where they can call witnesses, and they can also challenge the evidence in support of the charge, including through cross-examination of the ECB’s witnesses.
It is entirely the decision of defendants if they choose not to take advantage of this opportunity.
“At the end of the hearing it is for the independent CDC Panel, not the ECB, to determine guilt or otherwise and any sanction.”
Rafiq said: “Over the past two years I have been vindicated time and again.
“This has included a legal investigation that confirmed I was a victim of racial harassment and bullying; a Yorkshire commissioned panel that concluded I suffered discrimination; numerous apologies, both public and private, from people who witnessed or were involved in this behaviour; and others have come forward to confirm the culture in the wider game.
“I was grateful to Matthew Hoggard for calling me to apologise shortly after I went public in 2020. It is regrettable, though, that these defendants are not willing to go to a public hearing and face what happened.”