Ivan Toney was pulled to the floor by Marcos Senesi shortly before half-time, giving referee Jarred Gillett what looked a simple penalty decision.
However, Cherries boss O’Neil insisted VAR should have spotted Toney actually held on to the defender’s arm until they reached the box.
But the decision stood and Toney, still to miss a penalty for Brentford, duly took his unblemished record to 21 successful spot-kicks from 21 attempts.
Mathias Jensen added the second with a cool finish from the edge of the area to condemn Bournemouth to a sixth consecutive defeat, all since O’Neil was made permanent manager.
O’Neil said: “Ivan Toney has managed to lock Marcos’ arm under him and pulled him down.
“In this day and age with VAR it’s scandalous. I’ve spoken to the referee and told him I expect an apology.
“When I watched it I thought, ‘They’re definitely overturning that’. I thought it was a terrible decision.
“We had the same referee in the play-off semi-final here two years ago and I thought he was equally as bad this evening as he was then. I thought it was a strange game to give him.
“But he should have been helped by VAR. Live I can see how it was given, but how we can’t then come to the right decision is scandalous.”
Bournemouth are now just a point above the relegation zone, but O’Neil added: “I’d rather talk about the players and performance, they worked their socks off to the end. I didn’t think there was much in the game against a good side.
“What I saw today is they are still fully committed. I thought they were ready to go this evening and they competed with Brentford. I thought they were unfortunate to come away with nothing.”
Brentford climbed up to eighth in the Premier League after a third straight win and stretched their unbeaten run to seven matches.
Boss Thomas Frank said: “I’m very pleased with the boys. I knew it would be a difficult game.
“I don’t believe in defining games of the season, but I said to the boys, ‘This is a defining-ish game’.
“Bournemouth are a similar team to us, similar sized club, similar budget, and we had to make sure we are absolutely bang on, and I can’t praise the lads enough for doing that.”
As for O’Neil’s reaction to the penalty, Frank added: “I think scandalous is a big word to use. I’d be disappointed with my defenders if they defended like that.”