Casey Affleck Describes Significance of Longtime Friends Like Matt Damon: 'Bread of Life's Leaven' (Exclusive)

"People you've known for a long time become more and more important the older you get, and they are for me for sure," Casey tells PEOPLE

Jason Mendez/Getty; Kevin Mazur/Getty Casey Affleck during the Center at West Park's 2nd Annual Fall Benefit in New York City on Nov. 14, 2024; Matt Damon in New York City on Sept. 19, 2024

Jason Mendez/Getty; Kevin Mazur/Getty

Casey Affleck during the Center at West Park's 2nd Annual Fall Benefit in New York City on Nov. 14, 2024; Matt Damon in New York City on Sept. 19, 2024

Casey Affleck is reflecting on his bond with longtime friends like Matt Damon.

The 49-year-old actor spoke with PEOPLE during the Center at West Park’s 2nd Annual Fall Benefit in New York City on Thursday, Nov. 14, where he was asked about the significance of his friendship with Damon, who was an honorary chair at the evening's event.

"Someone said old friends are the bread of life's leaven," Casey tells PEOPLE. "The older I get, the more that means to me."

"People you've known for a long time become more and more important the older you get, and they are for me for sure," he continues.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human-interest stories.

Kevin Mazur/WireImage Matt Damon and Casey Affleck in Santa Monica, California, on Dec. 11, 2016

Kevin Mazur/WireImage

Matt Damon and Casey Affleck in Santa Monica, California, on Dec. 11, 2016

Related: Casey Affleck 'Still Felt Like an Outsider' While Living with Brother Ben and Matt Damon After Moving to L.A.

As for his presence at the benefit, which celebrates the arts, the Manchester by the Sea star says in part, "[Art] was something that I just loved early on, and I think it does a lot for society."

"Arts, they entertain us and they keep culture going, and the arts are the mirror that we look into when we want to understand who we are and what we're doing," Casey adds.

Casey and Damon, 54, have been friends for more than four decades, along with Casey's older brother Ben Affleck. The trio have collaborated numerous times, including on 1997's Good Will Hunting and, more recently, this year's The Instigators.

“We have this 43-year shared history,” Damon told PEOPLE in August of his longtime bond with the Affleck brothers. “We grew up together.”

The PEOPLE Puzzler crossword is here! How quickly can you solve it? Play now!

Todd Williamson/Getty From L: Matt Damon, Casey Affleck and Ben Affleck in Beverly Hills, California, on Nov. 14, 2016

Todd Williamson/Getty

From L: Matt Damon, Casey Affleck and Ben Affleck in Beverly Hills, California, on Nov. 14, 2016

Related: Casey Affleck Asks Matt Damon to Direct His Movies 'Every Time I Get a Script,' but He Always Says No

The Boston natives’ “shared experience and relationship and trust and love,” as Damon put it, is what makes collaborating such a natural process.

“There's an underlying understanding of who we are,” he told PEOPLE, adding, “In our business, there's a whole language that gets created around trying to protect people's feelings because people's egos are involved."

He also said that he and Casey "don't waste any time on diplomacy, which I love.”

Damon’s memories of pursuing Hollywood dreams alongside Casey and Ben, 52, in the years leading up to the Oscar-winning success of Good Will Hunting included not being “afraid to be vulnerable with each other as young men” in Boston.

As he recalled to PEOPLE, “Part of being an actor and putting yourself in those positions and getting rejected constantly — knowing that somebody else is living that same life and making those same sacrifices, that shared sense of experience, that's why we would clump up and hang out together and talk about it."