I am summoned, thus I shall appear.
OK, so I think a fair amount of this is really the product of technology. Putting aside changes in user expectation for a moment, I think we need to dig hard into a few 'facts of life' about things.
It wasn't always the case that PHP was the dominant language of choice for forums; before that, there was a history of Perl based platforms.
The two obvious contenders that come to mind in that era, are YABB (of which YABB SE was a PHP-based rewrite, which would later become SMF) and Ikonboard, developed by a young man named Matt Mecham. Yes, THAT Matt M.
Both of those originally didn't even use a database, just storing the posts as files on disk, because that was how tech rolled back in the pre-2000 era, but Perl has always been a real bear to run safely. You have the drama of the cgi-bin folder, executable permissions and all that caper... and you can't readily throttle resources used by it.
This is why PHP positively blossomed in the early 2000s - it didn't need special privileges, you could throttle it, lock it down and everyone supported it, even moreso than today. And since many of the PHP forum platforms have their ancestry in that 1998-2005 era, it's hardly surprising that they used what was everywhere at the time. (Same reason for WordPress, really.)
And there's always the .NET ecosystem - YAF.NET has been around since 2003, Community Server (later Telligent Community, later Verint Community) since 2004 - but you won't see these on average hobbyist environments because they want a Windows Server which is of course far more expensive than your average PHP shared hosting. They're still out there, though, for those Windows-environment folks who want such things (because, honestly, running PHP/MySQL on Windows *sucks*)
Then we enter the modern era; there's no shortage now of people doing other things, but I think a lot of that comes from the fact that it's not 2010 any more, commodity hosting options have broadened massively.
It's now more than possible to get an actual VPS with 1GB RAM for $5 US a month. I remember back in the day when it wasn't even *close* to that (my first VPS in 2006, was 160MB RAM at $40/month and that was *cheap*). The rise in availability of properly sandboxed virtualisations and how cheap it's gotten to do that means it's affordable to set the more exotic things up without needing to run your own physical server configured just so.
Combine that with the rise in things like Docker - no small part in why Discourse flourished vs its immediate peers (seriously, installing NodeBB from scratch is hard work compared to 'install this Docker image, done'), and you have a recipe for people trying new things. On the other end, Heroku is more flexible than ever for the Rails folks and even Python-geared hosting isn't that hard to find.
But yeah, it's mostly been driven by 'what tech is out there that is affordable for the scale' and 'what people are motivated to build on' - PHP was so dominant for so long because it was (and is) everywhere. The dial's moving, but not rapidly.