Forum server upgraded

[emk]

OK, my huge apologies for how long it took to get the forum working again. I knew it was going to be an involved process, because I last did a full, ground-up rebuild two laptops ago. But then we lost power for all of Saturday.

We're back. We're running on c6g.medium, which is pretty beefy, instead of the pokey old t2.micro. We no longer have a "burst mode"; this server runs at full speed all the time. We should be able to survive even aggressive crawlers.

I plan to transfer the lowest-level parts of the tech over to rdearman, since he has more time to work on them than I do, and because he has an easier route to accepting donations than I ever did.

Once again, my apologies for the slowly degrading service. Mea maxima culpa.

[tommus]

Great to have the Forum back. Thanks so, so much emk. Really appreciated!!!!!

[CDR]

Yeah, thank you so much for you work @emk! and @rdearman for taking on more and more. I am also very happy to have the forum back!

Thanks to @iguanamon for posting on the old forum with updates as well, I wouldn't have known about the progress on the issue!

[emk]

OK, take 2! We had to switch to a m1.small instance. This is more expensive than the old server, but it also has consistent performance. Sorry for the second, shorter outage. I'm going to check in again tomorrow morning and see if we're still doing OK.

[emk]

As of this morning, the server is still running fine, and it feels pretty snappy.

Spam rates, unfortunately, seem to have increased. Maybe the slow server was discouraging spammers, too.

At this point, I estimate the new configuration will cost about $70-75/month to run, up from about $60/month for the last few years. The breakdown is something like $32/month for the server, $20/month for the database, and $20/month for the load balancer. If I rebuilt our Docker image to work with an ARM processor, I might be able to switch to a $22/month server with slightly better performance, getting us back down to our old costs. But this has been a long-term expense for me, and it's time to figure out a better plan. rdearman has proposed figuring out a donation system. I suspect that something like Patreon or a similar site would work. If only 15 people gave $5/month, for example, then we'd be set. We just need to pay for the hosting costs.

Also, I can easily do full maintenance on the system again. This had been difficult for a while, because I'd replaced my laptop, and I'd needed to update some tools. At the moment, I can do a complete server rebuild with 60 seconds of work and 10 minutes of waiting for everything to automatically deploy.

There are some major improvements we could make if we have someone willing to spend a weekend preparing a PR on GitHub to fix the "Proxy" headers. This requires some PHP knowledge and elbow grease. But it would fix (1) the logouts, (2) the excessive prompting with security questions, (3) some of the spam, and (4) our historical problems with crawlers. If anyone here knows how to program and wants to get their sleeves dirty, I'd be happy to explain the problem in detail and to review and deploy your PR.

[golyplot]

I do know how to program, but unfortunately don't have experience with configuring server headers in PHP, so I doubt I'd be able to help any more than googling Stack Overflow would.

[grayson]

At this point, I estimate the new configuration will cost about $70-75/month to run, up from about $60/month for the last few years. [...] this has been a long-term expense for me, and it's time to figure out a better plan. rdearman has proposed figuring out a donation system.

My gourd, I had no idea it was this expensive. Clearly I've never run a popular forum before. I'd love for this information to be part of whatever new registrants see when they sign up -- "this forum is a labor of love that costs XXX a month .... it is and always will be free for everyone to use ... if you want and can responsibly within your means contribute, go here" kind of thing. And to be repeated in a sticky somewhere. Better yet, in a header or footer note, for unregistered visitors who find themselves often ending up here to answer questions they've typed into Google. I can't be the only person who gave zero thought to the cost and is more than willing to contribute now that I know.

On that note: if the donation system ends up netting positive, I suggest we use that to first pay back emk for the years of out-of-pocket expense. After that, an annual "courtesy nod" to emk and rdearman (and whoever else is involved that I don't yet know about) for ongoing maintenance and moderation. After that, perhaps a poll to see what people would like to spend it on -- an LLORG event, a server upgrade, LLORG swag (stickers! magnets!), .....

[sfuqua]

Glad to donate.
These problems have reminded me how important this site is to my sanity.

It is very cool to see the site up.

[Le Baron]

I had some idea of the costs, because I run an economics discussion forum. It's much less open to all-comers than this forum and still it can get out of hand when more features are added and has already moved once. I appreciate that a lot of work goes on which is unseen, plus costs.

The choice to have this forum ad-free and well-maintained against abuse is much appreciated.

[coldrainwater]

I do know how to program, but unfortunately don't have experience with configuring server headers in PHP, so I doubt I'd be able to help any more than googling Stack Overflow would.

Ditto. To the extent that modifications can safely be made in an open-source capacity, I wouldn't mind following along with changes as they are made to learn the ropes. Like golyplot, with no PHP experience, I doubt my current level of knowledge would be of assistance. I am unsure if the relevant code is still housed at github.com/language-learners/. If it is, putting the issue/problem statement there might be reasonable to allow for more participation. Though of little help for the current issues, I am particularly interested in being able to spin up a local copy of the source code in a way that would allow me to tinker and gain the type of familiarity that could be helpful in the future. Relatively minor edits can often yield major benefits once familiar with the code base and configuration.

I am also open to donating as others have mentioned.