Forum server upgraded

Discuss technical problems and features here
User avatar
emk
Black Belt - 1st Dan
Posts: 1733
Joined: Sat Jul 18, 2015 12:07 pm
Location: Vermont, USA
Languages: English (N), French (B2+)
Badly neglected "just for fun" languages: Middle Egyptian, Spanish.
Language Log: viewtopic.php?f=15&t=723
x 6919

Forum server upgraded

Postby emk » Tue Jan 16, 2024 1:43 am

OK, my huge apologies for how long it took to get the forum working again. I knew it was going to be an involved process, because I last did a full, ground-up rebuild two laptops ago. But then we lost power for all of Saturday.

We're back. We're running on c6g.medium, which is pretty beefy, instead of the pokey old t2.micro. We no longer have a "burst mode"; this server runs at full speed all the time. We should be able to survive even aggressive crawlers.

I plan to transfer the lowest-level parts of the tech over to rdearman, since he has more time to work on them than I do, and because he has an easier route to accepting donations than I ever did.

Once again, my apologies for the slowly degrading service. Mea maxima culpa.
64 x

User avatar
tommus
Blue Belt
Posts: 957
Joined: Sat Jul 04, 2015 3:59 pm
Location: Kingston, ON, Canada
Languages: English (N), French (B2), Dutch (B2)
x 1939

Re: Forum server upgraded

Postby tommus » Tue Jan 16, 2024 2:07 am

Great to have the Forum back. Thanks so, so much emk. Really appreciated!!!!!
14 x
Dutch: 01 September -> 31 December 2020
Watch 1000 Dutch TV Series Videos : 40 / 1000

User avatar
CDR
Orange Belt
Posts: 114
Joined: Fri May 06, 2022 10:47 pm
Languages: English (N), Portuguese (B2?), Japanese (A2?)
Language Log: https://forum.language-learners.org/vie ... 15&t=18051
x 479

Re: Forum server upgraded

Postby CDR » Tue Jan 16, 2024 2:32 am

Yeah, thank you so much for you work @emk! and @rdearman for taking on more and more. I am also very happy to have the forum back!

Thanks to @iguanamon for posting on the old forum with updates as well, I wouldn't have known about the progress on the issue!
11 x

User avatar
emk
Black Belt - 1st Dan
Posts: 1733
Joined: Sat Jul 18, 2015 12:07 pm
Location: Vermont, USA
Languages: English (N), French (B2+)
Badly neglected "just for fun" languages: Middle Egyptian, Spanish.
Language Log: viewtopic.php?f=15&t=723
x 6919

Re: Forum server upgraded

Postby emk » Tue Jan 16, 2024 4:22 am

OK, take 2! We had to switch to a m1.small instance. This is more expensive than the old server, but it also has consistent performance. Sorry for the second, shorter outage. I'm going to check in again tomorrow morning and see if we're still doing OK.
23 x

User avatar
emk
Black Belt - 1st Dan
Posts: 1733
Joined: Sat Jul 18, 2015 12:07 pm
Location: Vermont, USA
Languages: English (N), French (B2+)
Badly neglected "just for fun" languages: Middle Egyptian, Spanish.
Language Log: viewtopic.php?f=15&t=723
x 6919

Re: Forum server upgraded

Postby emk » Tue Jan 16, 2024 1:34 pm

As of this morning, the server is still running fine, and it feels pretty snappy.

Spam rates, unfortunately, seem to have increased. Maybe the slow server was discouraging spammers, too. :?

At this point, I estimate the new configuration will cost about $70-75/month to run, up from about $60/month for the last few years. The breakdown is something like $32/month for the server, $20/month for the database, and $20/month for the load balancer. If I rebuilt our Docker image to work with an ARM processor, I might be able to switch to a $22/month server with slightly better performance, getting us back down to our old costs. But this has been a long-term expense for me, and it's time to figure out a better plan. rdearman has proposed figuring out a donation system. I suspect that something like Patreon or a similar site would work. If only 15 people gave $5/month, for example, then we'd be set. We just need to pay for the hosting costs.

Also, I can easily do full maintenance on the system again. This had been difficult for a while, because I'd replaced my laptop, and I'd needed to update some tools. At the moment, I can do a complete server rebuild with 60 seconds of work and 10 minutes of waiting for everything to automatically deploy.

There are some major improvements we could make if we have someone willing to spend a weekend preparing a PR on GitHub to fix the "Proxy" headers. This requires some PHP knowledge and elbow grease. But it would fix (1) the logouts, (2) the excessive prompting with security questions, (3) some of the spam, and (4) our historical problems with crawlers. If anyone here knows how to program and wants to get their sleeves dirty, I'd be happy to explain the problem in detail and to review and deploy your PR.
29 x

golyplot
Black Belt - 1st Dan
Posts: 1800
Joined: Mon Jan 16, 2017 9:41 pm
Languages: Am. English (N), German, French, ASL (abandoned), Spanish, Dutch, Italian, Japanese (N2)
Language Log: https://forum.language-learners.org/vie ... 15&t=12230
x 3660

Re: Forum server upgraded

Postby golyplot » Tue Jan 16, 2024 4:15 pm

I do know how to program, but unfortunately don't have experience with configuring server headers in PHP, so I doubt I'd be able to help any more than googling Stack Overflow would.
3 x

grayson
Yellow Belt
Posts: 66
Joined: Sat Sep 17, 2022 1:46 pm
Location: Netherlands
Languages: .
x 403

Re: Forum server upgraded

Postby grayson » Tue Jan 16, 2024 4:58 pm

emk wrote:At this point, I estimate the new configuration will cost about $70-75/month to run, up from about $60/month for the last few years. [...] this has been a long-term expense for me, and it's time to figure out a better plan. rdearman has proposed figuring out a donation system.


My gourd, I had no idea it was this expensive. Clearly I've never run a popular forum before. I'd love for this information to be part of whatever new registrants see when they sign up -- "this forum is a labor of love that costs XXX a month .... it is and always will be free for everyone to use ... if you want and can responsibly within your means contribute, go here" kind of thing. And to be repeated in a sticky somewhere. Better yet, in a header or footer note, for unregistered visitors who find themselves often ending up here to answer questions they've typed into Google. I can't be the only person who gave zero thought to the cost and is more than willing to contribute now that I know.

On that note: if the donation system ends up netting positive, I suggest we use that to first pay back emk for the years of out-of-pocket expense. After that, an annual "courtesy nod" to emk and rdearman (and whoever else is involved that I don't yet know about) for ongoing maintenance and moderation. After that, perhaps a poll to see what people would like to spend it on -- an LLORG event, a server upgrade, LLORG swag (stickers! magnets!), .....
18 x
Much madness is divinest sense, to a discerning eye; much sense, the starkest madness. —Emily Dickinson

User avatar
sfuqua
Black Belt - 1st Dan
Posts: 1654
Joined: Sun Jul 19, 2015 5:05 am
Location: san jose, california
Languages: Bad English: native
Samoan: speak, but rusty
Tagalog: imperfect, but use all the time
Spanish: read
French: read some
Japanese: beginner, obsessively studying
Language Log: https://forum.language-learners.org/vie ... =15&t=9248
x 6382

Re: Forum server upgraded

Postby sfuqua » Tue Jan 16, 2024 7:26 pm

Glad to donate.
These problems have reminded me how important this site is to my sanity. :D

It is very cool to see the site up.
12 x
荒海や佐渡によこたふ天の川

the rough sea / stretching out towards Sado / the Milky Way
Basho[1689]

Sometimes Japanese is just too much...

User avatar
Le Baron
Black Belt - 3rd Dan
Posts: 3708
Joined: Mon Jan 18, 2021 5:14 pm
Location: Koude kikkerland
Languages: English (N), fr, nl, de, eo, Sranantongo,
Maintaining: es, swahili.
Language Log: https://forum.language-learners.org/vie ... 15&t=18796
x 9895

Re: Forum server upgraded

Postby Le Baron » Tue Jan 16, 2024 9:07 pm

I had some idea of the costs, because I run an economics discussion forum. It's much less open to all-comers than this forum and still it can get out of hand when more features are added and has already moved once. I appreciate that a lot of work goes on which is unseen, plus costs.

The choice to have this forum ad-free and well-maintained against abuse is much appreciated.
11 x
To have talked much and read much is of more value in learning to speak and write well than to have parsed and analysed half a library.

User avatar
coldrainwater
Blue Belt
Posts: 690
Joined: Sun Jan 01, 2017 4:53 am
Location: Magnolia, TX
Languages: EN(N), ES(rusty), DE(), FR(studies)
Language Log: https://forum.language-learners.org/vie ... =15&t=7636
x 2417

Re: Forum server upgraded

Postby coldrainwater » Tue Jan 16, 2024 9:54 pm

golyplot wrote:I do know how to program, but unfortunately don't have experience with configuring server headers in PHP, so I doubt I'd be able to help any more than googling Stack Overflow would.


Ditto. To the extent that modifications can safely be made in an open-source capacity, I wouldn't mind following along with changes as they are made to learn the ropes. Like golyplot, with no PHP experience, I doubt my current level of knowledge would be of assistance. I am unsure if the relevant code is still housed at github.com/language-learners/. If it is, putting the issue/problem statement there might be reasonable to allow for more participation. Though of little help for the current issues, I am particularly interested in being able to spin up a local copy of the source code in a way that would allow me to tinker and gain the type of familiarity that could be helpful in the future. Relatively minor edits can often yield major benefits once familiar with the code base and configuration.

I am also open to donating as others have mentioned.
7 x


Return to “Technical Support and Feature Requests”

Who is online

Users browsing this forum: No registered users and 2 guests