Login is really slow

[golyplot]

I was finally able to access the site again this morning, and so far everything has been working great.

[iguanamon]

Thank you, rdearman and emk for fixing the forum... at least for now. Have we forgotten to have a place where people can go to find out if and why the forum is down? This can help to inform people and maybe keep some members until problems can be fixed. Now, with twitter in a shambles, perhaps we need to come up with a site/place to let people know what's going on. Just a thought.

Edit: back to 504 Gsteway Time-out again. Sigh

[Le Baron]

Bah! I opened Opera browser today on a different computer which has this website as quick launch. So I clicked it and saw everything moving superfast. Then I logged in....

Hello 504! And slow loading!

Judging by everyone else's positive experiences, I'm taking this as a personal conspiracy against me.

[Iversen]

It's more a question of impersonal luck - or not. Yesterday everything went really well, but already today it's somewhat slower, and heaven knows whether anything works tomorrow.. I use three different browsers on two different computers, and I can't see anything that changes between them. EIther you are lucky or you aren't.

[Ug_Caveman]

Had a "502" bad gateway today... Not sure if the number going up or down is a good or bad change...

[ryanheise]

Given the nature of the problem, we can expect the forum to go through cycles of working and then not working, so it may well be that nothing has changed in the setup and by chance you are experiencing a good or bad part of that cycle.

If it were me, I would temporarily flip the switch in the AWS console to "unlimited mode" just in the short term until emk can implement his plan to rebuild the setup (happy to help with this). Unlimited mode may potentially come with an increased cost which is easily manageable if it is temporary, and it looks enough of us are willing to chip in and support this financially.

(The following comment is from 2 weeks ago):

If it doesn't do that soon then I'm fairly certain that even legit longtime users will loose faith and abandon the site. The present situation is as bad as that which basically killed off HTLAL, and the only consolation is that rDearman still is active and knows about the situation. We didn't have the same confident feelings about FX at HTLAL, and that's what ultimately lead to the exodus from that once glorious site.

I am pleased to see that both financial and technical support has been offered by the community. Although I would note that neither rdearman nor emk have responded to the offers for support despite weeks passing and rdearman has not commented on the situation since 3 weeks ago. I'm sure that work is being done behind the scenes, but I would rather like to see more communication to reassure us that things are happening. At the very least, there should be a pinned announcement prominently displayed at the top of the general discussion area where everyone can see it, just so that they know what's going on, and so they are less likely to give up and leave the community. And if the community is offering to help, it would be nice to acknowledge that because it is possible that with community support, the problem could be solved sooner.

[tastyonions]

Is there any way to whitelist only the known “well-behaved” bots? Or else blacklist the bad ones.

I’m a programmer myself but don’t know much about admin stuff.

[ryanheise]

You can instruct which bots you want to allow or disallow in the robots.txt file but bad bots will ignore your instructions and do it anyway. For well-behaved bots, there are sometimes bot-specific ways that you can also rate limit them.

However, for bad bots, you need a more advanced solution since they will tend to make it hard for you to blacklist them. E.g. by not using a stable IP address, or indeed by using a distributed crawler involving lots of different IP addresses.

I would suggest using something like Cloudflare which can intercept all requests and only forward through to your server those requests that do not come from a bad bot. From the server's perspective, it never sees the bad bots and so it won't incur any CPU credit cost for dealing with them.

[golyplot]

I think my last post jinxed it. Right afterwards, the forums started 504ing again, although fortunately they're working at least a little this morning as I was able to make this post.

[Chmury]

I'm still getting 504 messages too. Really hope the issue can be resolved soon as like many of us here, I love this community and find a lot of value, fun, connection, motivation and support on this forum. I sadly have zero programming skills, but would be up for helping financially if it were to help ensure the usability and longevity of the forum.

And I agree that it'd be great if we had a place to go when the forum is experiencing technical issues, just so we can all stay connected and understand what's going on and timelines to when things may get fixed etc. Not sure what that could be though. Facebook group? Unsure how many people are wedded to staying anonymous though.