Search found 878 matches

by emk
Wed Oct 18, 2017 11:11 am
Forum: Technical Support and Feature Requests
Topic: We're hitting our server limits this week
Replies: 6
Views: 497

Re: We're hitting our server limits this week

Um just saying that Yandex is as big a deal in Russia as Google in the US. Isn't it possible to limit its crawling capacity instead of banning it entirely? Yes, it is possible. The snag is that I need to configure it as a user (bot) so that the forum software gives out the pages in a nice consisten...
by emk
Tue Oct 17, 2017 12:06 pm
Forum: Technical Support and Feature Requests
Topic: We're hitting our server limits this week
Replies: 6
Views: 497

Re: We're hitting our server limits this week

Just an update on the performance tuning. With Yandex disabled, and several other search engines forbidden to index more than 1 page every 10 seconds, our CPU credit balance seems to be accumulating nicely: llo-cpu-credit-with-new-robots-txt.png As long as this number stays well above 10 credits or ...
by emk
Mon Oct 16, 2017 2:42 pm
Forum: General Language Discussion
Topic: Forum running costs
Replies: 15
Views: 727

Re: Forum running costs

For some pretty charts of what the forum costs to run, see my thread this morning. rdearman and I have split the costs various ways in the past, and it's not a huge deal at $1/day. But if our needs grow, we can definitely look into other options.
by emk
Mon Oct 16, 2017 2:34 pm
Forum: Technical Support and Feature Requests
Topic: We're hitting our server limits this week
Replies: 6
Views: 497

Re: We're hitting our server limits this week

OK, it looks like our CPU credits have stabilized nicely for the time being, since I banned Yandex crawls and slowed down Bing, etc.: llo-cpu-credit-balance.png So we can keep running on the cheap server for now, and performance should stay reasonably high with any luck. It shouldn't take a big serv...
by emk
Mon Oct 16, 2017 1:42 pm
Forum: Technical Support and Feature Requests
Topic: We're hitting our server limits this week
Replies: 6
Views: 497

Re: We're hitting our server limits this week

As a temporary workaround, I've replaced our server (again), giving us a new 30-credit balance on CPU credits, and I've set an alarm to notify me if the balance goes below 10 credits, so maybe I can catch this problem before it becomes critical. I've also asked most crawlers to slow down their crawl...
by emk
Mon Oct 16, 2017 1:02 pm
Forum: Technical Support and Feature Requests
Topic: We're hitting our server limits this week
Replies: 6
Views: 497

We're hitting our server limits this week

While working on the recent maintenance issues , I realized that we've maxed out our server for the moment. In order to keep the costs for this site low ($30/month for everything), we run on an Amazon t2.micro "burst" server. This is normally quite fast for a cheap server, but this is beca...
by emk
Mon Oct 16, 2017 11:56 am
Forum: Technical Support and Feature Requests
Topic: Maintenance underway on Saturday
Replies: 9
Views: 355

Re: Maintenance underway on Saturday

OK, Yandex is banned, which helped some. But I'm still seeing terrible performance. :-( Working notes: ⋅ Performance is slow even when we're not running too many Apache instances. It's extremely fast for a few minutes after restarting Apache (and maybe for a longer period after replacing t...
by emk
Mon Oct 16, 2017 10:54 am
Forum: Technical Support and Feature Requests
Topic: Maintenance underway on Saturday
Replies: 9
Views: 355

Re: Maintenance underway on Saturday

Figure out who's hitting our site with lots of traffic and block them. The "Yandex" search engine indexing bot is just hammering the site, with zero sense of politeness. I'm going to try blocking it using robots.txt and see if it takes a hint. I still need to tune the PHP Apache image to ...
by emk
Mon Oct 16, 2017 10:28 am
Forum: Technical Support and Feature Requests
Topic: Maintenance underway on Saturday
Replies: 9
Views: 355

Re: Maintenance underway on Saturday

Somebody is hitting the site with a ton of traffic, and the site is not responding gracefully. Check this out: [ec2-user@ip-172-31-19-62 ~]$ uptime 10:12:52 up 1 day, 16:56, 1 user, load average: 152.63, 153.32, 153.24 "load average: 152" means that we've got 150 processes waiting for 4 CP...
by emk
Sun Oct 15, 2017 12:23 am
Forum: Technical Support and Feature Requests
Topic: Maintenance underway on Saturday
Replies: 9
Views: 355

Re: Maintenance underway on Saturday

OK, I'm sad. :cry: The X-Forward-For fix didn't work. I'll need to fix it in the web server, not PHP.

Go to advanced search