A language learners’ forum

Have made admins aware and am awaiting them fixing this. It takes time as people aren't always immediately available. We will be as quick as we can!

Rhian wrote:Have made admins aware and am awaiting them fixing this. It takes time as people aren't always immediately available. We will be as quick as we can!

Sigh, somebody ate all our CPU credits again:



Before fixing this, I'm going to dig through our logs and try to find out what crawler or other system is using up all our CPU credits. This forum is not currently spec'ed for heavy downloads, hour after hour. I mean, it's nice that the search engines want to index us, but somebody's not behaving well.

Oh, look, it's Yandex again (among others), despite the fact that I followed all of Yandex's instructions to ban their crawlers completely until I had time to try to configure them for slow crawling only. Not happy. Not cool, Yandex.

Seems to be working again. Thanks to Emk! Cheers for giving up your Saturday morning!!

Thanks for getting on this so quickly, emk! Sorry that your hard work has all been undone.

Rhian wrote:Seems to be working again. Thanks to Emk! Cheers for giving up your Saturday morning!!

It's only temporary, sadly. I've upgraded to a t2.small, which refreshed the CPU quota, and I'm trying to tell Googlebot to slow down using Google's webmaster tools. And I'm updating robots.txt to keep the dumber bots from hitting the signup page thousands of times per day, though that might be spammers.

emk wrote:

Rhian wrote:Seems to be working again. Thanks to Emk! Cheers for giving up your Saturday morning!!

It's only temporary, sadly. I've upgraded to a t2.small, which refreshed the CPU quota, and I'm trying to tell Googlebot to slow down using Google's webmaster tools. And I'm updating robots.txt to keep the dumber bots from hitting the signup page thousands of times per day, though that might be spammers.

Yandex bot is showing in active users atm

I've upgraded to a t2.small, bringing our total costs to an estimated $40/month. And I've limited GoogleBot from downloading more than a page every 10 seconds. Once again, we see a similar pattern. Here's the new t2.small:


For comparison, here's the old t2.micro again, with a total cost around $30/month, and more crawlers enabled:



In both pictures, you can see that our CPU credits go up sharply after initial launch, then take a sudden hit. Once the "hit" is passed, then things drift for a while. The new instance has a much larger total "reserve", and it regains CPU credits twice as fast. And even when the credits are exhausted, the baseline performance is twice as high as before.

So far, we're holding steady: Our credit balance is remaining level, but it's not climbing back up towards its maximum. I'm crossing my fingers that this t2.small will be enough. We could avoid this whole "CPU credit" system by switching to an m3-class instance, where we get 100% power at all times.

How much does an M3-class cost per month?

We might be fine now but I'd like to know, have you any idea?

Rhian wrote:How much does an M3-class cost per month?

We might be fine now but I'd like to know, have you any idea?

You can look at different server costs using this handy online tool. Here are the t2 and m3 type instances. m4 is also nice, but even the cheapest versions are fairly pricey. We don't need a lot. We probably want to avoid "reserved" instances unless we know that we won't need to upgrade.

For the t2-class instances, here are our options:


Once you know what the server costs, you need to add in the other expenses. Amazon is reasonably priced, but they meter and bill for everything. This includes the database (and the automatic database backups), the bandwidth, the disk volume (and its backups), etc. Here's what the total costs looked like half way through the month, using a t2.micro:



RDS is the managed database, which doesn't seem to need any upgrades right now. (We want a managed database, because that makes it far more difficult to screw up backups and accidentally lose the entire forum.) EC2 is the server and its disks. Route53 is the DNS server that translates "language-learners.org" into a numeric Internet address. Sometimes there's an extra dollar a month for CodePipeline (which does the builds), or an extra ~$12 once a year to renew the domain name. I did a whole bunch of extra work to get 100% of costs through Amazon and not spread them around.

The difference in costs between the t2.micro and the t2.small is about $8 or $9/month, which is how I estimated $40 after upgrading to the t2.small.