How's the new server performing?

Discuss technical problems and features here
User avatar
emk
Black Belt - 1st Dan
Posts: 1620
Joined: Sat Jul 18, 2015 12:07 pm
Location: Vermont, USA
Languages: English (N), French (B2+)
Badly neglected "just for fun" languages: Middle Egyptian, Spanish.
Language Log: viewtopic.php?f=15&t=723
x 6323
Contact:

How's the new server performing?

Postby emk » Fri Apr 21, 2017 11:52 am

The forum has been moved from a shared PHP host to Amazon Web Services (AWS). The actual site is served up by two virtual machines:

  • A "t2.micro" serving the site. This is managed using ECS (the EC2 Container System) and a Docker container built from the source code on GitHub. We're probably going to automate building and deploying using AWS CodePipeline at some point, so we'll just be able to push to a GitHub branch and update the site's software.
  • A "db.t2.micro" hosting the database. This is a dedicated database VM managed and backed up by AWS.
The main "t2.micro" operates using a "burst" system. Like a cheetah, it can go very fast for a short while, and then it slows down. We get 6 CPU credits per hour, and a "reserve" of roughly 150 credits. If we use up all our hourly credits, we'll draw on the reserve. If the reserve goes to 0, then the site slows to 10% of 1 CPU. At the moment, it doesn't look like we're even close—we might want to downgrade to a t2.nano, which gets 3 credits per hour, and which only costs half as much. Anyway, here's what it looked like this morning:

llo-database-performance.png

The database seems to be ticking along pretty nicely without too much load, at least for the moment:

llo-server-cpu-usage.png

We're not using very much CPU—only about 5 to 10% of what we've been allocated. But I'm a little concerned by the IOPS. We currently have 100 IOPS, and it looks like we're already using a third of that. Again, there's a burst mode available, up to about 3000 IOPS, but once we use it up, we go back to 100.

In both cases, we can upgrade the underlying servers if we want to go faster (or downgrade to save money). It would take literally a couple of hours. But faster servers cost more money. You can play around with the various server sizes (look in the "Linux on Demand" column for the price) and database server sizes (look in the "MySQL on Demand" column for the price).

So, how's forum performance looking right now? Is everything reasonably fast and snappy? Do you ever see sudden, massive slowdowns, as if we exhausted our CPU credits or IOPS?
You do not have the required permissions to view the files attached to this post.
3 x

User avatar
Adrianslont
Blue Belt
Posts: 827
Joined: Sun Aug 16, 2015 10:39 am
Location: Australia
Languages: English (N), Learning Indonesian and French
x 1936

Re: How's the new server performing?

Postby Adrianslont » Fri Apr 21, 2017 11:55 am

So far it's all fine for me.

Thanks for your efforts, guys.

Edit: better than before actually - this comment just posted without the usual lag I get when I make a post.

Edit: typo and to check the lightening speed again!
2 x

User avatar
tommus
Blue Belt
Posts: 957
Joined: Sat Jul 04, 2015 3:59 pm
Location: Kingston, ON, Canada
Languages: English (N), French (B2), Dutch (B2)
x 1937

Re: How's the new server performing?

Postby tommus » Fri Apr 21, 2017 12:03 pm

Based on a sample of two check-ins and reading several posts, I think the new Forum setup is working perfectly. Very fast. Thanks again to emk and rdearman for their stellar work on our behalf. I'm sure it will become a separate thread but we the membership need to step forward and financially bear the cost of running the Forum.
4 x
Dutch: 01 September -> 31 December 2020
Watch 1000 Dutch TV Series Videos : 40 / 1000

User avatar
iguanamon
Black Belt - 2nd Dan
Posts: 2353
Joined: Sat Jul 18, 2015 11:14 am
Location: Virgin Islands
Languages: Speaks: English (Native); Spanish (C2); Portuguese (C2); Haitian Creole (C1); Ladino/Djudeo-espanyol (C1); Lesser Antilles French Creole (B2)
Studies: Catalan
Language Log: viewtopic.php?t=797
x 14189

Re: How's the new server performing?

Postby iguanamon » Fri Apr 21, 2017 12:06 pm

The speed is smoking fast here in the Caribbean right now.
2 x

User avatar
emk
Black Belt - 1st Dan
Posts: 1620
Joined: Sat Jul 18, 2015 12:07 pm
Location: Vermont, USA
Languages: English (N), French (B2+)
Badly neglected "just for fun" languages: Middle Egyptian, Spanish.
Language Log: viewtopic.php?f=15&t=723
x 6323
Contact:

Re: How's the new server performing?

Postby emk » Fri Apr 21, 2017 2:42 pm

iguanamon wrote:The speed is smoking fast here in the Caribbean right now.

The server is currently running in one of Amazon's Virginia data centers, which have excellent connectivity, especially to the US and Europe. You might see a little more network latency in Asia, but not much—especially not compared to the lags introduced by the old database server.

We're now using only 3 IOPS, down from about 30 earlier. Basically the database is mostly asleep. We can handle 100 IOPS sustained, and 3000 for a brief burst. So performance should be great, except possibly during the daily backup. But backups are important!
2 x

rlnv
Orange Belt
Posts: 228
Joined: Mon Jul 20, 2015 3:07 pm
Languages: English
x 454

Re: How's the new server performing?

Postby rlnv » Fri Apr 21, 2017 3:45 pm

Thanks for taking the time to share the technical details of the new setup. Great choice in AWS. It certainly looks like you admins have taken performance seriously. Excellent speed where I sit. Thank you!
1 x

Arnaud
Blue Belt
Posts: 984
Joined: Sat Jul 18, 2015 11:57 am
Location: Paris, France
Languages: Native: French
Intermediate: English, Russian, Italian
Tourist : Breton, Greek, Chinese, Japanese, German, Spanish, Latin
Language Log: viewtopic.php?t=1524
x 2172

Re: How's the new server performing?

Postby Arnaud » Fri Apr 21, 2017 4:05 pm

As fast as Speedy Gonzales :mrgreen:
1 x

User avatar
jeff_lindqvist
Black Belt - 3rd Dan
Posts: 3135
Joined: Sun Aug 16, 2015 9:52 pm
Languages: sv, en
de, es
ga, eo
---
fi, yue, ro, tp, cy, kw, pt, sk
Language Log: viewtopic.php?f=15&t=2773
x 10462

Re: How's the new server performing?

Postby jeff_lindqvist » Fri Apr 21, 2017 4:54 pm

A lot faster than before. :)
1 x
Leabhair/Greannáin léite as Gaeilge: 9 / 18
Ar an seastán oíche: Oileán an Órchiste
Duolingo - finished trees: sp/ga/de/fr/pt/it
Finnish with extra pain : 100 / 100

Llorg Blog - Wiki - Discord

User avatar
smallwhite
Black Belt - 2nd Dan
Posts: 2386
Joined: Mon Jul 06, 2015 6:55 am
Location: Hong Kong
Languages: Native: Cantonese;
Good: English, French, Spanish, Italian;
Mediocre: Mandarin, German, Swedish, Dutch.
.
x 4876

Re: How's the new server performing?

Postby smallwhite » Sat Apr 22, 2017 1:36 am

Browsing noticeably faster then before. Australia. Thanks a lot.
0 x
Dialang or it didn't happen.

User avatar
emk
Black Belt - 1st Dan
Posts: 1620
Joined: Sat Jul 18, 2015 12:07 pm
Location: Vermont, USA
Languages: English (N), French (B2+)
Badly neglected "just for fun" languages: Middle Egyptian, Spanish.
Language Log: viewtopic.php?f=15&t=723
x 6323
Contact:

Re: How's the new server performing?

Postby emk » Sat Apr 22, 2017 2:32 am

I'm seeing some bad performance at the moment, and I'm investigating. It doesn't seem to be the database or the server. Both have adequate CPU credits and IOPS in reserve. More when I figure out what's up, and my apologies!

I need to make sure that there isn't a database backup or something like that killing performance.

(Sigh. New servers are always a bit tricky and we set this one up in a hurry.)
0 x


Return to “Technical Support and Feature Requests”

Who is online

Users browsing this forum: No registered users and 2 guests