Super Challenge bot

Discuss technical problems and features here
User avatar
rdearman
Site Admin
Posts: 7231
Joined: Thu May 14, 2015 4:18 pm
Location: United Kingdom
Languages: English (N)
Language Log: viewtopic.php?f=15&t=1836
x 23120
Contact:

Re: Super Challenge bot

Postby rdearman » Mon Aug 14, 2017 6:33 pm

solanderdog wrote:
rdearman wrote:
I have tweeted this also. If you see any problems post here.


It's probably due to my not knowing what I'm doing, but I can't seem to get it to add any books. I added three films, that I watched recently, but every time I try to add a book it tells me, "You don't seem to be studying Tibetan. Register first using the #register tag!" I don't get it.

I tried all sorts of things, including:

@langchallenge I just finished reading a #fr #book with 588 pages.

@langchallenge I just finished reading a book in French with 558 pages. It was called "Soul Breakers" and don't recommend it.

@langchallenge I just finished reading a book with 558 pages. It was called "Soul Breakers" and don't recommend it.

@langchallenge I just read a #book in #french. It was called "Soul Breakers"

It always tells me that I'm not studying Tibetan.


That is because the use of #book has been depreciated after we added in the language of Tibetan which has the code of #bo so please use #read instead.
1 x
: 0 / 150 Read 150 books in 2024

My YouTube Channel
The Autodidactic Podcast
My Author's Newsletter

I post on this forum with mobile devices, so excuse short msgs and typos.

User avatar
solanderdog
White Belt
Posts: 17
Joined: Sun Aug 06, 2017 11:04 pm
Location: Vancouver, Canada
Languages: English (N), French (B2), Norwegian (Beginner)
Language Log: https://forum.language-learners.org/vie ... =15&t=6513
x 30
Contact:

Re: Super Challenge bot

Postby solanderdog » Mon Aug 14, 2017 6:38 pm

Thanks!
0 x

User avatar
Brun Ugle
Black Belt - 2nd Dan
Posts: 2273
Joined: Mon Jul 27, 2015 12:48 pm
Location: Steinkjer, Norway
Languages: English (N), Norwegian (~C1/C2), Spanish (B1/B2), German (A2/B1?), Japanese (very rusty)
Language Log: https://forum.language-learners.org/vie ... 15&t=11484
x 5821
Contact:

Re: Super Challenge bot

Postby Brun Ugle » Mon Aug 14, 2017 7:00 pm

solanderdog wrote:
rdearman wrote:
I have tweeted this also. If you see any problems post here.


It's probably due to my not knowing what I'm doing, but I can't seem to get it to add any books. I added three films, that I watched recently, but every time I try to add a book it tells me, "You don't seem to be studying Tibetan. Register first using the #register tag!" I don't get it.

I tried all sorts of things, including:

@langchallenge I just finished reading a #fr #book with 588 pages.

@langchallenge I just finished reading a book in French with 558 pages. It was called "Soul Breakers" and don't recommend it.

@langchallenge I just finished reading a book with 558 pages. It was called "Soul Breakers" and don't recommend it.

@langchallenge I just read a #book in #french. It was called "Soul Breakers"

It always tells me that I'm not studying Tibetan.

The #book tag doesn't work. Use the tag #read instead.
1 x

User avatar
solanderdog
White Belt
Posts: 17
Joined: Sun Aug 06, 2017 11:04 pm
Location: Vancouver, Canada
Languages: English (N), French (B2), Norwegian (Beginner)
Language Log: https://forum.language-learners.org/vie ... =15&t=6513
x 30
Contact:

Re: Super Challenge bot

Postby solanderdog » Mon Aug 14, 2017 7:29 pm

Brun Ugle wrote:The #book tag doesn't work. Use the tag #read instead.


Thanks! The instructions I was reading haven't been updated to reflect that!
0 x

User avatar
emk
Black Belt - 1st Dan
Posts: 1619
Joined: Sat Jul 18, 2015 12:07 pm
Location: Vermont, USA
Languages: English (N), French (B2+)
Badly neglected "just for fun" languages: Middle Egyptian, Spanish.
Language Log: viewtopic.php?f=15&t=723
x 6315
Contact:

Re: Super Challenge bot

Postby emk » Mon Aug 14, 2017 10:08 pm

Serpent wrote:edit: it does render correctly starting from May 27, but that's only the last half a year of the challenge :cry:
unless there's a connection to the security issues i kinda don't understand why this had to be done so late in the challenge, and couldn't be put off until the next one began :?

The superchallenge bot has three parts:

  1. The database, which stores everybody's content data.
  2. The web app.
  3. The "update" task, which contacts Twitter every 5 minutes to see if there are new tweets.
Here's a rough timeline of what happened:

  1. (earlier this year) The old site was broken into, and all the user accounts, passwords, forum code, etc., were at risk of being compromised.
  2. We shut down the old site, posted a warning message, and migrated the whole site to a new dedicated web server and database (which is why we don't get constant timeouts any more).
  3. When we migrated the site, I we also migrated the database. The old database was configured to use ISO Latin 1 (a western European character set), but it was actually storing UTF-8 data with each byte being treated as a Latin 1 character. I fixed it using a variation of this approach. Unfortunately, even though we migrated the database, we didn't have enough time to properly package the web app and the update task for the new server, and rdearman ran them someplace else.
  4. At some point, I think the Super Challenge database might have been imported a second time, or some other code wasn't fixed properly, and we went back to confused encodings.
  5. Last week, rdearman's old hosting company told him that they no longer supported the "update" task, which would leave the bot unable to poll Twitter.
  6. This weekend, rdearman and spent about 5 hours on Google Hangouts moving the web app and the "update" task to the new server. This means that anybody can submit code for new features, we can review it, and we like the changes, we can deploy them in just a few minutes.
We couldn't have just left this alone until the end of the challenge, because that would have meant either storing the database on a compromised server in April), or it would have meant that the bot could no longer read new tweaks (last week).

Ideally, what we need to do at this point is identify when the bad tweets end, and the good tweets begin, preferably to the nearest day. Then I would need to go into the database, identify the last broken tweet and the first good one, and write a whole bunch of SQL to fix the double-encoding for some database records but not others. Unfortunately, this is pretty fiddly work and it usually takes me a while to sort it out.

I apologize to everybody doing the challenge! I'm sorry that it's taken so long to get the bot sorted out. :-(

EDIT: Oh, yeah. If anybody is interested, all the code for the site is publicly available. I can also provide a few sample good and bad rows from the database if anybody wants to try their hand at this.
5 x

User avatar
Serpent
Black Belt - 3rd Dan
Posts: 3657
Joined: Sat Jul 18, 2015 10:54 am
Location: Moskova
Languages: heritage
Russian (native); Belarusian, Polish

fluent or close: Finnish (certified C1), English; Portuguese, Spanish, German, Italian
learning: Croatian+, Ukrainian; Romanian, Galician; Danish, Swedish; Estonian
exploring: Latin, Karelian, Catalan, Dutch, Czech, Latvian
x 5179
Contact:

Re: Super Challenge bot

Postby Serpent » Mon Aug 14, 2017 11:29 pm

The "bad" tweets are all that were sent before May 27 (this year). Well, among my tweets the earliest affected tweet is from May 11 last year, but there are also no "good" tweets with diacritics before that.

The bot was originally broken from April 21 to May 3, and it replied to tweets starting from May 4th. At the same time rdearman instructed us to repost everything from these dates. All this time the bot's webpage was down. Rdearman put up a temporary one in late June and then we discovered that the tweets from April 21 to May 27 were still missing, even the ones the bot had replied to.

May 27 was shortly before the polyglot gathering. I think you were in the process of fixing the bot at the time. Everyone got accidentally banned on May 30.
1 x
LyricsTraining now has Finnish and Polish :)
Corrections welcome

jeffers
Blue Belt
Posts: 848
Joined: Sat Aug 22, 2015 4:12 pm
Location: UK
Languages: Speaks: English (N), Hindi (A2-B1)

Learning: The above, plus French (A2-B1), German (A1), Ancient Greek (?), Sanskrit (beginner)
Language Log: https://forum.language-learners.org/vie ... 15&t=19785
x 2774
Contact:

Re: Super Challenge bot

Postby jeffers » Thu Aug 17, 2017 2:34 pm

I've just started tweeting again and the bot hasn't picked up all my tweets. I sent five, one after the other, and only the first and 5th were responded to. I wondered if it was because I sent too many at once, so I resent the other three but only sent a tweet after the bot responded to me. Two of the three worked, but the last one gained no response. Is the bot having issues?
0 x
Le mieux est l'ennemi du bien (roughly, the perfect is the enemy of the good)

French SC Books: 0 / 5000 (0/5000 pp)
French SC Films: 0 / 9000 (0/9000 mins)

User avatar
rdearman
Site Admin
Posts: 7231
Joined: Thu May 14, 2015 4:18 pm
Location: United Kingdom
Languages: English (N)
Language Log: viewtopic.php?f=15&t=1836
x 23120
Contact:

Re: Super Challenge bot

Postby rdearman » Thu Aug 17, 2017 6:01 pm

jeffers wrote:I've just started tweeting again and the bot hasn't picked up all my tweets. I sent five, one after the other, and only the first and 5th were responded to. I wondered if it was because I sent too many at once, so I resent the other three but only sent a tweet after the bot responded to me. Two of the three worked, but the last one gained no response. Is the bot having issues?

Not that I'm aware of. What was the content of the one which wasn't picked up?
0 x
: 0 / 150 Read 150 books in 2024

My YouTube Channel
The Autodidactic Podcast
My Author's Newsletter

I post on this forum with mobile devices, so excuse short msgs and typos.

jeffers
Blue Belt
Posts: 848
Joined: Sat Aug 22, 2015 4:12 pm
Location: UK
Languages: Speaks: English (N), Hindi (A2-B1)

Learning: The above, plus French (A2-B1), German (A1), Ancient Greek (?), Sanskrit (beginner)
Language Log: https://forum.language-learners.org/vie ... 15&t=19785
x 2774
Contact:

Re: Super Challenge bot

Postby jeffers » Thu Aug 17, 2017 6:19 pm

rdearman wrote:
jeffers wrote:I've just started tweeting again and the bot hasn't picked up all my tweets. I sent five, one after the other, and only the first and 5th were responded to. I wondered if it was because I sent too many at once, so I resent the other three but only sent a tweet after the bot responded to me. Two of the three worked, but the last one gained no response. Is the bot having issues?

Not that I'm aware of. What was the content of the one which wasn't picked up?


The tweet which was ignored was:
@langchallenge #hi #read"RIHR 02 (6th-8th times)" 6 pages (trying again)

Previously it had missed the following three tweets:
@langchallenge #hi #read"RIHR 02 (6th-8th times)" 6 pages
@langchallenge #hi #read"RIHR 03 (6th-8th times)" 6 pages
@langchallenge #hi #read"RIHR 04 (6th-8th times)" 6 pages

I resent all three (with the additional bracket to keep the tweets clear to myself) and it accepted two of them. Just now I resent the missing tweet as follows, and it worked:
@langchallenge #hi #read"RIHR 02 (6th-8th times)" 6 pages (third tweet attempt)
0 x
Le mieux est l'ennemi du bien (roughly, the perfect is the enemy of the good)

French SC Books: 0 / 5000 (0/5000 pp)
French SC Films: 0 / 9000 (0/9000 mins)

User avatar
rdearman
Site Admin
Posts: 7231
Joined: Thu May 14, 2015 4:18 pm
Location: United Kingdom
Languages: English (N)
Language Log: viewtopic.php?f=15&t=1836
x 23120
Contact:

Re: Super Challenge bot

Postby rdearman » Thu Aug 17, 2017 7:26 pm

jeffers wrote:
rdearman wrote:
jeffers wrote:I've just started tweeting again and the bot hasn't picked up all my tweets. I sent five, one after the other, and only the first and 5th were responded to. I wondered if it was because I sent too many at once, so I resent the other three but only sent a tweet after the bot responded to me. Two of the three worked, but the last one gained no response. Is the bot having issues?

Not that I'm aware of. What was the content of the one which wasn't picked up?


The tweet which was ignored was:
@langchallenge #hi #read"RIHR 02 (6th-8th times)" 6 pages (trying again)

Previously it had missed the following three tweets:
@langchallenge #hi #read"RIHR 02 (6th-8th times)" 6 pages
@langchallenge #hi #read"RIHR 03 (6th-8th times)" 6 pages
@langchallenge #hi #read"RIHR 04 (6th-8th times)" 6 pages

I resent all three (with the additional bracket to keep the tweets clear to myself) and it accepted two of them. Just now I resent the missing tweet as follows, and it worked:
@langchallenge #hi #read"RIHR 02 (6th-8th times)" 6 pages (third tweet attempt)


Don't know if it is just the copy and paste, but is there a space between #read and " ???
1 x
: 0 / 150 Read 150 books in 2024

My YouTube Channel
The Autodidactic Podcast
My Author's Newsletter

I post on this forum with mobile devices, so excuse short msgs and typos.


Return to “Technical Support and Feature Requests”

Who is online

Users browsing this forum: No registered users and 2 guests