No internet connection
  1. Home
  2. Support

URGENT: Talkyard production server search broken (!!)

By Christian Scheuer @chrscheuer
    2022-04-14 16:26:41.368Z

    Hi @KajMagnus,

    I'm seeing reports that searches on our forum has started to fail:

    • 11 replies
    1. C
      Christian Scheuer @chrscheuer
        2022-04-14 16:27:31.665Z

        This was somebody just searching on the forum in a browser (Safari), so not related to the API.

        1. CChristian Scheuer @chrscheuer
            2022-04-14 16:28:22.063Z
            1. CChristian Scheuer @chrscheuer
                2022-04-14 17:56:09.565Z

                We're receiving very wide spread reports from users now - many users are seeing this issue now

                1. CChristian Scheuer @chrscheuer
                    2022-04-14 18:07:55.935Z

                    @KajMagnus this is really getting critical. How do we get a hold of you when things stop working? I've tried sending an email.

                    If we can't get this fixed asap we'll have to start looking at self hosting this. We can't have a service down for several hours.

                    1. I'll have a look at the search problem.

                      Maybe there could be a Telegram chat

                      1. CChristian Scheuer @chrscheuer
                          2022-04-14 21:38:28.798Z

                          We had to implement a workaround so I used the HTML/CSS engine to insert a Google-provided search box instead now, so you might have difficulty testing this on at the moment

                          1. Wow, that was a fast workaround. Sorry for the troubles.

                            The problem was, I think, caused by me increasing the search rate limits, which I did earlier today — and this I can test on localhost. (Didn't realize that that could cause this problem. If that's the problem)

                            1. CChristian Scheuer @chrscheuer
                                2022-04-14 21:47:28.951Z

                                5 hours is a long downtime for any service, so I had to get creative.

                                Yea my guess is also that it should be related to your recent rate limit changes.

                                1. Ok. I tried on localhost, and could reproduce it (same error & stack trace) — and the problem disappeared, on localhost, when I removed the new rate limit setting.

                                  I've removed the setting in Prod now too, so I think search works again.
                                  Still, maybe better to leave things as is until tomorrow when we're rested?

                                  I'll reply to the other topic, I mean, topics, tomorrow ( = Firiday), & I'll fix the bug and release a new version ... should be in Prod tomorrow or the day after.

                                  1. CChristian Scheuer @chrscheuer
                                      2022-04-14 22:23:31.166Z

                                      Thank you! Yes now that we have a workaround, waiting for the fix to be better tested is preferable.

                    2. In reply tochrscheuer:

                      Fixed now, marked as Done. ( & added auto tests)

                      1. Progress
                        with handling this problem
                      2. @KajMagnus marked this topic as Done 2022-04-18 02:24:09.639Z.