No internet connection
  1. Home
  2. Issues

Google Search: a sitemap is required

By @Pitre
    2025-08-04 13:47:55.037Z2025-08-05 08:44:14.336Z

    Hello,

    I was wondering why I was unable to see the forum in Google Search and contrary to an old post I found about this subject (Is there Sitemap.xml and is it needed?) it does seem a sitemap is indeed required.

    Here is what I see in Google Search Console

    Page Indexing
    Page not indexed: Crawled, currently not indexed

    Discovery
    Sitemaps
    No referring sitemaps detected

    I can, at least temporarily, manually create a sitemap.xml, but, especially with a docker compose installation publishing it at the /sitemap.xml location of the page is far from easy. Is there another way to realize it ?

    I think adding a sitemap.xml by default would be a good help for the global visibility of the talkyard websites.

    Reply
    • 6 replies
    1. KajMagnus @KajMagnus2025-08-05 07:32:06.101Z

      Yes seems that'd be good, ChatGPT says: "You don’t _need_ a sitemap, but it’s strongly recommended for large forums, JS-heavy UIs, or content with varying visibility. Google has explicitly said sitemaps help them find and index important content faster."

      About the /sitemap.xml location. I'm thinking an okay way to manage sitemaps, is if Talkyard periodically generates a sitemap.xml file and places somewhere the HTTP server (Nginx) can serve it from.

      Maybe as a first step, I could just add a location to Nginx, and then you could place a sitemap.xml at the sitemap location, and Nginx will find it. And, later, Talkyard keeps the .xml file up-to-date itself.

      I'll have a look and get back to you ...

      Reply
      1. In reply toPitre⬆:
        KajMagnus @KajMagnus2025-08-05 07:47:08.278Z

        Are you self-hosted b.t.w.? (A bit quicker to make an Nginx location { ... } work for self-hosted sites, I think, than for the SaaS, as a first step.)

        Reply
        1. P@Pitre
            2025-08-05 08:40:08.521Z

            Hello,

            Just a planned location in the nginx.conf for this file would make wonder.

            I am self-hosted and I tried myself to do this exact same change with some help from Gemini, once I successfully managed to find the public folders of the service (a specific folder for the sitemap could work too of course).

            sudo docker exec -it 942f5fc4f838 nano /etc/nginx/server-locations.conf
And adding
location = /sitemap.xml {
  root /opt/talkyard/ty-media/;
  add_header Cache-Control "max-age=86400, s-maxage=86400, public";
}

            However when restarting the services my modification was wiped.
            I think it would have worked otherwise.

            Reply
            1. KajMagnus @KajMagnus2025-08-06 22:44:09.010Z2025-08-06 23:47:38.026Z

              Wow, Gemini could be of any use in Talkyard's code base? I thought it was too large — or did you ask it to look only at the Nginx file?

              Something like that (I mean your location = .... snippet) is what I have in mind, yes. I think there'll be a new sever some time at the end of   next week, with this included.

              However when restarting the services my modification was wiped.

              If you want to try with a sitemap now directly, you could copy server-locations.conf to the host, edit it, and in /opt/talkyard/docker-compose.yml, bind-mount both your edited server-locations.conf and your sitemap.xml at the correct paths in the Web container. But if you forget to remove the bind mounts, you might run into troubles later, if there's a new Talkyard version with changes to the server-locations.conf.

              Reply
              1. P@Pitre
                  2025-08-07 07:12:41.557Z2025-08-07 07:19:50.401Z

                  I specifically asked him to find a way for me to publish the sitemap I made with https://www.mysitemapgenerator.com/ on the location /sitemap.xml and he already knew about Talkyard so once I told him the server was using Docker Compose I received some commands on how to find the container ID and use it to execute commands inside the docker environment.

                  The following steps were standard nginx I could manage based on the content of nginx.conf.

                  I am going to wait for the next week change, I prefer minimizing risks. Thank you for the quick update and answer by the way.

                  It will be my colleague who will follow-up on this conversation from Tuesday as I will change job from this moment on.

                  Btw I am this same guy, I just switched account to be able to answer once I lose access to my previous email:
                  Free css to give a modern look to Talkyard
                  So feel free to check if you want to see what is the result of the CSS I spoke about : https://forum.ioi.online/categories

                  Reply
                  1. KajMagnus @KajMagnus2025-08-12 04:59:40.597Z

                    Thanks for the link, interesting to see how it looks :- )  I remember that post some years ago. Best wishes at the new job.

                    Code reviewing the next version right now ...

                    Reply
            2. Progress
              with handling this problem
            3. @KajMagnus marked this topic as Started 2025-08-05 07:32:53.145Z.
            Reply (discussion)Add progress note