Google Search: a sitemap is required
Hello,
I was wondering why I was unable to see the forum in Google Search and contrary to an old post I found about this subject (Is there Sitemap.xml and is it needed?) it does seem a sitemap is indeed required.
Here is what I see in Google Search Console
Page Indexing
Page not indexed: Crawled, currently not indexed
Discovery
Sitemaps
No referring sitemaps detected
I can, at least temporarily, manually create a sitemap.xml, but, especially with a docker compose installation publishing it at the /sitemap.xml location of the page is far from easy. Is there another way to realize it ?
I think adding a sitemap.xml by default would be a good help for the global visibility of the talkyard websites.
KajMagnus @KajMagnus2025-08-05 07:32:06.101ZYes seems that'd be good, ChatGPT says: "You don’t _need_ a sitemap, but it’s strongly recommended for large forums, JS-heavy UIs, or content with varying visibility. Google has explicitly said sitemaps help them find and index important content faster."
About the
/sitemap.xmllocation. I'm thinking an okay way to manage sitemaps, is if Talkyard periodically generates asitemap.xmlfile and places somewhere the HTTP server (Nginx) can serve it from.Maybe as a first step, I could just add a
locationto Nginx, and then you could place asitemap.xmlat the sitemap location, and Nginx will find it. And, later, Talkyard keeps the.xmlfile up-to-date itself.I'll have a look and get back to you ...
- In reply toPitre⬆:KajMagnus @KajMagnus2025-08-05 07:47:08.278Z
Are you self-hosted b.t.w.? (A bit quicker to make an Nginx
location { ... }work for self-hosted sites, I think, than for the SaaS, as a first step.)Hello,
Just a planned location in the nginx.conf for this file would make wonder.
I am self-hosted and I tried myself to do this exact same change with some help from Gemini, once I successfully managed to find the public folders of the service (a specific folder for the sitemap could work too of course).
sudo docker exec -it 942f5fc4f838 nano /etc/nginx/server-locations.conf And adding location = /sitemap.xml { root /opt/talkyard/ty-media/; add_header Cache-Control "max-age=86400, s-maxage=86400, public"; }However when restarting the services my modification was wiped.
I think it would have worked otherwise.
KajMagnus @KajMagnus2025-08-06 22:44:09.010Z2025-08-06 23:47:38.026ZWow, Gemini could be of any use in Talkyard's code base? I thought it was too large — or did you ask it to look only at the Nginx file?
Something like that (I mean your
location = ....snippet) is what I have in mind, yes. I think there'll be a new seversome timeat the end of next week, with this included.However when restarting the services my modification was wiped.
If you want to try with a sitemap now directly, you could copy
server-locations.confto the host, edit it, and in/opt/talkyard/docker-compose.yml, bind-mount both your editedserver-locations.confand yoursitemap.xmlat the correct paths in the Web container. But if you forget to remove the bind mounts, you might run into troubles later, if there's a new Talkyard version with changes to theserver-locations.conf.I specifically asked him to find a way for me to publish the sitemap I made with https://www.mysitemapgenerator.com/ on the location /sitemap.xml and he already knew about Talkyard so once I told him the server was using Docker Compose I received some commands on how to find the container ID and use it to execute commands inside the docker environment.
The following steps were standard nginx I could manage based on the content of nginx.conf.
I am going to wait for the next week change, I prefer minimizing risks. Thank you for the quick update and answer by the way.
It will be my colleague who will follow-up on this conversation from Tuesday as I will change job from this moment on.
Btw I am this same guy, I just switched account to be able to answer once I lose access to my previous email:
Free css to give a modern look to Talkyard
So feel free to check if you want to see what is the result of the CSS I spoke about : https://forum.ioi.online/categories- KajMagnus @KajMagnus2025-08-12 04:59:40.597Z
Thanks for the link, interesting to see how it looks :- ) I remember that post some years ago. Best wishes at the new job.
Code reviewing the next version right now ...
- KajMagnus @KajMagnus2025-08-17 09:47:16.782Z
@Pitre, I'm still code reviewing, almost done ... Looks fine. (Related to sth else, namely embedded forum fixes, which will be in the same new version). Sorry it's taking a bit longer, if [the colleague who will follow-up on this conversation] is waiting
- KajMagnus @KajMagnus2025-08-25 01:56:00.503Z
Now I've deployed the new server version to here, Talkyard .io. Will bump the production version number tomorrow probably, then you could try it out. (I'll write some instructions then too.)
- KajMagnus @KajMagnus2025-08-31 05:38:08.659Z
Hi @Pitre, sorry for the late reply. Now there's a new version.
If you place a
sitemap.xmlhere, on the host:/opt/talkyard/data/uploads/sitemap-test/sitemap.xmlTalkyard will serve it at
https://your talkyard server/sitemap.xml.(Details: I added a Nginx
locationblock that looks in that.../sitemap-test/sitemap.xmlin the uploads dir, which is bind-mountend on the host already. Makes sense for self-hosted sites only)
- Progresswith handling this problem