Self-hosted installation - docker containers guidance
[Edit by KajMagnus] Now (Mars 2019), I added instructions here about how to integrate Talkyard into one's own Docker-Compose stack, which was what this question was mostly about. [/Edit]
Hi,
I think I am ready to move from Disqus comments to Talkyard comments for my blog. I would like to use self-hosted solution.
I don't want to run any scripts/compose file offered, mainly because I already have my website running using docker-compose with some customization to config files (like nginx).
I was trying to understand what every container in your compose file is doing, but without a description for every container it is not very straightforward.
For example, I don't need nginx and certbot, because I already have that.
Could you explain what every container is needed for and provide some guidance how to migrate to self-hosted comments in my case? Is there something else that can be excluded if I don't care about the forum portion?
Thanks in advance!
Linked from:
- KajMagnus @KajMagnus2018-05-08 05:08:56.263Z2018-05-08 05:22:24.580Z
(Sorry for the late reply. This was more tricky than the other questions :- ))
Your Nginx server run inside a Docker container or directly on the host?
I don't want to run any scripts/compose file offered
I think it'd be hard to do this in any other way, than using Talkyard's compose file &
.env
file and upgrade script. Talkyard's compose file uses a version number in the.env
file in the same directory. And there's a script, supposed to be called daily from Cron, that checks fo new versions, upgrades the version number, downloads new images & restarts. — If one (for example) copy-pastes thedocker-compose.yml
contents to another compose file ... then the version number in the.env
will thereafter be missing. Or if one copies the.env
too — then, auto-upgrade won't work (it assumes Talkyard is installed in/opt/talkyard
).If you have ideas / thoughts about how to do this, in a better way (& that doesn't break auto-upgrade) — that would be interesting to hear :- )
Could you [...] provide some guidance how to migrate to self-hosted comments in my case?
Ok. First, the only Talkyard container that can be removed, without breaking something, is the
certgen
container. (It's supposed to generate HTTPS certs, but it currently does nothing, not yet implemented.) If, however, you copy-paste the Talkyard Nginx config, to your own Nginx config files — then things will break, later when I make changes to the Nginx config, but your config files won't get auto-updated with those changes.Now. How to self-host. I think reverse-proxying all Talkyard stuff behind your Nginx, is the way to go. Here's how to reverse-proxy a Talkyard site, with the address
talkyard.yourblog.com
, behind a Nginx server on the host. (I haven't tested this myself though.)(First, note: If your Nginx is inside a Docker container, then, would be tricky to connect it to Talkyard — since Talkyard's containers are in a different docker-compose file on a different docker network. Probably you would instead need to add another Nginx server, directly on the host, then. )
You edit Talkyard's docker-compose.yml file and change the public ports from 80 and 443 (which would collide with Nginx on the host), to 8080 and 8443, like so:
web: ... ports: - '80:80' - '443:443'
change to:
ports: # This makes Talkyard's 'web' container listen on 8080 on the host, # and send to Nginx inside the container on port 80. - '8080:80' - '8443:443'
Then, you configuring Nginx on the host, to reverse-proxy requests to
talkyard.yourblog.com
, to ports 8080 and 8443. Something like this: (in your Nginx config)server { listen 443 ssl; listen [::]:443 ssl; server_name talkyard.yourblog.com; # SSL config ... # Reverse proxy to Talkyard: location / { proxy_pass http://talkyard.yourblog.com:8080/; proxy_redirect http://talkyard.yourblog.com:8080/ https://talkyard.yourblog.com; proxy_http_version 1.1; proxy_set_header Host $http_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_read_timeout 120; }
There's one thing I need to fix though. Talkyard's Nginx server in the Docker container, also does:
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; <--- oops
That means (I think) that the Talkyard Nginx server discards the
X-...
headers from your Nginx, so the Talkyard app server will (among other things) believe the request is insecure over HTTP, enven though it's HTTPS. This might make Talkyard's app server throw some security error. Seems I can fix that, as described here: https://serverfault.com/questions/515957/how-to-have-nginx-forward-the-http-x-forwarded-proto-headerAbout the containers
Could you explain what every container is needed
Here're the different containers and what they do:
- web = Nginx. Reverse-proxies the application server. Serves uploaded files (e.g. images) from disk, without passing through the app server. Websocket and Long polling, for real time things (like chat and notifications).
- app = The application server: Play Framework, running in the JVM.
- cache = In-memory cache: Redis.
- certgen = In the future, is supposed to generate HTTPS certs automatically, via Let'sEncrypt.
- rdb = Relational database: PostgreSQL.
- search = Full text search database: ElasticSearch.
Only for developers: (not included in the
talkyard-prod-one
repo, only in thetalkyard
repo)- gulp = Transpiles & minifies Javascript (well, Typescript) and CSS.
- app-dev = A development build of the application server. Includes non-minified JS and CSS.
- DDmitry Figol @dmfigol
Thanks for reply.
Sounds a little bit more complicated than I would love to. So probably I will try some other service :(
For your self-upgrade problem, I'd recommend taking a look on https://github.com/v2tec/watchtower
For automatic lets encrypt certs - you have two options:- set up cronjob on the server which will run certbot in webroot mode with proper webroot directory mounted or
- install certbot into the nginx image and run with --nginx. there should be a crontab installed with certbot installation that should be double-checked.
- KajMagnus @KajMagnus2018-06-08 03:11:16.834Z
Ok, and thanks for the ideas :- )
Watchtower. This project looks interesting. And also a bit dangerous, to me, if used by an app with many images and containers. Then, the containers might get upgraded independently of each other. Maybe the web server image, gets upgraded, and starts sending requests to new endpoints on the app server — slightly before the app server is done upgrading. That'd result in errors. (Having all images being compatible with the previous version of all other images = that can be problematic, in some cases ... and hard to test properly.)
I'd like Watchtower to upgrade all images at once, not one at a time independently of each other.
Another issue with W: Most images I push, are work-in-progress images, that people aren't supposed to use. (Only for me, on the servers I maintain.) Currently, the upgrade scripts looks in this file: https://github.com/debiki/talkyard-versions/blob/master/version-tags.log, upgrades to the latest version that is not a "WIP" or "test" or "beta" or something like that. ... Don't see any such functionality in Watchtower. (Creating different Docker image repos for "WIP", "tetst", "beta" etc — hmm, maybe ... or maybe there'd be a bit too many repos? hmm)
Anyway could be nice to use a standard thing like W, instead of the scripts. I asked the Watchtower developers: Upgrade all images & restart containers at the same time?.
Cron & Let'sEncrypt. Yes something like that is what I've been planning to do. I use certbot in webroot mode right now, on the servers I manage. I've installed certbot on the host.
And I've been thinking, maybe it's better to have certbot in a container instead (like you mentioned in 2.: "install certbot into the nginx image"). — I'd prefer it if Talkyard didn't install any apps on the host, except for Docker. Maybe someone uses a Let'sEncrypt tool other than Certbot? And wouldn't want certbot too, directly on the host?
In Talkyard's case, the tricky thing, with certs, is that there's SaaS hosting: The app server needs to automatically tell Certbot that a new site has been created at a custom domain, and Certbot then needs to automatically create & maintain a cert, for that new site. And eventually stop renewing it, if the domain gets changed again. — There needs to be a process, other than Cretbot, that talks with the app server and invokes Certbot. (Or maybe I could add Certbot in the app server image and call Certbot directly from Scala? Hmm)
- GJerry Koerkenmeier @gkoerk
Many of us are keen to host Talkyard behind a reverse-proxy such as Traefik, which automatically obtains/serves/renews SSL certs for all of the apps it proxies! Would it be possible for those who don't need certbot to use the individual containers together in a compose file? I know that unlike many, I'm interested in running Talkyard containers as services in a Docker Swarm stack (which uses a nearly identical compose.yml file), due to Swarm's magic with container maintenance and availability, etc. I'm willing to help in any way I can to get talkyard's docker-compose and .env ported to Docker Swarm if there's any interest by the Talkyard devs.
- KajMagnus @KajMagnus2018-10-23 07:30:44.387Z2018-10-23 08:17:48.219Z
Hi Jerry, sorry for the late reply, I was fixing bugs and preparing a new release. (I ought to get better at replying sooner, hmm this didn't take that long to write)
Someone is currently using Talkyard behind Caddyserver, which (I suspect you know already) is also written in Golang and automatically provisions Let'sEncrypt certs. I can ask him if he's interested in sharing his Docker-Compose file (I think he uses Compose, not Swarm).
Maybe his setup can serve as a starting point, for creating a Traefik + Talkyard Docker Swarm config file?
And someone else (Mustafa, @mcku in this topic) has created a Docker-Compose file that includes Talkyard and his other services too, in separate Docker networks.
I'll ask them, and get back to you.
- KajMagnus @KajMagnus2018-11-01 10:13:52.660Z
Hi again Jerry @gkoerk, I created a topic about Swarm etc + Talkyard, here: https://www.talkyard.io/-163/k8s-swarm-traefik-and-talkyard — it'd be interesting to hear if you have any thougts about that.
- In reply toKajMagnus⬆:MMustafa Kuscu @mcku
I have exactly the same motivation as @dmfigol. Luckily, I was able to migrate it to my app's docker-compose. However, the volumes in docker-compose was very confusing therefore I have switched to the long syntax for volume specifications. And upgraded the docker-compose file syntax to the most recent version, user defined networking etc all work fine, and will comply with the license and will describe any modifications I have done.
Aside, the build workflow is kind of split among sbt and bash and therefore was confusing to me, can a pure sbt approach simplify that? But the value of this project is so high therefore I will be happy to put effort to integrate it to my project.
Specifically, the modules folder has some stubs for external dependencies, but have to be managed manually if I understand correctly. Now I have issues like "
sanitizeHtml
" being missing in the prod bundle. Because it did not complain at compile time. When trying to create a community, I get:Something went wrong: [DwE500REX] java.lang.RuntimeException: Bad class: String [TyERCMR01] at com.debiki.core.Prelude$.die(Prelude.scala:136) at com.debiki.core.Prelude$.dieIf(Prelude.scala:150) at debiki.Nashorn.$anonfun$renderAndSanitizeCommonMark$3(Nashorn.scala:243) at debiki.Nashorn.withJavascriptEngine(Nashorn.scala:375) at debiki.Nashorn.renderAndSanitizeCommonMark(Nashorn.scala:232)
Still working on it ;)
- KajMagnus @KajMagnus2018-10-11 07:30:26.615Z
Hi Mustafa! Just a quick reply: I'll write more later.
1) I found an error message bug: The "
Bad class: String
" should be a completely different and more descriptive error message. 2) There's a Javascript gulp-watch bug: If Javascript sanitizer files are edited or added, Gulp won't notice until after restart (because I incorrectly didn't include those files in the watch file list), ... so maybe some changes you did, didn't have any effect.I'll push a fix to the
master
branch ... within 3 - 4 hours I would think.- KajMagnus @KajMagnus2018-10-11 15:32:01.333Z
Hi again @mcku! Thanks for describing the errors you encounter. I've now pushed a bugfix to the
master
branch, so instead of "Bad class: String" there'll be a more descriptive error. You'll find it (after havinggit merge --ff-only master
) in the app server's log file if you search for the error code "TyERCMEX
". View the logs like so:sudo s/d-logs app
or open./docker/data/app-logs/talkyard-app.log
in dev builds And in prod builds, you should see in Docker-Compose where the app server's log file gets mounted.... Hmm the error happens only in prod builds? Did you run
sudo s/d-gulp release
? That should happen automatically in the release scripts/build-and-release.sh
which callss/impl/release-as-root.sh
. I don't think anyone has attempted to run these script before except for I.What about posting the Docker-Compose file in a new topic here (with any private things removed), + the step by step commands you run to build everything? And send me build log output messages from the app container in a private message?
B.t.w. to have the server reload the server side Javascript files, in dev mode, you need to:
sudo s/d-restart-web-app
. (Client side Javascript gets auto reloaded, on browser refresh. However the server side script engines, which caches the server side scripts on server startup, won't get recreated and reload the scripts, until the server is restarted. Play Framework's internal reload isn't enough, right now)
Aside, the build workflow is kind of split among sbt and bash and therefore was confusing to me, can a pure sbt approach simplify that? But the value of this project is so high therefore I will be happy to put effort to integrate it to my project.
I was originally using SBT more — Play Framework has its own assets compilation pipeline (maybe you know about that? I'm getting the impression you have used SBT before? maybe a lot?). I fairly soon stopped using Play for all that though, because to me everything then got confusing, and I wanted to customize the build in ways that was simple with Gulp etc, and ... hmm, confusing to me, with SBT. And all instructions on the Internet (when I found a new Javascript lib I needed), were assuming one was using something like Gulp or Grunt or Webpack or things like that.
Anyway yes I think it's a bit unfortunate to have to understand the SBT config, and the Gulpfile for Javascript, ... and also how to start SBT and Gulp with various Bash + Docker commands. If you have more thoughts about how everything can be simplified, it'd be interesting to hear :- )
Yes it'd be interesting to see the changes you did in the Docker-Compose file. (Did you configure your own network?)
The reason some dependencies are maintained manually, is that there isn't, or wasn't, any Javascript npmjs module for those dependenies. Or that the Javascript npmjs library assumes a certain module system (like RequireJS or AMD or ES6), and an UMD-like build was available only directly in the Git repo. Or that I did some changes to the source code — which is the case with sanitizeHtml; by default, it excludes all? most? HTML5 tags for example. — Otherwise I try to use just
yarn add ....
for everything :- )- MMustafa Kuscu @mcku
Thanks for the explanations. That's fine. I will post a new topic on the docker-compose and send you a message. I had taken down some notes but all in Turkish :)
BTW, I would like to add Turkish translations hopefully, once i have more time for it.
Regarding Play pipeline, I went through that path which was as you described. But finally it was possible to do webpack and docker from with sbt, with the help of a typescript-react-webpack starter, if i remember correctly, this one: https://github.com/digital-wonderland/play-webpack-typescript-react
Of course I have broken and still occasionally break my head working with SBT. But I have kind of obsessive relationship with that and i like it.
- In reply tomcku⬆:GJerry Koerkenmeier @gkoerk
Mustafa,
Would you be willing to share your docker-compose.yml with me?
Jerry
- KajMagnus @KajMagnus2018-11-06 03:46:21.032Z
It's in this topic: https://www.talkyard.io/-153/upgrade-docker-compose-file-draft — however I think Mustafa @mcku did fairly many changes to the build process, namely, moved more build steps to SBT (a build tool for Scala), so Jerry if you try to use that Compose file, I think it won't work in your case.
- MMustafa Kuscu @mcku
Hi @gkoerk, if you have any questions that is fine and you are welcome.
My configuration has a reverse gateway (openresty) and that directs Talkyard traffic to Talkyard's nginx, which acts as a web server for the frontend app and its assets, and a firewall. I have not integrated nginx nchan yet, therefore websocket updates do not work as of yet. I don't know much about Træfik, but the concepts should be similar.
And as @KajMagnus mentioned, it would be a wise choice to stick with the official build tool. What I did was experimental and cannot be supported independently. But if you need more details on that because of curiosity, I understand and will try to provide with info.
Regards,
- In reply toKajMagnus⬆:H@Happyfeet01
Nice Guide for an Nginx Reverse Proxy in Host.
I try it also, but can only access to Talkyard over http://Forum.domain.tld:9001Is it also possible to import the letsencrypt certs into Talkyard?
- In reply toKajMagnus⬆:H@Happyfeet01
I also tried to deploy talkyard with Nginx Reverse Proxy.
The web container gives me an error:
nginx: [alert] detected a LuaJIT version which is not OpenResty's; many optimizations will be disabled and performance will be compromised (see https://github.com/openresty/luajit2 for OpenResty's LuaJIT or, even better, consider using the OpenResty releases from https://openresty.org/en/download.html)
Can anyone help me?
- KajMagnus @KajMagnus2020-05-13 09:57:35.869Z
It's harmless — it's because Talkyard builds Nginx + Lua itself, instead of using OpenResty. Maybe, like the message says, some optimizations are disabled, but Nginx is still faster than fast enough.
Here's a comment about that message, in Talkyard's source:
https://github.com/debiki/talkyard/blob/9204455b17b52260f1a95ae6ca640543173dc1ef/images/web/nginx.conf#L85I'm about to switch to OpenResty now soon. That'll remove that alert message, + will also make it possible to use OpenResty Lua plugins, e.g. for automatic LetsEncrypt certs.
- H@Happyfeet01
I got an blank page when accessing http://ipadress:9015
- KajMagnus @KajMagnus2020-05-14 09:41:27.897Z
What listens on port 9015, protocol
http:
? (Talkyard's Nginx server by default listens on port 80 and 443)If Talkyard listens on 9015 — what if you have a look in the Dev Tools console? Or in the Nginx or Talkyard logs? (
/var/log/nginx/
error.log
andaccess.log
, and/var/log/talkyard/talkyard-app.log
)- H@Happyfeet01
I have edited the Nginx and got this
404 Not Found There is no site with hostname 'talk.dasnetzundich.de' [TyE404HOSTNAME]
When I have access to my pc I look at the server for the Hostname.
- KajMagnus @KajMagnus2020-05-14 11:46:51.322Z
Edit
/opt/talkyard/conf/play-framework.conf
, thetalkyard.hostname
line so it becomes:# Required settings # --------------------- # Fill in your email address ... # Fill in your website address ... # ... talkyard.hostname="talk.dasnetzundich.de" <——— look
Then restart the app server:
cd /opt/talkyard/ docker-compose restart app # might take 20 seconds
The error message:
no site with hostname ... [TyE404HOSTNAME]
is from Talkyard's app (application) server. Seems the Nginx config is ok, since Nginx reaches the app server — but the app server config needs to be edited too (as shown above).- H@Happyfeet01
It works! 🤗👍🎉
- KajMagnus @KajMagnus2020-05-15 02:55:25.504Z
Ok :- )
- Progress