Talkyard setup in Docker Container

2020-01-06 16:18:28.521Z

Hello,

I tried to setup talkyard on my Server. I used the official documentation on GitHub. The Container starts and running fine, but I think I have a wrong Nginx setup.

I can access http://comments.dasnetzundich.de:9001 but not
https://comments.dasnetzundich.de

When I access the port directly, I can setup the first admin user. But the E-Mail is wrong. But I setup the same emailadress in conf/play-framework.conf

Can anyone help me?

Solved in post #16, click to view

Reply

20 replies

There are 20 replies. Estimated reading time: 26 minutes

KajMagnus @KajMagnus
core-dev
support-team
2020-01-08 16:18:31.940Z
I'll have a look tomorrow
Reply 1 Like
1. KajMagnus @KajMagnus
  core-dev
  support-team
  2020-01-10 17:21:57.121Z
  Sorry, will have to be tomorrow Saturday
  Reply 1 Like

In reply toHappyfeet01⬆:

KajMagnus @KajMagnus

2020-01-11 07:38:49.458Z

Hi again, you followed these instructions?: https://github.com/debiki/talkyard-prod-one
(but not these?: https://github.com/debiki/talkyard — that's for development only)

You use CloudFlare? I'm wondering if there's something going on with the CloudFlare config — maybe CloudFlare doesn't forward the traffic to the Talkyard server / to the correct address?

About CloudFlare and Talkayrd:

If you use CloudFlare, either 1) configure CloudFlare to send the traffic directly to Talkyard, bypassing CloudFlare, or 2) use Full SSL or Full SSL (Strict). But don't use Flexible SSL — that would result in a redirect loop (because Talkyard upgrades from http to https).

(this is from a change-server-address help dialog in the Talkyard admin area. Maybe this should be in the readme too, hmm.)

***

I'm surprised something replies on port 9001 — Talkyard doesn't listen on that port, and only exposes ports 80 an 443. Could 9001 be some CloudFlare thing?

Which Nginx config did you have in mind — is it for enabling HTTPS for Talkyard, or ... some other Nginx config?

From where does this port 9001 come :- )

It seems to me HTTPS works, ... and, it's CloudFlare's HTTPS, right?, the cname points to CloudFlare (104.27.152.104 is a CloudFlare ip):

$ curl -v -v https://comments.dasnetzundich.de/
*   Trying 104.27.152.104...
* Connected to comments.dasnetzundich.de (104.27.152.104) port 443 (#0)
...
* ALPN, offering http/1.1
* SSL connection using TLS1.2 ...
* 	 server certificate verification OK
* 	 server certificate status verification SKIPPED
* 	 common name: sni.cloudflaressl.com (matched)
* 	 server certificate expiration date OK
* 	 server certificate activation date OK
* 	 certificate public key: EC
* 	 certificate version: #3
* 	 subject: C=US,ST=CA,L=San Francisco,O=Cloudflare\, Inc.,CN=sni.cloudflaressl.com
* 	 start date: Mon, 09 Dec 2019 00:00:00 GMT
* 	 expire date: Fri, 09 Oct 2020 12:00:00 GMT
* 	 issuer: C=US,ST=CA,L=San Francisco,O=CloudFlare\, Inc.,CN=CloudFlare Inc ECC CA-2
* 	 compression: NULL
* ALPN, server accepted to use http/1.1
> GET / HTTP/1.1
> Host: comments.dasnetzundich.de
> User-Agent: curl/7.47.0
> Accept: */*
>

H @Happyfeet01

2020-01-11 08:51:39.080Z2020-01-11 09:04:22.110Z

Thanks for helping me.
i have set up with this Docker-compose.yml

web:
    image: ${DOCKER_REPOSITORY}/talkyard-web:${VERSION_TAG}
    # dockerfile: https://github.com/debiki/talkyard/blob/master/images/web/Dockerfile
    restart: always
    volumes:
      - ./conf/sites-enabled-manual/:/etc/nginx/sites-enabled-manual/:ro
      - ./data/sites-enabled-auto-gen/:/etc/nginx/sites-enabled-auto-gen/:ro
      - ./data/certbot/:/etc/certbot/:ro
      - ./data/certbot-challenges/.well-known/:/opt/nginx/html/.well-known/:ro
      - ./data/uploads/:/opt/talkyard/uploads/:ro
      # Mount here so standard monitoring tools looking for Nginx logs will work.
      - /var/log/nginx/:/var/log/nginx/
    ports:
      - '9001:80'
      - '4448:443'
    networks:
      internal_net:
        ipv4_address: ${INTERNAL_NET_WEB_IP}
    depends_on:
      - app
    #environment:
    #  X_PULL_KEY: '...'
    #  CDN_PULL_KEY: '...'
    # SECURITY COULD drop capabilities, see: http://rhelblog.redhat.com/2016/10/17/secure-your-containers-with-this-one-weird-trick/
    # Ask at Hacker News: which caps can I drop for an Nginx container? A JVM appserver?
    # Asked here about Nginx:
    #   https://stackoverflow.com/questions/43467670/which-capabilities-can-i-drop-in-a-docker-nginx-container
    # For all containers, not just 'web'.
    #cap_drop:
    #  - DAC_OVERRIDE
    #  ... many more?
`` 

i Think that is correct. I used this tutorial. https://github.com/debiki/talkyard-prod-one
Yes i use Cloudflare as DNS Provider, now you can access the ports directly.


Edit:

After change the Settings in Cloudflare i can connect to the Port and Setup Talkyard, but can only Access the port, but not with Nginx Reverse Proxy.

H @Happyfeet01

2020-01-11 09:18:35.470Z

Thats also strange.

sshadmin@larsmueller:/opt/talkyard$ sudo certbot certonly --dry-run --config-dir /opt/talkyard/data/certbot/ --email kontakt@dasnetzundich.de --webroot -w /opt/talkyard/data/certbot-challenges/ -d comments.dasnetzundich.de
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for comments.dasnetzundich.de
Using the webroot path /opt/talkyard/data/certbot-challenges for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. comments.dasnetzundich.de (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://comments.dasnetzundich.de/.well-known/acme-challenge/gwTdnBz2aeqGF3CYVPrTr85MXW0cqw6LhgTv96a3qNc [2a01:4f8:221:487::2]: "<html>\r\n<head><title>404 Not Found</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>404 Not Found</h1></center>\r\n<hr><center>"

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: comments.dasnetzundich.de
   Type:   unauthorized
   Detail: Invalid response from
   http://comments.dasnetzundich.de/.well-known/acme-challenge/gwTdnBz2aeqGF3CYVPrTr85MXW0cqw6LhgTv96a3qNc
   [2a01:4f8:221:487::2]: "<html>\r\n<head><title>404 Not
   Found</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>404
   Not Found</h1></center>\r\n<hr><center>"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.

KajMagnus @KajMagnus

2020-01-11 14:59:49.507Z2020-01-11 15:09:49.348Z

Now it seems to work better? I can load https://comments.dasnetzundich.de, https looks fine in Chrome and via cURL.

You need to set talkyard.secure=true , in /opt/talkyard/conf/play-framework.conf:

# Read in docs/setup-https.md about how to generate a HTTPS certificate.
# Once done, set this to true:
talkyard.secure=false     <—— here, change to true

Then, restart:

docker-compose restart app

Until then, assets like CSS and Javascript won't load.

Hmm, now I attempted to download the CSS via https: https://comments.dasnetzundich.de/-/assets/v0.6.51-WIP-1/styles-bundle.min.css
but I got a 404 Not Found from Nginx.

What if you posted the Nginx config? That should be /opt/talkyard/conf/sites-enabled-manual/talkyard-servers.conf There's nothing more involved except for CloudFlare and Talkyard? (no other reverse proxies?) What's the reason you use ports 9001 and 4448?
(Thanks for posting the Docker config.)

H@Happyfeet01
2020-01-11 16:45:39.478Z
Hi,

I use an other port because I use 80 and 443 for my Ghost CMS Blog. https://dasnetzundich.de
Cloudflare only is for DNS Lookup, nothing else.
Reply

In reply toKajMagnus⬆:

H @Happyfeet01

2020-01-11 16:52:05.938Z

Here is my config

## To enable HTTPS:
## In section HTTPS Server Nr 1 below, replace  forum.example.com  with your hostname## (at 3 places). And comment in that section.
##
## To redirect HTTP to HTTPS:
## Comment out the 'include /etc/nginx/...' lines in the HTTP server (not the HTTPS server).
## Comment in the 'return 302 ...' line.
##
## To add more HTTPS servers:
## Copy the HTTPS Server Nr 1 `server {...}` block to a Nr 2, and remove 'backlog=8192'
## from the listen directive in Nr 2 — otherwise there'll be a "duplicate listen options"
## Nginx error. (The backlog should be the same as net.core.somaxconn in /etc/sysctl.conf,
## namely 8192, set in /opt/talkyard/scripts/prepare-ubuntu.sh  [BACKLGSZ]
## — but one may specify this in only one place; that's why you need to remove it.)
##


## HTTP Server.
server {
  listen 80      backlog=8192;   # about backlog: see above [BACKLGSZ]
  # Using ipv6 here, can prevent Nginx from starting, if the host OS has disabled ipv6,
  # Nginx then won't start and says:
  #    [emerg] socket() [::]:80 failed (97: Address family not supported by protocol)  #listen [::]:80 backlog=8192;

  server_name _;

  ## To redirect to HTTPS, comment out these includes, and comment in "return 302 ..." below.
  include /etc/nginx/server-limits.conf;
  include /etc/nginx/server-locations.conf;

  ## Redirect from HTTP to HTTPS.
  ## Use temp redirect (302) not permanent (301) in case you'll need to revert to http for
  ## a short while some day in the future.
  # return 302 https://$http_host$request_uri;
}


## HTTPS Server Nr 1.
## Replace  forum.example.com  with your hostname.
#server {
#  listen 443      ssl backlog=8192;   # [BACKLGSZ]
#  listen [::]:443 ssl backlog=8192;

#  server_name comments.dasnetzundich.de;

#  ssl_certificate         /etc/certbot/live/comments.dasnetzundich.de/fullchain.pem;#  ssl_certificate_key     /etc/certbot/live/comments.dasnetzundich.de/privkey.pem;
#   ssl_certificate /etc/letsencrypt/live/comments.dasnetzundich.de/fullchain.pem;
#   ssl_certificate_key /etc/letsencrypt/live/comments.dasnetzundich.de/privkey.pem;
##   ssl_trusted_certificate /etc/letsencrypt/live/comments.dasnetzundich.de/chain.pem;

#  include /etc/nginx/server-ssl.conf;
#  include /etc/nginx/server-limits.conf;
#  include /etc/nginx/server-locations.conf;
#}

F
In reply toHappyfeet01⬆:
Ayla Fernandes @fernandes.ayla
2020-01-30 16:55:17.236Z
Hi!

There's a way to use behind a proxy reverse ? Since I can't use the port 433 or 80.

Thank you!
Reply
1. KajMagnus @KajMagnus
  core-dev
  support-team
  2020-01-31 17:55:03.677Z
  Hi, yes you can use any reverse proxy, like Nginx, Traefik, Caddy Server.
  
  What if you create a different topic here in the forum, and describe your setup a bit more, over there? Also it'd be interesting to hear what the reasons are ports 80 and 443 are unavailable in your case.
  Reply
Progress
KajMagnus @KajMagnus
core-dev
support-team
2020-01-12 02:22:24.876Z
Thanks for the info & config.

I use an other port [9001 /Magnus] because I use 80 and 443 for my Ghost CMS Blog

Am I understanding it correctly, that Ghost and Talkyard run on the same virtual machine / server, and Ghost listens to 80 and 443, and Talkyard to 9001 and 4448? And there's no reverse proxy in front of them? (except for CloudFlare)

1. HTTPS

If so, then that (the above) explains why it wasn't possible to configure https: LetsEncrypt wants to connect to your Talkyard server on port 80 and verify that it really controls the domain name (by looking at the contents of http://comments.dasnetzundich.de/.well-known/acme-challenge/...) — however, on port 80, LetsEncrypt instead gets a response from Ghost, which says it doesn't know about any challenge (Ghost replies 404 Not Found).

So, with Talkard listening on 9001 and 4448 (instead of 80 and 443), it's not possible to configure LetsEncrypt https (or at least not so easy).

(Note to myself about why LetsEncrypt requires access to the server on port 80, not only 443: https://community.letsencrypt.org/t/renew-certificate-using-https-port-443-or-alternative-port-eg-8000/66981/6 "[some sharing hosting providers] have circumstances where using HTTPS for verifications will allow one customer to satisfy a Let’s Encrypt challenge for another customer’s domain name" )

(Note 2 to tmyself about enabling LetsEncrypt HTTPS for Talkyard, when CloudFlare has already been activated: That's fine, see: https://support.cloudflare.com/hc/en-us/articles/214820528-Validating-a-Let-s-Encrypt-Certificate-on-a-Site-Already-Active-on-Cloudflare — CloudFlare forwards traffic on port 80 to the Talkyard server, so the web root challenge (which Talkyard uses) works.)

2. CloudFlare

When CloudFlare sees a request to http://comments.dasnetzundich.de, it needs to forward this to the Talkyard server — however, the request to CF arrives on port 80 or 443. I tried to find out if there's a way to configure CF to re-map these requests to ports 9001 and 4448? (Or maybe you did this already somehow?)

3. What you can do

I think you need to choose one of these:

disable HTTPS between CloudFlare and Talkyard, or

move Talkyard to a separate server, so it can listen on port 80, or

add a reverse proxy in front of Ghost and Talkyard, which listens on ports 80 and 443, and looks at each request's HOST header — if it's comments.dasnetzundich.de, then, it'd send the request to Talkyard (on port 9001 and 4448), otherwise to Ghost. I suppose you'd need to configure Ghost to listen on some ports other than 80 and 443 too (since the reverse proxy would listen on those ports).

Personally, I would have chosen alt 3 — then I could use HTTPS all the way from the browser to the Talkyard server, and I wouldn't need to pay for a 2nd Virtual Machine. However, this requires installing e.g. your own Nginx or something, and editing config files. (Hmm maybe I could find time to write instructions about how to do this)

What do you think?

B.t.w. your server, which operating system does it run? (I'd guess it's Ubuntu? Debian?)
Reply
H
@Happyfeet01
2020-01-12 06:02:59.558Zreplies toKajMagnus⬆:
Am I understanding it correctly, that Ghost and Talkyard run on the same virtual machine / server, and Ghost listens to 80 and 443, and Talkyard to 9001 and 4448? And there's no reverse proxy in front of them? (except for CloudFlare)

Only for my Ghost Blog and other Sites like Nextcloud, an selfhosted Pastebin, Collabora Office and many other Sites. Some in Docker, some installed on Host System.
It seems to work, when i look at the Site comments.dasnetzundich.de, but i must fix these Mixed Content Problem.

Is it possible to load the Certificates into the Container? like

volume:

/etc/letsencrypt/live/comments.dasnetzundich.de/:/etc/certbot:ro

BTW: i use Ubuntu 18.04 LTS on my Server
Reply
KajMagnus @KajMagnus
core-dev
support-team
2020-01-14 05:41:44.839Zreplies toHappyfeet01⬆:
Is it possible to load the Certificates into the Container? like

Yes, that's how things work already (albeit the cert would be located in /opt/talkyard/data/certbot/ not in /etc/letsencrypt/).

(Look at https://github.com/debiki/talkyard-prod-one/blob/master/docker-compose.yml#L25.)

However, LetsEncrypt still requires Talkyard to listen on port 80 not 9001. (Otherwise you cannot generate a cert.) Until Talkyard listens on port 80, you'll need to disable HTTPS between CloudFlare and the Talkyard server.

***

I don't know why this: https://comments.dasnetzundich.de/-/assets/v0.6.51-WIP-1/more-bundle.min.js
isn't found. I just installed Talkyard locally to verify that the url path is correct, and yes it is (this works for me: http://localhost/-/assets/v0.6.51-WIP-1/more-bundle.min.js ).

So I'm wondering if CloudFlare somehow forwards that request, to the wrong URL (or port).

Talkyard's Nginx server logs messages to: /var/log/nginx/access.log (on the host, not inside the container) and /var/log/nginx/error.log. Maybe you could private-message me those files?

***

You also need to set: talkyard.secure=true in /opt/talkyard/conf/play-framework.conf.
Reply
H
@Happyfeet01
2020-01-14 06:16:27.370Zreplies toKajMagnus⬆:
I try it again, when I install talkyard on a separate server.

I wrote again when it is installed
Reply
KajMagnus @KajMagnus
core-dev
support-team
2020-01-14 07:20:47.390Zreplies toHappyfeet01⬆:
Ok. Actually, looking at the things you self host on the same server — maybe you'll want to install even more other things in the future? I'm wondering if some of those will also want to listen on port 80 and 443. With that in mind, if you have time and think it seems like fun, it could be a good idea to learn how to configure Nginx as a reverse proxy. So anything that wants, can listen on 80 and 443, in the future.

B.t.w. the other services — you mentioned: "Nextcloud, an selfhosted Pastebin, Collabora Office and many other Sites" — what about them and https? You found a way to configure https for them, although I suppose they cannot listen on port 80 because Ghost has "taken" that port?
Reply
H
@Happyfeet01
2020-01-19 06:41:44.507Zreplies toKajMagnus⬆:
Okay installed on a seperate Server. But when i try to activate https. I get the Error Message that the certificate can notbe found. But the Certs are still present under the Cert location.
Reply
KajMagnus @KajMagnus
core-dev
support-team
2020-01-19 15:35:07.946Zreplies toHappyfeet01⬆:
What commands did you run to generate the cert? And from which directory?

I'm thinking maybe you followed these instructions:
https://github.com/debiki/talkyard-prod-one/blob/master/docs/setup-https.md

then, at which step did the error message appear?
Would you like to copy-paste the error message here?

when i try to activate https

Is that by editing the Nginx config files, and running nginx -t? or nginx -s reload?

the Certs are still present under the Cert location

What's the locatoin — is it /opt/talkyard/data/certbot/ or /etc/letsencrypt/ or something else?

Would you like to you post a tree directory listing of that location?
Send as a PM if it's private. Like so, in Bash: tree -a /path/to/certs/
Reply Solution
H
@Happyfeet01
2020-01-19 21:32:15.740Zreplies toKajMagnus⬆:
Okay,

All things working. The latest Problem was the slow server 2 vCPU and 2 GB RAM.

Now the forum works. Now I can export from hosted Version to self-hosted.

Export the json. I have read it -/export-site-json at the end of the URL. But how can I import?
Reply
KajMagnus @KajMagnus
core-dev
support-team
2020-01-19 23:12:38.263Zreplies toHappyfeet01⬆:
I replied here: https://www.talkyard.io/-285#post-6

Can I ask, which hosting provider do you use? Talkyard is a bit slow directly after startup, before Nashorn has gotten warmed up. (Nashorn is a Javascript engine that runs in the Java Virtual Machine, and it gets just-in-time compled to Java bytecode or somethng like that, and ... that happens when the first comments get posted, and, before that, Talkyard can be a bit slow.)
Reply
H
@Happyfeet01
2020-01-20 05:13:05.997Zreplies toKajMagnus⬆:
I use an Hetzner Cloud Server for Talkyard. https://www.hetzner.de/cloud

the first one. i resized it with 4 GB RAM and it works.
Reply

Reply (discussion)Add progress note

Talkyard setup in Docker Container

1. HTTPS

2. CloudFlare

3. What you can do