Can Talkyard be self-hosted with Apache?
Where is information for the web server configuration? I am already using Apache2.
Thank you,
Dana
KajMagnus @KajMagnus2019-08-02 05:43:29.747ZYes that works fine — and you then need to configure Apache as a reverse proxy that forwards traffic to Talkyard. It's simpler to skip Apache and allow access directly to Talkyard though — then you won't need to read about how to configure Apache.
***
What's your sitution? You have a server with Apache, which listens on ports 80 and 443? (http and https) ... And you want to install Talkyard on the same server? Then you'd also need to configure Talkyard to use other ports than 80 and 443 (since Apache uses those ports). Is this what you want to / need to do? Look here:
1) There's this section in Talkyard's docker-compose file that you need to change:
services: web: image: ... ... ports: - '80:80' - '443:443'and 2) you also need to add a config value
talkyard.port=....to a fileplay-framework.conf. And 3) add a HTTPS cert to your Apache server.- DDana Johnson @dana
I changed docker-compose.yml
ports:
#- '80:80' #GG
#- '443:443' #GG
- '8080:8080' #GG
- '8443:8443' #GGand talkyard-servers.conf
server {
listen 8080 backlog=8192; # about backlog: see above [BACKLGSZ]server {
listen 8443 ssl backlog=8192; # [BACKLGSZ]and play-framework.conf
talkyard.port=8080 #GGdocker-compose ps still shows these ports
talkyard_web_1 /bin/sh -c /etc/nginx/run- ... Up 443/tcp, 80/tcp, 0.0.0.0:8080->8080/tcpI am able to access Talkyard directly through http:8080 but not https:8443
I already have an apache2 server running other websites. I added this to redirect from 80 and it is working:
ProxyRequests off
ProxyPass / http://ip:8080/
ProxyPassReverse / http://ip:8080/Also, is there documentation on the embedded block options?
Thank you,
Dana
KajMagnus @KajMagnus2019-08-14 06:46:48.686ZHmm 1) I think I shouldn't have mentioned
talkyard.port, sorry. If your Apache listens on port 80 or 443, and forwards to Talkyard, then one shouldn't configuretalkyard.portto anything. Instead, Talkyard will use port 80 or 443 by default, which is what one wants (since Apache listens on 80 and 443). Buttalkyard.port=8080will result in broken links, in reply notification email, when Apache listens to 80 or 443. — So, if your forum is going to work at an address likehttps://forum.example.comwithout a port number (i.e. not sth likehttps://forum.example.com:1234), then, removetalkyard.port.2) Actually you shouldn't need to reconfigure Nginx. Instead, you can do like this, and continue using ports 80 and 443 inside the Docker stack:
- '8080:80' # instead of '8080:8080' - '8443:443'... and only remap the Docker stack "external" ports to 8080 and 8443. — Then you don't need to use port 8080 in talkyard-servers.conf. But the way you did this, should also work fine.
3)
I am able to access Talkyard directly through http:8080 but not https:8443
I think 8443 isn't needed. You can add a HTTPS cert to Apache instead? For Apache's port 443? And then send plain HTTP traffic (decrypted HTTPS) to the Talkyard stack, port 8080.
Concerning this:
server { listen 8443 ssl backlog=8192; # [BACKLGSZ]Did you also generate a HTTPS cert via LetsEncrypt, and add the related directives? (if you're unsure, most likely you didn't.) Otherwise that's not going to work. But as mentioned, I think it's simpler if you let Talkyard's Nginx server (the one you configure in
talkyard-servers.conf) listen to port 80 (or 8080) only, but not 8443. And have Apache do HTTPS offload instead (that is, you install a cert in Apache, which then sends plain HTTP traffic to Talkyard, as mentioned above.)If you've configured a HTTPS cert for Apache, you should set
talkyard.secure=truein play-framework.conf, so Talkyard will generate https links in reply notification emails.Also, is there documentation on the embedded block options?
What do you have in mind with "embedded blocks"? Is that embedded comments / embedded categories? Or things to embed in a post, e.g. syntax for embedding a youtube video?
- DDana Johnson @dana
Ok, I seem to be really close now. I did have ssl set up under docker but I removed that. Now apache handles all of the ssl and passes to :8080.
Currently (in various config files listed above):
- '8080:80' talkyard.secure=true server { listen 8080;# backlog=8192; # about backlog: see above [BACKLGSZ]For apache:
Redirect / https://example.com/and:
SSLProxyEngine on ProxyPreserveHost On ProxyRequests off ProxyPass / http://example.com:8080/ ProxyPassReverse / https://example.com/ RequestHeader set X-Forwarded-Proto "https" RequestHeader set X-Forwarded-Port "443"I can open with http://example.com or https://example.com or directly with http://example.com:8080 and it works great.
So I am down to the embedded comments. I want to create a category for my Ghost blog and various topics and have a section added to a post to open a particular topic. Perhaps this isn't how it is supposed to work....
In order to do this, I thought I could specify the data-discussion-id based on the URL I was seeing in Talkyard. So I've tried variations on this. It comes up but not tied to anything but the links do open my Talkyard site.
<section class="post-full-comments"> <script>talkyardServerUrl='https://example.com';</script> <script async defer src="https://example.com/-/talkyard-comments.min.js"></script> <!-- You can specify a per page discussion id on the next line, if your URLs might change. --> <div class="talkyard-comments" data-discussion-id="-11" style="margin-top: 45px;"> <noscript>Please enable Javascript to view comments.</noscript> <p style="margin-top: 25px; opacity: 0.9; font-size: 96%">Comments powered by<a href="https://www.talkyard.io"> Talkyard</a>.</p> </div> </section>Once again, any hints would be appreciated.
Thank you,
Dana- KajMagnus @KajMagnus2019-08-19 05:52:34.609Z
Hmm you have
server { listen 8080; ...combined with- '8080:80'? That shouldn't work — I think if the Web container gets deleted and restarted, things will break, e.g. if the Talkyard server auto upgrades itself ...... because after a recreation, Docker will send incoming traffic on 8080 to the Web container's port 80 — but Nginx in the Web container listens on 8080. Then probably you'll get a request timeout. I think you should change
listen 8080;tolisten 80;(or change- '8080:80'to- '8080:8080'). I think you'll need to rundocker-compose kill web ; docker-compose rm web, and then
docker-compose up -d ; docker-compose logs -f
for these changes to take effect.... Could be good to do now, so you'll know that things work ... instead of something breaking, later when the server auto upgrades itself and recreates the containers.
***
It comes up but not tied to anything
A new version of Talkyard I have in mind to release later today, will associate blog comments, with the URL path to the blog post page, e.g.
/2019-08-19/blog-post-title. So, unless one moves the blog post to a different URL path, the comments should stay "glued" to the blog post "forever" (also if one gets a new domain name).***
About the page id in the Ghost blog comments section: You can use the Talkyard page id, however, the dash
-token should be exclued (it's not part of the id) and the html attribute for this, isdata-talkyard-page-id=...instead ofdata-discussion-id=....However Ghost has its own internal comments id that they recommend you to use. The html snippet you posted is in a Ghost template file, right? Then you can insert Ghost's own comments id like so:
<div class="talkyard-comments" data-discussion-id="ghost-{{comment_id}}" style="margin-top: 45px;">and you won't have to think about Talkyard's internal page ids.
(Or is there some specific reason why you had in mind to use Talkyard's page ids? No one has tried that before :- ))
- DDana Johnson @dana
I had gone through many iterations so I am glad to get it straightened out now while I have some memory. I changed to:
listen 80;I ran:
docker-compose kill web ; docker-compose rm web docker-compose up -d ; docker-compose logs -fdocker-compose ps now shows the updated ports so it seems to be working, but the docker-compose rm web seemed to try to ask [yN] but skip past it. Not really sure if that step worked or not.
I updated my ghost link to use data-talkyard-page-id= and it seems to make the correct request but the answer is:
403 Forbidden Not an embedded comments page [EdE2F6UHY3]So page ids are not expected through the embedded comments process.
What I am trying to do is have general subjects with which each post can be associated. For example: tech, nutrition, exercise, etc. Each post would not have individual comments, instead there would be comments for nutrition and all posts on that subject would show the same comment thread.
This makes sense for me because I expect low volume and I didn't want comments to be isolated and lost.
I would also like all comments to be listed on the main page of the blog but I don't know how to make that happen yet (other than have multiple sections, one for each subject). I did see a long term plan to have an entire category listed which gave me the idea to make the blog comments a category and then have each subject be a topic. Then I would tie back to a topic for each subject and eventually put the whole category on the main page.
Category with topics on Talkyard:
Blog Comments Tech Nutrition ExerciseIf you have any thoughts, let me know. I will try just using specific discussion ids (ghost-tech) and put in sample comments and see where they show up in Talkyard. Maybe I can organize them after they are created...
Thank you,
Dana- DDana Johnson @dana
I'm now using the same discussion-id for all posts (data-discussion-id=blog-example) and the same blog comments appear on all pages. So that is working. When I posted the first comment and each additional comment, I'm getting an error message. The comments are posting and after clearing the error, everything displays perfectly.
When I post a comment, I'm getting this message:
Mixed Content: The page at 'https://example-blog.com/' was loaded over HTTPS, but requested an insecure XMLHttpRequest endpoint 'http://example.com/-/mark-as-seen?pageId=0'. This request has been blocked; the content must be served over HTTPS.This appears to be the the culprit in the header script "thePageJson"
"remoteOriginOrEmpty":"http://example.com"I think this is being generated by Talkyard but I'm not sure.
I've included the complete original header script but commented, you can see it when editing this comment.
Also found this in the log:
app_1 | TRACE s1: Page found in db cache, reusing: 11, render params: PageRenderParams(Medium,false,http://example.com,None,Some(1),None), db cache version: site: 2, page: 3 | app: 0.00.64, hash: lYvq6R9l9AOgFt010IQPO_8LEM0 current version: site: 2, page: 3 | app: 0.00.64, hash: lYvq6R9l9AOgFt010IQPO_8LEM0 [TyMREUSEDB] kvs: null- KajMagnus @KajMagnus2019-08-21 09:44:00.363Z
I ran:
docker-compose kill web ; docker-compose rm web
docker-compose up -d ; docker-compose logs -fThose are actually 4 separate Bash commands, separated by
;and newline. Sorry that I didn't clarify this. They can be run two at a time — but after the first two, then, you'll get a question like "Really delete the container? y / n", and then you need to press 'y'. If you copy-paste all four at the same time, probably the container won't get deleted. However it does get restarted (since it was killed) and, since things seem to work for you now, apparently that was enough.About:
data-talkyard-page-id=
403 Forbidden
Not an embedded comments page [EdE2F6UHY3]There're different types of pages, and the page currently must be of type "EmbeddedComments". Page ids for those pages, work fine, but not other pages e.g. those that you created manually. (They'll be of a different type, namely "Discussion", not intended for embeding. Hmm maybe I should re-think how this works?)
I'm now using the same discussion-id for all posts (data-discussion-id=blog-example) and the same blog comments appear on all pages
Yes that whas what I was going to suggest. When you use
data-discussion-id=..., then, pages of type "EmbeddedComments" will get created automatically and all will be fine.What I am trying to do is [...]
Thanks for explaining :- ) Now things start to make sense to me.
I would also like all comments to be listed on the main page of the blog
There is a recent comments RSS (well, Atom) feed, which you can use to showcase the most recent comments, on the blog. Right now, the links in the feed point back to the Talkyard blog comments site, but I'm going to make them point to your blog instead (which is probably what you want, so people stay at the blog).
eventually put the whole category on the main page
Does that mean you want to embed a whole category, on the blog? Rather than showing "only" the most recent comments?
***
The error you're seeing:
When I posted the first comment and each additional comment, I'm getting an error message. The comments are posting and after clearing the error, everything displays perfectly.
Mixed Content: The page at 'https://example-blog.com/' ....
... but requested an insecure XMLHttpRequest endpoint 'http://example.com/-/mark-as-seen....Looks as if the blog comments are served from an insecure HTTP domain, but your blog is served from a secure HTTPS domain. Then, the browser will show warnings / block requests, because secure HTTPS websites (your blog), should connect only to other also secure HTTPS things.
You've configured HTTPS in Apache? Then I think all you need to do, is to change
http://tohttps://in the Ghost config value that pont to the Talkyard blog, ... that'd be this:<script>talkyardServerUrl='https://your-talkayrd-comments-server-placed-behind-Apache.example.com';</script>- DDana Johnson @dana
The script in Ghost only refers to https:// otherwise it would always be complaining about mixed http/https. I just checked it again to confirm.
I'm only seeing this immediately after replying to a comment. The new reply is added correctly so the http value is appearing in the result of adding the reply.
- KajMagnus @KajMagnus2019-08-21 14:32:00.724Z
Hi again, can you private-message me a link to your blog? (click my username.) Or if it's public you can type the address here ... Then I can try to post test comments myself and investigate? (which you can delete later)
- DDana Johnson @dana
It's public, just haven't done anything other than work on structure so only system posts.
https://goblincat.com- DDana Johnson @dana
As I attempt to move from one area to the next, I've found an issue with email.
Ghost works with a fairly simple:
"mail": { "from": "Talk <user@example.com>", "transport": "SMTP", "options": { "port": 465, "host": "mail.example.com", "secureConnection": true, "auth": { "user": "user@example.com", "pass": "pwd123" } } },For Talkyard, I currently have:
talkyard.smtp.host="mail.example.com" talkyard.smtp.port="25" talkyard.smtp.requireStartTls=true talkyard.smtp.tlsPort="465" talkyard.smtp.connectWithTls=true talkyard.smtp.checkServerIdentity=false talkyard.smtp.insecureTrustAllHosts=true talkyard.smtp.user="user@example.com" talkyard.smtp.password="pwd123" talkyard.smtp.fromAddress="user@example.com"I've tried changing the arguments in several ways with no success.
Am I experiencing a lot more problems than most? I was not familiar with Docker when I started but my understanding is slowly increasing. If you tire of helping me work this out, just let me know.
Thank you,
Dana- KajMagnus @KajMagnus2019-08-22 04:04:49.849Z2019-08-22 04:17:37.932Z
About https://goblincat.com ( & with
http://) — the website doesn't load in my browser, not in Chrome nor in Firefox. This:telnet goblincat.com 80andGET /works though. Other websites and Ghost blogs load just fine. — I'm wondering if most people in fact cannot access the website? ... Or is it unavailable for certain countries / locations? Browser user agents? Some Apache plugin?***
About the mail config: Talkyard has been configured to both 1) use StartTLS on port 25, and also 2) to connect directly with TLS on port 465. But one can do only one of those things. (
requireStartTls=truemeans connecting over an unencrypted SMTP connection, and then upgrade to secure TLS (this is called StartTLS). WhilstconnectWithTls=truemeans connecting directly over an encrypted connection.)Looking at Ghost's config, I think you should comment out the lines with
port="25"andrequireStartTls=true,
and keeptlsPort="465"andtalkyard.smtp.connectWithTls=true. — Let me know if this works?Afterwards, you can try commenting out:
talkyard.smtp.checkServerIdentity=false talkyard.smtp.insecureTrustAllHosts=truefor better security. If this will work, depends on if the mail server has been configured correctly with a TLS certificate — which I would think is the case for all external email sending services you can sign up for.
(Notes to myself: Maybe I can add a comment in the config file that one should configure either
requireStartTlsorconnectWithTlsbut not both — that could be helpful for other people in the future. ... Hmm, also, there could be a message visible to admins somehow, about this. )***
Yes you're running into a bit more problems than others, and, also, you're doing an unusual thing, I mean: "have general subjects with which each post can be associated".
Getting these kind of questions is good for me — then I can improve the docs and the user interface and fix bugs :- )
- DDana Johnson @dana
Emails are getting through now:
talkyard.smtp.host="mail.example.com" #talkyard.smtp.port="25" talkyard.smtp.requireStartTls=false talkyard.smtp.tlsPort="465" talkyard.smtp.connectWithTls=true talkyard.smtp.checkServerIdentity=false talkyard.smtp.insecureTrustAllHosts=true talkyard.smtp.user="user@example.com" talkyard.smtp.password="pwd123" talkyard.smtp.fromAddress="user@example.com"An issue that popped up was that the Docker environment is a new network that is not listed as part of my network. I didn't change that, I refined the alias mysql table query. The email I was using for Talkyard is forwarded to my email address so it was not showing up as having permissions to itself (even though it had logged in with the username/password).
I also need to add a certificate to postfix to resolve the checkServerIdentity and insecureTruckAllHosts tests. Letsencrypt has taken away the price barrier to a more secure internet so I need to move forward.
I haven't had a chance to look at the external access issue, this was news to me so that you for helping me.
Thank you,
Dana- KajMagnus @KajMagnus2019-08-23 02:27:10.710Z2019-08-23 02:34:30.683Z
Ok, I'm glad it works. (Thanks for posting your updated config, good to know what worked.)
a new network that is not listed as part of my network
Talkyard creates its own Docker network, and assigns static ips to the various containers. Otherwise there's a risk that when a container restarts, Docker would give it a new ip, and the other containers then couldn't communicate with it.
(Unless they're also restarted so all things that have cached the old stale ip, picks up the new ip instead. Or unless one starts using something like Traefik ... A statc ip is simpler.)
So yes, there'll be a new Docker network. You can choose the IP range in the
.envfile — there's this variable:INTERNAL_NET_SUBNTET=172.26.0.0/25Maybe I can add a note about this in the installation instructions.
I refined the alias mysql table query
I don't understand that, maybe you could rephrase? :- )
mysql? That's for another app you're running on the same server? (Talkyard uses PostgreSQL)I also need to add a certificate to postfix
That's something I could mention that one needs to do, if one has one's own mail server — I'll add a comment in the config file.
Do you maybe have a link to instructions about how to generate a cert for one's mail server, with LetsEncrypt?
- DDana Johnson @dana
SSL was not open in the firewall but it is now. I only recently discovered Let's Encrypt and started using it. Blog is now visible (goblincat.com).
The mysql changes were in reference to email. I use postfix with a mysql database. I found that the older style configuration is deprecated and has changed to actual SQL statements for the query. I know SQL, so good for me.
I plan on adding a certificate to my email server but hasn't happened yet. I spent the weekend falling back to using my modems firewall and setting up a server to do DHCP. My pfSense box SG-2440 died and I've ordered a new one but it'll be a week. Not as fancy but it works for now.
I'm fine with a little subnet inside of Docker. I would say it helps refine my configurations since I have another test besides does it work for me and does it work outside the network. It just brought up a local tolerance for email addresses that I wasn't aware of before.