How do you want to build and upgrade, yourself?

2018-10-23 04:00:37.892Z2018-10-23 04:07:19.846Z

For now, to build your own Talkyard images, you can do as described in: docs/testing-images-in-vagrant.md, however, the build script is nowadays called s/build-and-release.sh (not just s/release.sh). The testing-images-in-vagrant.md doc doesn't, however, describe any good way to deploy the images in production.

To which Docker registry would you want to push images you build yourself? Would you setup a Docker registry on your intranet? (Here's Docker's docs about how to do that.). Or you'd use Docker's official registry, or something else? E.g. Google has a Docker registry. (Maybe I don't need to know; I'm just curious :- ))

Would you want your Talkyard server to upgrade automatically once you have run s/build-and-release.sh to build new Docker images and push to the registry? Or you'd like the upgrade-and-server-restart, to be a separate step that you maybe do the next day "manually"?

Currently the automatic upgrades works like so: there's this script that polls a Git repository (once per day), which contains a file with latest version numbers. And when a new version is detected, Docker images with that version number in the Docker image tag, are downloaded, the server restarts, and thereafter uses the new images. (What do you think about that?)

Reply2 Likes

4 replies

A
Adrián López @adrianlzt
2018-10-24 10:53:40.835Z
I'm quite confused with this instructions.

I was thinking that doing this:

git clone https://github.com/debiki/talkyard.git cd talkyard git submodule init git submodule update docker-compose build

I will get a production-ready images and the app running.
But actually is more like a dev environment, right?

Maybe we could have a docker-compose-prod.yml that builds the images ready for production, without links to local paths and, maybe, using docker volumes, leaving the decision to map them to the user.

So, if I want a custom build, I will do:

git the code

modify it

run docker-compose -f docker-compose-prod.yml build to build the images

tag and move the images to my custom registry (or maybe I don't have one and use docker save)

modify the docker-compose-prod.yml to use my tagged images (and remove the build:)

run docker-compose -f docker-compose-prod.yml up in my production host

(btw, I tried to do the build with docker compose and get an error: app_1 | [error] sbt.librarymanagement.ResolveException: unresolved dependency: community.ed#ed-logging_2.12;0.0.2: not found)
Reply 1 Disagree 2 Likes
1. KajMagnus @KajMagnus
  core-dev
  support-team
  2018-10-25 14:35:39.287Z
  I like those steps for building.
  
  Right now I'm using this script: https://github.com/debiki/talkyard/blob/master/docker/build-app-prod.sh to build production app-server images. It copies as-few-files-as-possible into a directory, plus a Dockerfile, and then runs docker build. In this way, the prod image becomes smaller. I suppose it'd be better if this script was run from inside a Dockerfile, instead of the script itself calling docker build.
  
  There's also this: s/d-gulp release to build a release version of the Typescript code, and this: s/d-cli test dist, to build a release build of the Scala code. — This could also be done from inside an appserver production Dockerfile I suppose. (Currently done here, in s/build-and-release.sh, before calling build-app-prod.sh )
  
  (Maybe there could be a help script, for tagging all images, and pushing to one's registry.)
  
  What about running tests? After step 2, "modify it", there could be a "run unit tests and end-to-end tests" step? And after step 3, "run docker-compose build", there could be a step to test all images: to run e2e tests again, and performance and security tests? I suppose [how important tests are], depends on what type of changes you're doing. If you just edit those email things, then ... maybe no need to run all tests.
  
  The unresolved dependency error — you'll find info about that, here, step 3: https://github.com/debiki/talkyard#the-instructions
  
  3. Compile and SBT publishLocal a logging library. (Search for [7SBMAQ2P] in this Git repo, to find out why.) sudo s/d-cli project edLogging publishLocal # then CTRL+D to exit
  
  I think I'll try to remove this edLogging module — it's needed only for log messages formatting in dev mode (because of how Play Framework's class loader works, in dev mode, with auto-reload on changes), but now I'm thinking it's more important to make the build process simpler, ... and that it's fine if log messages in dev builds look different.
  
  If I may ask, what changes do you have in mind to do, currently? Is it the email things?
  
  (Sorry about the weird Like and Disagree votes. Someone was having fun and posting weird comments and votes. I should implement delete-this-person's-votes functionality, for staff.)
  Reply
Progress

KajMagnus @KajMagnus

2018-10-31 10:24:03.377Z2018-10-31 14:45:19.596Z

@adrianlzt I've now removed the ed-logging build step that caused the app_1 | [error] sbt.librarymanagement.ResolveException: unresolved dependency error. (And now the log messages in dev builds, are a bit different from in prod builds.)

I think there have to be some separate steps, for compiling code. Something like:

git clone https://github.com/debiki/talkyard.git
cd talkyard
git submodule init --update

# Specify image tag (the VERSION_TAG value).
vi .env

# Compile and package code, using the default Docker-Compose.yml — it downloads
# build tools, e.g. Simple Build Tool for Scala, and Gulp for Javascript and CSS.
docker-compose run --rm sbt clean compile test dist
docker-compose run --rm gulp build-prod

# Build images, will make use of the packages built above.
docker-compose -f prod/docker-compose.yml build
docker-compose -f prod/docker-compose.yml push

But that's a bit complicated? What about this, with Make: (GNU Make)

git clone https://github.com/debiki/talkyard.git
cd talkyard
make build-images
make tag-and-push TAG=v1.2.3 REPO=your.docker.repo:5000

(And git submodule init --update, and other build steps, could be done by Make, as needed.)

(I'm going to read this article where someone "praises" Make + Docker + Compose.)

Reply 1 Like

Reply (discussion)Add progress note