Blog Taxonomy

Posts tagged with 'docker'

Creating a Docker container to run as a command

kinow @ Aug 04, 2018 22:00:36

For the past two weeks at work I have been assigned to work on PHP projects. Though I used PHP some time ago - especially with Code Igniter and Laravel - I have not used it in a few years. And have been doing mostly Java nowadays.

The complete project setup was done by co-workers. I had a PHP project, using Symfony, several bundles and libraries, and Postgres. But it required just running a few commands to set up AWS settings, and then fire up Docker Compose.

Besides Docker Compose starting a web container, and another Postgres container, we used the web container to run Composer. In the past, I would see the PHP project mapped as a folder in the running container, and then composer would be executed in the host.

I noticed then that I could do the same to an image I created in 2016 to run Cylc. The previous version would be used to start a container in a terminal, then run some commands while maintaining the state within the container.

The new version now maps a volume to the version of cylc, installs the dependencies, and then allows the state to be maintained solely in the mapped volume.

Furthermore, the entrypoint of the image is the cylc command. Which means that you do not have to execute a terminal in order to run commands to the container (or change the command/entrypoint).

...
VOLUME "/opt/cylc" "/tmp" "/run" "/var/run" # we have the state stored only in /opt/cylc
WORKDIR "/opt/cylc"
ENV PATH /opt/cylc/bin:$PATH
WORKDIR /opt/cylc
ENTRYPOINT ["cylc"] # here's the trick!

It uses the same approach from the PHP image I am using for Composer, with a few additional improvements from the Best practices for writing Dockerfiles documentation.

$ docker run -ti -v "$(pwd -P)"/cylc:/opt/cylc cylc validate /opt/cylc/etc/examples/tutorial/oneoff/basic/
Valid for cylc-7.7.2
$ docker run -ti -v "$(pwd -P)"/cylc:/opt/cylc cylc register /opt/cylc/etc/examples/tutorial/oneoff/basic/
REGISTER /opt/cylc/etc/examples/tutorial/oneoff/basic/
$ docker run -ti -v "$(pwd -P)"/cylc:/opt/cylc cylc start --non-daemon --debug /opt/cylc/etc/examples/tutorial/oneoff/basic/
            ._.                                                       
            | |                 The Cylc Suite Engine [7.7.2]         
._____._. ._| |_____.           Copyright (C) 2008-2018 NIWA          
| .___| | | | | .___|  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
| !___| !_! | | !___.  This program comes with ABSOLUTELY NO WARRANTY;
!_____!___. |_!_____!  see `cylc warranty`.  It is free software, you 
      .___! |           are welcome to redistribute it under certain  
      !_____!                conditions; see `cylc conditions`.       
2018-07-30T12:10:32Z INFO - Suite starting: server=3714d1a17999:43040 pid=1
2018-07-30T12:10:32Z INFO - Cylc version: 7.7.2
2018-07-30T12:10:32Z INFO - Run mode: live
2018-07-30T12:10:32Z INFO - Initial point: 1
2018-07-30T12:10:32Z INFO - Final point: 1
2018-07-30T12:10:32Z INFO - Cold Start 1
2018-07-30T12:10:32Z DEBUG - [hello.1] -released to the task pool
2018-07-30T12:10:32Z DEBUG - BEGIN TASK PROCESSING
2018-07-30T12:10:32Z DEBUG - [hello.1] -waiting => queued
2018-07-30T12:10:32Z DEBUG - 1 task(s) de-queued
2018-07-30T12:10:32Z INFO - [hello.1] -submit-num=1, owner@host=localhost
2018-07-30T12:10:32Z DEBUG - [hello.1] -queued => ready
2018-07-30T12:10:32Z DEBUG - END TASK PROCESSING (took 0.00642895698547 seconds)
2018-07-30T12:10:32Z DEBUG - ['cylc', 'jobs-submit', '--debug', '--', '/opt/cylc/etc/examples/tutorial/oneoff/basic/log/job', '1/hello/01']
2018-07-30T12:10:33Z DEBUG - [client-connect] root@3714d1a17999:cylc-message privilege='full-control' 7b05596a-5971-4733-82de-28528f702ff0
2018-07-30T12:10:33Z INFO - [client-command] put_messages root@3714d1a17999:cylc-message 7b05596a-5971-4733-82de-28528f702ff0
2018-07-30T12:10:33Z DEBUG - [jobs-submit cmd] cylc jobs-submit --debug -- /opt/cylc/etc/examples/tutorial/oneoff/basic/log/job 1/hello/01
    [jobs-submit ret_code] 0
    [jobs-submit out]
    [TASK JOB SUMMARY]2018-07-30T12:10:32Z|1/hello/01|0|61
    [TASK JOB COMMAND]2018-07-30T12:10:32Z|1/hello/01|[STDOUT] 61
2018-07-30T12:10:33Z INFO - [hello.1] -(current:ready) submitted at 2018-07-30T12:10:32Z
2018-07-30T12:10:33Z INFO - [hello.1] -job[01] submitted to localhost:background[61]
2018-07-30T12:10:33Z DEBUG - [hello.1] -ready => submitted
2018-07-30T12:10:33Z INFO - [hello.1] -health check settings: submission timeout=None
2018-07-30T12:10:33Z DEBUG - BEGIN TASK PROCESSING
2018-07-30T12:10:33Z DEBUG - 0 task(s) de-queued
2018-07-30T12:10:33Z DEBUG - [hello.1] -forced spawning
2018-07-30T12:10:33Z DEBUG - END TASK PROCESSING (took 0.000992059707642 seconds)
2018-07-30T12:10:33Z INFO - [hello.1] -(current:submitted)> started at 2018-07-30T12:10:33Z
2018-07-30T12:10:33Z DEBUG - [hello.1] -submitted => running
2018-07-30T12:10:33Z INFO - [hello.1] -health check settings: execution timeout=None
2018-07-30T12:10:34Z DEBUG - BEGIN TASK PROCESSING
2018-07-30T12:10:34Z DEBUG - 0 task(s) de-queued
2018-07-30T12:10:34Z DEBUG - END TASK PROCESSING (took 0.000984191894531 seconds)
2018-07-30T12:10:43Z DEBUG - [client-connect] root@3714d1a17999:cylc-message privilege='full-control' 524d658e-9932-4135-9523-3c2ace3990cf
2018-07-30T12:10:43Z INFO - [client-command] put_messages root@3714d1a17999:cylc-message 524d658e-9932-4135-9523-3c2ace3990cf
2018-07-30T12:10:43Z INFO - [hello.1] -(current:running)> succeeded at 2018-07-30T12:10:43Z
2018-07-30T12:10:43Z DEBUG - [hello.1] -running => succeeded
2018-07-30T12:10:43Z INFO - Suite shutting down - AUTOMATIC
2018-07-30T12:10:44Z INFO - DONE
$

If you have a utility in your computer that requires a few dependencies, but that can store the state/data in a mapped volume, the same kind of container may be useful.

Order of containers in Docker Compose

kinow @ Mar 14, 2017 21:22:03

In Docker Compose you are able to control the startup order of the containers via the depends_on statement. This is documented in Controlling startup order in Compose.

If you have a simple setup, with Tomcat and Postgres, sometimes Postgres will start first, but Compose will initialize Tomcat before Postgres has fully booted. When that happens, you may receive 401, 404, or other application errors.

You can fix it by combining depends_on with a healthcheck. For example:

# File: docker-compose.yml
version: '2.1'
services:
  db:
    container_name: twpg
    build:
      context: .
      dockerfile: Dockerfile.postgres
    restart: always
    ports:
      - "5432:5432"
    healthcheck:
      test: "pg_isready -h localhost -p 5432 -q -U postgres"
      interval: 10s
      timeout: 5s
      retries: 5

  web:
    container_name: twtc
    build:
      context: .
      dockerfile: Dockerfile.tomcat
    restart: always
    depends_on:
      db:
        condition: service_healthy
    links:
      - db
    ports:
      - "80:8080"

In the example docker-compose.yml, there are two containers, db and web. web is running a Tomcat, and db is running Postgres. Web depends on db (see depends_on), and uses a condition service_healthy. Which indicates it depends that that container is healthy.

The healthcheck entry under the db container settings define how to check whether Postgres is running or not. In this case, we are using pg_isready, which is available in the vanilla Postgres 9 container.

It will try 5 times, with a 10 seconds interval, and will time out after 5 seconds. You may have to tune these parameters for your application.

This code snippet is from a pull request submitted to Foxoncz/docker-thingworx.

♥ Open Source

Trying SaltStack with Docker

kinow @ Apr 17, 2016 22:06:03

Some weeks ago I started learning SaltStack for a project at work. But I couldn’t find a good Docker image for that and I had to ask the Ops team for some VM’s. We are having a rainy weekend in Auckland, so I decided to have another look at the Jenkins SaltStack Plug-in.

But now since I was at home, I couldn’t use the VM’s that I had access to at work. So decided to look again at Docker or Vagrant images. After playing with a few images, I found bbinet/salt-master. It not only sets up a master, but also provides an easy way to enable the cherrypy API (necessary for the Jenkins plug-in).

This post describes the steps that I took to have a running Salt Master with the API enabled. First you need to create some directories and files to use with the image.

shell$ mkdir ~/master && cd ~/master
shell$ mkdir -p config/master.d/
shell$ vim config/master.d/api.conf

The api.conf contains the SaltStack API configuration. You can change port, user and other settings if necessary. Just remember to add a credential in Jenkins for the plug-in.

# File: api.conf
external_auth:
  pam:
    saltapiuser:
      - .*
      - '@runner'
      - '@wheel'
      - '@jobs'
rest_cherrypy:
  port: 8000
  host: 0.0.0.0
  disable_ssl: True
  static: /opt/molten
  static_path: /assets
  app: /opt/molten/index.html
  app_path: /molten

The image also conveniently provides a script that is executed before the entry point (if provided). So we can also create a user for the API automatically when the image is created.

shell$ vim config/before-exec.sh
#!/bin/bash
# File: before-exec.sh
useradd saltapiuser
echo -e "nosecret\nnosecret\n" | passwd saltapiuser
exit 0

Also make the script executable.

chmod +x config/before-exec.sh

And finally start the container.

docker run --name salt-master -v $PWD/config:/config \
    -p 4505:4505 -p 4506:4506 -p 443:443 -p 8000:8000 \
    bbinet/salt-master

Once the container is running, you can go to http://localhost:8000 and log in as saltapiuser:nosecret, and also configure your plug-in in Jenkins.

Happy hacking!

Deploying WAR files to Tomcat with Jenkins

kinow @ Mar 20, 2016 15:29:03

Table of Contents

A co-worker asked me this week about how to deploy a WAR file to Tomcat with Jenkins. In my team we are currently maintaining and deploying about 10 Java web systems, but we have no consistent way of deploying the applications to Tomcat yet. In the past I used Ant, Maven, Cargo, Grunt, and Jenkins, so I decided to write this short post to show a few different ways it can be achieved, à la Perl’s TMTOWTDI motto.

#1 Deploying with custom scripts

At first you may be tempted to write your own script to deploy to Tomcat with some Shell, Perl, Python or Java. But I think I would choose this option only because either I needed some feature that is not available in the other options, or in order to call other tasks or debug some problem.

Example:

$ docker run -d -p 8888:8080 jeanblanchard/tomcat:8
$ git clone https://github.com/spring-projects/spring-petclinic.git && cd spring-petclinic && mvn package
$ curl --upload-file target/petclinic.war "http://admin:admin@localhost:8888/manager/text/deploy?path=/spring-petclinic&update=true"
OK - Deployed application at context path /spring-petclinic

( Read more ... )