Saturday, August 12, 2017

Practices for building a docker image

The first thing docker build does is to send build context to Docker daemon, if the build context is big, it can take a long time – so long that you doubt if it hangs. This blog shares some practices I am doing to speed up building a docker image.

Speed up Vagrant

if you are hosting vagrant VMs on Windows, accessing sync folder will be slow. There is a plugin https://github.com/winnfsd/vagrant-winnfsd that adds NSF to Windows, unfortunately, it doesn’t work for me: my McAfee blocks it.

There are some configurations that can speed up Vagrant:

config.vm.provider "virtualbox" do |v|
               v.memory =2048
               v.cpus = "2"
                
               # change the network card hardware for better performance
               v.customize ["modifyvm", :id, "--nictype1", "virtio" ]
               v.customize ["modifyvm", :id, "--nictype2", "virtio" ]

               # suggested fix for slow network performance
               # see https://github.com/mitchellh/vagrant/issues/1807
               v.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
               v.customize ["modifyvm", :id, "--natdnsproxy1", "on"] 
end 
   

.dockerignore

Add .dockerignore in the build context, files defined in the file will be ignored by docker when sending build context to the docker daemon.

echo ./server/log > .dockerignore
echo ./server/tmp >> .dockerignore

Change files in a running container

After you spend a lot of time building the docker image, you may find that it doesn’t work out as you expected, perhaps some configuration files are not setup correctly, you change those files, and then you need to rebuild the image again, which is unbearable waste of time.

My approach is:
1)      Add a volume to the image, which will be used to hold files to be copied to the container
VOLUME ["/942-docker-volume"]
Build the image

2)      Run a container, in the run command, mount the volume, and switch the entrypoint command. In the replaced entrypoint command, copy files from the mounted volume to replace files in the container. 

sudo docker run -it --name pm942 -p 8080:8080  -v /942-docker-volume/home:/942-docker-volume/home --entrypoint /bin/sh pm942 -c "cp /942-docker-volume/home/*.conf /pm942/ && cp /942-docker-volume/home/bin/*.sh /pm942/bin/ && cd ./bin && ./Start.sh"

Notice the --entrypoint parameter, whose value is “/bin/sh”. Also notice the argument to “/bin/sh” is placed at the end, after the image name pm942. This --entrypoint command copies files from the mounted volume and replace files accordingly inside the container.

Or you can ignore the argument, and thus log into the container to manipulate it:

sudo docker run -it --name pm942 -p 8080:8080  -v /942-docker-volume/home:/942-docker-volume/home --entrypoint /bin/sh pm942



Docker Tutorial


Install docker on Ubuntu

Installation guild is in:

1.       
To install Docker CE, you need the 64-bit version of one of these Ubuntu versions:
  • Zesty 17.04
  •  Xenial 16.04 (LTS)
  •  Trusty 14.04 (LTS)

check OS version:
vagrant@exp:/devops/PPM/942$ cat /etc/*-release
 2.       
wget https://download.docker.com/linux/ubuntu/dists/xenial/pool/stable/amd64/docker-ce_17.06.0~ce-0~ubuntu_amd64.deb  --verbose -P /tmp

 3.   

If run into this issue:
dpkg: dependency problems prevent configuration of docker-ce:
 docker-ce depends on libltdl7 (>= 2.4.6); however:
  Package libltdl7 is not installed.

Solution:
sudo apt-get update
sudo apt-get install libltdl7

Setup proxy

When you download a docker image, and run into issue docker:
Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers).

This is because you are running docker behind a proxy, and need to setup proxy for docker

Solution:

sudo mkdir -p /etc/systemd/system/docker.service.d
 
      sudo vim /etc/systemd/system/docker.service.d/http-proxy.conf

[Service]
Environment="HTTP_PROXY=YOUR_PROXY" "HTTPS_PROXY=YOUR_PROXY" "http_proxy=YOUR_PORXY" "https_proxy=YOUR_PROXY" "no_proxy=localhost,192.168.33.11,127.0.0.1" "NO_PROXY=localhost,192.168.33.11,127.0.0.1"

Flush changes
sudo systemctl daemon-reload
 
Check changes are effective
systemctl show --property=Environment docker
 
Restart docker service
sudo systemctl restart docker

Run Tomcat

Choose an image:

docker run command:

Run tomcat:
sudo docker run -it --name mytomcat1 -p 8080:8080 tomcat:8.0


This may take a long time to download the image.

Check container status:
sudo docker ps -a
(-a check all containers, without it, will only check running containers)

Check image:
sudo docker image ls

Start/stop container:
sudo docker start mytomcat1
sudo docker stop mytomcat1
sudo docker restart mytomcat1

Check container logs:
sudo docker logs -f mytomcat1

Find out the process running inside docker:
ubuntu@exp1:~$ sudo docker top mytomcat1
UID                 PID                 PPID                C                   STIME               TTY                 TIME                CMD
root                5566                5549                0                   07:34               ?                   00:00:04            /docker-java-home/jre/bin/java -Djava.util.logging.config.file=/usr/local/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Djava.endorsed.dirs=/usr/local/tomcat/endorsed -classpath /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/usr/local/tomcat -Dcatalina.home=/usr/local/tomcat -Djava.io.tmpdir=/usr/local/tomcat/temp org.apache.catalina.startup.Bootstrap start

Go inside the container:  
sudo docker exec -it mytomcat1 /bin/sh
#pwd
(working directory from Dockerfile)

Find out the processing running inside docker:
# ps -fwww 1
UID        PID  PPID  C STIME TTY      STAT   TIME CMD
root         1     0  1 07:40 ?        Ssl    0:03 /docker-java-home/jre/bin/java -Djava.util.logging.config.file=/usr/local/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Djava.endorsed.dirs=/usr/local/tomcat/endorsed -classpath /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/usr/local/tomcat -Dcatalina.home=/usr/local/tomcat -Djava.io.tmpdir=/usr/local/tomcat/temp org.apache.catalina.startup.Bootstrap start

#cd logs
#touch abc.abc
#exit

Restart and go inside mytomcat1, logs/abc.abc is still there

Remove container:
sudo docker rm mytomcat1

Recreate container:
sudo docker run -it --name mytomcat1 -p 8080:8080 tomcat:8.0


Access tomcat

Run another tomcat container with volume:

sudo docker run -d --name mytomcat2 -p 8081:8080 -v /devops/mytomcat/logs:/usr/local/tomcat/logs tomcat:8.0

Check host folder /devops/mytomcat/logs, see it has tomcat logs

Share volumes among dockers:
sudo docker run -d --name mytomcat2 -p 8081:8080 -v /devops/mytomcat/logs:/usr/local/tomcat/logs tomcat:8.0

sudo docker run -d --name mytomcat3 -p 8082:8080 --volumes-from mytomcat2 tomcat:8.0

Debug Docker Daemon

Change debug to true:
sudo vim /etc/docker/daemon.json

{
  "debug": true,
}


Restart docker:
sudo systemctl restart docker

check docker log:
sudo journalctl -fu docker.service


Vagrant ssh prompts for password



By default, Vagrant’s user/password is vagrant/vagrant. This is now true with the box ubuntu/xenial64, whose default user is ubuntu. This is not an issue if you download the box from the public Vagrant server, you will run into issues if you try to add a packaged box based on ubuntu/xenial64.

vagrant package outputs package.box.

Now add it:
vagrant box add exp1 package.box

In Vagrantfile, use exp1 as the base box of the VM:  
config.vm.box ="exp1"

vagrant up fails with:

    default: SSH username: vagrant
    default: SSH auth method: private key
    default: Warning: Connection reset. Retrying...
    default: Warning: Connection aborted. Retrying...
    default: Warning: Connection reset. Retrying...
    default: Warning: Connection aborted. Retrying...
    default: Warning: Remote connection disconnect. Retrying...
    default: Warning: Connection aborted. Retrying...
    default: Warning: Connection reset. Retrying...
    default: Warning: Authentication failure. Retrying...
    default: Warning: Authentication failure. Retrying...
    default: Warning: Authentication failure. Retrying...
    default: Warning: Authentication failure. Retrying...
    default: Warning: Authentication failure. Retrying...
    default: Warning: Authentication failure. Retrying...
    default: Warning: Authentication failure. Retrying...
    default: Warning: Authentication failure. Retrying...
    default: Warning: Authentication failure. Retrying...
    default: Warning: Authentication failure. Retrying...
    default: Warning: Authentication failure. Retrying...
    default: Warning: Authentication failure. Retrying...
    default: Warning: Authentication failure. Retrying...
Timed out while waiting for the machine to boot. This means that
Vagrant was unable to communicate with the guest machine within
the configured ("config.vm.boot_timeout" value) time period.


This is because vagrant uses user vagrant, while ubuntu/xenial64 uses user ubuntu.  ubuntu/xenial64 user/password can be found in file C:\Users\admin\.vagrant.d\boxes\ubuntu-VAGRANTSLASH-xenial64\20170523.1.0\virtualbox\Vagrantfile:

  config.ssh.username = "ubuntu"
  config.ssh.password = "5605ab75c7b6cc6baa497b18"

Add these lines to Vagrantfile, vagrant up now succeeds.



This is just the first hurdle, vagrant ssh now runs into an issue.

$ vagrant ssh
==> default: The machine you're attempting to SSH into is configured to use
==> default: password-based authentication. Vagrant can't script entering the
==> default: password for you. If you're prompted for a password, please enter
==> default: the same password you have configured in the Vagrantfile.

vagrant ssh-config shows something weird, it doesn’t output IdentityFile, which normally should be like:
IdentityFile C:/devops/exp2/.vagrant/machines/default/virtualbox/private_key

$ vagrant ssh-config
  Host default
  HostName 127.0.0.1
  User ubuntu
  Port 2201
  UserKnownHostsFile /dev/null
  StrictHostKeyChecking no
  PasswordAuthentication no
  IdentitiesOnly yes
  LogLevel FATAL

The problem is in file C:\Users\admin\.vagrant.d\boxes\exp1\0\virtualbox\Vagrantfile:

Vagrant.configure("2") do |config|
config.ssh.private_key_path = File.expand_path("../vagrant_private_key", __FILE__)
end

Comment out config.ssh.private_key_path line, now vagrant ssh works.