Udemy Docker notes

I was doing a course on Docker at Udemy ( https://www.udemy.com/docker-mastery/ ), here are my notes about it.
Kudos to Brett Fisher (https://twitter.com/bretfisher) who has done an amazing job creating this course. Thanks!

http://labs.play-with-docker.com 

New command syntax

• docker container ls
• docker container ls -a
• docker container start
• …

What does docker container start actually do?

1. Looks for the image in the image cache
2. Then looks in remote hub (docker hub default)
3. Downloads latest version
4. Starts a new container based on that image
5. Gives it a virtual IP !!INSIDE!! the docker engine on a private network
6. Opens up port 80 on host and forwards to port 80 in container (with the --publish command)
7. Starts container using the CMD in the image Dockerfile

Containers aren’t really VM-s: THEY ARE JUST PROCESSES

CLI process monitoring

• docker container top: list running processes in a specific container
• docker container inspect [ID/NAME]: gets metadata for that container (like volume, config, etc..
• docker container stats: shows live performance measures for all the containers running

Getting a shell inside a container

• docker container run -it: start new container interactively (with CLI, e.g. with bash)
• docker continer exect -it: run additional command in existing container (no ssh needed!!)

-it means: interactive (keeps session open to keep terminal input), -t pseudo-tty (simulates a real terminal, like what ssh does)

Docker networks
-p: exposing ports
BUT: you don’t have to expose all the time, you can create sub virtual networks which are going to “understand” and “see” each other, so you can just define a communication without exposing ports.
docker container inspect --format '{{ .NetworkSettings.IPAddress }}' webhost: just the IP . NEAT
Communicating with two virtual networks (subnetworks) is only able to do by going out to the exposed ports to the outside and communicate over there.

Docker network CLI

• docker network ls: networks
• docker network inspect: get metadata from a network
• docker network create --driver: creates a network
• docker network connect / disconnect

With docker swarm this is easier to do!

Docker container DNS
Docker daemon has a built-in DNS server that container use by default.
Note: IP-s are not good way to communicate, use names instead! Apps can fail and get new IP address, but it can fall back to a name with a different ID still!

Container images
What’s in an image:

• App binaries and dependencies
• Metadata about the image data and how to run the image

Download an image from Docker Hub.

docker pull [OPTIONS] NAME[:TAG|@DIGEST]

Image layers

• Image layers:
• docker image history nginx


• history of the image layers
• basically an image layer is a new command / change to the previous one, e.g. like the base image layer is ubuntu, and then apt-get -ing something (like mysql), and that’s going to be the next image layer
  
• On the picture above you can see how caching is done: if 2 images are using jessie image layer, it is not going to be dupicated, both images will use the same jessie image (layer)
• Image layers are saved and basically attahed together for an image (so it can save space)
• docker image inspect nginx
• Image tagging and pushing to docker hub
• a very similar process like git
• Dockerfile
• instructions how to build our image
• FROM: which is the starting image
• ENV: set environment variable
• RUN: run commands
• EXPOSE: expose port  (you still have to run -p if you would like to expose this to the outside) - so basically I am ALLOWING the image to be exposed, but I still have to do it explicitly!
• CMD: run a command when container is run
• General best practice: keep the less changing things on the top of the Dockerfile and and the more changing at the bottom

Persistent Data

Volumes
Persistent data for images.
docker volume ls

Bind Mounting
Maps a host file or directory to a container file or directory.
Basically just two locations pointing to the same file(s).
CAN’T USE IN DOCKERFILE, MUST BE IN container run .
That’s really good for development - binding local files to the containers

Docker-compose
You can use sub containers to communicate with each other. Neat!
docker compose up - this is starting all the containers which are defined in docker-compose.yml file.
With docker-compose you can manage a multi-container environment easily.

Docker swarm

Server clustering solution.
Can orchestrate the lifecycle of your containers.
docker swarm init - enables swarm mode in an environment (so you can run your own swarm in your computer)
With the swarm API we are not directly communicating with containers, rather we are just communicating with the orchestration which will make sure the commands will be executed, and e.g. if a service dies, it will make sure it re-runs it!

Routing mesh
The routing mesh enables each node in the swarm to accept connections on published ports for any service running in the swarm, even if there’s no task running on the node. The routing mesh routes all incoming requests to published ports on available nodes to an active container.

Stacks

Basically compose files for swarms.
docker stack deploy
Docker compose ignores deploy and swarm ignores build. Separation of concerns.
docker-compose cli not needed on Swarm server!
1 stack → 1 swarm!

Secrets
Easiest “secure” solution for storing secrets in Swarm.
Secret:

• usernames, passwords
• TLS certificates and keys
• SSH keys
• Any data you would prefer not be “on front page of news”

SWARM & STACK & SECRETS are cool!

Writing your own container in Go

This artice is about creating a tool which is like Docker, but much more simplified.

First of all, why do we need containers?

I like to think about containers as separate little planets who live in a universe. There is a planet for database management, other one is for an operating system, etc.. They don't care about each other, however you can create communication between them via e.g. radio waves. This could be  the interface between them.
In my world, what is good about these planets is that you can copy the exact same environment which it had, but who is going to live on these planets is up to that specific instance. (E.g. you can achieve that with quantum entanglements - this is getting a bit too fantasy-like haha.)
Let's get back to our containers. With these you can assure that if it works on your computer, it is going to work on any of them in the world - in theory. Saying that you can easily automate your processes for deployment, make development easier and more productive.
So - long story short - containers are really important for modern development environments.

Ok. How can I create one?

It's not that complex! Really!

First of all, I would like to shout out to Liz Rice, whose talk I saw at Craft Conference in Budapest few months ago. (similar video - twitter)

First, you have to understand two core features: control groups (cgroups) and namespaces.
So what are namespaces?
Well, in it's fundamental, namespace is what a user can see of it's environment: process id-s, file system, users, networking, hostname, etc.. And it's all yours, noone else's!
Ok now, let's see what cgroups are!
If we are sticking to the analogy before, cgroups are what you can use, like CPU, memory, disk I/O, etc.. So basically speaking about resources.
And now let's jump into coding!

As you can see it's not that big of a source code. Only 56 lines!
The basic idea is that we are going to run a system call inside a system call.  At the main function it is going to jump into the run function since first we passed the "run" parameter. (go run main.go run)
Basically what os.exec does is wrapping external commands so they can run in their own namespace. Uhm, excuse me? I have heard about this word before.
Well, no surprise, with this line of code we are going to achieve our own namespace, and we are going to have our own process id-s. Great!
You can see here that we are passing the new argument "child" which means it is going to call the child function.
The next interesting part is the 33rd line where we are passing a bunch of flags for the newly started process, these are:

• CLONE_NEWUTS is for namespace
• CLONE_NEWPID is for new process id-s
• CLONE_NEWNS - this means unshare the mount namespace, so that the calling process has a private copy of its namespace which is not shared with any other process.

Neat! So now we now how we want to run our child process which has it's own namespace and cgroup, but it's still not working in it's own "world". We have to give it a filesystem which behaves like an ubuntu filesystem would work.

So that's where one another trick comes in: we have to have a linux root filesystem which will be the playground of our container. You can learn about the restrictions for the root filesystem here: http://www.tldp.org/LDP/sag/html/root-fs.html .

With this in our toolbelt we can change the root of our child execution to that particular root filesystem directory (46th line), and with the 48th line we are mounting /proc because it's a special kind of directory, and THAT'S IT, if you run go main.go run /bin/bash you will have your own little planet which is completely separate (well, now you know, it's not that separate, but it's acting like it is ;) ).

Enjoy and feel free to play around if you like these kinds of little aspects of modern software development / containerization!