Changes

From SME Server
Jump to navigationJump to search

Docker (view source)

Revision as of 14:56, 20 October 2020

10,569 bytes added ,  14:56, 20 October 2020
→‎Repo setup
Line 1: Line 1:  
{{WIP box}}
 
{{WIP box}}
 
Placeholder for anything to do with Docker (https://docker.com)
 
Placeholder for anything to do with Docker (https://docker.com)
−
{{Note box| All info is based on SME Server 9 only.
+
{{Note box| All info is based on SME Server 9 only but I have just started to test on v10 - see notes below
 
Your host server must have internet access.
 
Your host server must have internet access.
   −
All testing below has been done on a virtual SME9 under Virtualbox. Yes, Docker also run on VM's :-)
+
Some testing below was originally done on a virtual SME9 under Virtualbox. Yes, Docker also run on VM's :-)
 +
 
 +
The contrib has been built from the original notes and I use it to permanently run Rocketchat.
 +
}}
 +
 
 +
'''You can discuss all things related to this page on the forums [http://forums.contribs.org/index.php/topic,51181.0.html here]'''
 +
 
 +
'''There is a separate page that addresses the design of a Docker contrib [http://wiki.contribs.org/Docker_design_concept here]'''
 +
 
 +
'''There is also a page to discuss on how to create a Docker image of SME [https://wiki.contribs.org/Docker_Image_of_SME here]'''
   −
By NO means this is an official thing or even applicable at all. Tinkering phase only here!}}
   
==About==
 
==About==
 
[[File:Docker_logo.png]]
 
[[File:Docker_logo.png]]
Line 14: Line 22:     
===Why Docker on SME Server?===
 
===Why Docker on SME Server?===
−
Docker containers hold one or more applications (and all it's dependecies) and can be started and stopped at will. The containers, when activated, use the Linux kernel namespaces and are operating isolated from the rest of your server, except for storage/mount points and networking, depending on the configuration of the container. Some applications require special PHP versions or other modifications to your server settings that are not desirable and may effect yum updates and upgrades. Docker containers is a way to have such an application packed with all it's dependencies and run it isolated. You can have multiple containers running, depending on your server hardware capacity.
+
Docker containers hold one or more applications (and all it's dependencies) and can be started and stopped at will. The containers, when activated, use the Linux kernel namespaces and are operating isolated from the rest of your server, except for storage/mount points and networking, depending on the configuration of the container. Some applications require special PHP versions or other modifications to your server settings that are not desirable and may effect yum updates and upgrades. Docker containers is a way to have such an application packed with all it's dependencies and run it isolated. You can have multiple containers running, depending on your server hardware capacity.
    
Examples:
 
Examples:
Line 38: Line 46:  
  sharedfolders-2.1.1-smeserver-9.0-fws
 
  sharedfolders-2.1.1-smeserver-9.0-fws
 
  frontaccounting-3.2.1-smeserver-8.1-contribsorg
 
  frontaccounting-3.2.1-smeserver-8.1-contribsorg
 +
 +
Why the SME Server version in the naming convention if it's all inside the container? Well, it could well be that the application inside the container will use some of SME Server specifics such as the db, templates or perl interaction. In that case we need to make sure that we know for which SME Server the image was build.
 +
 +
 
* Verification (checksum) of available images
 
* Verification (checksum) of available images
 
* Setting up trusted docker repo's
 
* Setting up trusted docker repo's
Line 43: Line 55:       −
==Challenges==
+
==Installation==
−
* How to interact with localhost PAM or LDAP from within a container?
+
 
−
* Many more...
+
===Contrib===
 +
There is a contrib that will set up a lot of this for you in the Reetp Repo:
 +
 
 +
https://wiki.contribs.org/User:ReetP
 +
 
 +
Add the reetpTest repo:
 +
 
 +
yum --enablerepo=reetpTest,epel install smeserver-docker
 +
 
 +
Most of the settings in the Manual Installation below are replicated into the contrib with templates
 +
 
 +
====Docker Images====
 +
 
 +
These can be pulled using docker itself as per Manual Installation below. Note that some require a higher version of docker. Regrettably I can't change that!
 +
 
 +
Alternatively you can create a docker-compose.yml file directly, or via templates, in:
 +
 
 +
/home/e-smith/files/docker/configs
 +
 
 +
docker-compose will automatically run this at boot.
 +
 
 +
Create you compose file and then run this from the configs directory
 +
 
 +
docker-compose up -d
 +
 
 +
====Networking====
 +
 
 +
{{Note box | See Issues and other comments further down re host/bridged networking}}
 +
 
 +
Docker attempts to guess what network to use and sets a bridged interface for it.
 +
 
 +
Access to the container.
 +
 
 +
This allows access to any local services, and any ports in the container will appear locally
 +
 
 +
--net="host"
 +
 
 +
This maps container port 80 to host port 8088
 +
 
 +
# container:host
 +
    ports:
 +
    - 8080:8080
 +
 
 +
So if you ran an Apache container service on port 80, you can connect to it from the host using
 +
 
 +
container.ip.add:8088
 +
 
 +
Using --net="host" means it is easier to connect to the container using the local IP address. Simple port forwarding/opening will suffice.
 +
 
 +
However, it exposes all ports on the container locally, and there may also be conflicts with local ports.
 +
 
 +
Using a port mapping is preferred, but your SME server will then block access container access to local services such as DNS.
 +
 
 +
The answer is probably to statically set the Docker network, and then add the network to 'Local Network'. You can then expose ports via the docker config entry eg:
 +
 
 +
docker=service
 +
  status=enabled
 +
  UPDPort=1234
 +
  TCPPort=8088
 +
 
 +
I am working on this currently but the LocalNetworking approach doesn't work. It probably need manipulation of the firewall with templates.
 +
 
 +
==== Login to container====
 +
 
 +
If permitted, most containers can be logged into using this:
 +
 
 +
docker exec -t -i -u root <container_name> /bin/bash
 +
 
 +
===Manual Installation===
    +
Note that most of the following is now in the contrib. See above.
   −
==Installation==
   
Docker requires some RPM's that are not available in the default upstream repo's. So we need to enable the epel repo first. See ''[[epel]]''
 
Docker requires some RPM's that are not available in the default upstream repo's. So we need to enable the epel repo first. See ''[[epel]]''
    
Then we can install Docker and it's dependencies:
 
Then we can install Docker and it's dependencies:
 
  yum install docker-io --enablerepo=epel
 
  yum install docker-io --enablerepo=epel
 +
 +
 +
Make the Docker service start at boot time:
 
  ln -s /etc/rc.d/init.d/e-smith-service /etc/rc7.d/S95docker
 
  ln -s /etc/rc.d/init.d/e-smith-service /etc/rc7.d/S95docker
 
  chkconfig docker on
 
  chkconfig docker on
−
  service docker start
+
  config set docker service
−
and then
+
config setprop docker status enabled
−
  docker
+
 
−
to see the available command line options. But first and foremost read the [https://docs.docker.com/userguide/ excellent Docker documentation]
+
 
 +
====Configuration====
 +
Docker comes with a configuration file located at:
 +
/etc/sysconfig/docker
 +
In this file you can set default parameters which are applicable to all containers run by Docker. By default it holds no arguments. All arguments can also be set manually when starting a container, in which case each individual container can have it's specific parameters.  To see a list all available arguments that can be used in the Docker configuration file enter:
 +
docker -h
 +
 
 +
 
 +
====SME Server specifics====
 +
By default Docker will store all images, containers and other data in:
 +
  /var/lib/docker
 +
 
 +
For SME Server this is not ideal for we would like to incorporate all Docker data into the pre-defined backup procedure(s) that come with SME Server. The preferred location for Docker data would be:
 +
'''/home/e-smith/files/docker'''
 +
 
 +
We want this to be the default location for all Docker data on SME Server, so we add the '-g' argument and the desired path to the storage location to the docker configuration file like this:
 +
 
 +
# /etc/sysconfig/docker
 +
#
 +
# Other arguments to pass to the docker daemon process
 +
# These will be parsed by the sysv initscript and appended
 +
# to the arguments list passed to docker -d
 +
 +
other_args="-g /home/e-smith/files/docker -H unix:///var/run/docker.sock"
 +
 
 +
 
 +
Since the Docker service will always check this configuration file upon (re)start, it will automatically pick up the arguments you have provided and act accordingly. That also implies that you can have multiple (but ''not'' simultaneously) storage locations if you omit the configuration file and add arguments manually on the command line.
 +
 
 +
The second argument '-H unix:///var/run/docker.sock' will tell Docker where to find it's socket to bind.
 +
 
 +
 
 +
It is ''important'' that you adjust the config file ''before'' you start using docker otherwise it will create it's default storage location in /var/lib/docker.
 +
 
 +
You can still change the storage location in a later stage by copying all data to the new location you've defined with the -g argument.
      −
* Note: Although we've linked the docker as service in runlevel 7, it will not pick up the config files /etc/sysconfig/docker. So Shipyard will not be able to connect to docker. Why?
+
Once the above changes have been made, the Docker service can be started and Docker will create it's new storage layout in /home/e-smith/files/docker.
 +
service docker start
    +
You can check if the docker deamon is running:
 +
service docker status
 +
and if it created the storage layout correctly:
 +
ls -l /home/e-smith/files/docker/
    
==Using a Docker image==
 
==Using a Docker image==
 
By default, there are pre-build images available from the official [https://registry.hub.docker.com/ Docker Hub]. In our examples we will use the pre-build centos6 image.
 
By default, there are pre-build images available from the official [https://registry.hub.docker.com/ Docker Hub]. In our examples we will use the pre-build centos6 image.
   −
To get a list of all available centos images you can use:
+
To get a list of all available Centos images you can use:
 
  docker search centos
 
  docker search centos
 
You will be flooded with available images from the Docker hub. This is because everyone can have a free account on Docker hub and create one repository for him/herself. We limit our testing to the official Centos repo. With all the other images, you are on your own and usage is at your own risk.
 
You will be flooded with available images from the Docker hub. This is because everyone can have a free account on Docker hub and create one repository for him/herself. We limit our testing to the official Centos repo. With all the other images, you are on your own and usage is at your own risk.
   −
By default, downloaded images from the Docker hub to your local server will be places in /var/lib/docker. If you want to store these images elsewhere, then you can create a symbolic link from that storage location pointing to /var/lib/docker.
  −
  −
* Note: See if we can change this default location in another way, config file?
      
===Downloading a docker image===
 
===Downloading a docker image===
 
To download the centos6 image to your local server, issue the following command as root:
 
To download the centos6 image to your local server, issue the following command as root:
 
  docker pull centos:centos6
 
  docker pull centos:centos6
−
where the syntax is 'centos' as the main repository and 'centos6' the specific version. Would you issue only 'docker pull centos', then all centos version will be downloaded. So be specific.
+
where the syntax is 'centos' as the main repository and 'centos6' the specific version. Would you issue only 'docker pull centos', then all centos versions will be downloaded. So be specific.
    
Once the image has been downloaded, you can check your local images by issuing:
 
Once the image has been downloaded, you can check your local images by issuing:
 
  docker images
 
  docker images
 +
The listing included the Image ID and Name. These are important to run additional commands when the container is running.
       
===Running a docker container===
 
===Running a docker container===
−
Now that we have downloaded the centos6 image it's time to give it a spin. To start the cento6 container we can issue the follwoing command:
+
Now that we have downloaded the centos6 image it's time to give it a spin. To start the cento6 container we can issue the following command:
−
  docker run -t -i centos:centos6 bash
+
  docker run -t -i --net="host" centos:centos6 bash
−
This will tell docker to run the centos6 container interactiveley from the local centos repo and start bash. After a few seconds you will be presented with the bash prompt inside the centos6 container:
+
This will tell docker to run the centos6 container interactively from the local centos repo, use the host network interface and start bash. After a few seconds you will be presented with the bash prompt inside the centos6 container:
 
  bash-4.1#
 
  bash-4.1#
 
and to check if we are really inside the centos6 container we can display the release version:
 
and to check if we are really inside the centos6 container we can display the release version:
Line 100: Line 219:        +
===Copy docker images===
 +
Docker images are stored on your local server. If you want to run the image on another machine you first have to take the image out of your local image repository and save the image in a transferable format. For this the ''save'' the image in .tar format. To get a listing of all available images on your local server:
 +
docker images
 +
will result in (example):
 +
[root@sme9 ~]# docker images
 +
REPOSITORY                TAG                IMAGE ID            CREATED            VIRTUAL SIZE
 +
sme9                      6.5                55db4355a2de        46 minutes ago      854.7 MB
 +
leszekk/centos_minimalcd  6.5                bc56fa8f1204        8 months ago        452.6 MB
 +
To create a copy of our sme9 image and save it as 'copyofsme9 you need to enter the following command:
 +
docker save sme9:6.5 > /tmp/copyofsme9.tar
 +
which will result in a copyofsme9.tar file in your /tmp directory of your local server. You can now copy/move this file to another server or simply archive it for later usage.
 +
 +
To use the copyofsme9.tar file on another server and use it on that server with Docker, we can load it into the repository of the new server:
 +
docker load -i < /downloads/copyofsme9.tar
 +
After Docker has loaded the file, you can check it's availability by executing: docker images and you can use it just like any other image on your new server. You can use the ''save'' and ''load'' commands to clean up your local repository and share copies of your image.
   −
==Building your own images==
+
===Docker networking===
 +
 
 +
some thoughts to share on docker networking
 +
 
 +
* Network port mapping
 +
http://docs.docker.com/userguide/dockerlinks/
 +
* Network Configuration
 +
http://docs.docker.com/articles/networking/
 +
 
 +
'''Note:''' Could we use FWS webapps to create an apache sub domain where the docker web application can be reached and 'masquerade' an unusual http port? e.g.
 +
owncloud.mydomain.com vs mydomain.com:8000
 +
Using
 +
mydomain.com/owncloud
 +
would require ibay checking
 +
 
 +
 
 +
===Docker Name resolution===
 +
 
 +
Normally you could add the DNS directly in the file /etc/sysconfig/docker, if you don't do that, your docker container could ping an IP, but never do the domain name translation. This is the dns of opendns, but you could change them.
 +
 
 +
# cat /etc/sysconfig/docker
 +
 
 +
# /etc/sysconfig/docker
 +
#
 +
# Other arguments to pass to the docker daemon process
 +
# These will be parsed by the sysv initscript and appended
 +
# to the arguments list passed to docker -d
 +
 +
other_args="--dns 208.67.220.220 --dns 208.67.220.222"
 +
 
 +
or you could add directly from the command line
 +
docker run -i -t -dns 208.67.220.220 -dns 208.67.220.222  sme9_real:6.5 /bin/bash
 +
 
 +
==Challenges==
 +
* How to interact with localhost PAM or LDAP from within a container?
 +
 
 +
I think that you can access localhost services by adding:
 +
 
 +
--net="host" to docker run
 +
 
 +
This means any services on the docker container are equally valid 'localhost' services accessible from the server itself so you need to ensure the server is properly firewalled. See Issues below.
 +
 
 +
* Many more...
       +
==Building your own images==
 
* Notes
 
* Notes
 
Manual, or..
 
Manual, or..
 
https://github.com/docker/fig
 
https://github.com/docker/fig
 +
 +
==='Proposal test image:'===
 +
An application that requires Java, PHP, Apache, MySQL and LDAP. The localhost MySQL and localhost LDAP should be used by the application. The application should be publicly available either on a subdomain or specific port on the FQDN. The application should only be available between 08:00AM until 19:00PM.
 +
All application data should be incorporated by the default SME Sever backup mechanisms, including the image itself.
 +
 +
* Building the image based on centos6
 +
* Configure networking, bridges and ports
 +
* Start/restart and stop syntax of the application
 +
* Configure cron
      Line 121: Line 307:  
  status=enabled
 
  status=enabled
    +
==Docker Compose==
 +
 +
The binary is included in the smeserver-docker contrib.
 +
 +
The latest version that you can use with the installed version of docker (currently 1.7.1) is docker-compose version 1.5.2
 +
https://github.com/docker/compose/releases/tag/1.5.2
 +
 +
curl -L https://github.com/docker/compose/releases/download/1.5.2/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
 +
chmod +x /usr/local/bin/docker-compose
    
==Shipyard web GUI==
 
==Shipyard web GUI==
−
[http://shipyard-project.com/ Shipyard] is a separate GUI project hosted on [https://github.com/shipyard/shipyard Github]. It will allow you to manage Docker instances and images from your browser. It is not required to use and run Docker containers, for all Docker commands can be issued from the command line.
+
There is a separate page on how to install Shipyard, the Docker web GUI [http://wiki.contribs.org/Shipyard here]
 +
 
 +
 
 +
==Related articles of interest==
 +
* [http://jpetazzo.github.io/2014/01/29/docker-device-mapper-resize/ Container storage and (re)size]
   −
For Shipyard to be able to connect to the Docker deamon, the file /etc/sysconfig/docker has to be adjusted to enable socket support and TCP.
     −
Templated /etc/sysconfig/docker based on below db settings
+
==Things to do==
 +
* Get the [https://github.com/shipyard/shipyard shipyard GUI] going
 +
* A LOT more ;-)
   −
/etc/sysconfig/docker:
+
==Issues==
−
other_args="-H tcp://127.0.0.1:4243 -H unix:///var/run/docker.sock"
  −
127.0.0.1 (localhost) or 0.0.0.0 (public) based on db key shipyard Access
     −
The restart the Docker deamon:
+
You will find that if you use 'host' networking docker will set /sys as Read Only and you will get an error with the raid_check as per this bug
−
service docker restart
     −
For Shipyard to be able to manage (multiple) hosts, each and every hosts will need to have shipyard-agent installed.
+
https://bugs.contribs.org/show_bug.cgi?id=10660
    +
If you don't use host networking, you use the internal IP address set with docker, but this address is unknown as a local network to SME and it will block any queries emanating from the container. I am looking at this with the contrib.
   −
===Installation===
+
==Koozai SME v10==
−
Shipyard can be installed as a Docker application itself. We will use this way of installing Shipyard.
     −
Issue the follwoing command as root:
+
Some basic scratchpad notes as I go
−
docker run -i -t -v /var/run/docker.sock:/docker.sock shipyard/deploy setup
  −
This will make Docker pull the various layers and images and combine it into fully functional webserver with the Shipyard application installed.
     −
Loose notes:
+
https://docs.docker.com/engine/install/centos/
−
shipyard=service
  −
TCPPort=4243
  −
access=public
  −
status=enabled
      +
Don't use the extras repo to install
   −
===Shipyard agent===
+
https://download.docker.com/linux/centos/docker-ce.repo
−
About
      +
===Repo setup===
   −
====Installation====
+
db yum_repositories set docker-ce-stable repository \
−
TBA
+
    BaseURL 'https://download.docker.com/linux/centos/7/$basearch/stable' \
 +
    EnableGroups no \
 +
    GPGCheck yes \
 +
    GPGKey https://download.docker.com/linux/centos/gpg \
 +
    Name 'Docker Stable' \
 +
    Visible yes \
 +
    status disabled
   −
Loose notes
+
signal-event yum-modify
−
shipyard-agent=service
  −
TCPPort=8000;4500
  −
access=public
  −
status=enabled
      +
yum --enablerepo=extras,docker-ce-stable install docker-ce docker-ce-cli
   −
==Related articles of interest==
+
===Files to modify?===
−
* [http://jpetazzo.github.io/2014/01/29/docker-device-mapper-resize/ Container storage and (re)size]
      +
systemd unit file
   −
==Things to do==
+
[Unit]
−
* Get the [https://github.com/shipyard/shipyard shipyard GUI] going
+
Description=Docker Application Container Engine
−
* A LOT more ;-)
+
Documentation=https://docs.docker.com
 +
BindsTo=containerd.service
 +
After=network-online.target firewalld.service containerd.service
 +
Wants=network-online.target
 +
Requires=docker.socket
 +
 +
[Service]
 +
Type=notify
 +
# the default is not to use systemd for cgroups because the delegate issues still
 +
# exists and systemd currently does not support the cgroup feature set required
 +
# for containers run by docker
 +
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock -g /home/e-smith/files/docker
 +
ExecReload=/bin/kill -s HUP $MAINPID
 +
TimeoutSec=0
 +
RestartSec=2
 +
Restart=always
 +
 +
# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
 +
# Both the old, and new location are accepted by systemd 229 and up, so using the old location
 +
# to make them work for either version of systemd.
 +
StartLimitBurst=3
 +
 +
# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
 +
# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
 +
# this option work for either version of systemd.
 +
StartLimitInterval=60s
 +
 +
# Having non-zero Limit*s causes performance problems due to accounting overhead
 +
# in the kernel. We recommend using cgroups to do container-local accounting.
 +
LimitNOFILE=infinity
 +
LimitNPROC=infinity
 +
LimitCORE=infinity
 +
 +
# Comment TasksMax if your systemd version does not support it.
 +
# Only systemd 226 and above support this option.
 +
TasksMax=infinity
 +
 +
# set delegate yes so that systemd does not reset the cgroups of docker containers
 +
Delegate=yes
 +
 +
# kill only the docker process, not all processes in the cgroup
 +
KillMode=process
 +
 +
[Install]
 +
WantedBy=multi-user.target
 +
 +
[[Category:Containers]]
 +
 +
[[Category:Containers:Docker]]
Retrieved from "https://wiki.koozali.org/Special:MobileDiff/24989...38903"

Navigation menu