SRE and DevOps – Site Reliability Engineering

Both SRE and DevOps are methodologies addressing organizations’ needs for production operation management. But the differences between the two doctrines are quite significant: While DevOps raise problems and dispatch them to Dev to solve, the SRE approach is to find problems and solve some of them themselves. While DevOps teams would usually choose the more conservative approach, leaving the production environment untouched unless absolutely necessary, SREs are more confident in their ability to maintain a stable production environment and push for rapid changes and software updates. Not unlike the DevOps team, SREs also thrive on a stable production environment, but one of the SRE team’s goals is to improve performance and operational efficiency.

mobydock

devops.pm father

More Posts - Website

Follow Me:
TwitterFacebook

Rapsberry PI Docker Swarm and Portainer.io

Today we will try portainer.io to monitorize dowker containers created with docker swarm on multi piZERO workers with a RPI3 leader

to install docker on rpi :
curl -sSL https://get.docker.com | sh
then
sudo usermod -aG docker pi

To init the leader
docker swarm init

This will give you the command for the workers
docker swarm join \
--token SWMTKN-1-5awy2ej1d55mvgpq1obunnh6u2r8b0jjujel619es-7caoz16dxre2bkplp3sh \
xxx.xxx.xxx.xxx:2377

On the leader you can control your node :
docker node ls

————————–
PORTAINER.IO
————————–

To install to manage your swarm cluster you need to install it on the leader
docker service create \
> --name portainer \
> --publish 9000:9000 \
> --constraint 'node.role == manager' \
> --mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \
> portainer/portainer \
> -H unix:///var/run/docker.sock

Then enjoy by connecting to :

http://IP_LEADER:9000

mobydock

devops.pm father

More Posts - Website

Follow Me:
TwitterFacebook

Building a NGINX PHP7 PHP-FPM with docker and docker-compose

Setting up Nginx

We’ll start by getting a web server and based on our requirements this will be a container running the official Nginx image. Since we’ll be using Docker Compose, we will create the following docker-compose.yml file, which will run the latest Nginx image and will expose its port 80 to port 8080:

web:
image: nginx:latest
ports:
– « 8080:80 »
Now we can run

docker-compose up
This should give you the default Nginx screen on port 8080 for localhost or the IP of your docker machine.

Now that we have a server let’s add some code. First we have to update the docker-compose.yml to mount a local directory. I will use a folder called code, which is in the same directory as my docker-compose.yml file, and it will be mounted as root folder code in the container.

web:
image: nginx:latest
ports:
– « 8080:80 »
volumes:
– ./code:/code
The next step is to let Nginx know that this folder exists.
Let’s create the following site.conf on the same level as the docker-compose.yml file:

server {
index index.html;
server_name php-docker.local;
error_log /var/log/nginx/error.log;
access_log /var/log/nginx/access.log;
root /code;
}
If you don’t have a lot of experience with Nginx, this is what we define here – index.html will be our default index, the server name is php-docker.local

To activate this setup we need to apply yet another modification to our docker-compose.yml file:

web:
image: nginx:latest
ports:
– « 8080:80 »
volumes:
– ./code:/code
– ./site.conf:/etc/nginx/conf.d/site.conf
This will add site.conf to the directory where Nginx is looking for configuration files to include. You can now place an index.html file in the code folder with contents that is to your heart’s delight. And if we run

docker-compose up
again, the index.html file should be available on php-docker.local:8080.

Adding PHP-FPM

Now that we have Nginx up and running let’s add the PHP in the game. The first thing we’ll do is pull the official PHP7-FPM repo and link it to our Nginx container. Our docker-compose.yml will look like this now:

web:
image: nginx:latest
ports:
– « 8080:80 »
volumes:
– ./code:/code
– ./site.conf:/etc/nginx/conf.d/site.conf
links:
– php
php:
image: php:7-fpm
The next thing to do is configure Nginx to use the PHP-FPM container for interpreting PHP files. Your updated site.conf should look like this:

server {
index index.php index.html;
server_name php-docker.local;
error_log /var/log/nginx/error.log;
access_log /var/log/nginx/access.log;
root /code;

location ~ \.php$ {
try_files $uri =404;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass php:9000;
fastcgi_index index.php;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
}
}
In order to test this let’s rename the index.html file to index.php and replace its content with the standard:

mobydock

devops.pm father

More Posts - Website

Follow Me:
TwitterFacebook

Splunk – universal forwarder on Raspberry Pi 3 and Splunk enterprise on Labtop

A little post to explain how i succeed to deploy a splunk forwarder on a raspberry pi 3

First install the client splunk enterprise on your labptop
Then configure it to accept receiving data on port 9997
In the upper right, click the dropdown for “Settings”. Under Data, click Forwarding and receiving, and you will be taken to the configuration page where you can set Splunk to listen for data from your Pi.

Click “configure receiving”, and you will be taken to the receive data configuration page. Assuming this is a brand new installation of Splunk, you will have no configurations. Click “New” and you will be taken to the new configuration wizard. For now, we will just add a new listener at port 9997, and click Save.

Then install the universal forwarder on your RPI :

Download the Universal Forwarder from http://apps.Splunk.com/app/1611 to your Pi

You’ll find some help on http://docs.splunk.com/Documentation/Splunk/6.0/Forwarding/Deployanixdfmanually

but it ‘s not necessary

just download the .tar file and use : tar -xvf …. to unzip it

One important thing to know when installing the Universal Forwarder on *nix, is that the default install does NOT autorun on boot.
You can set it to autostart running the following as root:
$SPLUNK_HOME/bin/Splunk enable boot-start

To start Splunk on your forwarder, navigate to $SPLUNK_HOME /bin/ and run ./splunk start. You’ll see the standard output for startup.
At the next prompt, run ./splunk version, and you should see the version output for ARM Linux.

Congratulations, the Splunk Universal Forwarder is running on your Raspberry Pi!

Now, create a script which will be played by splunk to extract data :

edit file : $SPLUNK_HOME/bin/scripts

#! /bin/sh
for c in $(seq 1 60)
do
TIME=$(date +"%D %T.%3N %Z")
TEMP=$(vcgencmd measure_temp)
VOLTS=$(vcgencmd measure_volts)
CPU_TEMP=`echo $TEMP | cut -d \' -f 1`
VOLTS=`echo $VOLTS | cut -d V -f 1`
echo "time=$TIME|$VOLTS|$CPU_TEMP"
sleep 1
done

Then edit :
$SPLUNK_HOME/etc/system/local/inputs.conf

[default]
host = raspberrypi
[script:///opt/Splunkforwarder/bin/scripts/cpu.sh]
index = main
interval = 60
source = cpu_details
sourcetype = pi_system_scripts

Then edit :
$SPLUNK_HOME/etc/apps/SplunkUniversalForwarder/default/outputs.conf

# Version 6.0
[tcpout]
defaultGroup=my_indexers
[tcpout:my_indexers]
server=XX.XXX.XX.XXX:9997

Where XX… is the IP of your labtop

BE CAREFUL to accept on your labtop firewall (Windows) the port 9997
if you don’t you will have in the log file on the rpi in $SPLUNK_HOME/var/log/splunk/splund.log
the error :

connection to ip=ip.address:9997 timed out

When it done just start splunk on the RPI :
$SPLUNK_HOME/bin/splunk start

and you ll receive data on the search screen of the plunk enterprise client installed on your labtop :
http://localhost:8000/fr-FR/app/search/search#fr-FR/app/search/search

CONGRATULATION !! and have fun

mobydock

devops.pm father

More Posts - Website

Follow Me:
TwitterFacebook

HowTo build an Docker infrastructure

What :

An infrastructure based on Varnish + Nginx + PHP-FPM + MariaDB

On Docker Containers

Why :

For a personnal blog or website

 

How : Build,Ship, Run

(Because the what is famous , we won’t do it from Dockerfiles . We will use Dockerhub images)

Step 1:

mkdir p $HOME/data/webstack/conf $HOME/data/webstack/www

 

Step 2: install NGinx

docker pull nginx

Controle your image with docker images

Launch your container

docker run name webstack_nginx_1 v $HOME/data/webstack/www:/usr/share/nginx/html:ro p 8080:80 d nginx
You can control by open your navigator and try
http://youserver:8080
Our container name is  webstack_nginx_1, the port TCP/8080 of our host will be tranfer to the port TCP/80 of the container (default port for NGinx) and it assigne in readonly the directory volume : /usr/share/nginx/html to the host directory $HOME/data/www .
To test we create some static pages :
echo « My first page » > $HOME/data/webstack/www/index.html
echo « <?php phpinfo(); ?>«  > $HOME/data/webstack/www/phpinfo.php
You can also confirm tha everythnig work fine by using :
curl http://localhost:8080
it will say : « My first page »
Step 3 : PHP-FPM

docker pull jprjr/phpfpm

Let run the container :

docker run name webstack_php_01 p 9000:9000 d jprjr/phpfpm

 

Step 4 : Make them talk together

Docker option : –link

We will copy the nginx configfile to the host

docker cp webstack_nginx_1:/etc/nginx/nginx.conf $HOME/data/webstack/conf/nginx.conf

 

 

vi $HOME/data/webstack/conf/nginx.conf

 

Add the server part

 

daemon off;
user  nginx;
worker_processes  1;
error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;
events {
    worker_connections  1024;
}
http {
    include       /etc/nginx/mime.types;
    default_type  application/octetstream;
    log_format  main  ‘$remote_addr – $remote_user [$time_local] « $request » ‘
                      ‘$status $body_bytes_sent « $http_referer » ‘
                      ‘ »$http_user_agent » « $http_x_forwarded_for »‘;
    access_log  /var/log/nginx/access.log  main;
    sendfile        on;
    #tcp_nopush     on;
    keepalive_timeout  65;
    #gzip  on;
    server {
        listen       80;
        server_name  localhost;
        location / {
            root   /usr/share/nginx/html;
            index  index.html index.htm;
        }
        #error_page  404              /404.html;
        # redirect server error pages to the static page /50x.html
        #
        #error_page   500 502 503 504  /50x.html;
        #location = /50x.html {
        #    root   /usr/share/nginx/html;
        #}
        # Pass PHP scripts to PHP-FPM
        location ~* \.php$ {
            fastcgi_index   index.php;
            fastcgi_pass    webstack_php:9000;
            #fastcgi_pass   unix:/var/run/php-fpm/php-fpm.sock;
            include         fastcgi_params;
            fastcgi_param   SCRIPT_FILENAME    /srv/http$fastcgi_script_name;
            fastcgi_param   SCRIPT_NAME        $fastcgi_script_name;
        }    
        # deny access to .htaccess files, if Apache’s document root
        # concurs with nginx’s one
        location ~ /\.ht {
            deny  all;
        }
    }
}

 

 

In our usecase we need that the nginx container webstack_nginx_1 can communicate with the PHP container (webstack_php_01).

We delete the existing containers

docker stop webstack_php_01 && docker rm webstack_php_01

docker stop webstack_nginx_1 && docker rm webstack_nginx_1

 

And we recreate them by using a docker-compose command

Before create a docker-compose.yml file

 

php:
image: jprjr/php-fpm
ports:
– « 9000:9000 »
volumes:
– $HOME/data/webstack/www:/srv/http

web:
image: nginx
ports:
– « 8080:80 »
volumes:
– $HOME/data/webstack/www:/usr/share/nginx/html
– $HOME/data/webstack/conf/nginx.conf:/etc/nginx.conf
links:
– php

 

Then run the command

docker-compose up

 

and try it :

curl http://localhost:8080

 

 

 

 

 

mobydock

devops.pm father

More Posts - Website

Follow Me:
TwitterFacebook

waffle.io an automatic kanban report

WAFFLE.IO

Waffle creates a free project management solution from your existing GitHub Issues.

Track Your Work Automatically
Your Waffle board shows your GitHub Issues and Pull Requests in real time. Never wonder if an Issue is still in progress or not. Waffle listens to the actions in your workflow to know when work is finished and updates your status automatically.

mobydock

devops.pm father

More Posts - Website

Follow Me:
TwitterFacebook

Otto a vagrant successor ?

oto

Otto is the single solution to develop and deploy any application, with first class support for microservices.

Otto automatically builds development environments without any configuration; it can detect your project type and has built-in knowledge of industry-standard tools to setup a development environment that is ready to go. When you’re ready to deploy, otto builds and manages an infrastructure, sets up servers, builds, and deploys the application.

With the growing trend of microservices, Otto knows how to install and configure service dependencies for development and deployment. It automatically exposes these dependencies via DNS for your application to consume.

mobydock

devops.pm father

More Posts - Website

Follow Me:
TwitterFacebook

Jhipster on docker

LINUX UBUNTU

 

Pull the JHipster Docker image:

sudo docker pull jdubois/jhipster-docker

Create a « jhipster » folder in your home directory:

mkdir ~/jhipster

Run The docker image, with the following options:

  • The Docker « /jhipster » folder is shared to the local « ~/jhipster » folder
  • Forward all ports exposed by docker (8080 for Tomcat, 3000 for BrowserSync from the « grunt serve » task, 3001 for the BrowserSync UI, and 22 for SSHD). In the following example we forward the container 22 port to the host 4022 port, to prevent some port conflicts:

sudo docker run -v ~/jhipster:/jhipster -p 8080:8080 -p 3000:3000 -p 3001:3001 -p 4022:22 -t jdubois/jhipster-docker

 

 

SSH configuration

You can now connect to your docker container with SSH. You can connect as « root/jhipster » or as « jhipster/jhipster », and we recommand you use the « jhipster » user as some of the tool used are not meant to be run by the root user.

Start by adding your SSH public key to the Docker container:

cat ~/.ssh/id_rsa.pub | ssh -p 4022 jhipster@localhost 'mkdir ~/.ssh && cat >> ~/.ssh/authorized_keys'

You can now connect to the Docker container:

ssh -p 4022 jhipster@localhost

Your first project

You can then go to the /jhipster directory in your container, and start building your app inside Docker:

cd /jhipster
yo jhipster

If the following exception occurs, you need to change the owner of the /jhipster directory. See next step.

...
? (13/13) Would you like to use the Compass CSS Authoring Framework?: No

/usr/lib/node_modules/generator-jhipster/node_modules/yeoman-generator/node_modules/mkdirp/index.js:89
throw err0;
^
Error: EACCES, permission denied '/jhipster/src'
at Object.fs.mkdirSync (fs.js:653:18)
at sync (/usr/lib/node_modules/generator-jhipster/node_modules/yeoman-generator/node_modules/mkdirp/index.js:70:13)
at sync (/usr/lib/node_modules/generator-jhipster/node_modules/yeoman-generator/node_modules/mkdirp/index.js:76:24)
at Function.sync (/usr/lib/node_modules/generator-jhipster/node_modules/yeoman-generator/node_modules/mkdirp/index.js:76:24)
at JhipsterGenerator.app (/usr/lib/node_modules/generator-jhipster/app/index.js:419:10)
at /usr/lib/node_modules/generator-jhipster/node_modules/yeoman-generator/lib/base.js:387:14
at processImmediate [as _immediateCallback] (timers.js:345:15)

You need to fix the jhipster folder to give the « jhipster » user ownership of the directory:

ssh -p 4022 jhipster@localhost
sudo chown jhipster /jhipster

Once your application is created, you can run all the normal grunt/bower/maven commands, for example:

mvn spring-boot:run

Congratulations! You’ve launched your JHipster app inside Docker!

On your host machine, you should be able to :

  • Access the running application at http://localhost:8080
  • Get all the generated files inside your shared folder

As the generated files are in your shared folder, they will not be deleted if you stop your Docker container. However, if you don’t want Docker to keep downloading all the Maven and NPM dependencies every time you start the container, you should commit its state.

mobydock

devops.pm father

More Posts - Website

Follow Me:
TwitterFacebook