Both SRE and DevOps are methodologies addressing organizations’ needs for production operation management. But the differences between the two doctrines are quite significant: While DevOps raise problems and dispatch them to Dev to solve, the SRE approach is to find problems and solve some of them themselves. While DevOps teams would usually choose the more conservative approach, leaving the production environment untouched unless absolutely necessary, SREs are more confident in their ability to maintain a stable production environment and push for rapid changes and software updates. Not unlike the DevOps team, SREs also thrive on a stable production environment, but one of the SRE team’s goals is to improve performance and operational efficiency.
DevOps for Jedi
Rapsberry PI Docker Swarm and Portainer.io
Today we will try portainer.io to monitorize dowker containers created with docker swarm on multi piZERO workers with a RPI3 leader
to install docker on rpi :
curl -sSL https://get.docker.com | sh
then
sudo usermod -aG docker pi
To init the leader
docker swarm init
This will give you the command for the workers
docker swarm join \
--token SWMTKN-1-5awy2ej1d55mvgpq1obunnh6u2r8b0jjujel619es-7caoz16dxre2bkplp3sh \
xxx.xxx.xxx.xxx:2377
On the leader you can control your node :
docker node ls
————————–
PORTAINER.IO
————————–
To install to manage your swarm cluster you need to install it on the leader
docker service create \
> --name portainer \
> --publish 9000:9000 \
> --constraint 'node.role == manager' \
> --mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \
> portainer/portainer \
> -H unix:///var/run/docker.sock
Then enjoy by connecting to :
http://IP_LEADER:9000
Building a NGINX PHP7 PHP-FPM with docker and docker-compose
Setting up Nginx
We’ll start by getting a web server and based on our requirements this will be a container running the official Nginx image. Since we’ll be using Docker Compose, we will create the following docker-compose.yml file, which will run the latest Nginx image and will expose its port 80 to port 8080:
web:
image: nginx:latest
ports:
– « 8080:80 »
Now we can run
docker-compose up
This should give you the default Nginx screen on port 8080 for localhost or the IP of your docker machine.
Now that we have a server let’s add some code. First we have to update the docker-compose.yml to mount a local directory. I will use a folder called code, which is in the same directory as my docker-compose.yml file, and it will be mounted as root folder code in the container.
web:
image: nginx:latest
ports:
– « 8080:80 »
volumes:
– ./code:/code
The next step is to let Nginx know that this folder exists.
Let’s create the following site.conf on the same level as the docker-compose.yml file:
server {
index index.html;
server_name php-docker.local;
error_log /var/log/nginx/error.log;
access_log /var/log/nginx/access.log;
root /code;
}
If you don’t have a lot of experience with Nginx, this is what we define here – index.html will be our default index, the server name is php-docker.local
To activate this setup we need to apply yet another modification to our docker-compose.yml file:
web:
image: nginx:latest
ports:
– « 8080:80 »
volumes:
– ./code:/code
– ./site.conf:/etc/nginx/conf.d/site.conf
This will add site.conf to the directory where Nginx is looking for configuration files to include. You can now place an index.html file in the code folder with contents that is to your heart’s delight. And if we run
docker-compose up
again, the index.html file should be available on php-docker.local:8080.
Adding PHP-FPM
Now that we have Nginx up and running let’s add the PHP in the game. The first thing we’ll do is pull the official PHP7-FPM repo and link it to our Nginx container. Our docker-compose.yml will look like this now:
web:
image: nginx:latest
ports:
– « 8080:80 »
volumes:
– ./code:/code
– ./site.conf:/etc/nginx/conf.d/site.conf
links:
– php
php:
image: php:7-fpm
The next thing to do is configure Nginx to use the PHP-FPM container for interpreting PHP files. Your updated site.conf should look like this:
server {
index index.php index.html;
server_name php-docker.local;
error_log /var/log/nginx/error.log;
access_log /var/log/nginx/access.log;
root /code;
location ~ \.php$ {
try_files $uri =404;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass php:9000;
fastcgi_index index.php;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
}
}
In order to test this let’s rename the index.html file to index.php and replace its content with the standard:
Splunk – universal forwarder on Raspberry Pi 3 and Splunk enterprise on Labtop
A little post to explain how i succeed to deploy a splunk forwarder on a raspberry pi 3
First install the client splunk enterprise on your labptop
Then configure it to accept receiving data on port 9997
In the upper right, click the dropdown for “Settings”. Under Data, click Forwarding and receiving, and you will be taken to the configuration page where you can set Splunk to listen for data from your Pi.
Click “configure receiving”, and you will be taken to the receive data configuration page. Assuming this is a brand new installation of Splunk, you will have no configurations. Click “New” and you will be taken to the new configuration wizard. For now, we will just add a new listener at port 9997, and click Save.
Then install the universal forwarder on your RPI :
Download the Universal Forwarder from http://apps.Splunk.com/app/1611 to your Pi
You’ll find some help on http://docs.splunk.com/Documentation/Splunk/6.0/Forwarding/Deployanixdfmanually
but it ‘s not necessary
just download the .tar file and use : tar -xvf
…. to unzip it
One important thing to know when installing the Universal Forwarder on *nix, is that the default install does NOT autorun on boot.
You can set it to autostart running the following as root:
$SPLUNK_HOME/bin/Splunk enable boot-start
To start Splunk on your forwarder, navigate to $SPLUNK_HOME /bin/ and run ./splunk start. You’ll see the standard output for startup.
At the next prompt, run ./splunk version, and you should see the version output for ARM Linux.
Congratulations, the Splunk Universal Forwarder is running on your Raspberry Pi!
Now, create a script which will be played by splunk to extract data :
edit file : $SPLUNK_HOME/bin/scripts
#! /bin/sh
for c in $(seq 1 60)
do
TIME=$(date +"%D %T.%3N %Z")
TEMP=$(vcgencmd measure_temp)
VOLTS=$(vcgencmd measure_volts)
CPU_TEMP=`echo $TEMP | cut -d \' -f 1`
VOLTS=`echo $VOLTS | cut -d V -f 1`
echo "time=$TIME|$VOLTS|$CPU_TEMP"
sleep 1
done
Then edit :
$SPLUNK_HOME/etc/system/local/inputs.conf
[default]
host = raspberrypi
[script:///opt/Splunkforwarder/bin/scripts/cpu.sh]
index = main
interval = 60
source = cpu_details
sourcetype = pi_system_scripts
Then edit :
$SPLUNK_HOME/etc/apps/SplunkUniversalForwarder/default/outputs.conf
# Version 6.0
[tcpout]
defaultGroup=my_indexers
[tcpout:my_indexers]
server=XX.XXX.XX.XXX:9997
Where XX… is the IP of your labtop
BE CAREFUL to accept on your labtop firewall (Windows) the port 9997
if you don’t you will have in the log file on the rpi in $SPLUNK_HOME/var/log/splunk/splund.log
the error :
connection to ip=ip.address:9997 timed out
When it done just start splunk on the RPI :
$SPLUNK_HOME/bin/splunk start
and you ll receive data on the search screen of the plunk enterprise client installed on your labtop :
http://localhost:8000/fr-FR/app/search/search#fr-FR/app/search/search
CONGRATULATION !! and have fun
HowTo build an Docker infrastructure
What :
An infrastructure based on Varnish + Nginx + PHP-FPM + MariaDB
On Docker Containers
Why :
For a personnal blog or website
How : Build,Ship, Run
(Because the what is famous , we won’t do it from Dockerfiles . We will use Dockerhub images)
Step 1:
mkdir –p $HOME/data/webstack/conf $HOME/data/webstack/www
Step 2: install NGinx
docker pull nginx
Controle your image with docker images
Launch your container
docker pull jprjr/php–fpm
Let run the container :
docker run —name webstack_php_01 –p 9000:9000 –d jprjr/php–fpm
Step 4 : Make them talk together
Docker option : –link
We will copy the nginx configfile to the host
docker cp webstack_nginx_1:/etc/nginx/nginx.conf $HOME/data/webstack/conf/nginx.conf
vi $HOME/data/webstack/conf/nginx.conf
Add the server part
In our usecase we need that the nginx container webstack_nginx_1 can communicate with the PHP container (webstack_php_01).
We delete the existing containers
docker stop webstack_php_01 && docker rm webstack_php_01
docker stop webstack_nginx_1 && docker rm webstack_nginx_1
And we recreate them by using a docker-compose command
Before create a docker-compose.yml file
php:
image: jprjr/php-fpm
ports:
– « 9000:9000 »
volumes:
– $HOME/data/webstack/www:/srv/http
web:
image: nginx
ports:
– « 8080:80 »
volumes:
– $HOME/data/webstack/www:/usr/share/nginx/html
– $HOME/data/webstack/conf/nginx.conf:/etc/nginx.conf
links:
– php
Then run the command
docker-compose up
and try it :
curl http://localhost:8080
waffle.io an automatic kanban report
Waffle creates a free project management solution from your existing GitHub Issues.
Track Your Work Automatically
Your Waffle board shows your GitHub Issues and Pull Requests in real time. Never wonder if an Issue is still in progress or not. Waffle listens to the actions in your workflow to know when work is finished and updates your status automatically.
Otto a vagrant successor ?
Otto is the single solution to develop and deploy any application, with first class support for microservices.
Otto automatically builds development environments without any configuration; it can detect your project type and has built-in knowledge of industry-standard tools to setup a development environment that is ready to go. When you’re ready to deploy, otto builds and manages an infrastructure, sets up servers, builds, and deploys the application.
With the growing trend of microservices, Otto knows how to install and configure service dependencies for development and deployment. It automatically exposes these dependencies via DNS for your application to consume.
Github at home , try gitblit
if you want your propoer github go gitblit http://gitblit.com/
DOCKER + VAGRANT
Jhipster on docker
LINUX UBUNTU
Pull the JHipster Docker image:
sudo docker pull jdubois/jhipster-docker
Create a « jhipster » folder in your home directory:
mkdir ~/jhipster
Run The docker image, with the following options:
- The Docker « /jhipster » folder is shared to the local « ~/jhipster » folder
- Forward all ports exposed by docker (8080 for Tomcat, 3000 for BrowserSync from the « grunt serve » task, 3001 for the BrowserSync UI, and 22 for SSHD). In the following example we forward the container 22 port to the host 4022 port, to prevent some port conflicts:
sudo docker run -v ~/jhipster:/jhipster -p 8080:8080 -p 3000:3000 -p 3001:3001 -p 4022:22 -t jdubois/jhipster-docker
SSH configuration
You can now connect to your docker container with SSH. You can connect as « root/jhipster » or as « jhipster/jhipster », and we recommand you use the « jhipster » user as some of the tool used are not meant to be run by the root user.
Start by adding your SSH public key to the Docker container:
cat ~/.ssh/id_rsa.pub | ssh -p 4022 jhipster@localhost 'mkdir ~/.ssh && cat >> ~/.ssh/authorized_keys'
You can now connect to the Docker container:
ssh -p 4022 jhipster@localhost
Your first project
You can then go to the /jhipster directory in your container, and start building your app inside Docker:
cd /jhipster
yo jhipster
If the following exception occurs, you need to change the owner of the /jhipster directory. See next step.
...
? (13/13) Would you like to use the Compass CSS Authoring Framework?: No
/usr/lib/node_modules/generator-jhipster/node_modules/yeoman-generator/node_modules/mkdirp/index.js:89
throw err0;
^
Error: EACCES, permission denied '/jhipster/src'
at Object.fs.mkdirSync (fs.js:653:18)
at sync (/usr/lib/node_modules/generator-jhipster/node_modules/yeoman-generator/node_modules/mkdirp/index.js:70:13)
at sync (/usr/lib/node_modules/generator-jhipster/node_modules/yeoman-generator/node_modules/mkdirp/index.js:76:24)
at Function.sync (/usr/lib/node_modules/generator-jhipster/node_modules/yeoman-generator/node_modules/mkdirp/index.js:76:24)
at JhipsterGenerator.app (/usr/lib/node_modules/generator-jhipster/app/index.js:419:10)
at /usr/lib/node_modules/generator-jhipster/node_modules/yeoman-generator/lib/base.js:387:14
at processImmediate [as _immediateCallback] (timers.js:345:15)
You need to fix the jhipster folder to give the « jhipster » user ownership of the directory:
ssh -p 4022 jhipster@localhost
sudo chown jhipster /jhipster
Once your application is created, you can run all the normal grunt/bower/maven commands, for example:
mvn spring-boot:run
Congratulations! You’ve launched your JHipster app inside Docker!
On your host machine, you should be able to :
- Access the running application at http://localhost:8080
- Get all the generated files inside your shared folder
As the generated files are in your shared folder, they will not be deleted if you stop your Docker container. However, if you don’t want Docker to keep downloading all the Maven and NPM dependencies every time you start the container, you should commit its state.