DevOps et Site Reliability Engineering (SRE)

Le DevOps s'inscrit dans la culture d'entreprise comme un ensemble de principes qu'une entreprise aspire et respecte à long terme. Les partisans de cette culture valorisent la collaboration, l’expérimentation et la volonté d’apprendre. Toutes les parties impliquées dans une culture DevOps se concentrent sur un objectif tout au long du cycle de vie d'un projet (pas seulement le développement et l'exploitation): la mise en œuvre rapide de logiciels stables et de haute qualité, du concept client jusqu'à l'expérience utilisateur.

L'automatisation du développement, des tests et du déploiement de logiciels par le biais du déploiement continue (CD) est un facteur clé reconnu de la méthodologie DevOps. L'automatisation permet une mise en œuvre logicielle plus rapide et garantit à la solution finale qualité, sécurité et stabilité.

Objectifs

Définition et contribution:

  • Indicateur de niveau de service (SLI)
  • Objectif de niveau de service (SLO)
  • Service Level Agreements (SLA)
  • Estimation des risques, des niveaux de disponibilité et du budget d'erreur

Collaboration

Expertise auprès des développeurs d'applications:

  • Gestion du changement
  • Définir des objectifs communs
  • Assurer la livraison de la production
  • Améliorer la fiabilité du système

Responsabilités

Implication et responsibilisation:

  • Surveillance et notifications
  • Estimation des capacités allouées
  • Latence, performance et efficacité
  • Réponse d'urgence et automatisation

Articles associés au DevOps

Local development environments with Terraform + LXD

Local development environments with Terraform + LXD

Catégories : Containers Orchestration, DevOps & SRE | Tags : Automation, DevOps, KVM, LXD, Virtualization, VM, Terraform, Vagrant

As a Big Data Solutions Architect and InfraOps, I need development environments to install and test software. They have to be configurable, flexible, and performant. Working with distributed systems…

Gauthier LEONARD

Par Gauthier LEONARD

1 juin 2023

How to build your OCI images using Buildpacks

How to build your OCI images using Buildpacks

Catégories : Containers Orchestration, DevOps & SRE | Tags : CNCF, OCI, CI/CD, Docker, Kubernetes

Docker has become the new standard for building your application. In a Docker image we place our source code, its dependencies, some configurations and our application is almost ready to be deployed…

Paul-Adrien CORDONNIER

Par Paul-Adrien CORDONNIER

9 janv. 2023

Big data infrastructure internship

Big data infrastructure internship

Catégories : Big Data, Data Engineering, DevOps & SRE, Infrastructure | Tags : Infrastructure, Hadoop, Big Data, Cluster, Internship, Kubernetes, TDP

Job description Big Data and distributed computing are at the core of Adaltas. We accompagny our partners in the deployment, maintenance, and optimization of some of the largest clusters in France…

Stephan BAUM

Par Stephan BAUM

2 déc. 2022

Introducing Trunk Data Platform: the Open-Source Big Data Distribution Curated by TOSIT

Introducing Trunk Data Platform: the Open-Source Big Data Distribution Curated by TOSIT

Catégories : Big Data, DevOps & SRE, Infrastructure | Tags : DevOps, Hortonworks, Ansible, Hadoop, HBase, Knox, Ranger, Spark, Cloudera, CDP, CDH, Open source, TDP

Ever since Cloudera and Hortonworks merged, the choice of commercial Hadoop distributions for on-prem workloads essentially boils down to CDP Private Cloud. CDP can be seen as the “best of both worlds…

Leo SCHOUKROUN

Par Leo SCHOUKROUN

14 avr. 2022

JS monorepos in prod 7: Continuous Integration and Continuous Deployment with GitHub Actions

JS monorepos in prod 7: Continuous Integration and Continuous Deployment with GitHub Actions

Catégories : DevOps & SRE, Front End | Tags : CI/CD, Monorepo, Node.js, Unit tests

The value of CI/CD lies in the ability to control and coordinate changes and feature addition in multiple, iterative releases while simultaneously having multiple services being actively developed in…

Alexander HOFFMANN

Par Alexander HOFFMANN

6 avr. 2022

Ansible variables: choosing the right location

Ansible variables: choosing the right location

Catégories : DevOps & SRE | Tags : Infrastructure, Ansible, IaC, YAML

Defining variables for your Ansible playbooks and roles can become challenging as your project grows. Browsing the Ansible documentation, the diversity of Ansible variables location is confusing, to…

Xavier HERMAND

Par Xavier HERMAND

15 mars 2022

GitOps in practice, deploy Kubernetes applications with ArgoCD

GitOps in practice, deploy Kubernetes applications with ArgoCD

Catégories : Containers Orchestration, DevOps & SRE, Adaltas Summit 2021 | Tags : Argo CD, CI/CD, Git, GitOps, IaC, Kubernetes

GitOps is a set of practices to deploy applications using Git. Application definitions, configurations, and connectivity are to be stored in a version control software such as Git. Git then serves as…

Paul-Adrien CORDONNIER

Par Paul-Adrien CORDONNIER

16 déc. 2021

JS monorepos in prod 6: CI/CD, continuous integration and deployment with Travis CI

JS monorepos in prod 6: CI/CD, continuous integration and deployment with Travis CI

Catégories : DevOps & SRE, Front End | Tags : CI/CD, Monorepo, Node.js, Unit tests

Implementing continuous integration CI and continuous deployment (CD) on a monorepo is quite complex due to the diversity of multiple responsibilities between developers and the need to coordinate…

David WORMS

Par David WORMS

6 déc. 2021

Running your Travis CI builds locally with Docker

Running your Travis CI builds locally with Docker

Catégories : DevOps & SRE, Front End | Tags : Bash, Tools, CI/CD, Node.js, Unit tests

Setting up the environment to run the tests on a CI/CD can take a few roundtrips between your host machine and the CI/CD running remotely. For every attempt, you’ll have to commit and publish your…

David WORMS

Par David WORMS

6 sept. 2021

Modern Python part 3: run a CI pipeline & publish your package to PiPy

Modern Python part 3: run a CI pipeline & publish your package to PiPy

Catégories : DevOps & SRE | Tags : CI/CD, Git, GitHub, Python, Release and features, Unit tests

To propose a well-maintained and usable Python package to the open-source community or even inside your company, you are expected to accomplish a set of critical steps. First ensure that your code is…

Faouzi BRAZA

Par Faouzi BRAZA

28 juin 2021

Modern Python part 2: write unit tests & enforce Git commit conventions

Modern Python part 2: write unit tests & enforce Git commit conventions

Catégories : DevOps & SRE | Tags : Git, pandas, Python, Unit tests

Good software engineering practices always bring a lot of long-term benefits. For example, writing unit tests permits you to maintain large codebases and ensures that a specific piece of your code…

Faouzi BRAZA

Par Faouzi BRAZA

24 juin 2021

Modern Python part 1: start a project with pyenv & poetry

Modern Python part 1: start a project with pyenv & poetry

Catégories : DevOps & SRE | Tags : Git, Python, Release and features, Unit tests

When learning a programming language, the focus is essentially on understanding the syntax, the code style, and the underlying concepts. With time, you become sufficiently comfortable with the…

Faouzi BRAZA

Par Faouzi BRAZA

9 juin 2021

JS monorepos in prod 5: merging Git repositories and preserve commit history

JS monorepos in prod 5: merging Git repositories and preserve commit history

Catégories : DevOps & SRE, Node.js | Tags : Bash, DevOps, Packaging, Git, GitHub, GitOps, JavaScript, Monorepo

At Adaltas, we maintain several open-source Node.js projects organized as Git monorepos and published on NPM. We shared our experience to work with Lerna monorepos in a set of articles: Part…

Sergei KUDINOV

Par Sergei KUDINOV

21 mai 2021

Bridging the DBnomics Swagger/OpenAPI schema with GraphQL

Bridging the DBnomics Swagger/OpenAPI schema with GraphQL

Catégories : DevOps & SRE, Front End | Tags : Data Engineering, JAMstack, GraphQL, JavaScript, Node.js, REST, Schema

While redacting a long and fastidious document today, I came across DBnomics, an open platform federating economic datasets. Browsing its website and APIs, I found their OpenAPI schema (aka Swagger…

David WORMS

Par David WORMS

8 avr. 2021

JS monorepos in prod 4: unit testing with Mocha and Should.js

JS monorepos in prod 4: unit testing with Mocha and Should.js

Catégories : DevOps & SRE, Front End | Tags : Automation, CI/CD, Git, GitOps, Monorepo, Node.js, Unit tests

Unit testing is essential for every long-term project and allows you to pull down functionalities of your code into isolated testable units. Indeed the main goal of a unit test is to verify if an…

David WORMS

Par David WORMS

25 févr. 2021

JS monorepos in prod 3: commit enforcement and changelog generation

JS monorepos in prod 3: commit enforcement and changelog generation

Catégories : DevOps & SRE, Front End | Tags : CI/CD, Git, JavaScript, Monorepo, Node.js, Release and features, Unit tests

Conventional Commits introduces a structured format for commit messages. It standardizes the messages among all the contributors. This makes them more readable and easy to automate. It simplifies the…

David WORMS

Par David WORMS

2 févr. 2021

JS monorepos in prod 2: project versioning and publishing

JS monorepos in prod 2: project versioning and publishing

Catégories : DevOps & SRE, Front End | Tags : CI/CD, Git, GitOps, JavaScript, Monorepo, Node.js, Release and features, Unit tests

One great advantage of a monorepo is to maintain coherent versions between packages and to automatize the version creation and the publication of packages. This article covers the versioning and…

David WORMS

Par David WORMS

11 janv. 2021

JS monorepos in prod 1: project initialization

JS monorepos in prod 1: project initialization

Catégories : DevOps & SRE, Front End | Tags : Git, GitOps, JavaScript, Monorepo, Node.js, Release and features

Every project journey begins with the step of initialization. When your overall project is composed of multiple projects, it is tempting to create one Git repository per project. In Node.js, a project…

David WORMS

Par David WORMS

5 janv. 2021

Data versioning and reproducible ML with DVC and MLflow

Data versioning and reproducible ML with DVC and MLflow

Catégories : Data Science, DevOps & SRE, Events | Tags : Data Engineering, Databricks, Delta Lake, Git, Machine Learning, MLflow, Storage

Our talk on data versioning and reproducible Machine Learning proposed to the Data + AI Summit (formerly known as Spark+AI) is accepted. The summit will take place online the 17-19th November…

Petra KAFERLE DEVISSCHERE

Par Petra KAFERLE DEVISSCHERE

30 sept. 2020

Version your datasets with Data Version Control (DVC) and Git

Version your datasets with Data Version Control (DVC) and Git

Catégories : Data Science, DevOps & SRE | Tags : DevOps, Infrastructure, Operation, Git, GitOps, SCM

Using a Version Control System such as Git for source code is a good practice and an industry standard. Considering that projects focus more and more on data, shouldn’t we have a similar approach such…

Grégor JOUET

Par Grégor JOUET

3 sept. 2020

Machine Learning model deployment

Machine Learning model deployment

Catégories : Big Data, Data Engineering, Data Science, DevOps & SRE | Tags : DevOps, Operation, AI, Cloud, Machine Learning, MLOps, On-premises, Schema

“Enterprise Machine Learning requires looking at the big picture […] from a data engineering and a data platform perspective,” lectured Justin Norman during the talk on the deployment of Machine…

Oskar RYNKIEWICZ

Par Oskar RYNKIEWICZ

30 sept. 2019

Rook with Ceph doesn't provision my Persistent Volume Claims!

Rook with Ceph doesn't provision my Persistent Volume Claims!

Catégories : DevOps & SRE | Tags : PVC, Linux, Rook, Ubuntu, Ceph, Cluster, Internship, Kubernetes

Ceph installation inside Kubernetes can be provisioned using Rook. Currently doing an internship at Adaltas, I was in charge of participating in the setup of a Kubernetes (k8s) cluster. To avoid…

Eyal CHOJNOWSKI

Par Eyal CHOJNOWSKI

9 sept. 2019

Spark Streaming part 3: DevOps, tools and tests for Spark applications

Spark Streaming part 3: DevOps, tools and tests for Spark applications

Catégories : Big Data, Data Engineering, DevOps & SRE | Tags : DevOps, Learning and tutorial, Spark, Apache Spark Streaming

Whenever services are unavailable, businesses experience large financial losses. Spark Streaming applications can break, like any other software application. A streaming application operates on data…

Oskar RYNKIEWICZ

Par Oskar RYNKIEWICZ

31 mai 2019

Monitoring a production Hadoop cluster with Kubernetes

Monitoring a production Hadoop cluster with Kubernetes

Catégories : DevOps & SRE | Tags : Thrift, Grafana, Shinken, Hadoop, Knox, Cluster, Docker, Elasticsearch, Kubernetes, Node, Node.js, Prometheus, Python

Monitoring a production grade Hadoop cluster is a real challenge and needs to be constantly evolving. The software we use today is based on Nagios. Very efficient when it comes to the simplest…

Paul-Adrien CORDONNIER

Par Paul-Adrien CORDONNIER

21 déc. 2018

Hadoop cluster takeover with Apache Ambari

Hadoop cluster takeover with Apache Ambari

Catégories : Big Data, DevOps & SRE, Adaltas Summit 2018 | Tags : Ambari, Automation, iptables, Kerberos, Nikita, Systemd, Cluster, HDP, Node, Node.js, REST

We recently migrated a large production Hadoop cluster from a “manual” automated install to Apache Ambari, we called this the Ambari Takeover. This is a risky process and we will detail why this…

Leo SCHOUKROUN

Par Leo SCHOUKROUN

15 nov. 2018

KVM machines for Vagrant on Archlinux

KVM machines for Vagrant on Archlinux

Catégories : DevOps & SRE | Tags : Arch Linux, KVM, Linux, Virtualization, VM, Vagrant

Vagrant supports different providers to manage virtualization. In a Linux environment, you can dramatically improve VM performance by using the libvirt provider and the KVM hypervisor. This tutorial…

Gauthier LEONARD

Par Gauthier LEONARD

19 sept. 2018

Publishing guidelines

Publishing guidelines

Catégories : DevOps & SRE | Tags : Arch Linux, KVM, VM, Vagrant, Markdown

This is as much a set of guidelines targeting everyone publishing content on the web as rules for reviewers to ensure no validation is forgotten before submitting for publication. It mostly targets…

David WORMS

Par David WORMS

28 févr. 2018

Ambari - How to blueprint

Ambari - How to blueprint

Catégories : Big Data, DevOps & SRE | Tags : Ambari, Automation, DevOps, Operation, Ranger, REST

As infrastructure engineers at Adaltas, we deploy Hadoop clusters. A lot of them. Let’s see how to automate this process with REST requests. While really handy for deploying one or two clusters, the…

Joris RUMMENS

Par Joris RUMMENS

17 janv. 2018

Multi-Repo, Multi-Node Gating at Massive Scale

Multi-Repo, Multi-Node Gating at Massive Scale

Catégories : Cloud Computing, DevOps & SRE, Open Source Summit Europe 2017 | Tags : Infrastructure, Jenkins, Red Hat, Zuul, Ansible, CI/CD, OpenStack

This is a recap and personal review of Monty Taylor’s presentation of OpenStack’s Continuous Integration tool Zuul at the OpenSource Summit 2017 in Prague (not to mix with Netflix’ Zuul project…

Joris RUMMENS

Par Joris RUMMENS

28 oct. 2017

Apache Thrift vs REST

Apache Thrift vs REST

Catégories : DevOps & SRE, Open Source Summit Europe 2017 | Tags : Thrift, gRPC, HTTP, JavaScript Object Notation (JSON), REST

Adaltas recently attended the Open Source Summit Europe 2017 in Prague. I had the opportunity to follow a presentation made by Randy Abernethy and Jens Geyer of RM-X, a cloud native consulting company…

Leo SCHOUKROUN

Par Leo SCHOUKROUN

28 oct. 2017

From Dockerfile to Ansible Containers

From Dockerfile to Ansible Containers

Catégories : Containers Orchestration, DevOps & SRE, Open Source Summit Europe 2017 | Tags : pip, Shell, Ansible, Docker, Docker Compose, YAML

This talk was an introduction to the Dockerfile format and to Ansible container’s tool and then a comparison of both. It was hold by Tomas Tomecek from Red Hat’s containerization team. The Dockerfile…

César BEREZOWSKI

Par César BEREZOWSKI

25 oct. 2017

MiNiFi: Data at Scales & the Values of Starting Small

MiNiFi: Data at Scales & the Values of Starting Small

Catégories : Big Data, DevOps & SRE, Infrastructure | Tags : MiNiFi, NiFi, C++, HDF, Cloudera, HDP, IOT

This conference presented rapidly Apache NiFi and explained where MiNiFi came from: basically it’s a NiFi minimal agent to deploy on small devices to bring data to a cluster’s NiFi pipeline (ex: IoT…

César BEREZOWSKI

Par César BEREZOWSKI

8 juil. 2017

HDP cluster monitoring

HDP cluster monitoring

Catégories : Big Data, DevOps & SRE, Infrastructure | Tags : Alert, Ambari, Metrics, Monitoring, HDP, REST

With the current growth of BigData technologies, more and more companies are building their own clusters in hope to get some value of their data. One main concern while building these infrastructures…

Joris RUMMENS

Par Joris RUMMENS

5 juil. 2017

Hive Metastore HA with DBTokenStore: Failed to initialize master key

Hive Metastore HA with DBTokenStore: Failed to initialize master key

Catégories : Big Data, DevOps & SRE | Tags : Infrastructure, Hive, Bug

This article describes my little adventure around a startup error with the Hive Metastore. It shall be reproducable with any secure installation, meaning with Kerberos, with high availability enabled…

David WORMS

Par David WORMS

21 juil. 2016

A fresh look at testing Node.js projects: Mocha, Should and Travis

A fresh look at testing Node.js projects: Mocha, Should and Travis

Catégories : DevOps & SRE, Node.js | Tags : DevOps, Mocha, CI/CD, JavaScript, Node.js, Unit tests

Today, I finally decided to spend some time around Travis. It’s been a few weeks since that little green image on top of many GitHub homepages has been buzzing me. Well, to be totally honest, this isn…

David WORMS

Par David WORMS

19 févr. 2012

Announcing Mecano, a set of functions for system deployment

Announcing Mecano, a set of functions for system deployment

Catégories : DevOps & SRE, Node.js | Tags : Automation, Infrastructure, CoffeeScript, JavaScript, Open source

Update July 2016, Mecano is now renamed Nikita. We are releasing Node Mecano on GitHub which gather common functions used while deploying systems. The idea was to group those functions into a…

David WORMS

Par David WORMS

12 févr. 2012

Canada - Maroc - France

Nous sommes une équipe passionnée par l'Open Source, le Big Data et les technologies associées telles que le Cloud, le Data Engineering, la Data Science le DevOps…

Nous fournissons à nos clients un savoir faire reconnu sur la manière d'utiliser les technologies pour convertir leurs cas d'usage en projets exploités en production, sur la façon de réduire les coûts et d'accélérer les livraisons de nouvelles fonctionnalités.

Si vous appréciez la qualité de nos publications, nous vous invitons à nous contacter en vue de coopérer ensemble.

Support Ukrain