Jonathan Lalou's blog

Posts Tagged ‘cloud’

At Devoxx France 2018, Arun Gupta and Tiffany Jernigan, both from Amazon Web Services (AWS), delivered a three-hour deep-dive session titled Compute options for Microservices on AWS. This hands-on tutorial explored deploying a microservices-based application using various AWS compute options: EC2, Amazon Elastic Container Service (ECS), AWS Fargate, Elastic Kubernetes Service (EKS), and AWS Lambda. Through a sample application with web app, greeting, and name microservices, they demonstrated local testing, deployment pipelines, service discovery, monitoring, and canary deployments. The session, rich with code demos, is available on YouTube, with code and slides on GitHub.

Microservices: Solving Business Problems

Arun Gupta opened by addressing the monolith vs. microservices debate, emphasizing that the choice depends on business needs. Microservices enable agility, frequent releases, and polyglot environments but introduce complexity. AWS simplifies this with managed services, allowing developers to focus on business logic. The demo application featured three microservices: a public-facing web app, and internal greeting and name services, communicating via REST endpoints. Built with WildFly Swarm, a Java EE-compliant server, the application produced a portable fat JAR, deployable as a container or Lambda function. The presenters highlighted service discovery, ensuring the web app could locate stateless instances of greeting and name services.

EC2: Full Control for Traditional Deployments

Amazon EC2 offers developers complete control over virtual machines, ideal for those needing to manage the full stack. The presenters deployed the microservices on EC2 instances, running WildFly Swarm JARs. Using Maven and a Docker profile, they generated container images, pushed to Docker Hub, and tested locally with Docker Compose. A docker stack deploy command spun up the services, accessible via curl localhost:8080, returning responses like “hello Sheldon.” EC2 requires manual scaling and cluster management, but its flexibility suits custom stacks. The GitHub repo includes configurations for EC2 deployments, showcasing integration with AWS services like CloudWatch for logging.

Amazon ECS: Orchestrating Containers

Amazon ECS simplifies container orchestration, managing scheduling and scaling. The presenters created an ECS cluster in the AWS Management Console, defining task definitions for the three microservices. Task definitions specified container images, CPU, and memory, with an Application Load Balancer (ALB) enabling path-based routing (e.g., /resources/greeting). Using the ECS CLI, they deployed services, ensuring high availability across multiple availability zones. CloudWatch integration provided metrics and logs, with alarms for monitoring. ECS reduces operational overhead compared to EC2, balancing control and automation. The session highlighted ECS’s deep integration with AWS services, streamlining production workloads.

AWS Fargate: Serverless Containers

Introduced at re:Invent 2017, AWS Fargate abstracts server management, allowing developers to focus on containers. The presenters deployed the same microservices using Fargate, specifying task definitions with AWS VPC networking for fine-grained security. The Fargate CLI, a GitHub project by AWS’s John Pignata, simplified setup, creating ALBs and task definitions automatically. A curl to the load balancer URL returned responses like “howdy Penny.” Fargate’s per-second billing and task-level resource allocation optimize costs. Available initially in US East (N. Virginia), Fargate suits developers prioritizing simplicity. The session emphasized its role in reducing infrastructure management.

Elastic Kubernetes Service (EKS): Kubernetes on AWS

EKS, in preview during the session, brings managed Kubernetes to AWS. The presenters deployed the microservices on an EKS cluster, using kubectl to manage pods and services. They introduced Istio, a service mesh, to handle traffic routing and observability. Istio’s sidecar containers enabled 50/50 traffic splits between “hello” and “howdy” versions of the greeting service, configured via YAML manifests. Chaos engineering was demonstrated by injecting 5-second delays in 10% of requests, testing resilience. AWS X-Ray, integrated via a daemon set, provided service maps and traces, identifying bottlenecks. EKS, later supporting Fargate, offers flexibility for Kubernetes users. The GitHub repo includes EKS manifests and Istio configurations.

AWS Lambda: Serverless Microservices

AWS Lambda enables serverless deployments, eliminating server management. The presenters repurposed the WildFly Swarm application for Lambda, using the Serverless Application Model (SAM). Each microservice became a Lambda function, fronted by API Gateway endpoints (e.g., /greeting). SAM templates defined functions, APIs, and DynamoDB tables, with sam local start-api testing endpoints locally via Dockerized Lambda runtimes. Responses like “howdy Sheldon” were verified with curl localhost:3000. SAM’s package and deploy commands uploaded functions to S3, while canary deployments shifted traffic (e.g., 10% to new versions) with CloudWatch alarms. Lambda’s per-second billing and 300-second execution limit suit event-driven workloads. The session showcased SAM’s integration with AWS services and the Serverless Application Repository.

Deployment Pipelines: Automating with AWS CodePipeline

The presenters built a deployment pipeline using AWS CodePipeline, a managed service inspired by Amazon’s internal tooling. A GitHub push triggered the pipeline, which used CodeCommit to build Docker images, pushed them to Amazon Elastic Container Registry (ECR), and deployed to an ECS cluster. For Lambda, SAM templates were packaged and deployed. CloudFormation templates automated resource creation, including VPCs, subnets, and ALBs. The pipeline ensured immutable deployments with commit-based image tags, maintaining production stability. The GitHub repo provides CloudFormation scripts, enabling reproducible environments. This approach minimizes manual intervention, supporting rapid iteration.

Monitoring and Logging: AWS X-Ray and CloudWatch

Monitoring was a key focus, with AWS X-Ray providing end-to-end tracing. In ECS and EKS, X-Ray daemons collected traces, generating service maps showing web app, greeting, and name interactions. For Lambda, X-Ray was enabled natively via SAM templates. CloudWatch offered metrics (e.g., CPU usage) and logs, with alarms for thresholds. In EKS, Kubernetes tools like Prometheus and Grafana were mentioned, but X-Ray’s integration with AWS services was emphasized. The presenters demonstrated debugging Lambda functions locally using SAM CLI and IntelliJ, enhancing developer agility. These tools ensure observability, critical for distributed microservices.

Choosing the Right Compute Option

The session concluded by comparing compute options. EC2 offers maximum control but requires managing scaling and updates. ECS balances automation and flexibility, ideal for containerized workloads. Fargate eliminates server management, suiting simple deployments. EKS caters to Kubernetes users, with Istio enhancing observability. Lambda, best for event-driven microservices, minimizes operational overhead but has execution limits. Factors like team expertise, application requirements, and cost influence the choice. The presenters encouraged feedback via GitHub issues to shape AWS’s roadmap. Visit aws.amazon.com/containers for more.

Links:

• AWS Official Site
• GitHub Repository
• YouTube Video

Hashtags: #AWS #Microservices #ECS #Fargate #EKS #Lambda #DevoxxFR2018 #ArunGupta #TiffanyJernigan #CloudComputing

Manually provisioning and managing infrastructure – whether virtual machines, networks, or databases – can be a time-consuming and error-prone process. As applications and their underlying infrastructure become more complex, automating these tasks is essential for efficiency, repeatability, and scalability. Infrastructure as Code (IaC) addresses this by allowing developers and operations teams to define and manage infrastructure using configuration files, applying software development practices like version control, testing, and continuous integration. Terraform, an open-source IaC tool from HashiCorp, has gained significant popularity for its ability to provision infrastructure across various cloud providers and on-premises environments using a declarative language. At Devoxx France 2017, Yannick Lorenzati presented “Terraform 101″, introducing the fundamentals of Terraform and demonstrating how developers can use it to quickly and easily set up the infrastructure they need for development, testing, or demos. His talk provided a practical introduction to IaC with Terraform.

Traditional infrastructure management often involved manual configuration through web consoles or imperative scripts. This approach is prone to inconsistencies, difficult to scale, and lacks transparency and version control. IaC tools like Terraform allow users to define their infrastructure in configuration files using a declarative syntax, specifying the desired state of the environment. Terraform then figures out the necessary steps to achieve that state, automating the provisioning and management process.

Declarative Infrastructure with HashiCorp Configuration Language (HCL)

Yannick Lorenzati introduced the core concept of declarative IaC with Terraform. He would have explained that instead of writing scripts that describe how to set up infrastructure step-by-step (imperative approach), users define what the infrastructure should look like (declarative approach) using HashiCorp Configuration Language (HCL). HCL is a human-readable language designed for creating structured configuration files.

The presentation would have covered the basic building blocks of Terraform configurations written in HCL:

• Providers: Terraform interacts with various cloud providers (AWS, Azure, Google Cloud, etc.) and other services through providers. Yannick showed how to configure a provider to interact with a specific cloud environment.
• Resources: Resources are the fundamental units of infrastructure managed by Terraform, such as virtual machines, networks, storage buckets, or databases. He would have demonstrated how to define resources in HCL, specifying their type and desired properties.
• Variables: Variables allow for parameterizing configurations, making them reusable and adaptable to different environments (development, staging, production). Yannick showed how to define and use variables to avoid hardcoding values in the configuration files.
• Outputs: Outputs are used to expose important information about the provisioned infrastructure, such as IP addresses or connection strings, which can be used by other parts of an application or by other Terraform configurations.

Yannick Lorenzati emphasized how the declarative nature of HCL simplifies infrastructure management by focusing on the desired end state rather than the steps to get there. He showed how Terraform automatically determines the dependencies between resources and provisions them in the correct order.

Practical Demonstration: From Code to Cloud Infrastructure

The core of the “Terraform 101″ talk was a live demonstration showing how a developer can use Terraform to provision infrastructure. Yannick Lorenzati would have guided the audience through writing a simple Terraform configuration file to create a basic infrastructure setup, perhaps including a virtual machine and a network configuration on a cloud provider like AWS (given the mention of AWS Route 53 data source in the transcript).

He would have demonstrated the key Terraform commands:

• terraform init: Initializes the Terraform working directory and downloads the necessary provider plugins.
• terraform plan: Generates an execution plan, showing what actions Terraform will take to achieve the desired state without actually making any changes. This step is crucial for reviewing the planned changes before applying them.
• terraform apply: Executes the plan, provisioning or updating the infrastructure according to the configuration.
• terraform destroy: Tears down all the infrastructure defined in the configuration, which is particularly useful for cleaning up environments after testing or demos (and saving costs, as mentioned in the transcript).

Yannick showed how Terraform outputs provide useful information after the infrastructure is provisioned. He might have also touched upon using data sources (like the AWS Route 53 data source mentioned) to fetch information about existing infrastructure to be used in the configuration.

The presentation highlighted how integrating Terraform with configuration management tools like Ansible (also mentioned in the description) allows for a complete IaC workflow, where Terraform provisions the infrastructure and Ansible configures the software on it.

Yannick Lorenzati’s “Terraform 101″ at Devoxx France 2017 provided a clear and practical introduction to Infrastructure as Code using Terraform. By explaining the fundamental concepts, introducing the HCL syntax, and demonstrating the core workflow with live coding, he empowered developers to start automating their infrastructure provisioning. His talk effectively conveyed how Terraform can save time, improve consistency, and enable developers to quickly set up the environments they need, ultimately making them more productive.

Links:

• HashiCorp: https://www.hashicorp.com/

Hashtags: #Terraform #IaC #InfrastructureAsCode #HashiCorp #DevOps #CloudComputing #Automation #YannickLorenzati

J’ai eu la chance d’assister a la derniere journee de DevoxxFR a Paris, le vendredi 29/03/2013, pour le compte de mon employeur StepInfo, sponsor de l’evenement. Voici quelques impressions en vrac

General

• C’est bien organise, il y a du monde, et excepte au moment du depart au niveau des vestiaires, il n’y a pas eu de gros souci de logistique.
• Les entreprises sponsors ont joue le jeu 😉
• Que ce soit au niveau des stands ou des conferences, la domination des Mac est ecrasante! Google a reussi a mettre en valeur ses ChromeBooks, mais j’ai vu peu de Windows et encore moins de Linux.
• J’ai pu assister a 4 conferences, toutes interessantes, mais d’un niveau heterogene, je reviens dessus plus en detail ci-dessous.

IDE Java : astuces de productivité pour le quotidien

La conference est animee par Xavier Hanin (@xavierhanin). Il y presente les trois IDE phares du monde Java: NetBeans, Eclipse et IntelliJ IDEA.

Maitrisant plutot bien IDEA, je n’ai pas appris de choses fondamentales sur mon IDE, si ce n’est le raccourci Ctrl+Shift+A pour afficher les intentions. J’ai connu NetBeans a ses debuts (Forte puis Sun ONE), mais j’en ai totalement perdu la maitrise depuis des annees. Quant a Eclipse, il m’arrive de le lancer quelques fois par an pour des problematiques clients specifiques, mais je suis contraint d’avoir recours a la souris en quasi-permanence ; sous IDEA c’est tres rare.

Un “sondage” assez grossier des personnes dans la salle a donne des resultats interessants: ~15% des developpeurs presents utilisent NetBeans, ~25% IDEA et ~90/95% Eclipse.

Quick Start avec le Cloud Google

La conference est animee par Didier Girard de SFEIR et Alexis Moussine-Pouchkine de Google France. Les principaux outils de Google pour le cloud sont presentes, en prenant pour hypothese de creer une startup: une solution “a l’ancienne”, avec serveurs d’applications, gestion materielle etc. est avancee, avant de ceder la place a une solution entierement sur le cloud de Google.

En un mot: c’est tres convaincant.
Je regrette que le sujet n’ait pas ete elargi vers des solutions cloud alternatives, comme celle d’Amazon.

• Didier Girard:
• Sur twitter: @DidierGirard
• Sur Google+: +Didier
• Alexis Moussine-Pouchkine:
• Blog: http://alexismp.wordpress.com/
• Twitter: @alexismp
• Google+: +Alexis

The Spring Update: Looking at Spring 3.1, 3.2, and 4.0

La presentation est menee par Josh Long de SpringSource. On se rend compte tres rapidement que ne serait-ce que pour la qualite du show, nous les Francais (voire les Latins) nous sommes largement en dessous de la qualite des Anglo-Saxons.

Josh enonce ses concepts, donne des exemples, met l’eau a la bouche. Au bout de 50′ de conference, je n’ai plus qu’une envie: retourner coder avec les dernieres versions de Spring!

Parmi toutes les nouveautes, j’en retiens deux:

• l’AnnotationConfigApplicationContext, qui m’evitera de taper d’ecrire un bloc comme:

      <context:component-scan annotation-config="true" base-package="lalou.jonathan.beans"/>

• l’integration future des websockets d’HTML5 pour obeir a la norme JEE7 d’ici la fin de l’annee.

Ce dernier point m’interesse particulierement, en raison d’un projet sur lequel je travaille actuellement, et dont nombre de problemes seraient resolus par les websockets de JEE7. Theoriquement, et potentiellement, d’ici la fin de l’annee 2013, nous pourrions integrer une brique “Spring JEE7″ avec websocket au sein d’un WebSphere 8 (donc non JEE7-compliant), au lieu d’etre dependant dans la prochaine release du serveur d’applications d’IBM.

Josh:

• sur twitter: @starbuxman
• sur son blog: http://joshlong.com/
• Google+: +Josh

Entre HPC et big data: un case study sur la simulation du risque de contrepartie

C’est la conference dont j’ai vu le moins le rapport avec Devoxx, mais ayant passe 7 ans dans la finance de marche j’y ai trouve mon interet. En passant, j’y ai vu le seul PC sous Windows de toutes les conferences :-D.

Le theme de cette conference est: comment deporter une partie du calcul intensif de Monte-Carlo du CPU vers les GPU? Ces derniers ont une taille et une structure de memoire radicalement differente de celles des CPU (CPU+RAM pour etre plus precis). Les GPU permettent d’effectuer des masses de calcul en un temps bien plus faible (jusqu’a 99% de gain) qu’en architecture classique, moyennant une reecriture du code C en un autre langage adapte, par exemple OpenCL.

Les deux speakers, de Murex, sont Adrien Tay Pamart (visiblement pas tres a l’aise en mode geek) et Jonathan Lellouche pour la partie technique.

Durant les questions, l’existence de ponts Java vers OpenCL, comme JavaCL est evoquee. Il est dommage de ne pas avoir plus de temps pour creuser ce theme.

5 ans et 500 releases en 50 minutes !

La presentation est dirigee par Freddy Mallet  et Olivier Gaudin de Sonar.

La demonstration est rudement bien faite. Les dirigeant de SonarSource retracent, dans un ordre plus ou moins chronologique, les problemes, ou les impediments a-t-on envie de dire, rencontres en 5 ans sur Sonar.
Quelques themes forts: le context switching est une plaie, on ne gere pas une entreprise commerciale avec des clients comme un simple projet open source, etc.

• Freddy:
• Sur twitter: @FreddyMallet
• Olivier:
• Sur twitter: @gaudol

En guise de conclusion

Devoxx a repondu aux attentes que j’avais en y entrant:

• une journee de formation intensive et motivante
• revoir des “anciennes tetes”
• echanger des cartes de visite

Qu’il me soit donc permis ici de remercier StepInfo pour m’avoir permis d’acceder a cette journee, ainsi que les organisateurs de Devoxx pour le travail qu’il ont accompli.

Vivement DevoxxFR 2014!