Global Cosmetics Leader Increases Developer Velocity and Satisfaction Through Automation

This blog is for anyone stretching beyond the limits of their infrastructure or platform engineering teams and looking for a way to streamline their processes to overcome common cloud native issues.

This piece outlines the digital transformation Grupo Boticário, one of the largest cosmetic companies in Brazil, has gone through in the last few years. It explains the core problems the company went through when reaching limits on their infrastructure team, as well as the solutions they invested in.

Who is Grupo Boticário?

Grupo Boticário is a global, sustainable cosmetics company headquartered in Brazil. With over 4,000 stores worldwide, they are the second largest cosmetics company in Brazil and the fifteenth largest in the world.

Below is a testimonial from one of Crossplane and Upbound’s core users from Grupo Boticário, Francisco Ribas, written on Grupo Boticário’s tech department’s blog here. It explains the process he and his team went through when looking at technologies such as Helm, ArgoCD, and more, in addition to Crossplane and Upbound. Read on to understand why one of the leading cosmetics companies in the world invested in Crossplane and Upbound to increase developer velocity, decrease deployment time, and more!

Abstract

By 2019, Grupo Boticário started its technological transformation, and internalized the development of many solutions. Due to this business change, the technology team grew substantially, from less than 500 in November 2019, to 1,600 people in December 2022.

As we know, during this period, the COVID pandemic changed the technology market, and good professionals, mainly with cloud and infrastructure skills, became hard to hire and development tasks were commonly blocked by pending provision of infrastructure and cloud services. Going forward, the increase of the maturity of architecture and security of our solutions included extra patterns to be achieved manually, bringing even more complexity and time spent on provisioning.

The solution was to automate the process of provisioning common services, using some well-known and some disrupting tools. The success of this solution is so successful that the time spent on the process dropped from 5 days to a maximum of 20 minutes.

Background: Multiteam and manual way

With more than 1,000 developers working on our solutions, it was a strategic decision to have a specific team to take care of cloud and infra topics, considering cost optimization and the difficult-to-hire infrastructure workforce.

The process used by that infra team is almost a market pattern and broadly adopted by large tech companies. It already adopts good practices and modern tooling, such as Infrastructure as Code, Terraform and HCL, GitHub Repositories, Pull Requests and Atlantis

Using the Gitops concept, the source of truth was GitHub Repositories with HCL defining the infrastructure. The repository contained the infra of a specific product and was maintained mainly by the infrastructure team, making changes and creating new resources on demand, fulfilling requests opened by the development team.

We can resume all processes using an activities diagram:

Despite this being a very good and elegant solution, it was common to have a conflicting prioritization between Development and Devops backlogs, and unexceptional delays occurred caused by blocked teams waiting for each other. This all, adding the fact of our restricted workforce on Devops team, made the problem escalate to a very painful grade, so, a solution was urgent for this situation.

Looking for a better process

More than just automating the infrastructure creation, the upcoming changes gave us the opportunity to enforce some other patterns that were not always implemented by the manual creation process, like security enforcements and roles and policies restructuring. And, of course, this solution must increase the user experience for all teams involved in the creation.

First off, we separate our infrastructure into two categories. Each category has its own stack, target team and a defined desired experience for each one.

The first category is structuring infrastructure that comprehends the base infrastructure to run our solution. It contains the definition of clusters, security groups, global roles and policies, for example. This category is maintained by the devops team and, to not change its experience, the stack remained unchanged, using the flow detailed above, with Terraform, HCL and Atlantis.

The second category is product infrastructure that contains all resources related directly to a product of a development team. For this category, we change the stack substantially. The most significant change was the adoption of Crossplane over Terraform and ArgoCD over Atlantis.

This stack change enabled us to have the same experience for both application and infrastructure deployment: both are defined into application repository, deployed by GitHub Actions and ArgoCD, and provide a unified status too.

Going forward, we abstract all complex parameters of cloud services, using helm charts. So the code of infrastructure into the repository is simple and readable, and all patterns of security and infrastructure are enforced by devops teams into the helm chart. We also automated the generation of infrastructure code, application base code and pipeline, using a template machine built by our platform team, called Scafflater. It all uses Backstage as an interface and the final experience for developers is a form with few fields to be filled, resulting in a repository ready to be deployed minutes after requested.

With this new process, we transform the way to work, mainly of devops teams. Now its work impacts all developers by template implementations instead of manual provision, focusing on how maintaining templates ensures that all company patterns are implemented: