During the last couple of years, a lot of organizations has made the transformation from Waterfall to Agile to DevOps. Continuous Integration & Continuous Delivery (CI/CD) is a well known method to this new way of working. Also, CI/CD has changed how developers and testers ship software. Teams are becoming self organized. Although a lot of companies did a good job, they still struggle to effectively make use of all of these new trends. To help overcome the most common pitfalls and to provide useful tips, this article provides practical CI/CD steps to accelerate your software development life-cycle.

On prem tools to start

When CI/CD and DevOps kicked in, a lot of companies started off with the “traditional tools” to fulfill the technical aspects. Think of popular tools like Jenkins as a CI-orchestrator and SonarQube to scan programming code for violations. Also, don’t forget Jfrog Artifactory with X-ray as a artifact repository and Octopus deploy to deploy their applications. These tools already put a lot of pressure to the organizations. but there is more.

Problems

Most of these tools require some investments up front like Virtual Machines being setup and maintained in on-prem data centers. Tools require servers which needs to be provisioned (consistently), secured and patched from time to time. These Virtual Machines require 24/7 uptime, even when (most probably) not used outside office hours. A lot of the tools mentioned above incur high license costs based on the number of hosts (sometimes even CPU cores), the number of users or projects. Or even worse: on the number of lines of code.

As more teams join the CI/CD journey, tools need to scale in terms of hardware and configuration. Not all teams use the same programming languages and build systems to create ready to deploy software applications. What about the setup and configuration of all of these tools. Teams require different configurations for their specific needs. Most of the time, they only use a subset of whatever is available to them. But it still requires attention to keep it all up and running without conflicts. When operating within an enterprise level, all of this becomes very complicated.

Central question is here: do these tools really help the self-organized DevOps teams? Given the increased popularity of Cloud offerings, the answer might be very well: no.

On demand infrastructure

Infrastructure and tools should support the principles of CI/CD and DevOps. Teams should be able to use the tools according to their internal way of working. The configuration of these tools should support their business applications. Costs of maintenance and licensing fees should be as low as possible. Cloud native services which are offered by cloud providers provide a lot of advantages to the problems mentioned above.

Scaling

Take the example of a CI server. Organizations which run a CI tool like Jenkins or Gitlab pay for these tools even when no developer uses them. Typical “build job” (e.g. compiling and packaging software applications) require a lot of processing power. Tools like this should be able to handle peak loads during office hours. How to add more memory or more disk space “on request”? It’s impossible, so infra is most of the time “over provisioned”. Capacity is either too low or too much. Cloud providers offer “scaling” on demand – when there is a high peak, infra resources can “expand” to the demand of that moment. It scales back when the load is less, say during midnight. You only pay for the resources which are in use.

One size fits all?

Using Infrastructure as Code help to create Virtual Machines in a consistent and controlled way. When Operating Systems need to be patched you just update your IaC scripts and apply it to your infra. All Virtual Machines are patched in a similar way. Think of the “desired state” of those servers. The Cloud provider has to figure out how to make sure all machines share the same configuration. When using these principles, configuration drift is avoided. It also reduces security risks, since there is better visibility in security related aspects.

Some examples of team specific configuration are the following:

  • Team A need to use Java version 8 and team B requires Java 9.
  • Team X uses JUnit for their unit tests and another team needs TestNG. Also take into account all of the plugins and different versions etc.

Team specific configuration options can also be achieved using the same principles as IaC. For example: when a tool is rolled out, the desired configuration is applied and the tool is ready to use.

Centralized maintenance?

A lot of large organizations have a centralized team to setup and maintain the on-prem tools. When just started, this looks like a good idea. But sooner or later they run into a lot of issues when teams demand “special cases”. Consider the following obstacles which teams might face when pushing their applications through the CI/CD processes:

  • Adjustments to configurations should be implemented by the centralized team. They have to request a “change” and wait for their user story to be picked up in their sprint. Deployments are delayed.
  • Detailed logs are stored within the context of the centralized tools. Only the centralized team has access to it. Development teams cannot view detailed information when their build fails. This creates extra delays since they depend on others to help them debug it. Sometimes the centralized tools are configured incorrectly, thus making it more difficult to pinpoint the exact problem.
  • When things go wrong and servers crash, it impacts a lot of teams. Downtime means delayed releases. Also think of maintenance windows of tools: “Friday between 5pm and Saturday 12AM the tool cannot be used”.
  • A dedicated team to maintain the tools has probably a lot of knowledge. It poses a risk when this knowledge is not available for whatever reason.

There is so much more for example being dependent on other teams like the storage team and the network team, the security team to handle secrets, etc.

Reduce license costs

A lot of organizations seek confidence in professional tool vendors to offer them the tools they need. Licenses for tools come in a lot of flavors. It’s sometimes difficult to predict the exact costs for licenses since this puts pressure on your budgets. When scaling up, license costs go up very fast. At the beginning of a (new) license period it’s difficult, if not, impossible to determine how much licenses you need. Even when you do know the number of licenses needed, the costs can be very high. Sometimes the open source tools available on the market provide enough to get started.

When using cloud based infrastructure and tools, licenses are very rare. You don’t need to pay for a license when spinning up a Windows server in the cloud. When running this in the on prem data-center this is not the case. In a lot of cases it’s just sufficient to have professional support, it should not directly being offered by the cloud vendor itself. No need to pay “customer success managers” 😉 Support costs go down immediately.

Conclusion

While big organizations are well on their way implementing CI/CD processes, they can use some extra tips. On prem infra and tools have their limitations when putting stuff into practice. Cloud providers offer lot of tools and benefits to overcome these limitations. Hope this article inspired you to move on to the next level of CI/CD into your organization.