As technology advances and morphs, startups continue to emerge, looking to differentiate themselves from the rest against incredible odds — some 90% of all startups fail, according to Forbes.

Companies to Watch in 2021

With broad support from industry leaders like Harness, Gruntwork, Spacelift, env0, Scalr, Digger, Terrateam, Massdriver, Terramate, and others, OpenTofu has received formal pledges spanning 140 ...

Linux Foundation Launches OpenTofu: A New Open Source Alternative to Terraform

BILBAO, Spain , Sept. 20, 2023 /PRNewswire/ -- Today, the Linux Foundation announced the formation of OpenTofu, an open source alternative to Terraform's widely used infrastructure as code ...

Backed by industry leaders and hundreds of developers, OpenTofu is set to become the go-to infrastructure as code solution.

With broad support from industry leaders like Harness, Gruntwork, Spacelift, env0, Scalr, Digger, Terrateam, Massdriver, Terramate, and others, OpenTofu has received formal pledges spanning 140+ organizations and 600+ individuals. OpenTofu will ensure the ...

And so we just thought that TF was a little bit too close to Terraform." OpenTofu arrives with the support of numerous companies – including Harness, Gruntwork, Spacelift, env0, Scalr, Digger, Terrateam, Massdriver, and Terramate, among others.

Terraform fork OpenTF renamed and relocated as OpenTofu

In addition to Gruntwork, the other members of the founding group include Harness Labs, Scalr, Env0 and Spacelift, all companies that rely on the open source version of Terraform as a basic ...

Terraform fork gets renamed OpenTofu, and joins Linux Foundation

env0

In the last several years, Kubernetes has become the “go to” standard for managing and orchestrating containerized workloads. Thanks to it’s vendor agnostic nature, you can easily run Kubernetes almost anywhere, and in fact, all the major cloud vendors offer a managed Kubernetes service (AWS EKS, Google GKE, and Azure AKS). With Kubernetes, one of the key advantages is the ease of managing multiple environments and workloads in a single cluster, by separating the cluster into logical areas using namespaces. This post will dive into how we can manage this by using Terraform to both manage the cluster provisioning as well as manage the namespaces. Multiple Environments in One Cluster When using Kubernetes for a team, you usually want to have an isolated environment for each developer, , or pull request. There are a few ways to achieve that using Kubernetes: one way is to create a full blown cluster for each division, but the way we’re focusing on is using . branch Kubernetes namespaces feature While the namespaces feature is quite powerful it’s not without its complications, especially with a large team of developers: How many namespaces do I have on my cluster? Can I remove them? Does anyone use them? Can I schedule them to automatically shut down during night time and weekend? Can I have policies on who can run what and where? Fortunately, with a good management platform, a lot of this can be alleviated. Why Manage Kubernetes using Terraform? As a cloud-native technology that is deployed at wide scale, it’s quite common to manage the deployment of the infrastructure of Kubernetes clusters using Terraform. And where you have many clusters to manage (Dev, Staging, Production, etc.), Terraform allows you to maintain a consistent configuration for the cluster and underlying infrastructure while creating as many clusters as you like with the same configuration in a reliable and easy way. Terraform’s multi-cloud approach lets you use any cloud provider you wish, including using the native managed services that I’ve mentioned above. Similarly, when managing namespaces there’s a few ways, but using Terraform is probably the best overall. The main advantages of using Terraform are: Use the same to provision the Kubernetes infrastructure and to deploy applications into it.Drift detection — `terraform plan` will always present you the difference between reality at a given time and the config you intend to apply.Full lifecycle management — Terraform doesn’t just initially create resources, but offers a single command for creation, update, and deletion of tracked resources without needing to inspect the API to identify those resources.Synchronous feedback — While asynchronous behaviour is often useful, sometimes it’s counter-productive as the job of identifying operation results (failures or details of created resource) is left to the user. e.g. you don’t have IP/hostname of load balancer until it has finished provisioning, hence you can’t create any DNS record pointing to it. — Terraform understands relationships between resources which may help in scheduling — e.g. if a Persistent Volume Claim claims space from a particular Persistent Volume, Terraform won’t even attempt to create the PVC if creation of the PV has failed. configuration language graph of relationships Putting It Into Action I’ve taken the Terraform code from a simple , popular for its ease, to provision a new EKS cluster in my AWS account. AWS EKS Terraform example To run the actual Terraform, I’m using which allows me to quickly deploy and manage my environments based on Terraform templates. If you’re just getting started with env0, you should first get your organization set up and create a new template (check out the for help there). env0 getting started documentation Now we will run the template which will provision the EKS cluster with env0 and get the outputs: Now that our EKS cluster is set up, we can create a new template for the namespaces and our deployment using the cluster name from the outputs. Set up Namespace + Deployment First we use the following code to authenticate against the Kubernetes cluster that we’ve created and create a new deployment with an ngnix: { { = labels = { = }
    namespace = random_string.namespace_name.result
  }

  spec { = selector { = { = }
    }
    template { { = { = }
      }
      spec { { = name  = port { = }

          resources { { = memory = }
            requests { = memory = }
          }
        }
      }
    }
  }
} resource "kubernetes_deployment" "nginx" metadata name "scalable-nginx-example" App "ScalableNginxExample" replicas 1 match_labels App "ScalableNginxExample" metadata labels App "ScalableNginxExample" container image "nginx:1.7.8" "example" container_port 80 limits cpu "0.5" "512Mi" cpu "250m" "50Mi" provider {
  version = region  = }

data {
  name = }

data {
  name = }

provider {
  host                   = data cluster_ca_certificate = base64decode(data . .data)
  token                  = data load_config_file       = false
} "aws" "~> 2.0" var .region "aws_eks_cluster" "eks_cluster" var .cluster_name "aws_eks_cluster_auth" "eks_cluster_auth" var .cluster_name "kubernetes" .aws_eks_cluster .eks_cluster .endpoint .aws_eks_cluster .eks_cluster .certificate_authority 0 .aws_eks_cluster_auth .eks_cluster_auth .token So now let’s make (and run) a new template using that Terraform code which will create the namespace and run the deployment: Running to get the namespace and the deployment and we can see the resources we’ve created inside the cluster: kubectl You can see that we use the and the data resource to get the authentication data, and to be able to create the namespace and the deployment in the cluster. aws_eks_cluster aws_eks_cluster_auth There is another way to achieve this and import the state file of the Kubernetes cluster you created, but for simplicity we took a simpler approach. You can read more about it . here Take advantage of More Features So now that we have a template we can create as many namespaces and deployments as we want, as well as leverage some of the more advanced features of our management platform. Scheduling First, I want to make sure that those resources aren’t running over night and weekends so let’s set up scheduling for those namespaces: Now, env0 will automatically destroy those namespaces on each workday at midnight and will deploy them at 9am when I start my workday. You can even do that on the entire Kubernetes cluster to save even more money. Dev Team Environments Next, now that we have created an easy way for self service provisioning of environments, we want to set up appropriate permissions for everyone on the team to deploy their own environments. In most cases, we want the EKS cluster to be managed by the DevOps team, while the rest of the dev team should be able to provision their own individual environments. You can achieve this by managing different projects with different permissions for each user. So now I will give our DevOps team the ability to provision the Kubernetes cluster by assigning them a “Deployer” role, but the developers will be assigned as a Planners so they can request changes on the cluster but will need the approval of the DevOps to make the changes. For the ephemeral namespaces Project the developers can have a “Deployer” role so they will be able to create environments whenever they like, but everything will be cost effective and no resource will be left behind. Summary As this is a simple example of provisioning an EKS cluster with relevant deployment, we actually created a balance between a self service way to get resources and governance and a cost efficient way to manage those cloud resources without losing control. In addition you can achieve this with any other managed kubernetes service like AKS and GKE, or even just a self managed cluster. You can also leverage env0’s API and CLI to fully automate a per pull request environment, so each PR will get its own namespace and deployment, and once the PR will be merged to master, it will automatically tear down those resources. You can read more about this concept in a great by Avner Sorek. blog post There is a lot more you can achieve with Kubernetes and Terraform using env0, so go ahead and try it yourself. Previously published at https://www.env0.com/blog/kubernetes-environments-using-namespaces-and-terraform

How Namespace and Terraform Can Manage Kubernetes Environments

In my previous , I went through how to create a maintenance mode page for your application, and how to implement it using Terraform and github pages. But the website is just one part of an Application, and often there’s also a public (or private) API that also needs to have a maintenance mode. Let’s see how we can do that using Terraform on API Gateway. blog post Architecture As previously mentioned, we host all of our infrastructure on AWS, with a clear separation between the Application and the API: The front end is a React application, hosted on S3 with CloudFront as a CDN. The Backend services are mostly Serverless using AWS Lambda with API Gateway that manages our public API. DNS is managed by Route 53. Implementing a Solution Beyond the requirements , we had one new one: When the API is in maintenance it should return “503 Service Unavailable” response This led us led us to the following solution: for our webpage maintenance mode with an informative message. Create a new API Gateway that will be dedicated to the maintenance mode This API will use a mock integration It will return the same response for all endpoints using proxy resource This mocked API will co-exists with our real API. We will use and change the base mapping between the real API and the mocked one. custom domain Based on that, let’s see the Terraform code in action: Mocked API Gateway Looking at the code you are able to see that we are using ANY method together with “{proxy+}” to achieve the ability to cover all resources with a simple configuration, and using the response template to generate the “503 Service Unavailable” response with a clear message to the end user. resource {
  name        = description = } {
  rest_api_id = aws_api_gateway_rest_api.maintenance_mode_api.id
  parent_id   = aws_api_gateway_rest_api.maintenance_mode_api.root_resource_id
  path_part   = } {
  rest_api_id   = aws_api_gateway_rest_api.maintenance_mode_api.id
  resource_id   = aws_api_gateway_resource.maintenance_mode_resource.id
  http_method   = authorization = } {
  rest_api_id          = aws_api_gateway_rest_api.maintenance_mode_api.id
  resource_id          = aws_api_gateway_resource.maintenance_mode_resource.id
  http_method          = aws_api_gateway_method.maintenance_mode_method.http_method = timeout_milliseconds = 29000
  passthrough_behavior = request_templates = { = <<EOF
    { : 503}
EOF
  }
} {
  rest_api_id = aws_api_gateway_rest_api.maintenance_mode_api.id
  resource_id = aws_api_gateway_resource.maintenance_mode_resource.id
  http_method = aws_api_gateway_method.maintenance_mode_method.http_method
  status_code = } {
  rest_api_id = aws_api_gateway_rest_api.maintenance_mode_api.id
  resource_id = aws_api_gateway_resource.maintenance_mode_resource.id
  http_method = aws_api_gateway_method.maintenance_mode_method.http_method
  status_code = aws_api_gateway_method_response.maintenance_mode_response_503.status_code

  # Transforms the backend JSON response XML
  response_templates = { = <<EOF EOF
  }
} {
  depends_on  = [aws_api_gateway_integration.maintenance_mode_integration]
  rest_api_id = aws_api_gateway_rest_api.maintenance_mode_api.id
  stage_name  = } "aws_api_gateway_rest_api" "maintenance_mode_api" "maintenance-mode-API" "This is my Maintenance mode API" resource "aws_api_gateway_resource" "maintenance_mode_resource" "{proxy+}" resource "aws_api_gateway_method" "maintenance_mode_method" "ANY" "NONE" resource "aws_api_gateway_integration" "maintenance_mode_integration" type "MOCK" "WHEN_NO_MATCH" "application/json" "statusCode" resource "aws_api_gateway_method_response" "maintenance_mode_response_503" "503" resource "aws_api_gateway_integration_response" "maintenance_mode_integration_response" to "application/json" #set($inputRoot = $input.path('$')) { "message" : "Sorry for the inconvenience but we are performing some maintenance at the moment. We will be back online shortly!" } resource "aws_api_gateway_deployment" "maintenance_mode_deployment" "prod" Now using a custom domain we can control the maintenance mode, so the base path mapping will either point to the real API and in case of a maintenance mode it will point to the mocked API. resource {
  certificate_arn = var.certificate_arn
  domain_name     = var.domain_name
} {
  api_id      = var.is_maintenance_mode ? aws_api_gateway_rest_api.maintenance_mode_api.id : aws_api_gateway_rest_api.api.id
  stage_name  = var.is_maintenance_mode ? aws_api_gateway_deployment.maintenance_mode_deployment.stage_name : aws_api_gateway_deployment.deployment.stage_name
  domain_name = aws_api_gateway_domain_name.domain_name.domain_name
  base_path   = } "aws_api_gateway_domain_name" "domain_name" resource "aws_api_gateway_base_path_mapping" "base_mapping" "/" Also in AWS, we want to ensure that Route53 is pointing in the custom domain we’ve created. As opposed to the application maintenance mode, here our custom domain switches on and off our maintenance mode and not our DNS record. resource {
  zone_id = name    = type    = alias {
    name                   = aws_api_gateway_domain_name zone_id                = aws_api_gateway_domain_name evaluate_target_health = false
  }
} "aws_route53_record" "api_dns_record" var .route53_zone_id var .route53_suffix_domain_name "A" .domain_name .regional_domain_name .domain_name .regional_zone_id ⚠️ Pay attention that this Terraform code does not create the Route53 hosted zone, nor the SSL certificates — you need to complete those as appropriate for your own setup ⚠️ Deployment Now that our system is all configured, all I have to do is change the Terraform variable of the maintenance mode to be true/false and deploy the environment (in our case via the env0 UI). The complete template source code can be found in this which includes all the Terraform code. We hope you find this useful, or get other ideas for more ways to use Terraform for your deployment workflows. github repo About env0 env0 lets your team manage their own environments in AWS, Azure and Google, governed by your policies and with complete visibility & cost management. You can learn more about env0 and you can also . here try it out yourself Feel free to drop us your thoughts below! Previously published at https://www.env0.com/blog/creating-a-maintenance-mode-for-your-api-gateway-with-terraform

Building Maintenance Mode for your API Gateway Using Terraform

As you may have seen, we just launched our of env0 last month. As part of the run-up to the launch, our dev team had to go through and make sure everything about our infrastructure was ready for ongoing public use: one element being creating a for both our Application and our . public beta maintenance mode public API Why do you need a maintenance mode for something like env0, which is built to be highly available? Well whether it’s human error (as in the case of ), a DDOS attack (like with ), or just a major upgrade (like ), even the most highly available applications from the most experienced providers sometimes need to be able to be taken offline for a short period of time. And when you have to do that, you want to make sure that your users understand what is happening and still have a good experience. AWS S3 github Zapier Here’s how our team put together our maintenance mode using Terraform, AWS, and Github Pages (including all the code at the end!) Architecture We host all of our infrastructure on AWS, with a clear separation between Application and API: React front end application is hosted on S3 with CloudFront as a CDN. The Backend services are mostly Serverless using AWS Lambda with API Gateway that manages our public API. DNS is managed by Route 53. Since we have a separation between the frontend application and our API, we need to have a maintenance mode for each of them, especially as we also have a public API that is used by our customers for integration in their CI/CD pipelines and other tools. Don’t forget this if you have multiple ways in which people access your services! Requirements As with any project, the first step is to lay out requirements and constraints. In this case, we came up with the following list of what the solution should do: Be implemented with Infrastructure as Code (IaC) as all of our stack is based on IaC Switch back and forth as fast as possible between normal and maintenance mode Switch automatically between normal and maintenance mode Be hosted by separate providers, where possible, from the current infrastructure in order to provide maximum redundancy Have an internal backdoor to allow our team to deploy a fix and test it before switching back Not override the current configuration with new deployments Communicate to an active user that we are in maintenance mode Implementing a Solution After investigating, we came to a few conclusions: We wanted to have a simple place to store the maintenance mode html, so we went with github pages. The switch itself will be done through the DNS, pointing to github pages when we are in maintenance mode. We will create a new DNS record and a Cloudfront distribution that will be our backdoor, and it will be always up and running. We will use our current push mechanism to notify existing users that are in the app that we are currently in maintenance mode. Based on that, we implemented the following at each piece of the system — each system contains a description of what we’re doing, along with the link to a gist with the actual Terraform code. Cloudfront Looking at the Cloudfront distribution code, you can see that we are creating 2 distributions, one is for the actual application and the other one is for a backdoor: resource { = }

resource { = = = = = origin { = = aws_s3_bucket.website_bucket.bucket_domain_name
    s3_origin_config { = aws_cloudfront_origin_access_identity.origin_access_identity.cloudfront_access_identity_path
    }
  }

  default_cache_behavior { = [ , ] = [ , ]

    forwarded_values { = = []
      cookies { = }
    } = = // day = // year = = = }

  restrictions {
    geo_restriction { = }
  }

  viewer_certificate { = count. == ? var.acm_certificate_arn : var.acm_certificate_arn_backdoor = = } = list(count. == ? var.dns_name : var.backdoor_dns_name)
} "aws_cloudfront_origin_access_identity" "origin_access_identity" comment " access identity" ${aws_s3_bucket.website_bucket.bucket} "aws_cloudfront_distribution" "website_cdn" count 2 # one for the actual cloudfront and one is for the backdoor enabled true price_class "PriceClass_200" http_version "http2" default_root_object "index.html" origin_id "origin-bucket- " ${aws_s3_bucket.website_bucket.id} domain_name origin_access_identity allowed_methods "GET" "HEAD" cached_methods "GET" "HEAD" query_string false headers forward "none" min_ttl 0 default_ttl 86400 1 max_ttl 31536000 1 target_origin_id "origin-bucket- " ${aws_s3_bucket.website_bucket.id} viewer_protocol_policy "redirect-to-https" compress true restriction_type "none" acm_certificate_arn index 0 ssl_support_method "sni-only" minimum_protocol_version "TLSv1.2_2018" aliases index 0 Route53 Also in AWS, we want to ensure that Route53 is pointing in the right direction. In case we are not in maintenance mode it should point to the Cloudfront distribution, and when in maintenance mode it should point our CNAME to the github page. In either case, we should have the backdoor pointing to the Cloudfront distribution. Additionally, we also set the TTL to be 60 seconds so it won’t take too long to move back and forth between maintenance mode and regular operation: resource {
  zone_id = var.aws_route53_zone_id
  name    = var.dns_name = records = list(var.maintenance_mode_enabled ? : aws_cloudfront_distribution.website_cdn[0].domain_name)
  ttl     = 60
} {
  zone_id = var.aws_route53_zone_id
  name    = var.backdoor_dns_name = alias {
    name                   = aws_cloudfront_distribution.website_cdn[1].domain_name
    zone_id                = aws_cloudfront_distribution.website_cdn[1].hosted_zone_id
    evaluate_target_health = }
} "aws_route53_record" "dns_record" type "CNAME" " .github.io" ${var.github_organization} resource "aws_route53_record" "dns_backdoor_record" type "A" false ⚠️ Pay attention that this Terraform code does not create the Route53 hosted zone, nor the SSL certificates — you need to complete those as appropriate for your own setup ⚠️ Git Repo Next, we need a git repo containing the html files for the new maintenance mode site. So we’ve created a simple Terraform code that will create the repo as well as add all existing files in the folder to the repo, which in our case is the maintenance mode html file: “maintenance_mode_website” resource { = var.github_repo_name = = = = }

resource { = github_repository.maintenance-mode-repo.name = var.github_username = = [github_repository.maintenance-mode-repo]
}

resource { = github_repository.maintenance-mode-repo.name = {
    for key, value fileset(path.root, ): => value
  }
  // There is a limitation here, Terraform file function supports UTF- files,
  // so any images other non text files can't be included the folder. = trimprefix(each.value, ) = file(each.value) = [github_repository_collaborator.users_repos]
} "github_repository" "maintenance-mode-repo" name description "Maintenance mode repo for " ${var.github_repo_name} private true auto_init true default_branch "master" "github_repository_collaborator" "users_repos" repository username permission "admin" depends_on "github_repository_file" "maintenance-mode-files" repository for_each in "maintenance_mode_website/*.*" value 8 or in file "maintenance_mode_website/" content depends_on Github Pages The last part is the trickiest to implement in Terraform, because Github pages configuration is not actually part of the , which means that I can’t really configure it with the Terraform code. However because github offers an API to , it can be done programmatically via the API. In our case, we can use the env0 to trigger those API calls once the deploy is finished: github Terraform provider configure github pages custom flows feature version: 1 deploy: steps: terraformOutput: after: - scripts/set_maintanance_mode.sh ${TF_VAR_github_organization} "$(terraform output github_repo_name)" "$(terraform output website_endpoint)" ${TF_VAR_github_username} ${TF_VAR_github_token} set -e

REPO_ORGANIZATION=$1
REPO_NAME=$2
CNAME=$3
GITHUB_USERNAME=$4
GITHUB_TOKEN=$5

if [[ ${REPO_NAME} != "" &&  CNAME != "" ]]; then

    echo "Going to set environment to maintenance mode, repo name: ${REPO_NAME}, CNAME: ${CNAME}"

    GIT_AUTH="${GITHUB_USERNAME}:${GITHUB_TOKEN}"

    curl --user "${GIT_AUTH}" \
     --request POST \
     --header "Content-Type: application/json" \
     --header "Accept: application/vnd.github.switcheroo-preview+json" \
     --url "https://api.github.com/repos/${REPO_ORGANIZATION}/${REPO_NAME}/pages" \
     --data "{ \"source\": { \"branch\": \"master\" } }"

     curl --user "${GIT_AUTH}" \
     --request PUT \
     --header "Content-Type: application/json" \
     --url "https://api.github.com/repos/${REPO_ORGANIZATION}/${REPO_NAME}/pages" \
     --data "{ \"cname\": \"${CNAME}\", \"source\": \"master\" }"
else
    echo "No need to set environment to maintenance mode"
fi # !/usr/bin/env bash Deployment Now that our system is all configured, all I have to do is change the Terraform variable of the maintenance mode to be true/false and deploy the environment (in our case via the env0 UI). Conclusion The complete template source code can be found in this which includes all the Terraform code, scripts, our env0.yml and the maintenance page html file. We hope you find this useful, or get other ideas for more ways to use Terraform for your deployment workflows. Our next blog post will give you a sneak preview on how we are creating a maintenance mode on our API using Terraform. github repo About env0 env0 lets your team manage their own environments in AWS, Azure and Google, governed by your policies and with complete visibility & cost management. You can learn more about env0 and you can also . Feel free to drop us your thoughts below! here try it out yourself

Building a “Maintenance Mode” with Terraform and Github Pages

Attributes offer the ability to add structured, machine-readable metadata information on declarations in code: Classes, methods, functions, parameters, properties, and class constants can be the target of an attribute. I believe the definition is on point, and I'm confident most developers reading this article have encountered attributes at least once. If you haven't, they are essentially metadata added to a class. At this point, you might be wondering how they differ from PHPDOCs then? Well, they are first-class citizens; they are actual PHP classes, and yes I know, it changes the whole game now; you don't have to write regular expressions to extract things from the PHPDocs, and you can even maintain some form of state within the properties. Since I am a bit late to the party, classic examples of attributes abound. So, why not build something cool with them instead? Making Routes Toggleable When working with a team, I often receive messages from other developers (frontend guys, I am looking at you), notifying me that a route isn't working as expected. At times, I wish I could easily disable the route for a specific environment, like the staging environment, while maintaining its functionality locally. This way, my fellow backend developers and I can work on it, push code, and maintain our typical workflow without concerns about accidental usage. Occasionally, it's simply a new route that needs to stay exclusive to the testing environment. So, pondering this, I thought it would be cool to mark an action as disabled or ignored. And guess what? With attributes, this turned out to be super easy and clean. Let's start by creating an attribute. I will name mine Ignore, and it will have a single property called in <?php

namespace App\Attributes;

use Attribute;

#[Attribute(Attribute::TARGET_CLASS | Attribute::TARGET_METHOD)]
class Ignore
{
    public function __construct(
        public array $in = ['production']
    ) {
    }
} That's it, you just created an attribute, you will also notice that we've limited its scope to classes and methods, allowing this attribute to be placed exclusively on those two entities. Now, we can use it like so namespace App\Http\Controllers;

use App\Attributes\Ignore;
use Illuminate\Http\Request;
use Illuminate\Routing\Controller;
use Symfony\Component\HttpFoundation\Response

class TwoFactorQrCodeController extends Controller
{
    #[Ignore(in: ['production', 'staging'])]
    public function show(Request $request): Response
    {
        if (is_null($request->user()->two_factor_secret)) {
            return [];
        }

        return response()->json([
            'svg' => $request->user()->twoFactorQrCodeSvg(),
            'url' => $request->user()->twoFactorQrCodeUrl(),
        ]);
    }
} You can see that this already reads well,;ignore it in production and staging. Still, we need to make this functional, and there are a few methods to achieve this. The simplest is using a middleware. Let's create a middleware, I will name it IsRouteIgnored, feel free to choose any name you prefer php artisan make:middleware IsRouteIgnored Now we can implement the logic; the idea is simple: we intercept the requests of the routes that use this middleware, and we then check if the action has the Ignore attribute, if it does, we check whether the current environment is permitted to have this route or not. For this, we will use the magic of the Reflection API, let's dive into the code. <?php

namespace App\Http\Middleware;

use Closure;
use ReflectionMethod;
use App\Attributes\Ignore;
use Illuminate\Http\Request;
use Illuminate\Routing\Route;
use Symfony\Component\HttpFoundation\Response;

class IsRouteIgnored
{
    public function handle(Request $request, Closure $next): Response
    {
        $route = $request->route();

        if (!($route instanceof Route) || $route->action['uses'] instanceof Closure) {
            return $next($request);
        }

        $reflection = new ReflectionMethod($route->getControllerClass(), $route->getActionMethod());

        $attributes = $reflection->getAttributes(Ignore::class);

        if (!empty($attributes) && in_array(config('app.env'), $attributes[0]->newInstance()->in)) {
            abort(404);
        }

        return $next($request);
    }
} We're creating a reflection of the method the route leads to so we retrieve the Ignore attribute. By default, attributes are not repeatable, meaning they can only be used once per entity. Since we've specified our interest solely in the Ignore attribute, we will end up with a single-element array. We can now instantiate the attribute by calling newInstance(), returning to the realm of regular classes. We then check the environments in which this route should be ignored within the in property. In this case, the route will return a 404 response for the production and staging environments but will function in the local and testing environments. Afterward, you can register the middleware globally or within the API routes, as you would normally do, and you can start ignoring routes by marking them with the attribute. Conclusion With just a few lines of code, we've enabled toggleable routes. While the implementation was relatively basic, the example was meant to showcase the power of attributes. I mean, come on, how cool is that? Toggling routes on and off within specific environments of your choice, you can even adjust the Ignore attribute to exclude the route from all environments except for the ones you specify, and the options are endless. Next time you ponder marking a class as something specific, consider giving Attributes a shot! 🪄 Attributes offer the ability to add structured, machine-readable metadata information on declarations in code: Classes, methods, functions, parameters, properties, and class constants can be the target of an attribute. Attributes offer the ability to add structured, machine-readable metadata information on declarations in code: Classes, methods, functions, parameters, properties, and class constants can be the target of an attribute. I believe the definition is on point, and I'm confident most developers reading this article have encountered attributes at least once. If you haven't, they are essentially metadata added to a class. At this point, you might be wondering how they differ from PHPDOCs then? Well, they are first-class citizens; they are actual PHP classes , and yes I know, it changes the whole game now; you don't have to write regular expressions to extract things from the PHPDocs, and you can even maintain some form of state within the properties. PHPDOCs actual PHP classes regular expressions Since I am a bit late to the party, classic examples of attributes abound. So, why not build something cool with them instead? Making Routes Toggleable When working with a team, I often receive messages from other developers (frontend guys, I am looking at you), notifying me that a route isn't working as expected. At times, I wish I could easily disable the route for a specific environment, like the staging environment, while maintaining its functionality locally. This way, my fellow backend developers and I can work on it, push code, and maintain our typical workflow without concerns about accidental usage. Occasionally, it's simply a new route that needs to stay exclusive to the testing environment. So, pondering this, I thought it would be cool to mark an action as disabled or ignored. And guess what? With attributes, this turned out to be super easy and clean. Let's start by creating an attribute. I will name mine Ignore , and it will have a single property called in Ignore in <?php

namespace App\Attributes;

use Attribute;

#[Attribute(Attribute::TARGET_CLASS | Attribute::TARGET_METHOD)]
class Ignore
{
    public function __construct(
        public array $in = ['production']
    ) {
    }
} <?php

namespace App\Attributes;

use Attribute;

#[Attribute(Attribute::TARGET_CLASS | Attribute::TARGET_METHOD)]
class Ignore
{
    public function __construct(
        public array $in = ['production']
    ) {
    }
} That's it, you just created an attribute, you will also notice that we've limited its scope to classes and methods, allowing this attribute to be placed exclusively on those two entities. Now, we can use it like so namespace App\Http\Controllers;

use App\Attributes\Ignore;
use Illuminate\Http\Request;
use Illuminate\Routing\Controller;
use Symfony\Component\HttpFoundation\Response

class TwoFactorQrCodeController extends Controller
{
    #[Ignore(in: ['production', 'staging'])]
    public function show(Request $request): Response
    {
        if (is_null($request->user()->two_factor_secret)) {
            return [];
        }

        return response()->json([
            'svg' => $request->user()->twoFactorQrCodeSvg(),
            'url' => $request->user()->twoFactorQrCodeUrl(),
        ]);
    }
} namespace App\Http\Controllers;

use App\Attributes\Ignore;
use Illuminate\Http\Request;
use Illuminate\Routing\Controller;
use Symfony\Component\HttpFoundation\Response

class TwoFactorQrCodeController extends Controller
{
    #[Ignore(in: ['production', 'staging'])]
    public function show(Request $request): Response
    {
        if (is_null($request->user()->two_factor_secret)) {
            return [];
        }

        return response()->json([
            'svg' => $request->user()->twoFactorQrCodeSvg(),
            'url' => $request->user()->twoFactorQrCodeUrl(),
        ]);
    }
} You can see that this already reads well,; ignore it in production and staging . Still, we need to make this functional, and there are a few methods to achieve this. The simplest is using a middleware. ignore it in production and staging Let's create a middleware, I will name it IsRouteIgnored , feel free to choose any name you prefer IsRouteIgnored php artisan make:middleware IsRouteIgnored php artisan make:middleware IsRouteIgnored Now we can implement the logic; the idea is simple: we intercept the requests of the routes that use this middleware, and we then check if the action has the Ignore attribute, if it does, we check whether the current environment is permitted to have this route or not. Ignore For this, we will use the magic of the Reflection API , let's dive into the code. Reflection API <?php

namespace App\Http\Middleware;

use Closure;
use ReflectionMethod;
use App\Attributes\Ignore;
use Illuminate\Http\Request;
use Illuminate\Routing\Route;
use Symfony\Component\HttpFoundation\Response;

class IsRouteIgnored
{
    public function handle(Request $request, Closure $next): Response
    {
        $route = $request->route();

        if (!($route instanceof Route) || $route->action['uses'] instanceof Closure) {
            return $next($request);
        }

        $reflection = new ReflectionMethod($route->getControllerClass(), $route->getActionMethod());

        $attributes = $reflection->getAttributes(Ignore::class);

        if (!empty($attributes) && in_array(config('app.env'), $attributes[0]->newInstance()->in)) {
            abort(404);
        }

        return $next($request);
    }
} <?php

namespace App\Http\Middleware;

use Closure;
use ReflectionMethod;
use App\Attributes\Ignore;
use Illuminate\Http\Request;
use Illuminate\Routing\Route;
use Symfony\Component\HttpFoundation\Response;

class IsRouteIgnored
{
    public function handle(Request $request, Closure $next): Response
    {
        $route = $request->route();

        if (!($route instanceof Route) || $route->action['uses'] instanceof Closure) {
            return $next($request);
        }

        $reflection = new ReflectionMethod($route->getControllerClass(), $route->getActionMethod());

        $attributes = $reflection->getAttributes(Ignore::class);

        if (!empty($attributes) && in_array(config('app.env'), $attributes[0]->newInstance()->in)) {
            abort(404);
        }

        return $next($request);
    }
} We're creating a reflection of the method the route leads to so we retrieve the Ignore attribute. By default, attributes are not repeatable, meaning they can only be used once per entity. Since we've specified our interest solely in the Ignore attribute, we will end up with a single-element array. We can now instantiate the attribute by calling newInstance() , returning to the realm of regular classes. We then check the environments in which this route should be ignored within the in property. In this case, the route will return a 404 response for the production and staging environments but will function in the local and testing environments. Ignore Ignore newInstance() in 404 Afterward, you can register the middleware globally or within the API routes, as you would normally do, and you can start ignoring routes by marking them with the attribute. Conclusion With just a few lines of code, we've enabled toggleable routes. While the implementation was relatively basic, the example was meant to showcase the power of attributes. I mean, come on, how cool is that? Toggling routes on and off within specific environments of your choice, you can even adjust the Ignore attribute to exclude the route from all environments except for the ones you specify, and the options are endless. Ignore Next time you ponder marking a class as something specific, consider giving Attributes a shot! 🪄 Next time you ponder marking a class as something specific, consider giving Attributes a shot! 🪄

PHP Development: Creating Toggleable Laravel Routes with Attributes

The OpenTelemetry Collector sits at the center of the OpenTelemetry architecture but is unrelated to the W3C Trace Context. In my , I use Jaeger instead of the Collector. Yet, it's ubiquitous, as in every OpenTelemetry-related post. I wanted to explore it further. tracing demo In this post, I explore the different aspects of the Collector: The data kind: logs, metrics, and traces Push and pull models Operations: reads, transformations, and writes First steps A long time ago, as we know it didn't exist; what we had instead was . Back then, monitoring was a bunch of people looking at screens displaying dashboards. Dashboards themselves consisted of metrics and only system metrics: mainly CPU, memory, and disk usage. For this reason, we will start with metrics. observability monitoring is one of the primary monitoring solutions. It works on a pull-based model: Prometheus scrapes compatible endpoints of your application(s) and stores them internally. Prometheus We will use the OTEL Collector to scrape a Prometheus-compatible endpoint and print out the result in the console. Grafana Labs offers a that generates random metrics to play with. For simplicity's sake, I'll use Docker Compose; the setup looks like the following: project version: "3"

services:
  fake-metrics:
    build: ./fake-metrics-generator                                         #1
  collector:
    image: otel/opentelemetry-collector:0.87.0                              #2
    environment:                                                            #3
      - METRICS_HOST=fake-metrics
      - METRICS_PORT=5000
    volumes:
      - ./config/collector/config.yml:/etc/otelcol/config.yaml:ro           #4 No Docker image is available for the fake metrics project; hence, we need to build it Latest version of the OTEL Collector at the time of this writing Parameterize the following configuration file Everything happens here As I mentioned above, the OTEL Collector can do a lot. Hence, configuration is everything. receivers:                                                                  #1
  prometheus:                                                               #2
    config:
      scrape_configs:                                                       #3
        - job_name: fake-metrics                                            #4
          scrape_interval: 3s
          static_configs:
            - targets: [ "${env:METRICS_HOST}:${env:METRICS_PORT}" ]

exporters:                                                                  #5
  logging:                                                                  #6
    loglevel: debug

service:
  pipelines:                                                                #7
    metrics:                                                                #8
      receivers: [ "prometheus" ]                                           #9
      exporters: [ "logging" ]                                              #9 List of receivers. A receiver reads data; it can be either push-based or pull-based. We use the pre-defined receiver prometheus Define pull jobs Job's configuration List of exporters. In contrast to receivers, an exporter writes data. The simplest exporter is to write data on the standard out Pipelines assemble receivers and exporters Define a metric-related pipeline The pipeline gets data from the previously-defined receiver and sends it to the exporter, , prints them prometheus logging i.e. Here's a sample of the result: 2023-11-11 08:28:54 otel-collector-collector-1     | StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
2023-11-11 08:28:54 otel-collector-collector-1     | Timestamp: 2023-11-11 07:28:54.14 +0000 UTC
2023-11-11 08:28:54 otel-collector-collector-1     | Value: 83.090000
2023-11-11 08:28:54 otel-collector-collector-1     | NumberDataPoints #1
2023-11-11 08:28:54 otel-collector-collector-1     | Data point attributes:
2023-11-11 08:28:54 otel-collector-collector-1     |      -> fake__embrace_world_class_systems: Str(concept)
2023-11-11 08:28:54 otel-collector-collector-1     |      -> fake__exploit_magnetic_applications: Str(concept)
2023-11-11 08:28:54 otel-collector-collector-1     |      -> fake__facilitate_wireless_architectures: Str(extranet)
2023-11-11 08:28:54 otel-collector-collector-1     |      -> fake__grow_magnetic_communities: Str(challenge)
2023-11-11 08:28:54 otel-collector-collector-1     |      -> fake__reinvent_revolutionary_applications: Str(support)
2023-11-11 08:28:54 otel-collector-collector-1     |      -> fake__strategize_strategic_initiatives: Str(internet_solution)
2023-11-11 08:28:54 otel-collector-collector-1     |      -> fake__target_customized_eyeballs: Str(concept)
2023-11-11 08:28:54 otel-collector-collector-1     |      -> fake__transform_turn_key_technologies: Str(framework)
2023-11-11 08:28:54 otel-collector-collector-1     |      -> fake__whiteboard_innovative_partnerships: Str(matrices)
2023-11-11 08:28:54 otel-collector-collector-1     | StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
2023-11-11 08:28:54 otel-collector-collector-1     | Timestamp: 2023-11-11 07:28:54.14 +0000 UTC
2023-11-11 08:28:54 otel-collector-collector-1     | Value: 53.090000
2023-11-11 08:28:54 otel-collector-collector-1     | NumberDataPoints #2
2023-11-11 08:28:54 otel-collector-collector-1     | Data point attributes:
2023-11-11 08:28:54 otel-collector-collector-1     |      -> fake__expedite_distributed_partnerships: Str(approach)
2023-11-11 08:28:54 otel-collector-collector-1     |      -> fake__facilitate_wireless_architectures: Str(graphical_user_interface)
2023-11-11 08:28:54 otel-collector-collector-1     |      -> fake__grow_magnetic_communities: Str(policy)
2023-11-11 08:28:54 otel-collector-collector-1     |      -> fake__reinvent_revolutionary_applications: Str(algorithm)
2023-11-11 08:28:54 otel-collector-collector-1     |      -> fake__transform_turn_key_technologies: Str(framework)
2023-11-11 08:28:54 otel-collector-collector-1     | StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
2023-11-11 08:28:54 otel-collector-collector-1     | Timestamp: 2023-11-11 07:28:54.14 +0000 UTC
2023-11-11 08:28:54 otel-collector-collector-1     | Value: 16.440000
2023-11-11 08:28:54 otel-collector-collector-1     | NumberDataPoints #3
2023-11-11 08:28:54 otel-collector-collector-1     | Data point attributes:
2023-11-11 08:28:54 otel-collector-collector-1     |      -> fake__exploit_magnetic_applications: Str(concept)
2023-11-11 08:28:54 otel-collector-collector-1     |      -> fake__grow_magnetic_communities: Str(graphical_user_interface)
2023-11-11 08:28:54 otel-collector-collector-1     |      -> fake__target_customized_eyeballs: Str(extranet) Beyond printing The above is an excellent first step, but there's more than printing to the console. We will expose the metrics to be scraped by a regular Prometheus instance; we can add a to visualize them. While it may seem pointless, bear with it, as it's only a stepstone. Grafana dashboard To achieve the above, we only change the OTEL Collector configuration: exporters:
  prometheus:                                                               #1
    endpoint: ":${env:PROMETHEUS_PORT}"                                     #2

service:
  pipelines:
    metrics:
      receivers: [ "prometheus" ]
      exporters: [ "prometheus" ]                                           #3 Add a exporter prometheus Expose a Prometheus-compliant endpoint Replace printing with exposing That's it. The OTEL Collector is very flexible. Note that the Collector is multi-input, multi-output. To both print data and expose them via the endpoint, we add them to the pipeline: exporters:
  prometheus:                                                               #1
    endpoint: ":${env:PROMETHEUS_PORT}"
  logging:                                                                  #2
    loglevel: debug

service:
  pipelines:
    metrics:
      receivers: [ "prometheus" ]
      exporters: [ "prometheus", "logging" ]                                #3 Expose data Print data The pipeline will both print data and expose them With the Prometheus exporter configured, we can visualize metrics in Grafana. Note that receivers and exporters specify their type every one of them must be unique. To comply with the last requirement, we can append a qualifier to distinguish between them, , and and i.e. prometheus/foo prometheus/bar. Intermediary data processing A valid question would be why the OTEL Collector is set between the source and Prometheus, as it makes the overall design more fragile. At this stage, we can leverage the true power of the OTEL Collector: data processing. So far, we have ingested raw metrics, but the source format may not be adapted to how we want to visualize data. For example, in our setup, metrics come from our fake generator, "business", and the underlying NodeJS platform, "technical." It is reflected in the metrics' name. We could add a dedicated source label and remove the unnecessary prefix to filter more efficiently. You declare data processors in the section of the configuration file. The collector executes them in the order they are declared. Let's implement the above transformation. processors The first step toward our goal is to understand that the collector has two flavours: a "bare" one and a contrib one that builds upon it. Processors included in the former are limited, both in number and in capabilities; hence, we need to switch the contrib version. collector:
  image: otel/opentelemetry-collector-contrib:0.87.0                        #1
  environment:
    - METRICS_HOST=fake-metrics
    - METRICS_PORT=5000
    - PROMETHEUS_PORT=8889
  volumes:
    - ./config/collector/config.yml:/etc/otelcol-contrib/config.yaml:ro     #2 Use the flavour contrib For added fun, the configuration file is on another path At this point, we can add the processor itself: processors:
  metricstransform:                                                         #1
    transforms:                                                             #2
      - include: ^fake_(.*)$                                                #3
        match_type: regexp                                                  #3
        action: update
        operations:                                                         #4
          - action: add_label                                               #5
            new_label: origin
            new_value: fake
      - include: ^fake_(.*)$
        match_type: regexp
        action: update                                                      #6
        new_name: $${1}                                                     #6-7
# Do the same with metrics generated by NodeJS Invoke the metrics transform processor List of transforms applied in order Matches all metrics with the defined regexp List of operations applied in order Add the label Rename the metric by removing the regexp group prefix Fun stuff: syntax is $${x} Finally, we add the defined processor to the pipeline: service:
  pipelines:
    metrics:
      receivers: [ "prometheus" ]
      processors: [ "metricstransform" ]
      exporters: [ "prometheus" ] Here are the results: Connecting receivers and exporters A connector is both a receiver an exporter and connects two pipelines. The example from the documentation receives the number of spans (tracing) and exports the count, which has a metric. I tried to achieve the same with 500 errors - spoiler: it doesn't work as intended. and Let's first add a log receiver: receivers:
  filelog:
    include: [ "/var/logs/generated.log" ] Then, we add a connector: connectors:
  count:
    requests.errors:
      description: Number of 500 errors
      condition: [ "status == 500 " ] Lastly, we connect the log receiver and the metrics exporter: service:
   pipelines:
     logs:
       receivers: [ "filelog" ]
       exporters: [ "count" ]
     metrics:
       receivers: [ "prometheus", "count" ] The metric is named , but its value stays at 1. log_record_count_total Logs manipulation Processors allow data manipulation; operators are specialized processors that work on logs. If you're familiar with the Elasticsearch Logstash Kibana stack, they are the equivalent of Logstash. As of now, the log timestamp is the ingestion timestamp. We shall change it to the timestamp of its creation. receivers:
  filelog:
    include: [ "/var/logs/generated.log" ]
    operators:
      - type: json_parser                                                   #1
        timestamp:                                                          #2
          parse_from: attributes.datetime                                   #3
          layout: "%d/%b/%Y:%H:%M:%S %z"                                    #4
        severity:                                                           #2
          parse_from: attributes.status                                     #3
          mapping:                                                          #5
            error: 5xx                                                      #6
            warn: 4xx
            info: 3xx
            debug: 2xx
      - id: remove_body                                                     #7
        type: remove
        field: body
      - id: remove_datetime                                                 #7
        type: remove
        field: attributes.datetime
      - id: remove_status                                                   #7
        type: remove
        field: attributes.status The log is in JSON format; we can use the provided JSON parser Metadata attributes to set Fields to read from Parsing pattern Mapping table Accept a range, , . The operator has a special interpreted value (and similar) for HTTP statuses. e.g. 501-599 5xx Remove duplicated data Logs At this point, we can send the logs to any log aggregation component. We shall stay in the Grafana Labs sphere and use Loki. exporters:
  loki:
    endpoint: "http://loki:3100/loki/api/v1/push" We can also use logs from the collector itself: service:
  telemetry:
    logs: Finally, let's add another pipeline: service:
  pipelines:
    logs:
      receivers: [ "filelog" ]
      exporters: [ "loki" ] Grafana can also visualize the logs. Choose Loki as a datasource. Conclusion In this post, we delved into the OpenTelemetry collector. While it's not a mandatory part of the OTEL architecture, it's a useful Swiss knife for all your data processing needs. In case you're not stuck to a specific stack or don't want to, it's a tremendous help. The complete source code for this post can be found on . GitHub To go further: OpenTelemetry Collector OpenTelemetry Operators Originally at on November 12th, 2023 published A Java Geek

Delving Into OpenTelemetry Collector

ওপেনটেলিমেট্রি সংগ্রাহকের মধ্যে যোগ করা

Eintauchen in OpenTelemetry Collector

Profundizando en OpenTelemetry Collector

Plonger dans le collecteur OpenTelemetry

ओपनटेलीमेट्री कलेक्टर में गहराई से जाना

OpenTelemetry コレクターの詳細を調べる

OpenTelemetry Collector 살펴보기

Aprofundando-se no OpenTelemetry Collector

Углубляемся в сборщик OpenTelemetry

OpenTelemetry Collector'ı İncelemek

Đi sâu vào Bộ thu thập từ xa mở

深入研究 OpenTelemetry Collector

Let's learn about via these 51 free stories. They are ordered by most time reading created on HackerNoon. Visit the to find the most read stories about any technology. Terraform /Learn Repo 1. 6 Free Terraform Tools to Facilitate a Multi-Cloud Infrastructure Have a multi-cloud infrastructure? These free Terraform tools will help you to reduce costs and make your multi-cloud efficient. 2. Developer Trends from Thoughtworks’ Technology Radar 27 Thoughtworks’ Technology Radar is a regular time to take a look at what trends are changing our industry. 3. Serverless API with Terraform: GO and AWS [Part 2] In part 2, we'll go over deploying serverless API to AWS Cloud with Terraform. 4. How a Unicorn Startup in Japan Leveraged the Power of Microservices Startup experience in Mercari. Road to microservices from monolith. 5. Getting Started With Creating, Managing an AWS ECS Cluster With Terraform In this tutorial, you'll define AWS resources using Terraform and how resource definitions translate to actual resources created on AWS 6. Basics of AWS Tags & Terraform with S3 - Part 1 In this video course, you'll learn the basics of AWS tags and Terraform with an S3 bucket. 7. How to Test Terraform the Right Way In this blog post, you'll learn about why you should be testing IaC and the best platforms to get started with. 8. Static Code Analysis  for Infrastructure as Code Using Azure DevOps Pipelines This article explains how to set up static code analysis for infrastructure as code using Azure DevOps pipelines. 9. Should Deploying Production Infrastructure be Hard? Below I'll explain why I built terra_boi to deploy Clientelify, why I think it was downloaded 1,000 times in the first 10 days (despite not promoting it anywhere), and why deploying an application to production shouldn't suck. 10. How To Manage Infrastructure With Terraform What is Terraform? 11. Getting Started Provisioning an AWS EKS Kubernetes Cluster with Terraform Learn how to provision an AWS EKS Kubernetes cluster with Terraform. See a full step-by-step tutorial. 12. How Infrastructure as Code Is Forcing a Revolution in Cost Management You might have read our beta launch blog post, which describes how Infrastructure as Code (IaC) is driving the third datacenter revolution (the first two being virtualization and the move to public clouds). In this post, I want to go deeper into looking at how IaC is driving a new need and approach to cost visibility and management. 13. Automating User Management in the Company Have you ever faced the challenge of automating users in your company? 14. Introducing Driftctl: Your IaC Security Belt We recently released the first versions of driftctl, a new open-source project for infrastructure developers, DevOps, SRE, and cloud practitioners, with the goal of helping manage all kinds of drifts. 15. Deploying a Terraform Remote State Backend with AWS S3 and DynamoDB In this article, we are going to be deep diving into remote state management in terraform. 16. Another Reason to Use Docker Recently I was working on a project which includes Terraform and AWS stuff. While working on that I was using my local machine for terraform code testing and luckily everything was going fine. But when we actually want to test it for the production environment we got some issues there. Then, as usual, we started to dig into the issue and finally, we got the issue which was quite a silly one 😜. The production server Terraform version and my local development server Terraform version was not the same. 17. Building Maintenance Mode for your API Gateway Using Terraform In my previous blog post, I went through how to create a maintenance mode page for your application, and how to implement it using Terraform and github pages. But the website is just one part of an Application, and often there’s also a public (or private) API that also needs to have a maintenance mode. Let’s see how we can do that using Terraform on API Gateway. 18. How to Deploy a Django Application on AWS with Terraform The final goal of this guide is to create a scalable and reproducible setup of the Django Web Application on AWS using Terraform. 19. How To Sync Terraform Config And .tfstate File For Existing AWS Resources Terraform is a great (and dominant) infrastructure automation tool. It is multi-cloud, can build all sorts of resources, and in some cases supports API calls to build resources before the native tooling from cloud providers does. 20. Managing Active Directory Objects with Azure AD Provider for Terraform Learn how to manage Active Directory Objects with Azure AD Provider for Terraform. See examples of how to authenticate and grant the correct permissions. 21. Why you should take care of Infrastructure Drift When talking about infrastructure drift, you often get knowing glances and heated answers. Recording gaps in your infra between what you expected to be and the reality of what is, is a well known and widespread issue bothering hundreds of DevOps teams around the globe. 22. Managing a GitHub Organization With Infrastructure as Code This post will share our experience in adopting infrastructure as code (IaC) to manage GitHub organization resources. 23. Using Jupyter/TensorBoard In Any Cloud With One Command Using Terraform Provider Iterative for bespoke Machine Learning & Data Science on AWS, Azure, GCP and K8s 24. The Top 5 DevOps Tools and Services to Consider as a Startup A collection of DevOps tools can significantly enhance your Software Development Life Cycle (SDLC) and increase engineering productivity. 25. Change the Name of your AWS S3 Bucket in Terraform Without Breaking Things Learn how to rename an AWS S3 bucket in Terraform. See renaming process of Amazon s3 bucket in a few simple steps. 26. Exporting Your GKE Cluster to Terraform Cloud: A Guide with Challenges and Solutions In this article, I'll share our journey at ANNA Money from using our Kubernetes installation in GCP to the managed Kubernetes service by Google and GKE. 27. Hashicorp's Terraform: A Introduction Join us in the journey of learning Terraform. This blog series aims to provides a guidance for essentials to be known while working with Terraform. 28. A Handy Guide to Migrating Terraform State to GitLab CI/CD Migrating Terraform state to GitLab CI/CD is a GitOps best practice that allows multiple engineers to work together to develop infrastructure. 29. How Namespace and Terraform Can Manage Kubernetes Environments In the last several years, Kubernetes has become the “go to” standard for managing and orchestrating containerized workloads. Thanks to it’s vendor agnostic nature, you can easily run Kubernetes almost anywhere, and in fact, all the major cloud vendors offer a managed Kubernetes service (AWS EKS, Google GKE, and Azure AKS). 30. Building a “Maintenance Mode” with Terraform and Github Pages As you may have seen, we just launched our public beta of env0 last month. As part of the run-up to the launch, our dev team had to go through and make sure everything about our infrastructure was ready for ongoing public use: one element being creating a maintenance mode for both our Application and our public API. 31. How To Migrate An Existing Infrastructure into Terraform Terraform is a powerful tool to have in your toolset. 32. How To Setup Continuous Integration Pipeline By Using Terraform And GitLab CI Terraform is a fantastic tool for managing your cloud infrastructure, especially if your assets are hosted on multiple cloud providers. 33. Learn How to Build a CI/CD Pipeline Using GitLab for your Website We'll run through how to build and configure a CI/CD pipeline to deploy your business's website using AWS, Terraform & Gitlab. 34. How to Optimize for Iteration: Choosing Your Early Startup's Tech Stack As we’ve discussed in a previous blog post, product development is hypothesis testing. This is especially true in the early stages of a company when you need to confirm or reject your hypothesis as quickly as possible. This process is then repeated until you (hopefully) reach product-market fit. To get there, your team needs to be able to work and build at a pace that allows for this constant and rapid iteration. 35. What Really Propelled The Cloud Revolution: A Developer's Perspective An overview of the main trends that have pushed the Cloud industry — and the world — forward, in the past decade and a half. 36. Sanity-Checks with Playwright in Fargate container on schedule There is often the need to check whether your production system is working as expected. Playwright is a great way to test if everything is going to plan. 37. From “Feature branches” to “Feature environments” with Terraform “The advantage of feature branching is that each developer can work on their own feature and be isolated from changes going on elsewhere.” (FeatureBranch) 38. Serverless API with Terraform, GO and AWS, Part 1 Knowing how to build REST API with latest tech is cool. You know what's even cooler? It's being able to deploy it to the cloud! I'll walk you through the process of building simple, server-less application using GO, AWS (Lambda, API Gateway, Dynamodb) and Terraform. 39. Improve Your IaC With a Specialized Management Platform Most often, there is no point in implementing an in-house solution as the cost and effort of building and maintaining it may exceed its potential benefits. 40. Kubernetes Terraforming on Linode: Linode + Terraform + Kubernetes = Cloud Nirvana Let's discuss the process of "Terraforming" a Kubernetes cluster on Linode using Terraform to help organizations efficiently manage and scale  infrastructure. 41. 10+ Best Resources to learn DevOps in Depth [Online Courses] DevOps is one of the most in-demand skills from employer and there are many job opportunities lying for full stack developers, distinguished engineers and DevOps professionals. If you are an experienced Java programmer or a full-stack web developer, and want to become a DevOps engineer then you have come to the right place. 42. Ad Hoc Environments for Django Applications With ECS, Terraform and GitHub Actions This article will show how software development teams can build on-demand environments for dog-food testing, quality review, internal and external demos and oth 43. Create Your New Reality With Wirehole And Other Awesome Automation Tools **UPDATE October 13, 2020 - The mjtechguy/terransible-wirehole Github repo now supports deploying to Digital Ocean and Hetzner Cloud as well! 44. GraphQL Is Not a Silver Bullet GraphQL is not a configuration language, also not a transformation language, a replacement for SQL, and definitely not Terraform. 45. Building an AWS EKS Fargate Cluster with Terraform How to build an AWS EKS Fargate cluster using terraform. 46. Terraform Configuration Syntax Overview All you need to know to get started about Terraform Syntax. 47. Is Terraform Still Useful? Terraform is an agentless infrastructure tool used mainly by DevOps for infrastructure management. Find out why you should use Terraform for infrastructure. 48. 6 DevOps Trends in 2022 That DevOps Engineers Should Adopt The role of DevOps engineers, their responsibilities, growth opportunities, a set of important soft & hard skills, and most importantly, DevOps trends in 2022. 49. Deploying an Open Source Secure Video Conference Server [A How To Guide] In this article, you'll create a new DigitalOcean account using a free credit link. Then, you will clone a GitHub repository, and use Terraform code to initialize, plan and apply resources to your account, using a preconfigured image from DigitalOcean and your custom domain name. Finally, you will access your account remotely via SSH and execute two scripts to configure both your video conference server and Let's Encrypt SSL to enable HTTPS. 50. Newsflash - "People with AWS Console Access Tend to Change Things" This blog post is a written transcript of the FOSDEM Talk: “Infrastructure drifts aren’t like Pokemon, you can’t catch ’em all”, by Stephane Jourdan – CTO and founder 51. Introduction to AWS Log Insights as CloudWatch Metrics A step-by-step description of how to create an AWS Lambda to convert Cloudwatch LogInsights into metrics Thank you for checking out the 51 most read stories about Terraform on HackerNoon. to find the most read stories about any technology. Visit the /Learn Repo

51 Stories To Learn About Terraform

Let's learn about via these 46 free stories. They are ordered by most time reading created on HackerNoon. Visit the to find the most read stories about any technology. Infrastructure As Code /Learn Repo 1. How To Configure CloudFront Using CloudFormation Template I recently worked on implementing CloudFront for s3 bucket files. Most of the tutorials were doing that using console management(UI) but we at Blue Sky Analytics prefer “code as infrastructure”. So I had to do this using CloudFormation. With the help of a few StackOverflow links and tutorials, I was able to write CloudFormation Template. 2. How to Test Terraform the Right Way In this blog post, you'll learn about why you should be testing IaC and the best platforms to get started with. 3. The Relevance of Infrastructure-as-Code Concepts in Security So what is Infrastructure-as-Code (IaC) anyway? IaC is a process that automates the provisioning and management of cloud resources. 4. Change the Name of your AWS S3 Bucket in Terraform Without Breaking Things Learn how to rename an AWS S3 bucket in Terraform. See renaming process of Amazon s3 bucket in a few simple steps. 5. Using a Product-Led Growth Mindset to Improve User Experience How changing the development mindset to a product-led-growth one can improve the user experience by focusing the thought process for the team's members. 6. What Really Propelled The Cloud Revolution: A Developer's Perspective An overview of the main trends that have pushed the Cloud industry — and the world — forward, in the past decade and a half. 7. Static Code Analysis  for Infrastructure as Code Using Azure DevOps Pipelines This article explains how to set up static code analysis for infrastructure as code using Azure DevOps pipelines. 8. 6 Free Terraform Tools to Facilitate a Multi-Cloud Infrastructure Have a multi-cloud infrastructure? These free Terraform tools will help you to reduce costs and make your multi-cloud efficient. 9. How to Create an Azure API Management Instance using Bicep Lang via Azure DevOps The more I use Bicep, the more I love it. This is what ARM Templates should have been. 10. Deploy Docker Using Ansible 11. Smart E-bike Kits: A Tour Inside Our Servers The Idea 12. Learn About Infrastructure as Code in 5 Minutes and Why You Should Use It Infrastructure as Code (IaC) is the process of managing and configuring an infrastructure using configuration files, rather than manually editing configurations 13. Improve Your IaC With a Specialized Management Platform Most often, there is no point in implementing an in-house solution as the cost and effort of building and maintaining it may exceed its potential benefits. 14. Managing a GitHub Organization With Infrastructure as Code This post will share our experience in adopting infrastructure as code (IaC) to manage GitHub organization resources. 15. Why Learn Any Other Programming Language When You Can Learn Typescript? Use TypeScript in all your projects and all use cases without any limitation. You can now do frontend, backend, infrastructure and machine learning in one progr 16. Code as Documentation and Assessment Documenting IT networking lab procedures in code rather than word-processor documents enables more authentic, equitable and consistent assessment. 17. AWS CloudFormation Template Basics Get started creating IaC using CloudFormation templates. 18. Why Understanding Infrastructure-as-code is Critical to Scaling Your Technology Infrastructure Infrastructure as code (IaC) is one of the most important trends in modern IT development. Read about the benefits of IaC and how to implement it. 19. How Infrastructure as Code Is Forcing a Revolution in Cost Management You might have read our beta launch blog post, which describes how Infrastructure as Code (IaC) is driving the third datacenter revolution (the first two being virtualization and the move to public clouds). In this post, I want to go deeper into looking at how IaC is driving a new need and approach to cost visibility and management. 20. How To Manage Infrastructure With Terraform What is Terraform? 21. Codify your SaaS Apps: The Answer to the Unmanaged SaaS Jungle A phenomenon we have encountered often, when helping companies overcome drift, is a common neglect of the entire SaaS toolchain. Learn why this is troubling. 22. Three Ways to Scale Your Apps with Render When your app is experiencing high traffic or utilization, you need to scale your service to handle that load. We’ll explore three ways to scale your apps. 23. Building Your Infrastructure With Just a Diagram This hands-on tutorial will teach you how to create infrastructure via the AWS Application Composer Console. 24. Check Out the Principles and Best Practices of Infrastructure as Code for 2023 Infrastructure-as-code is a very important concept to understand in the DevOps world today. 25. How To Migrate An Existing Infrastructure into Terraform Terraform is a powerful tool to have in your toolset. 26. Hashicorp's Terraform: A Introduction Join us in the journey of learning Terraform. This blog series aims to provides a guidance for essentials to be known while working with Terraform. 27. 2023 Will Be the Year of Kubernetes (and Other Predictions in the Cloud and Infrastructure Industry) Dive into the new year’s top cloud and infra-tech trends, straight from a cloud and infrastructure technology fanatic. 28. Introducing Driftctl: Your IaC Security Belt We recently released the first versions of driftctl, a new open-source project for infrastructure developers, DevOps, SRE, and cloud practitioners, with the goal of helping manage all kinds of drifts. 29. Cloud Application Infrastructure from Code (IfC) Presents a relatively new concept of Infrastructure from Code (IfC) as the next logical step in cloud automation 30. Project-as-Code: The Catalyst DevOps Needs If you are familiar with Docker, Terraform, and the CI platforms (eg: Jenkins, CircleCI, Codeship), you already know the power of Declarative DevOps. It can make development easier by being repeatable, predictable, and fast. Supporting technologies both hide complexities and offer important reuse by supporting simple structured syntax in an easy to create and read file. Each technology has codified much of its domain allowing developers to author and instrument with nothing more than a text editor. 31. I Got a New EKS Cluster Running And Terraform Code in 2 minutes AWS EKS + Terraform + Cloudskiff do the job 32. B2B Fintech Trends for 2021: Storage, Security, Cloud Migration, IaC As CTO at Altar.io, a large part of my work revolves around observing the forward-looking industry trends (including fintech trends) that affect how we create innovative products. 33. Why you should take care of Infrastructure Drift When talking about infrastructure drift, you often get knowing glances and heated answers. Recording gaps in your infra between what you expected to be and the reality of what is, is a well known and widespread issue bothering hundreds of DevOps teams around the globe. 34. The State of Cloud Automation and the Promise of Infrastructure Automation The Cloud has become a familiar feature of the tech stacks at most organizations and is becoming much more than a place for server resources. At present, almost everything in our digital landscape is connected to a Cloud in one way or another. 35. From “Feature branches” to “Feature environments” with Terraform “The advantage of feature branching is that each developer can work on their own feature and be isolated from changes going on elsewhere.” (FeatureBranch) 36. How To Setup Continuous Integration Pipeline By Using Terraform And GitLab CI Terraform is a fantastic tool for managing your cloud infrastructure, especially if your assets are hosted on multiple cloud providers. 37. Creating and Provisioning Azure Container Apps with Bicep Using Bicep, we can deploy Azure Container Apps quickly and easily! 38. Building a “Maintenance Mode” with Terraform and Github Pages As you may have seen, we just launched our public beta of env0 last month. As part of the run-up to the launch, our dev team had to go through and make sure everything about our infrastructure was ready for ongoing public use: one element being creating a maintenance mode for both our Application and our public API. 39. Application Deployment: Recipe for Great Serverless Strategy This is true for the most part, but Serverless apps also have a certain property that can make their deployment and maintenance time consuming. 40. Exporting Your GKE Cluster to Terraform Cloud: A Guide with Challenges and Solutions In this article, I'll share our journey at ANNA Money from using our Kubernetes installation in GCP to the managed Kubernetes service by Google and GKE. 41. What is Everything-as-Code? Examining  the Explosion of "as Code" Buzzwords If you’re confused when you read about “[some software term] as code” or “everything as code,” all you really need to know is that we’re talking about automation: The thing we use to do tedious tasks for us, or orchestrate tasks when they become too large and complex for manual methods. 42. Creating a Instance Scheduler using AWS CDK The AWS CDK is a software development framework to define cloud infrastructure as code and provision it through CloudFormation. The CDK integrates fully with AWS services and allows developers to use high-level construct to define cloud infrastructure in code. 43. Cloud-Native Yay or Nay: Reasons to Use Cloud-Native Technologies Cloud-native has been the talk of the town for quite some time now. Some developers think it’s just hyped way too much and will soon diminish from the limelight. Others think cloud-native will revolutionize software development and is here to stay! 44. Newsflash - "People with AWS Console Access Tend to Change Things" This blog post is a written transcript of the FOSDEM Talk: “Infrastructure drifts aren’t like Pokemon, you can’t catch ’em all”, by Stephane Jourdan – CTO and founder 45. How Do You Manage Helm Charts? Whether you love it or hate it, Helm is a ubiquitous tool for managing Kubernetes applications. You can use it in many different ways which is great, but can also be overwhelming. 46. Terraform Configuration Syntax Overview All you need to know to get started about Terraform Syntax. Thank you for checking out the 46 most read stories about Infrastructure As Code on HackerNoon. to find the most read stories about any technology. Visit the /Learn Repo

46 Stories To Learn About Infrastructure As Code

139 Stories To Learn About Cicd

Meet the Winners of Startups in North and Central America

This blog post is a written transcript of the FOSDEM Talk: “Infrastructure drifts aren’t like Pokemon, you can’t catch ’em all”, by Stephane Jourdan – CTO and founder This talk covers three major topics: Infrastructure as Code: all the good intentions and the ideal world each of us expected when we started using it, and how it’s actually going in everyday’s Ops life. We will see that how it started is probably different from how it is going and from what we expected. We will then “drift” together, using Terraform and AWS and share some war stories that we heard from infrastructure teams, and how things sometimes went really wrong for them. We will finally present driftctl, our open source answer to infrastructure drift problems. Infrastructure drift: definition Let’s first agree on what is infrastructure drift: infrastructure drift is what happens when the reality of your infrastructure doesn’t match your expectations. This phenomenon is also known as “Oh my god” in production when things unexpectedly go wrong. Infrastructure as code: an ideal world How it all started When it all started, we had this vision of an ideal world, and a bright future that Infrastructure as Code was to offer us, like Plato’s Republic was for society a while ago. With Infrastructure as code, we were supposed to have one single source of truth (the state file in the case of Terraform). We were supposed to have versioning of the code with git. The code was to be so great that it would be self-documenting. In this ideal world, we would have testing everywhere: on the pipelines before and after tdd and bdd etc, like the rest of engineering. Collaboration would be a bliss between teams. Infrastructure would be immutable: we could scale it up and down and throw the rest away. GitOps would be everywhere: CI/CD for infrastructure, no more locally running things. And basically all the credentials would be under control and multi-cloud would solve all our problems because you know what they say: “API everywhere”… and we would have the skills to handle all those APIs, and the time to do this greatly. How it is actually going Marcus Aurelius, the roman emperor a few centuries after Plato wrote to himself: . As it happens we are not Roman emperors but our ideal Republic is not to be expected either. How it started is not exactly how we see it going. “Do not expect Plato’s ideal Republic” In our team, many of us were quite experienced but we wanted more feedback on what we were building. So we just asked hundreds of infrastructure teams, SREs… etc around the world where they were standing in their Infrastructure as Code journey. Basically what they told us was that changes were still happening outside of their Infrastructure as Code: mainly manual changes and scripts that are changing tags, or lambdas updating things after deployment, etc… A lot of changes are still happening outside of their Infrastructure as Code scope, be it Terraform or Pulumi or whatever infrastructure tool they use, it doesn’t matter. Do not expect Plato's ideal Republic - Marcus Aurelius A bunch of other things go wrong for them as well. The scope of their changes is not very often fully committed because of broken processes. People are still changing things manually on their AWS web console and then backporting the changes in Terraform until Terraform starts “screaming”, which means that people are still running Terraform locally with the right access keys on their own laptop. Terrible processes…  and everybody knows that once you click everywhere on your console, a lot of other things are deployed on the background and so your Terraform code doesn’t reflect exactly what’s actually deployed online. Many people we talked to were aware of the good practices about testing requirements. They knew  frameworks like Terratest, etc… but most of them never had the time to either learn or write those tests. Sometimes they were just too costly to run for them. The lack of communication obviously makes things worse: even in these Covid19 times, “Async” communication is still a major issue between teams. Immutable infrastructure also is still a great struggle to achieve. A lot of the new projects are running more or less around this principle, but people also still have legacy to deal with, like thousands of VMs running every day. Lots of people are still gaining console access. And guess what ? unless you lock them down completely, which is sometimes just not possible because they are your boss or your customer, or because they are “this very special co-worker” who needs to do “that special thing” manually on the web console… That’s where problems happen. People with AWS console access tend to change things Many people are also facing skill issues. For example, if a full team very skilled on GCP has a new project that for some reason needs to run on Azure, as all the specifics of this provider are not immediately mastered, they often end up with bad practices in their Infrastructure as Code deployments. And basically for lots of them, everything that has an API is still not under infrastructure code: we heard terrible stories about GitHub accesses, or DNS records or anything that can be used with Terraform that wasn’t yet by far deployed with it. Drift happens All those  behaviors have something in common: they generate drift.  While drift is pretty cool for cars, the discrepancy between intention and reality can lead to major issues, disasters, or security misconfiguration. Drift can be a real major pain. To deal with it, most of the people that we met put something in place like and in a cron tab or in CI/CD somewhere before and after some commits, but were still missing things. terraform plan apply $ terraform apply
Apply complete! Resources: added, changed, destroyed 0 0 0 We love Terraform and all Hashicorp products. Terraform is an excellent provisioner but it’s not fair to use it for something it’s not meant to do. If anything is outside its scope, it’s just normal that Terraform doesn’t catch it. There are a lot of changes that cannot be seen by Terraform. Many of us expected in cron tabs or in CI/CD to report changes because some of the resources were managed by it, but unfortunately as we are going to see, it is not always the case. terraform plan Stories from the trenches The following stories are based on real stories that we heard while interviewing infrastructure teams. We simplified them to make them easy to understand with some code. How an intern with read-only access ended up with rogue administrative access and keys (without anyone noticing) The first story is about an intern with read-only access, ending up with access keys controlled by no one. Those keys were linked to an administrative policy, which nobody noticed. To gain a better understanding of this story, let’s look at this very simple Terraform configuration file: We have a user with a key and a policy attachment for read-only. Let’s apply this configuration: $ terraform apply
Apply complete! Resources: added, changed, destroyed 9 0 0 Now let’s move to the AWS console: here we can see again this intern user and the key that was created by our own code. The story goes like this: This intern went to the DevOps folks and complained that he couldn’t use his key for whatever reason. The team was in a rush, so they quickly logged in to the management console, they deactivated the first key and created a new one, perfectly sure they were going to remember it and add it to Terraform in just a few moments. On top of that, Terraform was to notify them if they forgot anyway. What do you think happened?  Let’s check with a on this user. terraform apply $ terraform apply
Apply complete! Resources: added, changed, destroyed 0 0 0 Basically nothing… While this very simple use case may have led you to think that this specific key and user would be noticed and reported it was not the case. So the story continues with this intern: the intern comes back one hour later complaining that he still couldn’t do what he needed to do. As the infrastructure team did not have the time to look into it, they just gave him administrative access so that he could change whatever he needed to, thinking they would revert through his permissions quickly after. What do you think Terraform found in this case with a , as we have a policy attached to the IAM user and everything is managed by terraform? terraform apply This team was sure that this was to be reported in or terraform plan terraform apply. But it wasn’t the case because basically it’s out of the control of Terraform. It’s just an attached resource, it doesn’t work like that. $ terraform apply
Apply complete! Resources: added, changed, destroyed 0 0 0 So the team forgot about it, they went on with the day and it was only weeks after that they realized they had an intern with an administrative key out there in the wild, uncontrolled by anyone, not even mentioned as “deactivated” on the code. How an intern with rogue Administrative access opened everything to anyone on IPv4 and IPv6 (still without anyone noticing) The next story starts with a process: as you know, you can have lambdas, scripts etcetera that have administrative access to some resources on a very limited scope. In this case someone with an administrative access just opened everything to anyone on IPv4 and IPv6 and no one noticed anything. So how did it go? Let’s go into the console > VPC > us-east-1 > Security groups So we have our super secure security group that we created using Terraform, with a rule that is attached to it. As we can see in the AWS console, we have our “super secure security group” and we have our ssh rule for allowing ssh to this local network For the sake of simplicity we are doing this demo manually here, but someone or something could change this specific super secure security group and could allow anything anywhere on any kind of networks on IPv4 etc… and name it very, very carefully “temp” (you know, like those temporary rules that we’ll remember soon to change…) So what happened here is that something, with the correct access rights allowed anything to anywhere on this network behind this security group. This team expected any kind of rules that was added or modified to be reported on terraform and and basically the support team went weeks or months with such a rule live on their system without any notification. while it is true for a rule it is not true for a new rule $ terraform apply
Apply complete! Resources: added, changed, destroyed 9 0 0 How a scripting issue created a billing nightmare Our last story is about a team that was building a lot of stuff on S3. They were putting a lot of their binaries there and removing them. So obviously they removed the versioning. For some reason (long story short) someone activated versioning on all the buckets for the company including this one but they were pretty sure that it wasn’t going to be an issue because it’s the default on Terraform to not be versioned. But the code didn’t explicitly say it so. Let log in on the AWS console and go to S3> bucket Now we are on this bucket that we created using the terraform code above, and basically we are going do manually what they did through script: we are just going to enable bucket versioning. So now Bucket versioning is enabled. Let’s check with a what was reported: terraform apply $ terraform apply
Apply complete! Resources: added, changed, destroyed 9 0 0 Nothing. So this poor team versioned for weeks hundreds and hundreds of terabytes a day of binaries until they received the bill… Solutions at hand There are a lot of solutions in our ecosystem to work around the kind of things that we just saw: CI/CD Integration (Jenkins, Terraform Cloud, Atlantis, env0…) Static Analysis (Checkov, TFLint, TFSec,…) are there to help you analyze your code before you actually push to your CI system (or if they are deploying the CI/CD systems, they are there to prevent you to push things like “allow anything to anyone” directly on the code) Testing & Verification like Terratest (written in Go), InSpec (for the “older” of us, it’s written in Ruby and comes from the Chef days). Policy & Compliance (Sentinel…) They are a very different set of tools, very cool and very protective. Driftctl That’s where drift control enters: we built driftctl to manage this drift and be alerted on all the discrepancies between the state and the reality on the AWS accounts. Let’s just use the same terminal and the same session that we had previously. drift control will basically report you all the changes that you could not see with or . terraform plan terraform apply $ driftctl scan -- tfstate: Terraform provider initialized (name=aws, alias=us-east )
Terraform provider initialized (name=aws, alias=eu-west )
Terraform provider initialized (name=aws, alias=eu-north )

Found deleted resources:
  - Table: rtb c5a9bb483ee6eaaf, Destination: / Found unmanaged resources:
  aws_security_group_rule:
  - Type: ingress, SecurityGroup: sg ca93f95fcc338, Protocol: All, Ports: All, Source: ::/ - Type: ingress, SecurityGroup: sg ca93f95fcc338, Protocol: All, Ports: All, Source: / aws_iam_access_key:
  - AKIASBXWQ3AYRSNQN5EY (User: INTERN mszmf)
Found drifted resources:
  - AKIASBXWQ3AYWWYX72GX (User: INTERN mszmf) (aws_iam_access_key):
     ~ Status: => (computed)
Found resource(s)
- % coverage
- covered by IaC
- covered by IaC
- deleted on cloud provider
- / drifted IaC from //./s3/terraform.tfstate --from tfstate://./vpc/terraform.tfstate --from tfstate://./iam/terraform.tfstate -1 -3 -1 -0 10.0 .1 .0 24 -0746 0 -0746 0.0 .0 .0 0 -8 -8 "Active" "Inactive" 26 84 22 3 not 1 1 22 from Now, we can see the rules that allowed “anything to anywhere” on IPv4 and IPv6 and we know that there are rules that were not expected. We also have the policy attachment of that intern we talked about. So probably with your eyes of administrative people, DevOps, whatever you are, it doesn’t match. It should ring a bell that something is wrong here actually: what about this new access key? Even if you’re not a security person, if you have a new access key created out of nowhere, reported to you, this should ring a bell. Let’s go ahead and create a new bucket and call it “Fosdem 2031”. Obviously this bucket is not in Terraform because we just created live on the console. It’s perfectly normal for Terraform not to report you that this specific bucket was created right now. But this is the job of driftctl to report that you have both unmanaged resources that were created and deltas from managed resources. $ driftctl scan -- tfstate: Terraform provider initialized (name=aws, alias=us-east )
Terraform provider initialized (name=aws, alias=eu-west )
Terraform provider initialized (name=aws, alias=eu-north )

Found deleted resources:
  aws_route:
  - Table: rtb c5a9bb483ee6eaaf, Destination: / Found unmanaged resources:
  aws_s3_bucket:
  - fosdem2031
  aws_security_group_rule:
  - Type: ingress, SecurityGroup: sg ca93f95fcc338, Protocol: All, Ports: All, Source: ::/ - Type: ingress, SecurityGroup: sg ca93f95fcc338, Protocol: All, Ports: All, Source: / aws_iam_access_key:
  - AKIASBXWQ3AYRSNQN5EY (User: INTERN mszmf)

Found drifted resources:
  - AKIASBXWQ3AYWWYX72GX (User: INTERN mszmf) (aws_iam_access_key):
    ~ Status: => (computed)

Found resource(s)
  - % coverage
  - covered by IaC
  - covered by IaC
  - deleted on cloud provider
  - / drifted IaC from //./s3/terraform.tfstate --from tfstate://./vpc/terraform.tfstate --from tfstate://./iam/terraform.tfstate -1 -3 -1 -0 10.0 .1 .0 24 -0746 0 -0746 0.0 .0 .0 0 -8 -8 "Active" "Inactive" 27 81 22 4 not 1 1 22 from Let’s focus on the VPC here: if we change an existing rule… … we can see it in the diffs in driftctl $ driftctl scan -- tfstate: Terraform provider initialized (name=aws, alias=us-east ) 
Terraform provider initialized (name=aws, alias=eu-west ) 
Terraform provider initialized (name=aws, alias=eu-north )


Found deleted resources: 

  aws_route: 
  - Table: rtb c5a9bb483ee6eaaf, Destination: / Found unmanaged resources: 
  aws_s3_bucket: 
  - fosdem2031 

  aws_security_group_rule: 
  - Type: ingress, SecurityGroup: sg ca93f95fcc338, Protocol: All, Ports: All, Source: ::/ - Type: ingress, SecurityGroup: sg ca93f95fcc338, Protocol: All, Ports: All, Source: / aws_iam_access_key: 
  - AKIASBXWQ3AYRSNQN5EY (User: INTERN mszmf) 


Found drifted resources: 

  - AKIASBXWQ3AYWWYX72GX (User: INTERN mszmf) (aws_iam_access_key): 
    ~ Status: => (computed) 

  aws_S3_bucket: 
  -fosdem2031


Found resource(s) 

- % coverage
- covered by IaC 
- covered by IaC 
- deleted on cloud provider 
- / drifted IaC from //./s3/terraform.tfstate --from tfstate://./vpc/terraform.tfstate --from tfstate://./iam/terraform.tfstate -1 -3 -1 -0 10.0 .1 .0 24 -0746 0 -0746 0.0 .0 .0 0 -8 -8 "Active" "Inactive" 14 50 7 4 not 1 1 14 from The other things we can do with drift control is ignore some resources that you do not want to see reported for whatever reason you have. So you can also filter using the same system as AWS CLI. For example, you can filter just for S3 buckets if you are not interested in the rest. $ driftctl scan --filter "Type=='aws_s3_bucket'" $ driftctl scan -- tfstate: Terraform provider initialized (name=aws, alias=us-east )
Terraform provider initialized (name=aws, alias=eu-west )
Terraform provider initialized (name=aws, alias=eu-north )

Found unmanaged resources:
  aws_s3_bucket:
  – fosdem2031

Found resource(s)
– % coverage
– covered by IaC
– not covered by IaC
– deleted on cloud provider
– / drifted IaC from //./s3/terraform.tfstate --from tfstate://./vpc/terraform.tfstate --from tfstate://./iam/terraform.tfstate --filter "Type=='aws_s3_bucket'" -1 -3 -1 2 50 1 1 0 0 1 from It’s pretty cool for auditing purposes, for monitoring purposes etc… And that's a wrap! Thank you very much for reading through all this talk transcript. If you wish to gain a better understanding of driftctl, please visit our or our . We also have a series of  short that you can watch to get started in minutes. product page advanced tutorial demo videos For any questions, feel free to open a discussion on GitHub or visit us . discord

Stories, News & Mentions about ENV0

Env0’s Stories on HackerNoon

Env0's News & Mentions