Serverless API with Terraform, GO and AWS, Part 1

Knowing how to build REST API with latest tech is cool. But you know what is even cooler? - Being able to deploy it to the cloud! I'll walk you through the process of building simple, server-less REST API using GO, AWS Lambda, API Gateway, DynamoDB and then deploy it all to AWS cloud with Terraform.

If you want to follow along, you'll need an AWS account with user that has programmatic access enabled, GO and Terraform. Installing these dependencies is relatively easy and there's plenty of online resources to guide you through. I would also recommend looking into tfenv to manage multiple versions of terraform locally.

For this project I'm using

go v1.17.1

terraform v1.0.11

Business specific logic will be kept in

api

iac

serverless-api/
  |-api/
    |-lambdas/
    |-internal/
  |-iac/
    |-prerequisites/
    |-modules/
    |-api/

Since I'll be using

go

/lambdas

/internal

iac/prerequisites

Terraform has the ability to store its state remotely in a variety of back-ends and since deployment will happen on AWS, I'll use S3 for that. This will allow me to easily manage lifecycle of the application as well as allow other participants to share and update same state. That way everything and everyone will be in sync.

Prerequisites is kept separate as it has to be deployed first. This is because our main applications

terraform > backend

# iac/prerequisites/main.tf

# ----------------------------------------------------
# configure aws provider
# ----------------------------------------------------
provider "aws" {
  region  = var.region
}

# ----------------------------------------------------
# create dynamo db for deployments locking 
# ----------------------------------------------------
resource "aws_dynamodb_table" "terraform_statelock" {
  name           = local.ddb_name
  read_capacity  = 20
  write_capacity = 20
  hash_key       = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }
}

# ----------------------------------------------------
# create S3 bucket that will be used for the backend
# ----------------------------------------------------
resource "aws_s3_bucket" "remote_state" {
  bucket        = local.state_bucket_name
  force_destroy = local.destroy_bucket
  acl           = "authenticated-read"

  versioning {
    enabled = true
  }

  tags = {
    Env = local.env
  }
}

Here I'm creating S3 bucket to hold shared state and DynamoDB table for locking deployments. Locking is needed for cases when two or more people are trying to perform deployment at almost exact same time and this is where things can get out of sync. Important thing to mention is DynamoDB

hash_key

LockID

Quick note before moving on. File names do not matter. Terraform will read all configuration files in top-level directory. Nested directories are treated as completely separate modules and will not be automatically included in the configuration. You don't even have to split your logic into variables.tf, locals.tf, main.tf like I do. It could all be placed within a single file. It is simply my personal preference to organise configuration by splitting logical units into separate files.

Next

variables.tf

locals.tf

versions.tf

# iac/prerequisites/variables.tf

variable "region" {
  default     = "eu-central-1"
  description = "region where all the resources will be deployed"
}

variable "prefix" {
  default     = "project-123"
  description = "organization or service name, has to be unique"
}

variable "ddb_statelock_table" {
  default     = "tf-statelock"
  description = "name of dynamo db table for terraform state locking"
}

variables.tf

TF_VAR_

export TF_VAR_region=us-east-1

# iac/prerequisites/locals.tf

locals {
  env               = terraform.workspace == "default" ? "dev" : terraform.workspace
  state_bucket_name = "${var.prefix}-remote-state"
  destroy_bucket    = contains(["prod", "staging"], local.env)
}

locals.tf

# iac/prerequisites/versions.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.66.0"
    }
  }

  required_version = "~> 1.0.11"
}

versions.tf

With all that in place I could try to deploy prerequisites stack to AWS, but there's a problem. Terraform will not have access to AWS api. To solve it, I need to export secret and access keys that I got from AWS when my user with programmatic access was created:

# replace ******** with appropriate values
export AWS_SECRET_ACCESS_KEY=********
export AWS_ACCESS_KEY_ID=********

Time for some action:

cd iac/prerequisites

# initialise terraform project
terraform init

# plan deployment, should output:
# Plan: 2 to add, 0 to change, 0 to destroy.
terraform plan

# deploy, should output:
# Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
terraform apply -auto-approve

One important thing to mention is that my user has full access rights to S3, DynamoDB, API Gateway and Lambda attached via AWS management console. I find it a bit easier to manage this way. If you'll run into errors, say

not authorized to perform: dynamodb:CreateTable

After running above commands, it's always a good idea to navigate back to AWS management console and ensure all resources have been created. It's also a good idea to destroy and recreate the stack to make sure everything is in right order:

# destroy the stack
# should print out: Destroy complete! Resources: 2 destroyed.
terraform destroy -auto-approve

# plan the stack
terraform plan

# deploy the stack
terraform apply -auto-approve

Sometimes terraform fails to execute. Most of the time the solution is just to rerun the last terraform command. If that does not help then your favourite search engine is always one click away and I'm pretty sure there will be plenty of answers on how to solve the issue.

I hope you have learned something useful. In part 2 I'll be creating configuration for api infrastructure. This is where above prerequisites backend will be used. You can find the source code and track the progress of the project here.

Got inspired? Share and inspire others!