One important capability in our QA strategy is to run smoke tests against remote service endpoints, to check that they are healthy. In this article, we share our approach, walking you through some examples.
IaC: AWS CDK vs. Terraform
Whether your cloud-native stack is predominantly serverless or containerised or a mixture of both, your life be much happier if you can choose the right provisioning tool. You might consider using Amazon’s tool, CDK, which is relatively new. With the reach and reputation of the Amazon Web Services brand, CDK can be expected to grow substantially in a short space of time.
Alternatively, you might consider the general abstraction of a multi-cloud tool like Terraform, with its flexibility, adaptability, and popular stablemates: Vault, Consul, Vagrant, Nomad, and Packer. These two heavyweights are not very complementary, so your choice will have a big impact on whether you can realise the dream of continuous deployment to AWS Cloud.
Business driving technological innovation
At AURENA Tech we recently conducted a lengthy evaluation of AWS CDK versus Terraform, and we will share the impressions we have, and the rationale for adopting one over the other. Luckily our auction platform aurena.at has very high, constant, and fanatical visitor traffic participating actively in our auctions. 2019 we celebrated: 859,771 placed bids; 135,609 sold lots; 19,169 new bidders.
As we projected greater usage demands on our system, it became imperative for us to improve scalability and reliability. For that matter, the impetus for us to move to the cloud was to benefit from horizontal scalability as our business continues to grow.
We decided to shift our workloads to AWS services like lambda or S3. We also determined that new and legacy elements should be embedded in a decoupled service-oriented architecture.
What we looked for in a provisioning tool were the abilities to:
- set up our infrastructure in a reproducible way
- scale with minimal change to our system configuration
- incorporate other provisioning and configuration management tools in the toolchain
We want to share with you our considerations on which tool to choose as IaC framework. Here is a quick question and answer tool, which will help you compare your situation to ours:
- Do you typically adopt new technologies on the basis that they will become the new norm within a year or two?
- Are you put off by problems with very new technologies, like breaking changes, lack of support?
- Will you move to multi-cloud sooner, later or never?
- If you are using AWS, are you already using Cloud Formation?
At AURENA, our evaluation criteria were as follows:
Now as promised, we will take a closer look at AWS CDK and Terraform, after which we’ll reveal which technology option we felt best met our needs.
The CDK was first released on 31 July 2018 (v0.8.0). Since then it was developed with great pace and is currently available in version 1.22.
The broad language support and the fast release cycles show AWS’s motivation to provide good tooling for the developers.
However, the downside is, that new releases often come with a lack of backward compatibility and leave outdated documentation pages in their wake.
AWS considered the migration path for those who were already working with CloudFormation (CFN) templates because they offer first class support for CFN .
You can import the template with the aws-cloudformation module .
On the other hand, CDK always uses CloudFormation under the hood when deploying to the cloud. Internally, the CDK code generates a CloudFormation template and runs the CloudFormation executor. This will give you the advantage of CloudFormation’s transactional deployments deployments, making rollbacks possible. The CloudFormation based deployment is indeed very stable but also quite slow.
When the CloudFormation template is generated, CDK also fills in a lot of defaults that you don’t have to specify explicitly in your code. In that respect, you can see the CDK as a templating engine for CloudFormation. If you want to have a look at the CloudFormation output, you can simply run cdk synth in the CLI, which synthesises and prints the CloudFormation template for this stack.
A great advantage of CDK is that you can enjoy the expressiveness of your preferred language alongside value-added benefits: the respective library ecosystem, code completion, and type-safety. For example, decisions based on environment variables or iterating over collections (like different Lambdas services) can be easily implemented.
An important concept in CDK is Constructs; a construct is a base for all elements in CDK. There are certain types of constructs:
- App: a special top-level construct / root node:
- Bracket around Stack constructs
- Stack: a special second-level)construct:
- Run cdk list to see all stacks in the app. A stack is a deployable unit in a CDK app. Tun cdk deploy stackname1 stackname2 to deploy the respective stacks by name. Note that nested stacks are not written out by cdk list.
- Constructs like Lambda or API Gateway are provided by AWS and represent your actual cloud resources. Think of your CDK app as a tree and those Constructs are the leaf nodes.
- Constructs can point to other constructs or hold the actual cloud resources (Composite pattern).
Besides the constructs provided by AWS in the Construct Library, you can build up personal libraries of reusable components. In general, your object-oriented programming (OOP) skills are a good match with the CDK.
Terraform was created by HashiCorp and has been available since 2014. Terraform is a declarative syntax and command line executor that performs cloud management through a native CLI bridge. For AWS it leverages the AWS SDK to manage cloud resources.
The Terraform CLI offers a nice planning step along with cost calculation. In contrast to CloudFormation, Terraform migrations are not transactional. They do run significantly faster. When errors occur though, this non-transactional nature can lead to weird infrastructure states which occasionally have to be resolved manually. Therefore, a solid pipelining infrastructure with different testing stages is crucial for a stable deployment process.
Terraform feels like a mature tool with solid documentation and has an active development team and community.
The counterpart to CDK’s Construct Library is the Terraform Module Registry. Those modules encapsulate certain sets of related cloud configurations while providing sane defaults. The usage of modules allows you to factor out similar code elements so that you don’t repeat yourself.
We recommend making frequent use of modules because otherwise, your Terraform infrastructure code will become bloated very quickly.
Terraform is accompanied by a domain-specific language called HashiCorp Configuration Language (HCL). This language was designed to manage cloud infrastructure:
- It has some visual similarities to JSON
- Resources are interconnected by reference to their identifiers
- Attributes determine the configuration of the resources in the cloud
I mentioned that we conducted a lengthy evaluation before committing one of these rival systems. The following table should give you a quick summary of our findings:
|AWS provisioning||via CloudFormation||via AWS SDK|
|Stackoverflow questions||approx. 300||approx. 4200|
|Speed of migrations||Slow||Fast|
|Language||Typescript, Java …||HCL|
After consideration, we opted for Terraform. But as I said, your mileage may differ, depending on what your management priorities and what aspects you value most in a tool. Hopefully, the framework that we have shared will help you determine what you want from an IaC tool, making the final choice clearer.
Until recently our engineering team was exclusively using Gitlab shared runners for our CI/CD pipelines. While they did (and partly still do) get the job done, recent outages have made us aware of how critical functioning CI/CD is to us in our day-to-day work.
QA is a cornerstone to ship reliably working products that exceed user expectations. Tatjana is going to build up our competence center to bring quality assurance at AURENA Tech to the next level.
#AngularGraz goes Leoben: Great talks and helpful insights in Angular Ivy and Testing in Angular, hosted at the AURENA Tech office.
To accelerate the expansion course, AURENA is bringing the recruiting and IT specialist Matt Pearce from Great Britain into the team.