We will use a custom resource written in Python that will be able to create ACM certificates with DNS validation. The custom resource will also automatically validate this certificate if the validation domain is managed by a Route53 hosted zone. We will also be able to specify an AWS region to create the certificate in, this region is independent of the Cloudformation stack region which for example makes it possible to deploy a certificate in region us-east-1 (to use with cloudfront) while deploying the stack in region eu-west-1. The resource will also provide the certificate arn as an output parameter so it can be used by other resources in the stack. Lastly when you delete the custom resource it will cleanup all validation records and the certificate itself.

Requirements:

• Python3
• Pip
• Bash
• Zip
• An S3 bucket to deploy the custom resource package on
• A hosted zone for the validation record

Implementation:

Let’s get started by downloading all the required code from our GitHub repository.

Step1: Uploading the custom resource package

In this step we are going to prepare the custom resource package and upload it to an S3 bucket.

First we go into the custom resource directory.
cd cloudar-acm-plus-custom-resource

Next we execute a script to install all required dependencies.
sh install_dependencies

Now we are ready to create the package.
sh pack_custom_resource

You will now find the zipfile ‘cloudar-acm-plus-custom-resource.zip’ in ‘cloudar-acm-plus-custom-resource/packed’, upload this zip file to your S3 bucket.

Step2: Creating a Cloudformation template

Now we can create a Cloudformation template in which we use this custom resource to create an ACM certificate.
You can use the template ‘cfn.yaml’ as an example.

First create a Lambda resource as following

Use the name of your bucket for the property ‘S3Bucket’ .

Next we create the custom resource.

We can set the following properties here:

• DomainName: (REQUIRED type:String) The domain name for the acm certificate.
• AdditionalDomains: (OPTIONAL type:List) Additional domains for the acm certificate
• ValidationDomain: (REQUIRED type:string) The domain name for the validation domain of the acm certificate
• HostedZoneId: (REQUIRED type:string) The hosted zone id for the validation domain of the acm certificate
• CertificateRegion: (REQUIRED type:string) The region to deploy the acm certificate in
• IdempotencyToken: (REQUIRED type:string pattern: \w+) The idempotency token for the create call of the acm certificate doc: https://docs.aws.amazon.com/acm/latest/APIReference/API_RequestCertificate.html#ACM-RequestCertificate-request-IdempotencyToken
• CertificateTags: (OPTIONAL type:list) The tags for the acm certificate

In order for the DNS record cleanup and delete certificate functionality to work when you delete the Cloudformation stack it is important to set the following output.

As you can see we can access the arn of the certifcate created by the custom resource with the GetAtt function on the resource.
!GetAtt CreateCertificateCustomResource.certificate_arn

Step3: Deploy the cloudformation

Finally the only thing left to do is deploy the Cloudformation template.
Once the deploy is started Cloudformation will create the Lambda containing the code from step1 and start a custom resource which will create the certificate and validation records. Once the status of the certificate becomes ‘ISSUED’ the custom resource will finish successfully and report the arn of the certificate back to Cloudformation. We can now further use this arn in other resources in the Cloudformation template.
When you delete the Cloudformation stack, the custom resource will cleanup the validation records in the hosted zone and delete the certificate.

CREATE_COMPLETE

The post Validate ACM certificates in Cloudformation appeared first on Cloudar.

In IT we have learned to embrace change and new technology. We all need to change if we want our businesses to stay relevant, don’t we? But change, and certainly positive change, doesn’t always happen overnight. It is a process in which we will need to make decisions that will define the desired output.

Since systems, including people and organizations, are interconnected, we have to approach change holistically to make it happen. Therefore, processes exist to guide you during the transformation and help you to arrive at the desired state. In our digital transformation journey and in Cloud Migration it’s not different.

Our approach towards Cloud Migration is a tried and true, structured approach, in which we will first prepare ourselves, then acquire the right methodologies and tools to lay solid foundations and from there on, start practicing and building. During this execution phase, we will iteratively ‘Slice and Dice’ your workloads and business capabilities into reasonable and practical actionables, following an agile approach. Practice makes perfect and before you know it, you’re migrating at scale.

In this perspective, the setup of a highly efficient Migration Factory is utterly important. Here we will make use of the technology and tools at hand to facilitate and automate the migration as much as we can. You can for example make use of CloudEndure, which delivers flexible and innovative migration solutions. Cloudendure offers Continuous block-level replication, cross-infrastructure machine conversion, does automated orchestration and this with a minimal footprint.

Once you’re in the AWS cloud, possibilities to improve, optimize and reinvent are numerous.

Business capabilities will be transformed with available Cloud-Native Services and Technology.

Choosing a partner or MSP that guides you and will help you make the right decisions during the journey, is a direction that you should make as early in the process as possible. It’s an important decision, because once you started your Cloud journey, you will start to see opportunities in which a trusted advisor is a valuable resource to ensure you make the right choices.

Do you want to know where you are in your Cloud journey or where to get started ? Contact your partner for an AWS Migration Readiness Assessment or a AWS Well-Architected Review. It’s a small step, but an essential one to keep your business moving forward in a fast-pace world.

The post Slice and Dice your Cloud Migration and Keep Moving appeared first on Cloudar.

Most often traditional managed services providers offered managed servers. There were some differences in how high in the stack your provider would support you (OS only, some SQL and AD or even full application support), but other than that you were given a server with an OS and what we call the Big Three of Managed Services: monitoring, patching and backups. The provider would also manage the underlying network infrastructure and storage boxes. But that was that. Infrastructure engineers would build your environment, hand it over to you, and you could start deploying your applications.

That infrastructure could be three things. Either dedicated physical servers, a multi-tenant virtualization platform or something that was called private cloud. It wasn’t really cloud, as it didn’t scale. It was just your own virtualization platform. A point also made by Bart Van Hecke: here.

Fast forward some years, and with public cloud, a big shift has been made. Managed infrastructure is no longer the be all and end all of a managed services company. Not only is the whole hardware layer that an MSP used to manage now managed by AWS, but a plethora of new services has been released. The result is that an MSP is now driven higher up in the stack. There is less to manage on the hardware side of things, but much more on PaaS and FaaS level.

A customer now gets a bucket full of Lego, and can start building. You can build beautiful things with Lego. But without any guidelines or help, you can also create the next Frankenstein. So what we see now is that unlike the early days where an engineer would throw a server over the fence and let the developer or application manager do his or her thing, it’s a journey. A journey where an engineer and a developer build things together. You’ve probably heard of this pretty thing called DevOps. It’s much more than just a buzz word. It quickly became the only proper way to move forward and work with the new technologies that AWS kept throwing at us.

As a next generation MSP, Cloudar also has a very solid consultancy business. And that is a match made in heaven. Not only do we take the burden of managing your servers out of your hands, but our skilled engineers help you in deploying your applications in a Well Architected Way. And the virtual infrastructure that comes from that, and is kept in version controlled CloudFormation or Terraform, will also be managed by us. And that is more than just an EC2 instance. It could be a PaaS platform like Beanstalk, ECS or EKS or it could be a FaaS Serverless platform based on Step Functions, Lambda, DynamoDB and API Gateway. Because in the end, developers want to develop and build new things, but they don’t want to manage and nurture what they have built 24*7.

So what we have seen now over the years during many customer engagements, is that we not only manage environments for customers, but we also teach customers how to work better. How to build efficient development pipelines instead of using FTP scripts and a local git to deploy to a static server. How to work together as dev and ops. And in return customers make us better, as they keep coming up with new requirements and new challenges.

The post About MSP’s and Lego appeared first on Cloudar.

Today that clear boundary has disappeared. System engineers are shifting more and more to automation and thus very basic development, or scripting. But at the other side of the spectrum, application engineers and developers are shifting more and more to the infrastructure side. Why? Infrastructure as code. CloudFormation. Boto. Terraform. AWS CDK. In a true public cloud environment, where every single administrative action on infrastructure level is a simple API call, things are changing, and DevOps is born.

You rarely saw devs running around the office with a screwdriver and rack mounts. If they needed a server, they’d ask one to be procured. And waited a few weeks. Now application engineers can quite easily build their own infrastructure to run their applications. It’s just code, right? And they do. Oh boy they do…

However, in most cases it is undesirable to have an application engineer create infrastructure. There is the principle of segregation of duties. Furthermore, it’s very rare a skilled dev has a strong infrastructure background. So while he can probably spin up resources to support his application, the question is: is it Well Architected? And that is where a next gen MSP comes into play. Cloudar was selected by AWS to perform Well Architected Reviews. Our AWS Solution Architect Professional engineers that have followed the required extra training are allowed to perform rigid reviews of existing business critical workloads. Needless to say, each environment we build, needs to adhere to the same Well Architected Principles.

But what if a developer wants to go full serverless, and change infrastructure components rapidly? Things like Lambda’s. Can he wait for an engineer at the MSP to create those resources, throw them away, reconfigure them? Probably not. The cloud allows for speed, and you don’t want your MSP to be a potential bottleneck. Now what?

Here, the answer is again: infrastructure as code. If you demand all infrastructure to be written in code, you can allow your devs to play around in development to deploy extra infrastructure resources. And then when they are happy, and they went through dev and test, and are ready to commit to UAT, insert a code review approval step. Have the engineers that will be actually supporting the environment 24×7 review the infrastructure code. Is what will be deployed to UAT and later on to PRD still Well Architected? Does it adhere to your internal support standards? And for the mental health of the on call engineers: will it be stable and not generate alerts at 3 am in the morning?

So you see that devs will need to work with sysops, and vice versa. Certainly in a project phase. You want to work as agile as possible, but still have a reliable and secure environment. One of the strong points of Cloudar is that we have both. We have a very solid managed services business, but we also have a flourishing consultancy practice. Our people love working with your devs. We like them. Sometimes we even want to cuddle them. Everything it takes to have your workload running 24*7, but reduce your time to market by allowing devs to go crazy until stuff moves to production. Because production, is serious business.

The post A new reality: Devs creating infrastructure. And how to deal with that… appeared first on Cloudar.

The post Cloudar as a Next Generation MSP appeared first on Cloudar.

Recently we were looking for a way to automate the following actions.

• Creating snapshots
• Starting and stopping instances
• Removing old snapshots

We already had a script and cronjob to take regular snapshots, but we wanted to centralise our automation to improve visibility and logging.

While this is possible using the existing Ansible modules, we needed a way to specify multiple parameters on a per instance basis. Specifically we needed it to be possible to:

• Change the start and stop times from within the AWS Console
• Allow users without access to (or knowledge of) Ansible to prevent the automatic shutdown of a machine

We quickly decided to use tags to specify these parameters. Because there’s a limit of ten tags per instance, and we wanted to store other information (like Environment and Project) too, it wasn’t possible to store each parameter in a separate tag.

We solved this by writing our own Ansible modules. Each module reads the same tag, where we store a JSON dictionary with all the needed variables. This way we keep room for other tags, while allowing everyone with the right to create tags to change things if they need to.

The source code is available on GitHub. You can install it by adding the path to your Ansible Library. Complete usage instructions and examples are included in the repository.

The post Flexible Instance and Snapshot management with Ansible appeared first on Cloudar.