In the AWS ecosystem, MSPs take vastly different approaches when building customer-specific landing zones and cloud management platforms. Some approaches preserve your freedom and flexibility. Others quietly build the walls of a gilded cage.

The Two Paths: Open Standards vs. Proprietary Platforms

The Open Approach: AWS Landing Zone Accelerator (LZA)

AWS Landing Zone Accelerator represents the gold standard for customer independence. As an open-source solution built on AWS CDK and CloudFormation, we deploy LZA to provide several critical advantages:

• Complete transparency: All infrastructure is defined as code that you can read, understand, and modify
• No licensing fees: Open-source with no proprietary components
• AWS-maintained: Continuously updated by AWS to keep up to date with new services and features
• Industry standard configuration: With multiple documented sample configurations, you do not need to start from scratch.
• Full customer ownership: Deployed directly into your environment with complete access to the Infrastructure as Code
• Exit-ready from day one: If you ever want to manage it yourself or switch MSPs, you own your landing zone configuration

At Cloudar, we’ve built an entire landing zone practice around LZA precisely because we believe customers should never feel trapped. Your AWS foundation should be an asset you own, not a chain that binds you to any single provider.

The Proprietary Approach: Custom Orchestration Platforms

In contrast, many MSPs have developed proprietary cloud management platforms that create significant lock-in. These are often marketed as “revolutionary” or “next-generation” platforms that promise to make cloud management easier with “just a few simple clicks” or web-based portals that abstract away complexity.

The convenience is real. The long-term cost is hidden.

Here’s what proprietary platforms typically involve:

• Black box deployment: Resources are created through proprietary tooling that abstracts away the underlying infrastructure
• Dependency on custom APIs: Your operations become dependent on the MSP’s platform rather than native AWS tools
• Limited portability: Moving to another MSP or bringing management in-house requires re-platforming
• Knowledge gap: Your team never develops deep expertise in AWS native tools because they’re shielded by the abstraction layer
• Commercial leverage: The MSP knows that switching costs are high, affecting pricing negotiations and service quality over time

The Lock-in Mechanisms You Need to Watch For

1. Custom Landing Zones Without Source Code Access

Some MSPs deploy your resources using “their” landing zone—a pre-configured multi-account setup built with proprietary Infrastructure as Code that remains their intellectual property. When you want to leave, you inherit an AWS environment you don’t fully understand, configured by tools you can’t access.

The Cloudar difference: We can deploy LZA directly into your AWS accounts. Every CloudFormation stack, every configuration file, every security baseline—you have access to it all.

2. Web-Based Orchestrators That Become Operational Chokepoints

Fancy web portals that let you “deploy with one click” sound appealing. Until you realize that every operational change must flow through the MSP’s platform. Want to modify a VPC? You’re dependent on their UI. Need to adjust security groups? Better hope their platform supports your use case.

These orchestrators create operational lock-in: You can’t effectively operate your AWS environment without the MSP’s tooling. You’ve traded AWS complexity for MSP dependency.

3. “Simplified” Interfaces That Hide AWS Reality

Abstraction layers that promise to “make AWS easy” can create a dangerous gap between what you think you’re deploying and what’s actually running in your account. When problems arise—and they always do—you discover that your team doesn’t understand the actual AWS infrastructure because they’ve only interacted with it through the MSP’s simplified interface.

The Real-World Impact of Lock-in

Scenario 1: The Price Increase

Your MSP announces a 30% price increase. With an LZA you own and open standards, you have options: negotiate from a position of strength, bring management in-house, or transition to another MSP in months. With a proprietary platform, you’re looking at risky and arduous re-platforming work—and your MSP knows it.

Scenario 2: The Service Quality Decline

Your MSP gets acquired. The new parent company shifts focus, key engineers leave, and service quality drops. With an open approach, you can transition smoothly. With lock-in, you’re stuck enduring declining service while planning an expensive migration.

Scenario 3: The Strategic Pivot

Your company wants to build internal cloud expertise and eventually self-manage. With LZA, your team can learn standard AWS tools and practices from day one. When you’re ready to transition, you already have the skills and the code. With proprietary platforms, your team has learned the MSP’s tools, not AWS—setting your in-house capability building back by years.

Scenario 4: The Platform Limitation

Your business needs evolve, and you need to implement a complex AWS architecture that isn’t supported by your MSP’s platform. You’re now in the worst position: paying for a platform that constrains you, unable to use native AWS capabilities, and facing the choice between living with limitations or undertaking an expensive re-platforming project.

How to Evaluate Your Current or Prospective MSP

Ask these critical questions:

1. “What landing zone solution do you use?”
   • Red flag: “Our proprietary solution” or vague answers. Subscription based landing zones (yes they exist!).
   • Green flag: “your own AWS Landing Zone Accelerator” or “a per-customer AWS Control Tower with Customizations for Control Tower”
2. “What happens to our infrastructure if we terminate the contract?”
   • Red flag: Vague answers about “transition planning” or “it depends”
   • Green flag: “You keep everything—we’ll help with knowledge transfer, and you’ll have all the code and documentation”
3. “Will our team learn AWS-native tools, or primarily your platform?”
   • Red flag: “Our platform abstracts AWS complexity away”
   • Green flag: “We teach AWS best practices and native tools”

The Cloudar Philosophy: Your Cloud, Your Terms

Here’s what that means in practice:

Full LZA Implementation

Every customer gets its own AWS Landing Zone Accelerator, deployed directly into their accounts with complete source code access to the LZA configuration. Deployments happen in your account, giving you end-to-end visibility on your Landing Zone.

AWS-Native Tooling

We use CloudFormation, CDK, AWS Config, Systems Manager—tools that work with or without us. If you hire another AWS expert or build an in-house team, they’ll recognize everything immediately.

Comprehensive Documentation

You can read about every configuration option today, in the documentation published by AWS. So while we pride ourselves in sharing our knowledge, you are not dependant on us to explain what is going on

Additionally, we write customer-specific documentation  in our Confluence – from architecture decisions to operational procedures.. If you decide to leave, we can provide you with an export of that information.

Open Book Operations

You have full Read Only access to your AWS environment —we’re partners, not gatekeepers. Want to check our work? Go ahead.

Standard AWS Best Practices

We follow AWS Well-Architected Framework principles and industry-standard patterns. No “special sauce” that only we understand.

We succeed by giving you excellent service so you want to stay, not by making it painful to leave.

The Economics of Freedom

Some argue that proprietary platforms are necessary to provide better service or lower costs. We disagree.

Lower costs come from:

• Automation that scales across customers (which we use)
• Deep AWS expertise (which we have)
• Efficient processes (which we’ve refined over years)

Not from locking customers into proprietary platforms.

Better service comes from:

• Highly skilled engineers (which we continuously train)
• Customer focus (which our retention rate proves)

Not from proprietary abstraction layers.

We’ve proven that you can deliver excellent MSP services at competitive prices while keeping customers completely free. In fact, we believe customer freedom makes us better—we can’t coast on lock-in, so we must continuously earn our customers’ business.

Making the Right Choice

Before signing with any MSP, ask yourself:

• Do I understand what I’m getting into? Can you clearly explain how your infrastructure will be deployed and managed?
• What’s my exit strategy if things don’t work out? Is it measured in weeks, months, or years?
• Am I choosing this approach for the right reasons? Is convenience masking a lack of control?
• Do I retain full ownership? Of code, configurations, documentation, and knowledge?

Red Flags in MSP Sales Processes

Be wary if you encounter:

• Heavy emphasis on “simplicity” with little discussion of the underlying AWS architecture
• Vague answers about exit strategies and transition processes
• Marketing focused on proprietary platforms as the primary differentiator
• Contracts that grant the MSP exclusive rights to infrastructure code
• Lack of clarity about what you actually own vs. what you’re licensing

Conclusion: Freedom as a Feature

In the rush to cloud transformation, it’s easy to prioritize speed and convenience. And yes, a well-designed proprietary platform can deploy faster than custom LZA implementation—at least initially.

But cloud strategy isn’t measured in weeks. It’s measured in years and decades. The question isn’t “who can get me to cloud fastest?” It’s “who can help me build sustainable cloud capabilities that serve my business long-term?”

The MSP industry has a pattern: some providers build their business model around customer stickiness achieved through proprietary tooling. They create beautiful interfaces and slick demos that abstract away AWS complexity. Then, months or years later, customers realize they’ve traded AWS vendor lock-in for MSP vendor lock-in—often worse, because at least AWS is standardized.

At Cloudar, we reject this model fundamentally. We believe that customer freedom isn’t a bug to work around—it’s a feature to build for. We’re proud to be an AWS Premier MSP Partner that wins business through excellence, not lock-in.

Your cloud infrastructure is too important to be held hostage by convenient abstractions. You deserve an MSP that treats you as a partner who will grow and evolve, not a captive customer who might someday try to escape.

Choose partners who believe you should always have the keys to your own kingdom. Choose partners who succeed by being valuable, not by being necessary.

Choose freedom.

The post The Hidden Cost of Convenience appeared first on Cloudar.

Interestingly, Roles Anywhere does not completely solve the problem of not needing to create long-lived credentials, instead it moves (and reduces) the problem from credential management (rotation) to certificate management (distribution).

Setting up a Certificate Authority (CA) that can create certificates can be done with AWS Private Certificate Authority, and there are some options out there to manage distribution (usually as part of a bigger Device Management solution), but wouldn’t it be nice if the government of Belgium would do that for us?

The Belgian eID

In Belgium every citizens gets an identity card. The government makes sure that these get distributed and renewed, and since 2005 they contain a chip that holds (among other things) a certificate that can be used for digital authentication. If we can tie this to Roles Anywhere, we don’t have to worry about certificate management ourselves.

This eID is the size of a bankcard and the chip can be interacted with using off the shelf smart card readers. Most information can be read without any additional authentication – if you have the correct middleware, but to sign something you need to enter a PIN code.

The different certificates

Every eID holds an “authentication certificate” that is tied to the owner of the eID and has been signed by the “Citizens CA”.  The Citizens CA’s certificate is in turn signed by the Belgium Root CA . The Root certificate is self-signed. If we trust the Belgium Root CA, we will be able to validate any authentication certificate. More information about this setup can be found at https://repository.eidpki.belgium.be/#/home

The authentication certificate has two pieces of information that are going to be useful to us:

• The Issuer, showing us that the certificate is indeed issued by the Citizen CA.
• The Subject, containing the name and National Number (a number that uniquely points to one person) of the owner.

AWS IAM Roles Anywhere

Roles Anywhere allows you to use a certificate and its private key to get temporary credentials. You can think of it as an AssumeRole call, but instead of starting from an Access Key, we start from a certificate.

There are a few components we need:

• Configuration in our AWS Account, to indicate which CA we trust, and which roles can be assumed by different Issuer and/or Subjects. This also includes the creation of IAM Roles.
• Software on our computer to interact with the Card Reader so we can prove we have access to the eID and know its PIN.

For our use case we are going to trust the Belgium Root CA, and create a Role that can be assumed with any certificate issued by the Citizen CA. We will than limit the permissions of that IAM Role, so it can only write to a unique prefix in S3.

We can do that because every component of the Subject in our certificate gets mapped to a principal tag on the role session. E.g. if the subject is “C=BE, SN=Doe, GN=John, serialNumber=70010112345, CN=John Doe (Authentication)”, we have the following principal tags, and we can use principal tags in any policy.

• x509Subject/C: BE
• x509Subject/SN: Doe
• x509Subject/GN: John
• x509Subject/serialNumber: 70010112345
• x509Subject/CN: John Doe (Authentication)

Putting it all together

In our AWS Account

We will create the following resources in our AWS Account:

• An S3 Bucket
• A trust anchor, configured to trust the Belgium Root CA certificate that is publicly available via https://crt.eidpki.belgium.be/eid/brca6.crt.  This way we can use any certificate created by the Root CA of any of its subordinate CAs (like the Citizen CA)
• An IAM Role, where we configure the trust policy to only allow usage of the Role if:
  • The principal is the Roles Anywhere service (rolesanywhere.amazonaws.com)
  • The Issuer is Citizen CA (aws:PrincipalTag/x509Issuer/CN must equal “Citizen CA”)
  • The source is our trust anchor (aws:SourceArn equals the ARN of the Root CA trust anchor)
• An IAM Policy on this role that allows PutObject acces to all objects that start with the unique National Number (aws:PrincipalTag/x509Subject/serialNumber)
• a (Roles Anywhere) profile that ties the trust anchor to the Role. We could configure more settings here, like a mapping of fields from the certificate or a session policy, but don’t need to.

Of course we do not have to do this by hand, we create a CloudFormation template for this. It can be found in our sample repository: https://github.com/WeAreCloudar/cloudformation-samples/tree/main/templates/roles-anywhere-eid

On our machine

On our machine, we will need – besides the eID and its PIN (at least on macOS, where I tested this):

• a SmartCard Reader
• the Roles Anywhere Credential Helper (downloadable from the AWS Website)
• The AWS CLI (or any other tool we want to use with our AWS Credentials)

We also need a few pieces of information:

• The ARNs of the trust anchor, profile and role we created
• Our own National Number and the name of the bucket (to test our policy)

Ideally we’d be able to configure the credential helper to read any eID we insert, and this is possible if you install the Belgian eID Middleware using the following command.

aws_signing_helper credential-process \
 --trust-anchor-arn $trust_anchor_arn \
--profile-arn $profile_arn \
--role-arn $role_arn \
--certificate "pkcs11:model=Belgium%20eID;object=Authentication" \
--pkcs11-lib '/Library/Belgium Identity Card/Pkcs11/beid-pkcs11.bundle/Contents/MacOS/libbeidpkcs11.dylib

However because of an issue with the helper, this only works on macOS if you build the helper yourself from source. Since we do not want to do that, we can plug in our eID and find and identifier using “aws_signing_helper read-certificate-data”. You should get something like

Matching identities
1) bf013dd57aa34f8f9d4e22ab89dd8eb2 "SERIALNUMBER=70010112345,CN=John Doe (Authentication),C=BE,..."
2) 786f7572198a4a8a82ee7ef3f205d7be "SERIALNUMBER=70010112345,CN=John Doe (Signature),C=BE,..."


Copying everything between quotes, allows us to create a new file, selector.json that looks like this:


[
{
"Key": "x509Subject",
"Value": "SERIALNUMBER=70010112345,CN=John Doe (Authentication),C=BE,..."
}
]

and use that to select our specific certificate:

aws_signing_helper credential-process \
 --trust-anchor-arn $trust_anchor_arn \
--profile-arn $profile_arn \
--role-arn $role_arn \
--cert-selector file://selector.json

Our smart card reader will now prompt us for a PIN. If we enter the right one, we get credentials as output:

{"Version":1,"AccessKeyId":"ASIA...","SecretAccessKey":"COA...","SessionToken":"IQoJ...","Expiration":"2025-03-28T14:16:07Z"}


We can put this command in our ~/.aws/config file, so the AWS CLI will execute it for us, but for testing purposes it’s slightly easier to use the “serve” option.  The AWS Documentation has a longer explanation about how you can use the credential-process: https://docs.aws.amazon.com/rolesanywhere/latest/userguide/credential-helper.html#credential-helper-examples

Testing with s3

In one terminal we can run a credential / metadata server that can be used by the AWS CLI:

~$ aws_signing_helper serve -process \
 --trust-anchor-arn $trust_anchor_arn \
--profile-arn $profile_arn \
--role-arn $role_arn \
--cert-selector file://selector.json


This gives us
2025/03/28 14:28:12 Local server started on port: 9911
2025/03/28 14:28:12 Make it available to the sdk by running:
2025/03/28 14:28:12 export AWS_EC2_METADATA_SERVICE_ENDPOINT=http://127.0.0.1:9911/

We can than use a second terminal to run:

AWS_EC2_METADATA_SERVICE_ENDPOINT=http://127.0.0.1:9911/ aws s3 cp example_file s3://$bucket/serialNumber/70010112345/example_file

and get our file in s3:

upload: example_file to s3://.../serialNumber/70010112345/example_file

Conclusion

This experiment shows that being able to use an external device withIAM Roles, is a very powerful feature, because it can remove a lot of tasks related to certificate and credential management. Being able to do this with something I had in my backpocket made it extra cool.

However, the Belgian eID is probably not the best way to go about this in a real scenario:

• I just gave everyone living in Belgium access to my S3 Bucket. I could use a more scoped-down trust policy, but I would eventually run into limitations of trust-policy size.
• I also didn’t tackle Certificate Revocation Lists (CRL). Every time someone loses their wallet, I would need to invalidate certain certificates – this also runs into size limitations
• I already have a good way to give users access to AWS Accounts, using Identity Center or Cognito.
• The pkcs11 support on macOS seems limited, and I would have to install the middleware everywhere,

This does not mean that Roles Anywhere is not a good solution for this kind of cases:

• If you are already putting certificates on machines (e.g using an MDM solution), AWS IAM Roles Anywhere is a great fit
• Instead of relying on the Belgian Government you can buy an hardware key like a yubikey and use it as your certificate source. This is as secure as using the eID (only the machine with the key plugged in will be able to request credentials), and you would be limiting it to the exact serial number(s) in your trust policy anyway – making CRLs mostly moot.

The post Sign in with your eID: Using AWS IAM Roles Anywhere with a SmartCard Reader appeared first on Cloudar.

We were recently working on an AWS setup which involved a Network LoadBalancer (NLB) with a TCP listener and a requirement for sticky sessions. As we were seeing some strange behavior which we couldn’t immediately explain and which might be linked to the session stickiness we decided to make a small test setup.

The problem

Unlike an ALB where session stickiness is accomplished with cookies, the NLB uses a built-in 5-tuple hash table in order to maintain stickiness across backend servers. We access the NLB through its DNS name, which actually returns the IPs of the two NLB endpoints in a round-robin fashion with a TTL of 60 seconds.

We were looking for an answer on the following question: if our end-user would resolve the DNS and pick the IP of the first NLB endpoint to start the connection, the session will be routed towards one of the backend servers. But after 60 seconds the client could potentially re-issue the DNS query and start its connection with the other NLB endpoint. How will the stickiness and cross-zone loadbalancing behave? Will our end-user connection be routed to the initial server again, even if that means crossing AZ boundaries?

The situation

We started with a classical setup comprising of an NLB with an endpoint in each AZ and a Targetgroup having one instance as target in each AZ.

Scenario #1

• Cross-Zone loadbalancing: Disabled
• TargetGroup Stickiness: Disabled

How does it behave?

1. Client connects to the IP of the first NLB node: the connection is redirected to the server in AZ 1.
2. Client connects to the IP of the second NLB node: the connection is redirect to the server in AZ 2.

Since there is only one healthy target per AZ and cross-zone loadbalancing is not enabled, this situation results in ‘AZ-stickiness’: the traffic remains in the AZ in which it arrived. The setup relies on DNS to distribute client connections evenly across both NLB endpoints, but there is nothing to guarantee that a specific user connection is always directed to the same NLB endpoint, let alone to the same backend server.

Scenario #2

• Cross-Zone loadbalancing: Enabled
• TargetGroup Stickiness: Disabled

Allowing cross-zone loadbalancing and not requiring any stickness. This should give us complete randomness, shouldn’t it?

And so it does. We’re now completely randomized and hit every backend server, irrespective of our NLB endpoint ‘point of entry’. Works as expected.

Scenario #3

• Cross-Zone loadbalancing: Disabled
• TargetGroup Stickiness: Enabled

We’ve enabled stickiness on the targetgroup now, and disabled the cross-zone loadbalancing again. Let’s hope our client connection is now sticky to a specific backend server.

1. Client connects to the IP of the first NLB node: the connection is redirected to the server in AZ 1.
2. Client connects to the IP of the second NLB node: the connection is redirect to the server in AZ 2.

Ok wait, we’ve asked our TargetGroup to be sticky, but still our connection is balanced over both backend servers? What’s going on?

The fact that our NLB is not allowing cross-zone loadbalancing seems to prevent the connection from reaching the same backend every time. The connection enters via NLB endpoint 1 but stickiness has decided that the connection should go to server in AZ 2? Stickiness fails, the disabled cross-zone loadbalancing wins…

With only one healthy backend per AZ this behaves the same as not enabling stickiness at all. We’re pretty sure that with more than one backend per AZ the stickiness is maintained…within that AZ only. Interesting!

Scenario #4

• Cross-Zone loadbalancing: Enabled
• TargetGroup Stickiness: Enabled

Let’s solve this. We’ve enabled both cross-zone loadbalancing and targetgroup stickiness. We should hit the same backend server every time now.

And so it does. Only now we reach true stickiness and hit the same backend server every time, no matter how hard we try by entering via the loadbalancer node in the other AZ.

The conclusion

If you don’t allow cross-zone loadbalancing, then stickiness is only active within AZ boundaries. As DNS round-robin could direct a client to a different point of entry after the TTL has expired, strict stickiness is not guaranteed.

So if you really need stickiness to a specific backend target, you need to allow cross-zone loadbalancing (and live with the extra cost of inter-AZ traffic). Only now do the different loadbalancer nodes share the hash table of “client-to-target” stickiness.

Kinda logic, though…

PS: NLB idle timeout for TCP connections is 350 seconds. Once the timeout is reached or the session is terminated, the NLB will forget the stickiness and incoming packets will be considered as a new flow and could be loadbalanced to a new target.

The post Why AWS NLB stickiness is not always sticky appeared first on Cloudar.

The post 10 random things you probably didn’t know about Cloudar appeared first on Cloudar.

It’s a familiar scene: as you’re waking up or making your morning coffee, you hear the garbage truck roaring away outside when you suddenly realize your garbage bags are still rotting away in the garage. Do you run after the truck like a lunatic, even though it’s too late (of course I’ve never done this, but I’ve heard it’s a thing)? Or do you just let it go and let the garbage fester even longer? Either way, it’s a frustrating start to the day.

Luckily for me I live in the modern and very civilised province of Limburg of Belgium where the local government has made a website available with a downloadable calendar showing all dates the garbage truck comes along: so handy! They even provide the calendar in a number of digital formats, including iCal. I could just import this into my phone and set an alert on it, but where’s the fun in that? After some fiddling around or “reverse engineering” as they call it, I found out that the data for the calendar files was provided by an undocumented but publicly available API.

Being the sucker for coding up and fooling around with APIs that I am (and given my state of utter boredom while under full lockdown in Belgium) I decided to code up a project based on this API that would send me an SMS notification the evening before garbage collection. Because that’s my idea of fun. And that way I would be reminded to take out the trash on time and keep my garage clean.

Want to know this neat trick to save you from rubbish resentment? Read on!

So for this project I partly used the AWS SAM framework and set up a CodePipeline to deploy my code changes instantly on AWS (stay tuned for a future blogpost on this).

First let’s code up the AWS SAM template file containing the necessary resources for this project.

The resources I used are:

• Type: AWS::Events::Rule
  A Cloudwatch event rule to invoke a lambda each evening around 8.
• Type: AWS::Lambda::Permission
  To give the Cloudwatch event rule permissions to invoke the lambda.
• Type: AWS::Logs::LogGroup
  A log group for the lambda logs.
• Type: AWS::Serverless::Function
  The lambda that will run the show.
• Type: AWS::IAM::Role
  An IAM role for the lambda to have the necessary rights like accessing SNS for the SMS notification.

Next we code up the lambda that will call the API and send us an SMS.

The API provides event data for each month by just requesting the year and month in the url path like this ‘/2020-03’. So we can easily get the data for the current month and loop through its events, checking whether any are occurring the next morning. If so we receive an SMS notifying us (note that you can provide the number of your girlfriend here, but it’s not recommended as I have found out it isn’t that effective).

And that’s all folks!

I’m aware that this trick is no rocket science, and that this isn’t even a full guide to get to the same result (leave a comment if you want some more details), but I thought it would be fun to share how I resolved one of life’s little problems using AWS.

As I’m moving toward two years of professional experience using the platform, I’m still amazed at how easily you can code some services together and create a working, scalable and highly available project in a matter of minutes or hours.

Hope you liked my trashy tale – keep your garages clean and your heads in the cloud!

The post Let’s trash talk! appeared first on Cloudar.

We will use a custom resource written in Python that will be able to create ACM certificates with DNS validation. The custom resource will also automatically validate this certificate if the validation domain is managed by a Route53 hosted zone. We will also be able to specify an AWS region to create the certificate in, this region is independent of the Cloudformation stack region which for example makes it possible to deploy a certificate in region us-east-1 (to use with cloudfront) while deploying the stack in region eu-west-1. The resource will also provide the certificate arn as an output parameter so it can be used by other resources in the stack. Lastly when you delete the custom resource it will cleanup all validation records and the certificate itself.

Requirements:

• Python3
• Pip
• Bash
• Zip
• An S3 bucket to deploy the custom resource package on
• A hosted zone for the validation record

Implementation:

Let’s get started by downloading all the required code from our GitHub repository.

Step1: Uploading the custom resource package

In this step we are going to prepare the custom resource package and upload it to an S3 bucket.

First we go into the custom resource directory.
cd cloudar-acm-plus-custom-resource

Next we execute a script to install all required dependencies.
sh install_dependencies

Now we are ready to create the package.
sh pack_custom_resource

You will now find the zipfile ‘cloudar-acm-plus-custom-resource.zip’ in ‘cloudar-acm-plus-custom-resource/packed’, upload this zip file to your S3 bucket.

Step2: Creating a Cloudformation template

Now we can create a Cloudformation template in which we use this custom resource to create an ACM certificate.
You can use the template ‘cfn.yaml’ as an example.

First create a Lambda resource as following

Use the name of your bucket for the property ‘S3Bucket’ .

Next we create the custom resource.

We can set the following properties here:

• DomainName: (REQUIRED type:String) The domain name for the acm certificate.
• AdditionalDomains: (OPTIONAL type:List) Additional domains for the acm certificate
• ValidationDomain: (REQUIRED type:string) The domain name for the validation domain of the acm certificate
• HostedZoneId: (REQUIRED type:string) The hosted zone id for the validation domain of the acm certificate
• CertificateRegion: (REQUIRED type:string) The region to deploy the acm certificate in
• IdempotencyToken: (REQUIRED type:string pattern: \w+) The idempotency token for the create call of the acm certificate doc: https://docs.aws.amazon.com/acm/latest/APIReference/API_RequestCertificate.html#ACM-RequestCertificate-request-IdempotencyToken
• CertificateTags: (OPTIONAL type:list) The tags for the acm certificate

In order for the DNS record cleanup and delete certificate functionality to work when you delete the Cloudformation stack it is important to set the following output.

As you can see we can access the arn of the certifcate created by the custom resource with the GetAtt function on the resource.
!GetAtt CreateCertificateCustomResource.certificate_arn

Step3: Deploy the cloudformation

Finally the only thing left to do is deploy the Cloudformation template.
Once the deploy is started Cloudformation will create the Lambda containing the code from step1 and start a custom resource which will create the certificate and validation records. Once the status of the certificate becomes ‘ISSUED’ the custom resource will finish successfully and report the arn of the certificate back to Cloudformation. We can now further use this arn in other resources in the Cloudformation template.
When you delete the Cloudformation stack, the custom resource will cleanup the validation records in the hosted zone and delete the certificate.

CREATE_COMPLETE

The post Validate ACM certificates in Cloudformation appeared first on Cloudar.

Video isn’t the best reference material if you want to quickly find one of those tips, so I will add a quick summary of them in this blogpost.

• By specifying source_profile in your .aws/config file, you can tell the AWS CLI to use credentials from one profile to assume the role_arn you configure in another profile. This also works in a chain of profiles and can include asking for an MFA token.
• You don’t have to get credentials from the .aws/credentials file, you can tell the CLI to get them from environment variables (this also works if you do not specify anything), instance metadata, or the ECS container role.
• There is a --debug flag, which gives you a lot of output to see everything that’s happening behind the scenes.
• In most cases you do not want that full debug output. Enabling history and using aws history show will give you the most relevant parts in an easy to read format.
• The CLI has the annoying habit of resolving all urls you give it. You can disable this with aws configure set cli_follow_urlparam false.
• You can make S3 uploads and downloads go a lot faster by tweaking the max_concurrent_requests. Or make it not exceed a certain speed with max_bandwith. See here for all options.
• The cli supports defining your own aliases. Look in this awslabs repository for examples.
• Instead of relying on an external tool (that needs to be installed), you can get specific fields from the response by using --query. Combine this with --output text to strip json-specific syntax.
• You can use completion in different shells.
• There are waiters you can use directly from the command line, so you do not have to write your own check-sleep loops in bash.
• Other useful cli tools include cfn-lint, aws-encryption-sdk, aws-shell and aws-vault
• There is a version 2 coming.

The post $ aws help # things you might not know about the AWS CLI appeared first on Cloudar.

In this blog we will use AWS Rekognition to search for ourselves in a collection of pictures.

First we need to acquire this collection, I simply used a Chrome extension to download all pictures of a specific Facebook album.

Next, we will create an S3 bucket to store the pictures, because this project reminds me a bit of where’s waldo I will name the bucket ‘awesome-waldo-bucket’.

Now we will upload all the pictures to this bucket using the command.

aws s3 sync . s3://awesome-waldo-bucket

Enjoy the terminal doing its job, casual snap some screenshots to non-IT friends. It scores well. #hackerMan

While the pictures are uploading we will create the AWS Rekognition collection with the following command.

aws rekognition create-collection –collection-id awesome-waldo-collection

After we create our collection and the pictures are uploaded we will use a Python script to add faces to the collection. The script will loop all the pictures in the S3 bucket and send them to the Rekognition collection with the ‘index faces’ operation, when a picture is added to the collection AWS will automatically process it and detect the faces, these faces will be stored in a map which we will be able to query later.

AWS documentation on this operation.

I added a hard-coded counter in this script just to follow progress.

import boto3
from boto.s3.connection import S3Connection

conn = S3Connection()
bucket = conn.get_bucket('awesome-waldo-bucket')
count = 1

for key in bucket.list():
    print("Added " + str(count) + " pictures out of 1697...")
    count = count + 1

    bucket='awesome-waldo-bucket'
    collectionId='awesome-waldo-collection'
    photo=key.name.encode('utf-8')
    client=boto3.client('rekognition')
    response=client.index_faces
    response=client.index_faces(CollectionId=collectionId,
                                Image={'S3Object':{'Bucket':bucket,'Name':photo}},
                                ExternalImageId=photo,
                                QualityFilter="AUTO")

Next execute the Python script.

You will find that it takes a very long time to process this number of pictures. For the sake of keeping it simple we did not do any batch processing in this blog but it is possible.

AWS documentation for batch processing.

After the images are done processing find a clear and recent picture of yourself, name it ‘me.png’ and upload it to the S3 bucket.

Be sure that it is a clear picture of your face and that you are the only one in it.
For example.

Now that we have all elements in place we can use the following command to retrieve faces matching our sample.

aws rekognition search-faces-by-image –image ‘{“S3Object”:{“Bucket”:”awesome-waldo-bucket”,”Name”:”me.png”}}’ –face-match-threshold 70 –collection-id “awesome-waldo-collection”

You can play with the ‘—face-match-threshold’ to get more or less correct results. The command will return json containing data on the matched faces.

You can repeat this with different sample pictures of your face to get more results.

And now the moment of truth, as it turns out I was not that popular with the photographers on the festival this year as I’m only in two pictures. Better luck next year!

As you can see, AWS Rekognition is a great service to find yourself in a picture even in a big crowd!

This picture surprised me the most, even when your face is very blury AWS Rekognition can still find you! What a gift of technology indeed!

I hope this blog has given you some insight in the possibilities with AWS Rekognition and wish you the best of luck in finding yourself!

The post Discover yourself appeared first on Cloudar.

aws glue create-job --name CloudTrailLogConvertor \
--description Convert and partition CloudTrail logs \
--role AWSGlueServiceRole-CrawlerS3 \	
--command Name=glueetl,ScriptLocation=s3://<scriptbucket>/sample_cloudtrail_job.py \	 	 
--default-arguments '	 	 
"--extra-py-files":"s3://<scriptbucket>/athena_glue_converter_<latest>.zip",
"--job-bookmark-option":"job-bookmark-enable" 
"--raw_database_name":"cloudtrail_logs", 
"--raw_table_name":"cloudtrail_raw", 
"--converted_database_name":"cloudtrail_logs", 	 
"--converted_table_name":"cloudtrail_optimized",	 	 
"--TempDir":"s3://<scriptbucket>/tmp", 	 
"--s3_converted_target":"s3://<logbucket>/converted/cloudtrail",	 	 
"--s3_source_location":"s3://<logbucket>/<account>/cloudtrail/"	 	 
'

aws glue start-job-run --job-name CloudtrailLogConvertor

aws glue get-job-runs --job-name CloudtrailLogConvertor

 "JobRuns": [	 	 
 {	 	 
 "Id": "jr_1cc3f9b8cf88a5abddd4f6957ec53ddfb70839773cc39292d5f8707ca19c7b6c",	
 "Attempt": 0,	 	 
 "JobName": "CloudtrailLogConvertor",	
 "StartedOn": 1541160714.424,	 	 
 "LastModifiedOn": 1541161359.587,	 	 
 "CompletedOn": 1541161359.587,	 	 
 "JobRunState": "SUCCEEDED",	 	 
 "PredecessorRuns": [],	 	 
 "AllocatedCapacity": 10	 	 
 }	 	 
 ]	 	 
}

create external schema cloudtrail_logs
from data catalog
database 'cloudtrail_logs'
iam_role 'arn:aws:iam::<accountnumber>:role/demo-redshift';

:~$ psql -h demo-cluster.cpbkebqgnfoo.eu-west-1.redshift.amazonaws.com -p 5439 -U demouser -d dev
Password for user demouser: 
psql (10.5 (Ubuntu 10.5-0ubuntu0.18.04), server 8.0.2)
SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off)
Type "help" for help.

dev=# SELECT COUNT(*) FROM cloudtrail_logs.cloudtrail_optimized;
 count  
--------
 119182
(1 row)

dev=# 

dev=# SELECT COUNT (*) AS TotalEvents, json_extract_path_text(json_useridentity,'type') AS usertype, eventname
dev-# FROM cloudtrail_logs.cloudtrail_optimized
dev-# WHERE eventtime >= '2017-01-01T00:00:00Z' 
dev-# AND usertype = 'Root'
dev-# GROUP BY eventname,json_useridentity
dev-# ORDER BY TotalEvents DESC
dev-# LIMIT 10;
 totalevents | usertype |         eventname         
-------------+----------+---------------------------
         588 | Root     | DescribeLoadBalancers
         392 | Root     | ListBuckets
         392 | Root     | GetBucketLocation
         336 | Root     | DescribeDBInstances
         168 | Root     | DescribeAutoScalingGroups
          90 | Root     | DescribeAccountLimits
          84 | Root     | GetTrailStatus
          84 | Root     | DescribeTrails
          84 | Root     | DescribeAccountAttributes
          84 | Root     | DescribeDBSecurityGroups
(10 rows)

The post Parse and query CloudTrail logs with AWS Glue, Amazon Redshift Spectrum and Athena appeared first on Cloudar.

The post Cloudar achieves AWS Premier Consulting Partner status appeared first on Cloudar.