Skip to content. Instantly share code, notes, and snippets. Code Revisions 1 Forks 1.AWS-Advanced ECS 2 - CodeBuild & CodePipeline for ECS
Embed What would you like to do? Embed Embed this gist in your website. Share Copy sharable link for this gist. Learn more about clone URLs. Download ZIP. Should give access all AWS resources needed by build and tests run by build. Arn -! GetAtt CodeBuildRole. Arn Pipeline for running build. GetAtt CodePipelineRole. Ref GithubRepoOwner Repo :! Ref GithubRepo Branch :! Sign up for free to join this conversation on GitHub.
Already have an account?
Sign in to comment. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Description : My example codepipeline template. CloudFormation Parameters. Parameters :.
ENV :. Description : Name of the environment for tag metadata. CodeBuildEnvironmentComputeType :. Type : String. AllowedValues :. CodeBuildImage :. Description : The codebuild runtime image to use in the Codebuild project. GithubRepoBranch :. Description : The git branch that will be polled for changes.
Default : " master ". GitHubOAuthToken :. NoEcho : ' true ' Do not display in CloudFormation.With TaskCat, you can run automated tests to learn of and fix any errors that arise in your CloudFormation templates. If you have been using CloudFormation for any period of time, you will learn that even if you have not made any changes to your templates, they might still fail.
This is typical of most software systems in which there are often version, configuration, API, or other changes to dependencies that can affect the operation of the CloudFormation template s. Using this approach, you can configure TaskCat to run with every code change and learn when errors occur in your CloudFormation templates. In this section, you will learn how to use manually run TaskCat automated tests on CloudFormation templates from the command line.
TaskCat is provided as a Python package that you will download.
Run AWS CloudFormation tests from CodePipeline using TaskCat
This example assumes you have access to an AWS account and have established the necessary permissions. If you are not, you should be able to simply modify the directory names accordingly.
To verify TaskCat is installed, type taskcat --version from the command line. You should see something like this returned from the command line:. For the purposes of these examples, I am assuming you are using version 0.
You can run TaskCat in several ways and there are a few command line options that the tool provides. I will take you through a simple example that is currently running on an open source repository that I own. In this section, you will create a new GitHub repository to store a CloudFormation so that you can run TaskCat against this and other CloudFormation templates. For more information, see Create a repo. From your Cloud9 terminal, copy and paste the following into your. This is the configuration file that TaskCat uses to know which CloudFormation templates to run and how to run them.
You can pass in parameters, use TaskCat tokens to generate passwords and other values, and perform other configuration. You use parameters to list the parameters and values when launching the CloudFormation stack. From your Cloud9 terminal, type the following command to run TaskCat against your CloudFormation template. TaskCat will create and delete CloudFormation stacks for all the files listed in the.
In this example, it will create and delete a total of two stacks — one for each listed AWS region in the. When successful, the results will look similar to the image below. To do this, right click on the index. A web page should display that looks similar to the image below. This way you can run TaskCat automatically without needing to manually type commands every time. There are four main steps in launching this solution: prepare an AWS account, create and store source files, launch the CloudFormation stack, and test the deployment.
Each is described in more detail in this section. Please note that you are responsible for any fees incurred while creating and launching your solution. Since we want to run all changes automatically and we want to be secure, you need to store this secrets in an encrypted location. Here are the steps:. Next, you will create a two source files that will be committed to your GitHub repository. From your AWS Cloud9 terminal, type the following to create and save two empty source files:.
Copy the source contents from the buildspec.Categories: AutomationDeveloper Tools.
I spent some time upgrading our Dromedary demo scripts to incorporate this new functionality. All of the code is freely available and open source. Click on the Launch Stack button below. Enter the required parameters. See the figure below and complete the rest of the steps to launch the stack.
Launching CloudFormation stack from command line. The steps I went through to create this CloudFormation template were fairly straightforward.
Finally, I ran through several testing scenarios. If you have any questions or comments, you can contact us at info stelligent.
Stelligent is an expert in deploying continuous delivery pipelines in AWS. If you are looking for help moving your applications into CodePipeline, or another type of continuous delivery pipeline, we can definitely help you with that.
But how to build a deployment pipeline? GitHub has been hosting source code for more than ten years. So to be more accurate, the title of this blog post should be: GitHub Actions vs. CodePipeline and CodeBuild. GitHub Actions offers an outstanding developer experience. As long as you host your source code on GitHub, the solution is flexible, not only because of the integrations aka actions offered in the open marketplace.
Being able to use IAM roles for authentication instead of fiddling around with access keys for IAM users is a big plus. No prior knowledge of Docker and AWS is required.
Get the first chapter for free! It worth mentioning that the integration with AWS CodeBuild allows you to run any script inside a container based on a pre-built or customized image.
This provides maximum flexibility. On top of that, a few 3rd party services are integrated as well. GitHub has taken a different approach: its open marketplace lists more than 2, integrations aka actions 7. Well known and trusted organizations e. A few examples:. Be careful when adding 3rd party actions to your deployment pipeline.
In theory, someone could insert malicious code or steal your AWS credentials by publishing a trojan horse to the GitHub Marketplace.
Also, a 3rd party action could be removed from the GitHub Marketplace, which will break your deployment pipeline out of a sudden. First of all, you need to define a Dockerfile. The Dockerfile defines the base image, installs build dependencies, and adds the entrypoint. Use the entrypoint.
It creates a pipeline with a CodeCommit repository as a source. I'd like to switch this to a Github repository. Here's the code that is defining this resource:.
How is GitHub defined as a resource and how is the authentication handled for a private repository? Learn more. Asked 1 year, 3 months ago. Active 1 year, 3 months ago. Viewed times. GetAtt PipelineExecutionRole. Ken J Ken J 3, 5 5 gold badges 41 41 silver badges 63 63 bronze badges. Active Oldest Votes. Sign up or log in Sign up using Google.
Sign up using Facebook. Sign up using Email and Password. Post as a guest Name. Email Required, but never shown. The Overflow Blog. Podcast Programming tutorials can be a real drag.GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.
If nothing happens, download GitHub Desktop and try again. If nothing happens, download Xcode and try again. If nothing happens, download the GitHub extension for Visual Studio and try again. This repository was created as part of a blog post I wrote for automated testing for CloudFormation templates. The full post can be found here. This is the most basic test you can run on a CloudFormation template.
Linting isn't necessarily a testing tool, but it is a great tool that allows for keeping code formatted in a clean consistent way. Having a tool like this run on every commit eliminates limits the number of code review comments related to style, allowing developers to focus in on the substance of the change instead of the style. Another linter, but this one goes beyond basic style checking.
The CloudFormation Linter is an open-source project currently maintained by the CloudFormation team and has been around since April It aims to check your CloudFormation template against the CloudFormation specification. Additionally, it will detect things like incorrect values for resource properties as well as looking for unused parameters.
Example 3: Create a GitHub Pipeline with AWS CloudFormation
It also supports some basic customization that allows you to tweak the rules to meet the specific needs of your project. Similar to yamllint, this tool is available as a Python module. Yet another linting tool, but this one looks at your template to ensure best practices are being followed.
This is an open-source project maintained by stelligent and is available as a Ruby gem and a Docker image. Another linting tool just kidding. The tool can be configured to run multiple different test scenarios with different input parameters to fully exercise your templates.
Subscribe to RSS
It also has the ability to generate values for your parameters that helps eliminiate some sticky parameters in your template that have to be globally unique e. Note: In my experience, this tool doesn't work on Windows, so I would highly recommend using the Docker image if that is your development operating system of choice. I know this one looks a bit scary, so let's break it down. The command s here will run taskcat and strip the color escape codes from the output. Finally, it will exit with the exit code from taskcat so that if the taskcat tests fail, the test stage will fail, ultimately stopping a deployment to production if an error was discovered.
The pipeline itself. Arguably the most important resource in the template it is in the name. This resource controls the steps that are taken, from checkout, to test, and ultimately deploy. While I won't go into the details of how each of these stages are configured, this resource controls the steps of the pipeline and how they are run. The [documentation for the Pipeline resource type][aws-codepipeline-pipeline-resource-type-documentation] is a great source of inspiration to determine what all is possible.
Additionally, the [documentation for the configuration of the various actions][codepipeline-action-configuration-documentation] is also helpful to figure out what is required for each action within a given stage. By supporting webhooks from GitHub, the pipeline can be triggered as soon as your code is committed to the repository.
This ensures that changes can work their way through the pipeline as quickly as possible and reduce the time it takes to get a change to production. I didn't really cover this previously, but the template also includes a CodeBuild project to run the same buildspec. This can give you earlier feedback on the test results before merging the changes into master. CodeBuild is also capable of reporting the status of the tests back to GitHub, so the results of these checks can be viewed directly from the pull request without having to log into AWS to determine if the tests are passing or failing.
While most of the roles and policies in this template shouldn't need to change, the CloudFormationStackPolicy resource definitely will to meet your purposes.
It is important to remember that this role will stay attach to the stack and be used on subsequent updates unless replaced with a different role. This means the permissions granted to CloudFormation also need to consider what permissions are required to delete the stack. Any time you modify the workload template, make sure to update this policy to include permissions to cover any new resource types or configuration that is being added.
Skip to content.I'm a big fan of the AWS Documentation in general. I also know first-hand many times over that it's not perfect; and realistically there's a limit to how much the service teams can provide. As someone who likes to get their hands dirty when learning a new service, I quickly jump to the quick starts and samplesbut I often find that these examples - while they're good enough for a "hello world" kind of experience - don't go far enough. Luckily, finding crowed sourced examples is not that hard; CloudFormation lends itself well to this because it has such a clearly defined language - I'd argue it's one of its selling points!
Combine that with GitHub Search 's prefix-based code search syntaxand you can quickly find real examples that can supplement the official examples. In this example I'm looking for inline examples of CodePipeline's pipeline stages syntax, as I'm always forgetting it and the CloudFormation docs are a bit light-on with detailed examples. By searching for the specific resource Type that you're implementing, you can see what's working for other people:.
I deliberately haven't included the " : " as GitHub Search ignores them anyway. You'd think adding the full resource type i. If you're getting a bunch of non-template results, force the search to show you only CloudFormation templates with:. The key to note here is that these two search terms are combined as a logical "OR" as opposed to an "AND" which is how many other terms work together ; a file obviously cannot have an extension of yaml AND yml.
This means you get results with both. Searching with the extension: prefix also has the benefit of switching your results to the "Code" results, rather than showing you repositories, issues, etc.
This search gives me a few results that I can reference, which is usually enough to get me productive, quickly:. Note that none of this is a guarantee of quality or security. Please please don't just download some templates from the Internet and launch them in your account, it's just not a Good Idea. Obviously this technique for searching GitHub is not unique to CloudFormation, and can be quickly and easily applied to other languages.
The reason why it works so well for CloudFormation because it has such a clearly defined language; User-determined variable names aka. Given the names are consistent within a language, at least then you can search for them with some similar parameters. Photo by Anthony Martino on Unsplash. Examples In this example I'm looking for inline examples of CodePipeline's pipeline stages syntax, as I'm always forgetting it and the CloudFormation docs are a bit light-on with detailed examples.
By searching for the specific resource Type that you're implementing, you can see what's working for other people: "type aws codepipeline" I deliberately haven't included the " : " as GitHub Search ignores them anyway.
Templates If you're getting a bunch of non-template results, force the search to show you only CloudFormation templates with: "AWSTemplateFormatVersion" Filetype Now that you're seeing templates, narrow it down to YAML results with: extension:yaml extension:yml The key to note here is that these two search terms are combined as a logical "OR" as opposed to an "AND" which is how many other terms work together ; a file obviously cannot have an extension of yaml AND yml.
You can limit your searches to SAM templates by searching for the transform definition: "Transform: AWS::Serverless" Again, the " : " is ignored, but there's no harm including it. Caveat Scrutator Note that none of this is a guarantee of quality or security. Other Code Obviously this technique for searching GitHub is not unique to CloudFormation, and can be quickly and easily applied to other languages.
Check your inbox and click the link to complete signin. Check your inbox and click the link to confirm your subscription. Close You've successfully subscribed to Rowan Udell.
Close Great! You've successfully signed up. Close Welcome back! You've successfully signed in. Close Success!