application-master-$account.json

Purpose

This configuration file holds infrastruction information for $account. Each AWS account in your pipeline would need a seperate application-master-$account.json file. If your account is named dev, you would want an application-master-dev.json file.

Example Configuration

{
    "app": {
        "app_description": null,
        "email": null,
        "eureka_enabled": false,
        "instance_profile": "{{ profile }}",
        "instance_type": "t2.micro",
        "lambda_environment": null,
        "lambda_memory": "128",
        "lambda_role": null,
        "lambda_timeout": "30",
        "canary": false
    },
    "asg": {
        "hc_type": "ELB",
        "hc_grace_period": 180,
        "app_grace_period": 0,
        "max_inst": 3,
        "min_inst": 1,
        "ssh_keypair": null,
        "subnet_purpose": "internal",
        "enable_public_ips": null,
        "provider_healthcheck": {
            "amazon": false
        },
        "scaling_policy": {}
    },
    "elb": {
        "certificate": null,
        "policies": [],
        "listener_policies": [],
        "backend_policies": [],
        "idle_timeout": null,
        "access_log": {},
        "connection_draining_timeout": null,
        "health": {
            "interval": 20,
            "threshold": 2,
            "timeout": 10,
            "unhealthy_threshold": 5
        },
        "i_port": 8080,
        "i_proto": "HTTP",
        "lb_port": 80,
        "lb_proto": "HTTP",
        "subnet_purpose": "internal",
        "target": "TCP:8080"
    },
    "qe": {
    },
    "regions": [
        "us-east-1"
    ],
    "deploy_strategy": "highlander",
    "security_group": {
        "description": "Auto-Gen SG for {{ app }}",
        "egress": "0.0.0.0/0",
        "elb_extras": [],
        "ingress": {
        },
        "instance_extras": []
    },
    "dns": {
        "ttl": 60,
        "failover_dns": true,
        "region_specific": true
    },
    "lambda_triggers": [],
    "s3": {
        "shared_bucket_master": false,
        "path": "/",
        "bucket_acl": "private",
        "bucket_policy": {},
        "website": {
            "enabled": true,
            "index_suffix": "index.html",
            "error_document": " "
        },
        "content_metadata": []
    },
    "datapipeline": {
        "description": "",
        "activate_on_deploy": false,
        "json_definition": {}
    }
}

Configuration Details

app Block

Top level key that contains information on the application and EC2 details

app_description

Describes the application.

Type: string
Default: null

eureka_enabled

Setting this value to true will not create an ELB, DNS record, and set the ASG health check to EC2.

Type: boolean
Default: false

instance_profile

The instance profile to start EC2 instances with. Foremast creates default instance profile based on the default string. Specifying a different profile name assumes the profile exists.

Type: string
Default: "${stack}_${app}_profile"

instance_type

The size/type of the EC2 instance. Uses Standard AWS instance names. See https://aws.amazon.com/ec2/instance-types/ for details

Type: string
Default: "t2.micro"

lambda_environment

Environment variables which are passed to the lambda function.

lambda_environment Keys

Variables : Dictionary of environment variables.

Type: object
Default: null
lambda_environment Example
{
    "environment": {
        "Variables": {
            "VAR1": "val1",
            "VAR2": "val2",
            "VAR3": "val3"
        }
    }
}

lambda_memory

The amount of memory to give a Lambda function

Type: string
Default: "128"
Units: Megabytes

lambda_role

Override the default generated IAM Role name.

Type: string
Default: "${stack}_${app}_role"

lambda_timeout

The timeout setting for Lambda function

Type: string
Default: "3600"
Units: Seconds

asg Block

Top level key containing information regarding application ASGs

hc_type

Note

See foremast.pipeline.construct_pipeline_block.construct_pipeline_block() for cases where the Health Check type is overridden to "EC2".

ASG Health check type (EC2 or ELB)

Type: string
Default: "ELB"
Options:
  • "ELB"
  • "EC2"

app_grace_period

App specific health check grace period (added onto default ASG healthcheck grace period) to delay sending of health check requests. This is useful in the event your application takes longer to boot than the default hc_grace_period defined in templates.

For example, hc_grace_period may be 180 seconds, but an app may need a variable amount of time to boot (say 30 seconds extra). This will add 180 + 30 to calculate the overall hc_grace_period of 210 seconds.

Type: number
Default: 0
Units: Seconds

max_inst

Maximum number of instances ASG will scale to.

Type: number
Default: 3

min_inst

Minimum number of instances your auto-scaling group should have at all times. This is also the default number of instances

Type: number
Default: 1

ssh_keypair

SSH key that your EC2 instances will use. Must already be created in AWS. This replaces the non-functional and deprecated app_ssh_key configuration key.

Type: string
Default: "{{ account }}_{{ region }}_default" - {{ account }} being the AWS account in the configuration name

subnet_purpose

Determines if the instances should be public (external) or non-public (internal).

Type: string
Default: "internal"
Options
  • "internal"
  • "external"

enable_public_ips

Determines if instances in an cluster should have public IPs associated. By default, this is set to null which means it uses default behavior configured for your subnets in your cloud provider.

Type: boolean
Default: null
Options
  • true
  • false

scaling_policy

Defines scaling policy to attach to ASG. If this block does not exist, no scaling policy will be attached

scaling_policy Keys

metric : The CloudWatch metric to trigger auto-scaling events.

Type: string
Default: "CPUUtilization"
Options:
  • "CPUUtilization"
  • "NetworkIn"
  • "NetworkOut"
  • "DiskReadBytes"

threshold : Metrics value limit for scaling up

Type: number

period_minutes : Time period to look across for determining if threshold was met

Type: number
Units: Minutes

statistic: Statistic to calculate at the period to determine if threshold was met

Type: string
Default: "Average"
Options:
  • "Average"
  • "Maximum"
  • "Minimum"
  • "Sum"
scaling_policy Example
{
    "scaling_policy": {
        "metric": "CPUUtilization",
        "threshold": 90,
        "period_minutes": 10,
        "statistic": "Average"
    }
}

elb Block

Top level key for ELB configuration

access_log

Access Log configuration block. Ensure S3 bucket has proper bucket policy to enable writing.

access_log Keys

bucket_name : Name of S3 bucket to write access log to

Type: string
Default: Null

bucket_prefix : Prefix to write to in the S3 bucket

Type: string
Default: Null

emit_interval : ELB Access Log write delay

Type: number
Range: 5-60
Units: seconds
Default: Null

connection_draining_timeout

Connection Draining Timeout to set on the ELB. This allows existing requests to complete before the load balancer shifts traffic away from a deregistered or unhealthy instance.

Type: number
Range: 1-3600
Units: seconds
Default: Null

certificate

Name of SSL certification for ELB. SSL certificate must be uploaded to AWS first.

Type: string
Default: Null

health

Health check configuration block

health Keys

interval : ELB health check interval

Type: number
Units: seconds
Default: 20

threshold : Number of consecutive health check succeses before declaring EC2 instance healthy.

Type: number
Default: 2

timeout : Health check response timeout

Type: number
Units: seconds
Default: 10

unhealthy_threshold : number of consecutive health check failures before declaring EC2 instance unhealthy

Type: number
Default: 5

idle_timeout

Idle Timeout to set on the ELB. This the time, in seconds, that the connection is allowed to be idle (no data has been sent over the connection) before it is closed by the load balancer.

Type: number
Range: 1-3600
Units: seconds
Default: 60

ports

Defines ELB listeners. Expects a list of listeners.

ports Keys

instance : The protocol:port of the instance

Type: string
Default: "HTTP:8080"

loadbalancer : the protocol:port of the load balancer

Type: string
Default: "HTTP:80"

stickiness : defines stickiness on ELB; if app, specify cookie_name, if elb, specify cookie_ttl

Type: object
Default: None
Supported Types: elb, app
Example app:
{
    "stickiness": {
        "type": "app",
        "cookie_name": "$cookiename"
    }
}
Example elb:
{
    "stickiness": {
        "type": "elb",
        "cookie_ttl": 300
    }
}

certificate : The name of the certificate to use if required

Type: string
Default: null

listener_policies : A list of listener policies to associate to an ELB. Must be created in AWS first.

Type: array
Default: []

backend_policies : A list of backend server policies to associate to an ELB. Must be created in AWS first.

Type: array
Default: []
Example: ["WebSocket-Proxy-Protocol"]`
ports Example
{
    "ports": [
        {
            "instance": "HTTP:8080",
            "loadbalancer": "HTTP:80",
            "stickiness": {
                "type": "app",
                "cookie_name": "cookie"
            }
        },
        {
            "certificate": "my_cert",
            "instance": "HTTP:8443",
            "loadbalancer": "HTTPS:443",
            "listener_policies": [
                "MyExamplePolicy"
            ],
            "stickiness": {
                "type": "elb",
                "cookie_name": 300
            }
        }
    ]
}

subnet_purpose

Determines if the load balancer should be public (external) or non-public (internal). When changing this option, the ELB and DNS Records must be manually destroyed before deployment. This is necessary because the ELB Scheme is not modifiable.

Type: string
Default: "internal"
Options:
  • "internal"
  • "external"

target

The check the ELB will use to validate application is online.

Type: string
Default: "TCP:8080"

regions Key

List of AWS regions that application will be deployed to.

Type: array
Default: [ "us-east-1" ]

deploy_strategy Key

Spinnaker strategy to use for deployments.

Type: string
Default: “highlander”
Options:
  • "highlander" - destroy old server group
  • "redblack" - disables old server group but do not destroy
  • "canary" - Only used in S3 deployments. Causes pipeline to first deploy to CANARY path
  • "alpha" - Only used in S3 deployments. Causes pipeline to first deploy to an ALPHA path

security_group Block

Hold configuration for creating application specific security group

description

Description of the security group. Used in AWS for creation

Type: string
Default: "Auto-Gen SG for {{ app }}"

elb_extras

A list of extra security groups to assign to ELB

Type: array
Default: []
Example: ["all_access", "test_sg"]`

instance_extras

A list of extra security groups to assign to each instance

Type: array
Default: []
Example: ["all_access", "test_sg"]`

ingress

Provides a list of other security groups and ports to allow inbound access to application

egress

Provides info about outbound access from application

Type: string
Default: "0.0.0.0/0"`

security_group Example

You can reference SG by name or by cidr block, you can also specify cross account SG by name by referring to the spinnaker environment name. To see an example of this see below:

{
    "security_group": {
        "ingress": {
            "examplesecuritygroupname": [
                {"start_port": 80, "end_port": 80, "protocol": "tcp"},
                {"start_port": 443, "end_port": 443, "protocol": "tcp"},
                {"start_port": 443, "end_port": 443, "protocol": "tcp", "env": "prod"}
            ],
            "192.168.100.0/24": [
                {"start_port": 80, "end_port": 80, "protocol": "tcp"}
            ]
        },
        "egress": {
            "192.168.100.0/24": [
                {"start_port": 80, "end_port": 80, "protocol": "tcp"}
            ]
        }
    }
}

dns Block

Top level key for dns settings

ttl

Defines DNS TTL for generated DNS records

Type: number
Units: seconds
Default: 60

lambda_triggers

A list of all events to trigger a Lambda function. See Lambda Triggers and Events for details

Type: array
Default: []

datapipeline Block

Top level key for AWS Data Pipeline settings. Only necessary for Data Pipeline deployments.

name

Name of the Data Pipeline. This defaults to the application name.

Type: string
Default: $appname

description

Description of the Data Pipeline.

Type: string
Default: ""

activate_on_deploy

Activates a Data Pipeline after deployment. Useful for OnDemand pipelines

Type: boolean
Default: false

json_definition

The exported JSON definition of the AWS Data Pipeline. You can get this by clicking “Export” in the AWS Console when creating the Data Pipeline.

Type: object
Default: {}

s3 Block

Holds settings related to s3 deployments

path

Path to upload assets to in a specified s3 bucket. Only works for S3 pipelines not using shared/master bucket setup. Refer to s3_bucket_master for more information.

Type: string
Default: "/"

bucket_acl

General ACL to apply to S3 bucket

Type: string
Default: "private"
Options:
  • "public"
  • "private"

bucket_policy

The S3 bucket policy in json format to apply to created S3 bucket. Must be a valid S3 bucket policy; use the AWS policy generator/simulator to test your policy. (https://awspolicygen.s3.amazonaws.com/policygen.html)

Type: json
Default: "{}"

content_metadata

S3 object metadata based on path. The “path” field should have NO leading or trailing slashes.

Type: object
Default: None
Example config:
[
    {
        "path": "assets/compressed",
        "content-encoding": "br"
    },
    {
        "path": "assets/gzip",
        "content-encoding": "gzip"
    }
]

website

S3 Website configuration block

website Keys

enabled : Enables/Disables an S3 bucket from being website enabled

Type: boolean
Default: false

index_suffix : Default index page

Type: string
Default: "index.html"

error_document : Default error page

Type: string
Default: "404.html"

shared_bucket_master

Setups up an S3 bucket as a shared target so other Spinnaker pipelines can upload to it. i

Type: boolean
Default: false
Example:
An example of this is having one s3 bucket for a given website. This website has a commercial and business webpage that are two unique deployment pipelines. Each of the unique apps would specify the shared_bucket_target to the Spinnaker application name of the shared_bucket_master pipeline. It is common to have a bare source repository for a master bucket with just Foremast pipeline and application configuration files.

shared_bucket_target

Shared bucket to deploy to. Refer to shared_bucket_master for use case and example.

Type: string
Default: None