Skip to main content

Bundle Development

Massdriver bundles wrap IaC tools like Terraform and Helm (more coming soon).

This guide will cover some of the key components and best practices for integrating your IaC module with the Massdriver UI and provisioning system.

note

A tl;dr full walkthrough on building an AWS SNS Topic bundle can be found here if you want to jump straight to the code.

For developing infrastructure bundles and applications you'll need the Massdriver CLI.

Where Are My Variable Files?​

Let's generate a bundle:

mass bundle new

You'll notice that in your ./src directory there are no Terraform variable files. Massdriver uses JSON Schema to present a rich user interface to end-users and uses that interface to also generate Terraform and Helm variable files.

You can generate your variables by running:

mass bundle build

Building the Right-Sized Bundle​

We recommend building single use-case scoped bundles rather than bundles that encapsulate an entire cloud service.

Complex infrastructure modules are confusing to end-users (S3 Buckets have over 90 attributes) and there are many fields in the cloud resource APIs that are incompatible with each other.

We've also seen how bloated general purpose modules can be. EKS modules

Bad scopes:

  • AWS RDS - End-users are looking for a specific database type; there are many fields in RDS configs that only work for MySQL or Postgres.
  • AWS S3 - As noted above, S3 has a lot of fields that may not apply to an end-user use case.

Good scopes:

  • AWS RDS MySQL
  • AWS RDS Postgres
  • AWS S3 CDN Bucket
  • AWS S3 Logging Bucket
  • AWS S3 Application Assets Bucket

Smaller scopes mean simpler use cases. This makes it easier for the user to configure and manage, while having simpler IAM policies that follow least privileges for their specific use case.

Massdriver bundles should do one thing and do it well.

Bundle Naming Guidelines​

Massdriver recommends (and follows) the following naming convention for bundles:

  • $cloud-$service-$use-case
  • $cloud-$service-$use-case-$component

Bad names:

  • rds
  • aws-rds

Good names:

  • aws-rds-mysql
  • aws-rds-postgres
  • aws-sns-pubsub-topic
  • aws-sqs-pubsub-subscription
  • aws-lambda-pubsub-subscriber

Massdriver Metadata​

Massdriver provides metadata to your IaC tool during provisioning. We publish a JSON Schema of the metadata and provide backwards compatibility in the data provided to your bundle.

The metadata will be a top-level value provided to your IaC tool.

For example, in Terraform:

var.md_metadata

There are various fields that your IaC tool can use to integrate with the Massdriver platform.

MD Metadata Properties:

  • name_prefix (string): Standardized cloud agnostic naming convention prefix to be used on all resources created by a bundle. The name prefix is the project slug, environment slug, manifest slug, and a random 4 character slug joined by hyphens. This should be the name/identifier of the primary resource in your bundle and used a prefix for other resources created. The name prefix also happens to be the Massdriver package name. Minimum: 33. Maximum: 33.
  • default_tags (object): Default tags to be applied to all bundle resources.
    • managed-by (string): Provisioning tool managing the resource. Must be one of: ['massdriver'].
    • md-manifest (string): Massdriver manifest slug. Minimum: 1. Maximum: 12.
    • md-package (string): Massdriver package name. Minimum: 33. Maximum: 33.
    • md-project (string): Massdriver project slug. Minimum: 1. Maximum: 7.
    • md-environment (string): Massdriver environment slug. Minimum: 1. Maximum: 7.
  • environment (object): Environment metadata for this deployment.
    • contact_email (string): The email address of the contact for this environment. This email address may be used by services like Lets Encrypt or to validate AWS SES domains.
  • observability (object): Observability integration metadata for this deployment.
    • alarm_webhook_url (string): A webhook URL to process metrics from AWS SNS Notifications, GCP Notification Channels, and Azure Monitor Action Groups. This will be a Massdriver URL used to present alerts from cloud resources.

Example MD Metadata:

{
  "default_tags": {
    "managed-by": "massdriver",
    "md-manifest": "caching",
    "md-package": "ecomm-prod-caching-1234",
    "md-project": "ecomm",
    "md-target": "prod"
  },
  "name_prefix": "ecomm-prod-caching-1234",
  "observability": {
    "alarm_webhook_url": "http://host.docker.internal:4000/alarms/a1ac80a5-b577-41a6-b247-682514aff51d/9a32bf1dd94f857fe57c264d7e0deaa8"
  },
  "organization": {
    "owner_email": "chauncy@kewld00d.info"
  }
}

Using md_metadata.name_prefix​

note

WIP : As identifier for primary resources, prefix for all other resource names/identifiers.

Integrated Monitoring & Alarms​

note

WIP : Alarm integrations using TF modules

Params & Connections​

note

WIP : How params and connections are routed to variable files. Using JSON Schema for rich validation.

Recommend Params Field Sets​

  • _resource_type_ i.e.: database for RDS, topic for SNS
  • networking
  • storage
  • backup
  • observability

Local Development​

info

Coming soon we will be releasing a UI visualizer for massdriver.yaml.

Two files are useful for local development:

  • _params.auto.tfvars.json
  • _connections.auto.tfvars.json

Artifacts

note

If you generated your bundle using mass bundle new a .gitignore file should have been created for you excluding all .tfvars.json files from git.

Artifacts​

Artifacts are outputs of a bundle that adhere to a specific type (artifact definition) and can be attached to other bundles.

There are two types of artifacts: provisioned and imported.

Provisioned Artifacts​

Provisioned artifacts are created during the deployment process, and therefore cannot be altered or removed outside of a provisioning run. Massdriver provides tooling to simplify the process of creating provisioned artifacts.

Massdriver Terraform Provider:

Massdriver maintains a terraform provider with a resource (massdriver_artifact) for creating provisioned artifacts within a terraform based bundle. Refer to the terraform provider documentation for more information.

JQ extraction

Massdriver provisioners have the ability to convert provisioning inputs (params, connections) and outputs (provsioner-specific) into an artifact using the powerful jq syntax for JSON querying and manipulation. Using this method, you can create a file named artifact_<name>.jq at the top level of the provisioner step. The <name> field refers to the name of the artifact in the massdriver.yaml file. The structure of the file should reflect the artifact structure, with jq syntax supported for querying. The input data will be a JSON object with the input params under a top level params key, connections under a connections key, and the outputs of the provisioning run under an outputs key.

Below is an example of an artifact file using this method:

./src/artifact_foo.jq
{
  "data": {
    "value": .outputs.someblock.somevalue,
    "another": .connections.database.data.field
  },
  "specs": {
    "version": .params.version
  }
}

In this example, we are creating the content for an artifact named foo that must be declared in the massdriver.yaml file. The artifact will be created as follows:

  • The .data.value field will be set to the value of someblock.somevalue from the provisioning output
  • The .data.another field is being set to the .data.field value from the incoming connection named database
  • The .spec.version field will be set to the value of the top-level parameter named version

Refer to the documentation on each provisioner for more information on the structure of the provisioner outputs.

Imported Artifacts​

Imported artifacts are created outside of the deployment process in the Massdriver platform. This type of artifact is useful for representing existing infrastructure that isn't managed by Massdriver, but you would like to connect to it with bundles managed inside the Massdriver platform.

Check out this guide for how to import artifacts.

Preset Configurations​

Presets allow you to provide pre-configured example configurations for your Massdriver bundle's parameters. These presets appear as selectable options in the Massdriver UI, making it easier for users to get started with recommended or common configurations. Presets are defined in your massdriver.yaml file under the params.examples key.

Why Use Presets?​

  • Onboarding: Help users quickly deploy with best-practice or common configurations.
  • Documentation: Show real-world use cases directly in the UI.
  • Consistency: Encourage standardization across environments or teams.

How Presets Work​

Each preset in the params.examples array consists of:

  • A __name: The friendly label shown in the UI dropdown.
  • The rest of the object: The actual configuration object that matches your bundle's parameter JSON schema for parameters.

When a user selects a preset in the UI, the parameter form is automatically populated with the values from the preset, which they can then further customize if needed.

Example: Adding Presets to massdriver.yaml​

Below are example snippets showing how to add presets to your bundle configuration.

Example 1: Minimal Example​

params:
  examples:
    - __name: Default
      foo: bar
      enabled: true

Example 2: S3 Bucket Presets​

params:
  examples:
    - __name: Logging Bucket
      bucket_type: logging
      versioning: true
      lifecycle_rules:
        - id: expire-logs
          enabled: true
          expiration_days: 30
    - __name: CDN Assets Bucket
      bucket_type: cdn
      versioning: false
      cors:
        - allowed_origins: ["*"]
          allowed_methods: ["GET"]

Example 3: Aurora MySQL Bundle​

This example is taken from the AWS Aurora MySQL bundle.

params:
  examples:
    - __name: Development
      backup:
        skip_final_snapshot: true
        retention_period: 1
      observability:
        enabled_cloudwatch_logs_exports: []
        enhanced_monitoring_interval: 0
        performance_insights_retention_period: 0
      networking:
        subnet_type: internal
      availability:
        min_replicas: 0
      database:
        instance_class: db.t4g.medium
        deletion_protection: false
    - __name: Production
      backup:
        skip_final_snapshot: false
        retention_period: 35
      observability:
        enabled_cloudwatch_logs_exports:
          - audit
          - error
          - general
          - slowquery
        enhanced_monitoring_interval: 60
        performance_insights_retention_period: 372
      networking:
        subnet_type: internal
      availability:
        min_replicas: 2
      database:
        instance_class: db.r6g.2xlarge
        deletion_protection: true

Tips for Creating Presets​

  • Be Descriptive: Use clear __name values to help users understand the intent of each preset.
  • Cover Common Use Cases: Include presets for production, development, and any other common scenarios.
  • Validate Values: Ensure the preset object matches your bundle's parameter JSON schema.
  • Keep It Up to Date: Update presets as your bundle evolves or as best practices change.