Self-hosting with AWS ECS

Running the JumpWire engine in your AWS account is straightforward with AWS Elastic Container Service and the Fargate infrastructure. The following guide will walk through the creation of an ECS cluster, task definition, and service to run the engine inside your own AWS account and VPC.

Note this guide uses the "old experience" for ECS in the AWS console. This is because at the time of this writing, the "new experience" does not allow for setting container environment variables from the Secrets Manager. A best practice is to store the token provided to the engine as a secret since it is used as an authn/z parameter.

Unless otherwise specified, we are using defaults provided by the AWS console when creating resources.

Create a new cluster for the service and task definition to run the JumpWire engine container.

In this example, we will use serverless Fargate infrastructure. So we start with a cluster template with networking only.

Minimal configuration here is necessary. We are not creating a VPC, because we want to launch our container into an existing VPC, which hosts our private RDS database. You can also create a new VPC just for the JumpWire engine if desired. This cluster is named "JumpWireEngine".

After a few seconds, the cluster is created and ready for deploying JumpWire. But first, we must create a task definition for the engine container.

The next step is to create a task definition that will run the JumpWire engine container. The task definition will contain a list of the configurations for ports exposed by the engine, and environment variables necessary to configure the container.

Navigate to "Task Definitions" in the left navigation, and click "Create new Task Definition".

In Step 1, select Fargate for the launch type compatibility.

In Step 2, enter a name for the task definition, and select Linux as the "Operating system family".

For the "Task Role" and "Task Execution Role", you can use the "ecsTaskExecutionRole", which AWS will create for you if it does not exist. Note: if you are storing the JumpWire engine token in Secrets Manager, you will need to update this role for access to read from Secrets Manager. More on this in the section below

Next, configure the container to run in the task. You can find all the details for running a container that will connect to the frontend and enforce your policies on the JumpWire configuration page. It's necessary to expose container ports for proxying connections within the VPC, and set environment variables to configure the engine application. You can read more about the configuration here.

Give the container a name, and reference the image listed from the JumpWire configuration page. The image URL should be something like `registry.hub.docker.com/jumpwire/jumpwire:latest`.

Set port mappings for the following ports, all as TCP - 4000, 3306, 5432, 9568

Next set up the environment variables provided to the container while running. The following environment variables are required: JUMPWIRE_CONFIG_ENDPOINT, JUMPWIRE_FRONTEND, and JUMPWIRE_TOKEN

In this example, we are using a secret that securely stores the JUMPWIRE_TOKEN variable, as it is used to authenticate with the frontend. To use a secret, be sure to select "ValueFrom" in the middle drop-down menu.

If you are not using Secrets Manager to store the token, you can also set the value of the environment variable directly to the token value.

Under "Storage and Logging", check "Auto-configure CloudWatch logs". The container engine reports informational messages to standard out, and this is helpful for debugging and verifying successful operation.

To complete the Task Definition, choose a task size of memory and CPU. Minimal sizes should be sufficient for running the engine, we recommend 1G of memory and 512 units of CPU. Click "Create" to complete the Task Definition. If you plan to decrypt large numbers of values at query time, more memory will be required.

Now we can create a service to run the JumpWire Task Definition.

In Step 1: select FARGATE and Linux as the service launch type and operating system family, then the definition we just created with the latest revision. Use the cluster created above and give it a name.

In Step 2: we are using an existing VPC for our network, that also has our private RDS instance already running. This eliminates the need to peer VPCs for the engine to access the database directly. However, you can create a dedicated VPC for the engine if desired. We are only deploying into the private subnets in the VPC, as all proxied connections will be coming from within the VPC.

In Step 3: we do not need autoscaling for this service, so skip this step.

In Step 4: Review and deploy!

_Coming Soon_

AWS Secrets Manager can inject environment variables at runtime for the JumpWire container. This will keep sensitive values more secure and allow for a separation of configuration from application definition. Since the JUMPWIRE_TOKEN is used by the engine to authenticate with the frontend and authorize access to your policies, it should be treated as a secret.

Secrets are created as key/value pairs in the AWS console, but they are stored as a JSON object in Secrets Manager. As each key in the JSON can be referenced as an environment variable, it may be convenient to store all of the container environment variables as secrets to simplify configuration.

To create a new secret to store environment variables, click "Store a new secret" from the Secrets Manager console.

In Step 1: select "Other type of secret". Enter JUMPWIRE_TOKEN as a key, and paste the token from your JumpWire configuration page as the value.

In Step 2: give the secret a name. We are using dev/jumpwire/engine/token

In Step 3: keep automatic rotation disabled, which is the default.

In Step 4: review and deploy.

To use values from the secret as environment variables, choose "ValueFrom" in the middle drop-down when setting variables in the container definition. The value of the environment variable should be set as the ARN of the secret you created, plus the name of the key - something like arn:aws:secretsmanager:us-east-2:1111:secret:dev/jumpwire/engine/token-aaBbCc:JUMPWIRE_TOKEN:: You can find the ARN for the secret you created by clicking on the details of the secret, and append with :JUMPWIRE_TOKEN:: to specify the key in the secret to use.

Update the ecsTaskExecutionRole that is created by ECS to grant IAM permissions for the task to read secrets from Secrets Manager. You can find the role definition under AWS IAM roles in the console. There is an AWS-managed policy called "SecretsManagerReadWrite" that can be assigned to the role. Click on "AddPermissions" > "AttachPolicies" and search for the AWS-managed policy.