Skip to content

AWS.EMRContainers reference

CancelJobRun

Section titled “CancelJobRun”

Source: src/AWS/EMRContainers/CancelJobRun.ts

Runtime binding for emr-containers:CancelJobRun.

Cancels a job run on the bound VirtualCluster. The virtual cluster ID is injected from the binding; pass the job run id returned by StartJobRun. Provide the implementation with Effect.provide(AWS.EMRContainers.CancelJobRunHttp).

CancelJobRun: Running Jobs

Section titled “CancelJobRun: Running Jobs”
// init
const cancelJobRun = yield* AWS.EMRContainers.CancelJobRun(virtualCluster);


// runtime
yield* cancelJobRun({ id: jobRunId });

DescribeJobRun

Section titled “DescribeJobRun”

Source: src/AWS/EMRContainers/DescribeJobRun.ts

Runtime binding for emr-containers:DescribeJobRun.

Reads a job run on the bound VirtualCluster — its state (PENDING → RUNNING → COMPLETED/FAILED/CANCELLED), failure reason, and the resolved driver/configuration. The virtual cluster ID is injected from the binding; pass the job run id returned by StartJobRun. Provide the implementation with Effect.provide(AWS.EMRContainers.DescribeJobRunHttp).

DescribeJobRun: Running Jobs

Section titled “DescribeJobRun: Running Jobs”
// init
const describeJobRun = yield* AWS.EMRContainers.DescribeJobRun(virtualCluster);


// runtime
const { jobRun } = yield* describeJobRun({ id: jobRunId }).pipe(
  Effect.repeat({
    schedule: Schedule.spaced("15 seconds"),
    until: (r) =>
      r.jobRun?.state === "COMPLETED" || r.jobRun?.state === "FAILED",
    times: 40,
  }),
);

DescribeJobTemplate

Section titled “DescribeJobTemplate”

Source: src/AWS/EMRContainers/DescribeJobTemplate.ts

Runtime binding for emr-containers:DescribeJobTemplate.

Reads the bound JobTemplate’s stored StartJobRun values — the execution role, release label, job driver, and parameter configuration. The template ID is injected from the binding. Provide the implementation with Effect.provide(AWS.EMRContainers.DescribeJobTemplateHttp).

DescribeJobTemplate: Job Templates

Section titled “DescribeJobTemplate: Job Templates”
// init — bind the operation to the template
const describeJobTemplate =
  yield* AWS.EMRContainers.DescribeJobTemplate(template);


// runtime
const { jobTemplate } = yield* describeJobTemplate();
yield* Effect.log(
  `template targets ${jobTemplate?.jobTemplateData.releaseLabel}`,
);

DescribeManagedEndpoint

Section titled “DescribeManagedEndpoint”

Source: src/AWS/EMRContainers/DescribeManagedEndpoint.ts

Runtime binding for emr-containers:DescribeManagedEndpoint.

Reads a managed endpoint (the EMR Studio gateway) on the bound VirtualCluster — its state, server URL, and security group. The virtual cluster ID is injected from the binding; pass the endpoint id. Provide the implementation with Effect.provide(AWS.EMRContainers.DescribeManagedEndpointHttp).

DescribeManagedEndpoint: Managed Endpoints

Section titled “DescribeManagedEndpoint: Managed Endpoints”
// init
const describeManagedEndpoint =
  yield* AWS.EMRContainers.DescribeManagedEndpoint(virtualCluster);


// runtime
const { endpoint } = yield* describeManagedEndpoint({ id: endpointId });
yield* Effect.log(`endpoint state: ${endpoint?.state}`);

DescribeVirtualCluster

Section titled “DescribeVirtualCluster”

Source: src/AWS/EMRContainers/DescribeVirtualCluster.ts

Runtime binding for emr-containers:DescribeVirtualCluster.

Reads the bound VirtualCluster’s live state — its lifecycle state (RUNNING/TERMINATING/TERMINATED/ARRESTED), the backing EKS container provider, and the attached security configuration. The virtual cluster ID is injected from the binding — the runtime callable takes no arguments. Provide the implementation with Effect.provide(AWS.EMRContainers.DescribeVirtualClusterHttp).

DescribeVirtualCluster: Virtual Clusters

Section titled “DescribeVirtualCluster: Virtual Clusters”
// init — bind the operation to the virtual cluster
const describeVirtualCluster =
  yield* AWS.EMRContainers.DescribeVirtualCluster(virtualCluster);


// runtime
const { virtualCluster } = yield* describeVirtualCluster();
yield* Effect.log(`virtual cluster is ${virtualCluster?.state}`);

GetManagedEndpointSessionCredentials

Section titled “GetManagedEndpointSessionCredentials”

Source: src/AWS/EMRContainers/GetManagedEndpointSessionCredentials.ts

Runtime binding for emr-containers:GetManagedEndpointSessionCredentials.

Mints a session token for connecting to a managed endpoint (EMR Studio gateway) on the bound VirtualCluster. The virtual cluster identifier is injected from the binding; pass the endpointIdentifier, the executionRoleArn to run as, and the credentialType (TOKEN). The returned credentials.token is a Redacted value — call Redacted.value(...) at the point of use. Provide the implementation with Effect.provide(AWS.EMRContainers.GetManagedEndpointSessionCredentialsHttp).

GetManagedEndpointSessionCredentials: Managed Endpoints

Section titled “GetManagedEndpointSessionCredentials: Managed Endpoints”
// init
const getSessionCredentials =
  yield* AWS.EMRContainers.GetManagedEndpointSessionCredentials(virtualCluster);


// runtime
const { credentials } = yield* getSessionCredentials({
  endpointIdentifier: endpointId,
  executionRoleArn: jobRoleArn,
  credentialType: "TOKEN",
  duration: "15 minutes",
});
const token = Redacted.value(credentials!.token);

JobTemplate

Section titled “JobTemplate”

Source: src/AWS/EMRContainers/JobTemplate.ts

An Amazon EMR on EKS job template — a stored set of StartJobRun values (execution role, release label, job driver, configuration) that can be referenced by ID when starting job runs, optionally with ${placeholder} parameters filled in per run.

Job templates are account-level and fully immutable: any change (including tags, which the tagging API does not support post-create for templates) replaces the template. They pair with the AWS.EMRContainers.StartJobRun binding — a Lambda can start a templated Spark job with just the template ID and parameter values.

JobTemplate: Creating Job Templates

Section titled “JobTemplate: Creating Job Templates”

A Spark Job Template

import * as AWS from "alchemy/AWS";


const template = yield* AWS.EMRContainers.JobTemplate("EtlTemplate", {
  jobTemplateData: {
    executionRoleArn: jobRole.roleArn,
    releaseLabel: "emr-7.5.0-latest",
    jobDriver: {
      sparkSubmitJobDriver: {
        entryPoint: "s3://my-bucket/scripts/etl.py",
        sparkSubmitParameters: "--conf spark.executor.instances=2",
      },
    },
  },
});

Parameterized Template

const template = yield* AWS.EMRContainers.JobTemplate("Parameterized", {
  jobTemplateData: {
    executionRoleArn: jobRole.roleArn,
    releaseLabel: "emr-7.5.0-latest",
    jobDriver: {
      sparkSubmitJobDriver: { entryPoint: "${EntryPoint}" },
    },
    parameterConfiguration: {
      EntryPoint: { type: "STRING" },
    },
  },
});
// StartJobRun with jobTemplateId + jobTemplateParameters: { EntryPoint: "s3://..." }

ListJobRuns

Section titled “ListJobRuns”

Source: src/AWS/EMRContainers/ListJobRuns.ts

Runtime binding for emr-containers:ListJobRuns.

Lists job runs on the bound VirtualCluster, optionally filtered by state, name, or creation time. The virtual cluster ID is injected from the binding. Provide the implementation with Effect.provide(AWS.EMRContainers.ListJobRunsHttp).

ListJobRuns: Running Jobs

Section titled “ListJobRuns: Running Jobs”
// init
const listJobRuns = yield* AWS.EMRContainers.ListJobRuns(virtualCluster);


// runtime
const { jobRuns } = yield* listJobRuns({
  states: ["PENDING", "SUBMITTED", "RUNNING"],
});
yield* Effect.log(`${jobRuns?.length ?? 0} active job runs`);

ListJobTemplates

Section titled “ListJobTemplates”

Source: src/AWS/EMRContainers/ListJobTemplates.ts

Runtime binding for emr-containers:ListJobTemplates.

Enumerates the account’s EMR on EKS job templates, optionally filtered by creation time. Account-level — no resource argument. Provide the implementation with Effect.provide(AWS.EMRContainers.ListJobTemplatesHttp).

ListJobTemplates: Job Templates

Section titled “ListJobTemplates: Job Templates”
// init — account-level binding, no resource argument
const listJobTemplates = yield* AWS.EMRContainers.ListJobTemplates();


// runtime
const { templates } = yield* listJobTemplates();
yield* Effect.log(`${templates?.length ?? 0} job templates`);

ListManagedEndpoints

Section titled “ListManagedEndpoints”

Source: src/AWS/EMRContainers/ListManagedEndpoints.ts

Runtime binding for emr-containers:ListManagedEndpoints.

Lists managed endpoints (EMR Studio gateways) on the bound VirtualCluster, optionally filtered by state or type. The virtual cluster ID is injected from the binding. Provide the implementation with Effect.provide(AWS.EMRContainers.ListManagedEndpointsHttp).

ListManagedEndpoints: Managed Endpoints

Section titled “ListManagedEndpoints: Managed Endpoints”
// init
const listManagedEndpoints =
  yield* AWS.EMRContainers.ListManagedEndpoints(virtualCluster);


// runtime
const { endpoints } = yield* listManagedEndpoints({ states: ["ACTIVE"] });

ListVirtualClusters

Section titled “ListVirtualClusters”

Source: src/AWS/EMRContainers/ListVirtualClusters.ts

Runtime binding for emr-containers:ListVirtualClusters.

Enumerates the account’s EMR on EKS virtual clusters, optionally filtered by state, container provider, or creation time. Account-level — no resource argument. Provide the implementation with Effect.provide(AWS.EMRContainers.ListVirtualClustersHttp).

ListVirtualClusters: Virtual Clusters

Section titled “ListVirtualClusters: Virtual Clusters”
// init — account-level binding, no resource argument
const listVirtualClusters = yield* AWS.EMRContainers.ListVirtualClusters();


// runtime
const { virtualClusters } = yield* listVirtualClusters({
  states: ["RUNNING"],
});
yield* Effect.log(`${virtualClusters?.length ?? 0} running virtual clusters`);

StartJobRun

Section titled “StartJobRun”

Source: src/AWS/EMRContainers/StartJobRun.ts

Runtime binding for emr-containers:StartJobRun.

Submits a Spark job run to the bound VirtualCluster. The virtual cluster ID is injected from the binding; pass a jobDriver + executionRoleArn + releaseLabel directly, or a jobTemplateId (+ jobTemplateParameters) to start from a JobTemplate. Returns the new job run’s id/arn for use with DescribeJobRun / CancelJobRun. Provide the implementation with Effect.provide(AWS.EMRContainers.StartJobRunHttp).

StartJobRun: Running Jobs

Section titled “StartJobRun: Running Jobs”
// init — bind the operation to the virtual cluster
const startJobRun = yield* AWS.EMRContainers.StartJobRun(virtualCluster);


// runtime
const { id } = yield* startJobRun({
  jobTemplateId: templateId,
  jobTemplateParameters: { EntryPoint: "s3://my-bucket/etl.py" },
});
yield* Effect.log(`job run started: ${id}`);

VirtualCluster

Section titled “VirtualCluster”

Source: src/AWS/EMRContainers/VirtualCluster.ts

An Amazon EMR on EKS virtual cluster — a registration of a Kubernetes namespace on an EKS cluster as an EMR job-submission target. Virtual clusters consume no resources themselves (no cost while idle); jobs submitted to the virtual cluster run as pods in the mapped namespace.

The underlying EKS cluster must grant Amazon EMR on EKS access to the namespace (via EKS access entries when the cluster’s authentication mode includes API, or the legacy aws-auth ConfigMap). Everything except tags is immutable — changes replace the virtual cluster.

VirtualCluster: Creating Virtual Clusters

Section titled “VirtualCluster: Creating Virtual Clusters”
import * as AWS from "alchemy/AWS";


const cluster = yield* AWS.EKS.Cluster("Cluster", {
  roleArn: clusterRole.roleArn,
  resourcesVpcConfig: { subnetIds },
  accessConfig: { authenticationMode: "API_AND_CONFIG_MAP" },
});


const virtualCluster = yield* AWS.EMRContainers.VirtualCluster("Spark", {
  containerProvider: {
    id: cluster.clusterName,
    info: { eksInfo: { namespace: "emr" } },
  },
});
// virtualCluster.virtualClusterId is passed to StartJobRun