AWS.EMR reference

AddInstanceFleet

Source: src/AWS/EMR/AddInstanceFleet.ts

Runtime binding for elasticmapreduce:AddInstanceFleet — adds a task instance fleet to the bound cluster (instance-fleet clusters only; TASK fleets can be added after launch).

AddInstanceFleet: Scaling the Cluster

const addFleet = yield* AWS.EMR.AddInstanceFleet(cluster);

const { InstanceFleetId } = yield* addFleet({
  InstanceFleet: {
    InstanceFleetType: "TASK",
    TargetSpotCapacity: 2,
    InstanceTypeConfigs: [{ InstanceType: "m5.xlarge" }],
  },
});

AddInstanceGroups

Source: src/AWS/EMR/AddInstanceGroups.ts

Runtime binding for elasticmapreduce:AddInstanceGroups — adds task instance groups to the bound cluster (instance-group clusters only). The cluster id is injected as JobFlowId.

AddInstanceGroups: Scaling the Cluster

const addGroups = yield* AWS.EMR.AddInstanceGroups(cluster);

const { InstanceGroupIds } = yield* addGroups({
  InstanceGroups: [{
    InstanceRole: "TASK",
    InstanceType: "m5.xlarge",
    InstanceCount: 2,
  }],
});

AddJobFlowSteps

Source: src/AWS/EMR/AddJobFlowSteps.ts

Runtime binding for elasticmapreduce:AddJobFlowSteps — submits work (Spark jobs, Hive queries, custom JARs) to the bound cluster as steps. The cluster id is injected as JobFlowId.

AddJobFlowSteps: Running Steps

const addSteps = yield* AWS.EMR.AddJobFlowSteps(cluster);

const { StepIds } = yield* addSteps({
  Steps: [{
    Name: "spark-pi",
    ActionOnFailure: "CONTINUE",
    HadoopJarStep: {
      Jar: "command-runner.jar",
      Args: ["spark-example", "SparkPi", "10"],
    },
  }],
});

CancelSteps

Source: src/AWS/EMR/CancelSteps.ts

Runtime binding for elasticmapreduce:CancelSteps — cancels pending or running steps on the bound cluster. Cancellation is asynchronous — poll DescribeStep for the final state.

CancelSteps: Running Steps

const cancelSteps = yield* AWS.EMR.CancelSteps(cluster);

const { CancelStepsInfoList } = yield* cancelSteps({
  StepIds: [stepId],
  StepCancellationOption: "SEND_INTERRUPT",
});

Cluster

Source: src/AWS/EMR/Cluster.ts

A provisioned Amazon EMR cluster (job flow) running open-source big-data frameworks such as Apache Spark and Hadoop on EC2 instances.

Clusters take roughly 10-15 minutes to reach WAITING and bill per instance-hour while they exist. Each cluster needs an EMR service role and an EC2 instance profile (job-flow role); destroy clusters you are not using, or set autoTerminationPolicy as a safety net.

Cluster: Creating a Cluster

Spark Cluster in a Default-VPC Subnet

const cluster = yield* Cluster("Analytics", {
  releaseLabel: "emr-7.5.0",
  applications: ["Spark", "Hadoop"],
  serviceRole: serviceRole.roleName,
  jobFlowRole: instanceProfile.instanceProfileName,
  logUri: Output.interpolate`s3://${logsBucket.bucketName}/logs/`,
  instances: {
    masterInstanceType: "m5.xlarge",
    coreInstanceType: "m5.xlarge",
    coreInstanceCount: 1,
    ec2SubnetId: subnetId,
  },
});

Cluster with an Auto-Termination Safety Net

const cluster = yield* Cluster("Batch", {
  releaseLabel: "emr-7.5.0",
  applications: ["Spark"],
  serviceRole: serviceRole.roleName,
  jobFlowRole: instanceProfile.instanceProfileName,
  autoTerminationPolicy: { idleTimeout: "1 hour" },
  stepConcurrencyLevel: 4,
});

Cluster: Applying a Security Configuration

const config = yield* SecurityConfiguration("Encryption", {
  securityConfiguration: {
    EncryptionConfiguration: {
      EnableInTransitEncryption: false,
      EnableAtRestEncryption: false,
    },
  },
});
const cluster = yield* Cluster("Secure", {
  releaseLabel: "emr-7.5.0",
  serviceRole: serviceRole.roleName,
  jobFlowRole: instanceProfile.instanceProfileName,
  securityConfiguration: config.securityConfigurationName,
});

CreatePersistentAppUI

Source: src/AWS/EMR/CreatePersistentAppUI.ts

Runtime binding for elasticmapreduce:CreatePersistentAppUI — creates a persistent application UI (Spark history, YARN timeline) for the bound cluster — the UI outlives the cluster. The cluster ARN is injected as TargetResourceArn.

CreatePersistentAppUI: Application UIs

const createAppUI = yield* AWS.EMR.CreatePersistentAppUI(cluster);

const { PersistentAppUIId } = yield* createAppUI();

DescribePersistentAppUI

Source: src/AWS/EMR/DescribePersistentAppUI.ts

Runtime binding for elasticmapreduce:DescribePersistentAppUI — reads a persistent application UI created for the bound cluster by CreatePersistentAppUI.

DescribePersistentAppUI: Application UIs

const describeAppUI = yield* AWS.EMR.DescribePersistentAppUI(cluster);

const { PersistentAppUI } = yield* describeAppUI({
  PersistentAppUIId: appUIId,
});

DescribeReleaseLabel

Source: src/AWS/EMR/DescribeReleaseLabel.ts

Runtime binding for elasticmapreduce:DescribeReleaseLabel — reads one EMR release label — the applications (with versions) it ships and the OS releases it supports.

DescribeReleaseLabel: Release Catalog

const describeReleaseLabel = yield* AWS.EMR.DescribeReleaseLabel();

const { Applications } = yield* describeReleaseLabel({
  ReleaseLabel: "emr-7.5.0",
});

DescribeStep

Source: src/AWS/EMR/DescribeStep.ts

Runtime binding for elasticmapreduce:DescribeStep — reads one step of the bound cluster — status, config, and failure details.

DescribeStep: Running Steps

const describeStep = yield* AWS.EMR.DescribeStep(cluster);

const { Step } = yield* describeStep({ StepId: stepId });
// Step.Status.State: PENDING | RUNNING | COMPLETED | FAILED | …

GetClusterSessionCredentials

Source: src/AWS/EMR/GetClusterSessionCredentials.ts

Runtime binding for elasticmapreduce:GetClusterSessionCredentials — mints temporary HTTP basic credentials for the bound cluster’s endpoints (runtime-role / fine-grained access control clusters). The returned Password is Redacted — unwrap with Redacted.value.

GetClusterSessionCredentials: Connecting to the Cluster

const getCredentials = yield* AWS.EMR.GetClusterSessionCredentials(cluster);

const { Credentials, ExpiresAt } = yield* getCredentials({
  ExecutionRoleArn: runtimeRoleArn,
});
const password = Redacted.value(
  Credentials!.UsernamePassword.Password! as Redacted.Redacted<string>,
);

GetManagedScalingPolicy

Source: src/AWS/EMR/GetManagedScalingPolicy.ts

Runtime binding for elasticmapreduce:GetManagedScalingPolicy — reads the bound cluster’s managed scaling policy (compute limits), if one is attached.

GetManagedScalingPolicy: Scaling the Cluster

const getScalingPolicy = yield* AWS.EMR.GetManagedScalingPolicy(cluster);

const { ManagedScalingPolicy } = yield* getScalingPolicy();

GetOnClusterAppUIPresignedURL

Source: src/AWS/EMR/GetOnClusterAppUIPresignedURL.ts

Runtime binding for elasticmapreduce:GetOnClusterAppUIPresignedURL — mints a presigned URL for a live application UI (Spark UI, YARN ResourceManager, Tez) on the bound cluster.

GetOnClusterAppUIPresignedURL: Application UIs

const getAppUrl = yield* AWS.EMR.GetOnClusterAppUIPresignedURL(cluster);

const { PresignedURL } = yield* getAppUrl({
  OnClusterAppUIType: "ApplicationMaster",
});

GetPersistentAppUIPresignedURL

Source: src/AWS/EMR/GetPersistentAppUIPresignedURL.ts

Runtime binding for elasticmapreduce:GetPersistentAppUIPresignedURL — mints a presigned URL for a persistent application UI of the bound cluster (works after the cluster terminates).

GetPersistentAppUIPresignedURL: Application UIs

const getAppUIUrl = yield* AWS.EMR.GetPersistentAppUIPresignedURL(cluster);

const { PresignedURL } = yield* getAppUIUrl({
  PersistentAppUIId: appUIId,
  PersistentAppUIType: "SHS",
});

GetSession

Source: src/AWS/EMR/GetSession.ts

Runtime binding for elasticmapreduce:GetSession — reads an interactive session of the bound cluster — state, engine configuration, and timeline.

GetSession: Interactive Sessions

const getSession = yield* AWS.EMR.GetSession(cluster);

const { Session } = yield* getSession({ SessionId: sessionId });
// Session.State: SUBMITTED | STARTING | IDLE | BUSY | …

GetSessionEndpoint

Source: src/AWS/EMR/GetSessionEndpoint.ts

Runtime binding for elasticmapreduce:GetSessionEndpoint — resolves the Spark Connect endpoint URL and a time-limited auth token for an interactive session on the bound cluster. The returned AuthToken is Redacted — unwrap with Redacted.value.

GetSessionEndpoint: Interactive Sessions

const getEndpoint = yield* AWS.EMR.GetSessionEndpoint(cluster);

const { Endpoint, AuthToken } = yield* getEndpoint({
  SessionId: sessionId,
});

ListBootstrapActions

Source: src/AWS/EMR/ListBootstrapActions.ts

Runtime binding for elasticmapreduce:ListBootstrapActions — lists the bootstrap actions the bound cluster ran at launch.

ListBootstrapActions: Inspecting the Cluster

const listBootstrapActions = yield* AWS.EMR.ListBootstrapActions(cluster);

const { BootstrapActions } = yield* listBootstrapActions();

ListClusters

Source: src/AWS/EMR/ListClusters.ts

Runtime binding for elasticmapreduce:ListClusters — lists the account’s EMR clusters (optionally filtered by state or creation window) — the building block of cluster-inventory automation.

ListClusters: Discovering Clusters

const listClusters = yield* AWS.EMR.ListClusters();

const { Clusters } = yield* listClusters({
  ClusterStates: ["RUNNING", "WAITING"],
});

ListInstanceFleets

Source: src/AWS/EMR/ListInstanceFleets.ts

Runtime binding for elasticmapreduce:ListInstanceFleets — lists the bound cluster’s instance fleets (instance-fleet clusters only) with target and provisioned capacities.

ListInstanceFleets: Inspecting the Cluster

const listInstanceFleets = yield* AWS.EMR.ListInstanceFleets(cluster);

const { InstanceFleets } = yield* listInstanceFleets();

ListInstanceGroups

Source: src/AWS/EMR/ListInstanceGroups.ts

Runtime binding for elasticmapreduce:ListInstanceGroups — lists the bound cluster’s instance groups (instance-group clusters only) with requested/running counts — the ids feed ModifyInstanceGroups and PutAutoScalingPolicy.

ListInstanceGroups: Inspecting the Cluster

const listInstanceGroups = yield* AWS.EMR.ListInstanceGroups(cluster);

const { InstanceGroups } = yield* listInstanceGroups();
const core = InstanceGroups?.find(
  (group) => group.InstanceGroupType === "CORE",
);

ListInstances

Source: src/AWS/EMR/ListInstances.ts

Runtime binding for elasticmapreduce:ListInstances — lists the bound cluster’s EC2 instances with state, private/public addresses, and group/fleet membership.

ListInstances: Inspecting the Cluster

const listInstances = yield* AWS.EMR.ListInstances(cluster);

const { Instances } = yield* listInstances({
  InstanceGroupTypes: ["CORE"],
  InstanceStates: ["RUNNING"],
});

ListReleaseLabels

Source: src/AWS/EMR/ListReleaseLabels.ts

Runtime binding for elasticmapreduce:ListReleaseLabels — lists the EMR release labels available in the region, newest first.

ListReleaseLabels: Release Catalog

const listReleaseLabels = yield* AWS.EMR.ListReleaseLabels();

const { ReleaseLabels } = yield* listReleaseLabels();
const latest = ReleaseLabels?.[0];

ListSessions

Source: src/AWS/EMR/ListSessions.ts

Runtime binding for elasticmapreduce:ListSessions — lists the bound cluster’s interactive sessions, optionally filtered by state.

ListSessions: Interactive Sessions

const listSessions = yield* AWS.EMR.ListSessions(cluster);

const { Sessions } = yield* listSessions({ SessionStates: ["IDLE"] });

ListSteps

Source: src/AWS/EMR/ListSteps.ts

Runtime binding for elasticmapreduce:ListSteps — lists the bound cluster’s steps, newest first (optionally filtered by state or id). Page with Marker.

ListSteps: Running Steps

const listSteps = yield* AWS.EMR.ListSteps(cluster);

const { Steps } = yield* listSteps({ StepStates: ["RUNNING"] });

ListSupportedInstanceTypes

Source: src/AWS/EMR/ListSupportedInstanceTypes.ts

Runtime binding for elasticmapreduce:ListSupportedInstanceTypes — lists the EC2 instance types a given EMR release supports in the region.

ListSupportedInstanceTypes: Release Catalog

const listInstanceTypes = yield* AWS.EMR.ListSupportedInstanceTypes();

const { SupportedInstanceTypes } = yield* listInstanceTypes({
  ReleaseLabel: "emr-7.5.0",
});

ModifyInstanceFleet

Source: src/AWS/EMR/ModifyInstanceFleet.ts

Runtime binding for elasticmapreduce:ModifyInstanceFleet — retargets the bound cluster’s instance fleet (on-demand/spot capacities, resize specifications).

ModifyInstanceFleet: Scaling the Cluster

const modifyFleet = yield* AWS.EMR.ModifyInstanceFleet(cluster);

yield* modifyFleet({
  InstanceFleet: {
    InstanceFleetId: fleetId,
    TargetOnDemandCapacity: 4,
  },
});

ModifyInstanceGroups

Source: src/AWS/EMR/ModifyInstanceGroups.ts

Runtime binding for elasticmapreduce:ModifyInstanceGroups — resizes or reconfigures the bound cluster’s instance groups (target counts, EC2 configurations, shrink policies).

ModifyInstanceGroups: Scaling the Cluster

const modifyGroups = yield* AWS.EMR.ModifyInstanceGroups(cluster);

yield* modifyGroups({
  InstanceGroups: [{ InstanceGroupId: coreGroupId, InstanceCount: 3 }],
});

PutAutoScalingPolicy

Source: src/AWS/EMR/PutAutoScalingPolicy.ts

Runtime binding for elasticmapreduce:PutAutoScalingPolicy — attaches a CloudWatch-driven automatic scaling policy to an instance group of the bound cluster.

PutAutoScalingPolicy: Scaling the Cluster

const putAutoScaling = yield* AWS.EMR.PutAutoScalingPolicy(cluster);

yield* putAutoScaling({
  InstanceGroupId: taskGroupId,
  AutoScalingPolicy: {
    Constraints: { MinCapacity: 0, MaxCapacity: 8 },
    Rules: [], // CloudWatch alarm-driven rules
  },
});

PutManagedScalingPolicy

Source: src/AWS/EMR/PutManagedScalingPolicy.ts

Runtime binding for elasticmapreduce:PutManagedScalingPolicy — attaches or replaces the bound cluster’s managed scaling policy — EMR then resizes the cluster within the configured compute limits.

PutManagedScalingPolicy: Scaling the Cluster

const putScalingPolicy = yield* AWS.EMR.PutManagedScalingPolicy(cluster);

yield* putScalingPolicy({
  ManagedScalingPolicy: {
    ComputeLimits: {
      UnitType: "Instances",
      MinimumCapacityUnits: 1,
      MaximumCapacityUnits: 10,
    },
  },
});

RemoveAutoScalingPolicy

Source: src/AWS/EMR/RemoveAutoScalingPolicy.ts

Runtime binding for elasticmapreduce:RemoveAutoScalingPolicy — detaches the automatic scaling policy from an instance group of the bound cluster.

RemoveAutoScalingPolicy: Scaling the Cluster

const removeAutoScaling = yield* AWS.EMR.RemoveAutoScalingPolicy(cluster);

yield* removeAutoScaling({ InstanceGroupId: taskGroupId });

RemoveManagedScalingPolicy

Source: src/AWS/EMR/RemoveManagedScalingPolicy.ts

Runtime binding for elasticmapreduce:RemoveManagedScalingPolicy — detaches the bound cluster’s managed scaling policy.

RemoveManagedScalingPolicy: Scaling the Cluster

const removeScalingPolicy =
  yield* AWS.EMR.RemoveManagedScalingPolicy(cluster);

yield* removeScalingPolicy();

SecurityConfiguration

Source: src/AWS/EMR/SecurityConfiguration.ts

An Amazon EMR security configuration — a reusable JSON document of encryption, authentication, and instance-metadata settings referenced by name when launching a Cluster.

Clusters capture the configuration at launch, so editing a configuration only affects clusters launched afterwards.

SecurityConfiguration: Creating a Security Configuration

Require IMDSv2 on Cluster Instances

const config = yield* SecurityConfiguration("Imds", {
  securityConfiguration: {
    InstanceMetadataServiceConfiguration: {
      MinimumInstanceMetadataServiceVersion: 2,
      HttpPutResponseHopLimit: 1,
    },
  },
});

Encryption Settings

const config = yield* SecurityConfiguration("Encryption", {
  securityConfiguration: {
    EncryptionConfiguration: {
      EnableInTransitEncryption: false,
      EnableAtRestEncryption: true,
      AtRestEncryptionConfiguration: {
        S3EncryptionConfiguration: { EncryptionMode: "SSE-S3" },
      },
    },
  },
});

SecurityConfiguration: Using with a Cluster

const cluster = yield* Cluster("Secure", {
  releaseLabel: "emr-7.5.0",
  serviceRole: serviceRole.roleName,
  jobFlowRole: instanceProfile.instanceProfileName,
  securityConfiguration: config.securityConfigurationName,
});

StartSession

Source: src/AWS/EMR/StartSession.ts

Runtime binding for elasticmapreduce:StartSession — starts an interactive Spark Connect session on the bound cluster (EMR 7.8+ with sessions enabled).

StartSession: Interactive Sessions

const startSession = yield* AWS.EMR.StartSession(cluster);

const { Id } = yield* startSession({
  Name: "adhoc-analysis",
  ExecutionRoleArn: runtimeRoleArn,
});

Studio

Source: src/AWS/EMR/Studio.ts

An Amazon EMR Studio — a web-based IDE for notebooks and interactive workloads that attaches to EMR clusters.

A Studio itself is free; you pay for the clusters it attaches to. Each Studio needs a VPC with subnets, a workspace and an engine security group, an IAM service role, and an S3 backup location.

Studio: Creating a Studio

IAM-Authenticated Studio

const studio = yield* Studio("Notebooks", {
  authMode: "IAM",
  vpcId: vpc.vpcId,
  subnetIds: [subnetA.subnetId, subnetB.subnetId],
  serviceRole: serviceRole.roleArn,
  workspaceSecurityGroupId: workspaceSg.groupId,
  engineSecurityGroupId: engineSg.groupId,
  defaultS3Location: Output.interpolate`s3://${bucket.bucketName}/studio/`,
});

Studio with Description and Tags

const studio = yield* Studio("Notebooks", {
  authMode: "IAM",
  vpcId: vpc.vpcId,
  subnetIds: [subnetA.subnetId],
  serviceRole: serviceRole.roleArn,
  workspaceSecurityGroupId: workspaceSg.groupId,
  engineSecurityGroupId: engineSg.groupId,
  defaultS3Location: Output.interpolate`s3://${bucket.bucketName}/studio/`,
  description: "Data-science notebooks",
  tags: { team: "analytics" },
});

TerminateSession

Source: src/AWS/EMR/TerminateSession.ts

Runtime binding for elasticmapreduce:TerminateSession — terminates an interactive session on the bound cluster.

TerminateSession: Interactive Sessions

const terminateSession = yield* AWS.EMR.TerminateSession(cluster);

yield* terminateSession({ SessionId: sessionId });