AWS.SageMaker reference

BatchGetRecord

Source: src/AWS/SageMaker/BatchGetRecord.ts

Runtime binding for sagemaker:BatchGetRecord — read a batch of records from a FeatureGroup’s online store in one call.

Bind this operation to a FeatureGroup inside a function runtime to get a callable that automatically scopes the batch identifiers to the bound feature group. Unknown identifiers are simply absent from Records (they are not errors).

BatchGetRecord: Reading Records

// init
const batchGetRecord = yield* AWS.SageMaker.BatchGetRecord(featureGroup);

// runtime
const { Records } = yield* batchGetRecord({
  RecordIdentifiersValueAsString: ["user-1", "user-2"],
});

BatchWriteRecord

Source: src/AWS/SageMaker/BatchWriteRecord.ts

Runtime binding for sagemaker:BatchWriteRecord — bulk-ingest records into a FeatureGroup’s online (and offline) store in one call.

Bind this operation to a FeatureGroup inside a function runtime to get a callable that automatically scopes every entry to the bound feature group. Per-record failures come back in the response’s Errors / UnprocessedEntries rather than failing the whole call.

BatchWriteRecord: Writing Records

// init
const batchWriteRecord = yield* AWS.SageMaker.BatchWriteRecord(featureGroup);

// runtime
const { Errors } = yield* batchWriteRecord({
  Entries: [
    {
      Record: [
        { FeatureName: "user_id", ValueAsString: "user-1" },
        { FeatureName: "event_time", ValueAsString: new Date().toISOString() },
        { FeatureName: "clicks", ValueAsString: "1" },
      ],
    },
  ],
});

Cluster

Source: src/AWS/SageMaker/Cluster.ts

An Amazon SageMaker HyperPod cluster — a resilient, persistent cluster of ML compute for distributed training and inference, orchestrated by Slurm or EKS, with automatic faulty-node recovery and deep health checks.

Provisioning a HyperPod cluster takes 10–25 minutes; instance groups are updated in place and removing a group from instanceGroups deletes it from the cluster.

Cluster: Creating Clusters

Slurm-Orchestrated Cluster

import * as AWS from "alchemy/AWS";

const cluster = yield* AWS.SageMaker.Cluster("TrainingCluster", {
  instanceGroups: {
    controller: {
      InstanceType: "ml.t3.medium",
      InstanceCount: 1,
      ExecutionRole: role.roleArn,
      LifeCycleConfig: {
        SourceS3Uri: `s3://${bucket.bucketName}/lifecycle`,
        OnCreate: "on_create.sh",
      },
    },
  },
});

EKS-Orchestrated Cluster

// The EKS cluster must use the `API` (or `API_AND_CONFIG_MAP`)
// authentication mode — pass `accessConfig` explicitly, EKS's own
// CONFIG_MAP default is rejected. LifeCycleConfig is required for
// EKS-orchestrated instance groups too.
const hyperpod = yield* AWS.SageMaker.Cluster("EksHyperPod", {
  orchestrator: { Eks: { ClusterArn: eksCluster.clusterArn } },
  vpcConfig: {
    SecurityGroupIds: [securityGroupId],
    Subnets: network.privateSubnetIds,
  },
  instanceGroups: {
    workers: {
      InstanceType: "ml.g5.xlarge",
      InstanceCount: 2,
      ExecutionRole: role.roleArn,
      LifeCycleConfig: {
        SourceS3Uri: `s3://${bucket.bucketName}/lifecycle`,
        OnCreate: "on_create.sh",
      },
    },
  },
  nodeRecovery: "Automatic",
});

// The keys carry through to the attributes — typed per key:
const workers = hyperpod.instanceGroups.workers;

Cluster: Running Workloads (Slurm)

# Slurm jobs are submitted on the cluster itself. Each node is an SSM
# target named sagemaker-cluster:<cluster-id>_<instance-group>-<instance-id>
# (list nodes with `aws sagemaker list-cluster-nodes`).
aws ssm start-session \
  --target sagemaker-cluster:6wl4at0i68c6_controller-i-0123456789abcdef0
# then, on the node:
sbatch --nodes=4 train.sbatch

Cluster: Running Workloads (EKS)

Low level: apply any Kubernetes manifest to the orchestrator

// HyperPod nodes are ordinary EKS nodes — target them from a raw
// manifest (a PyTorchJob CRD, a batch/v1 Job, ...) with the well-known
// node labels.
const job = yield* AWS.EKS.Manifest("RawTrainJob", {
  cluster: eksCluster,
  manifest: {
    apiVersion: "batch/v1",
    kind: "Job",
    metadata: { name: "raw-train", namespace: "default" },
    spec: {
      template: {
        spec: {
          nodeSelector: {
            "sagemaker.amazonaws.com/node-health-status": "Schedulable",
            "sagemaker.amazonaws.com/instance-group-name": "workers",
          },
          containers: [{ name: "train", image: "ghcr.io/acme/train:v3" }],
          restartPolicy: "Never",
        },
      },
    },
  },
});

High level: an effectful Job pinned to HyperPod nodes

// Kubernetes.Job / Kubernetes.Deployment run on HyperPod via the
// orchestrating EKS cluster in plain Kubernetes vocabulary — the
// HyperPod resources expose the derived values as attributes: the
// group's `nodeSelector`, the quota's governed `namespace` and Kueue
// `queueName`.
const evaluate = yield* Kubernetes.Job(
  "Evaluate",
  {
    cluster: eksCluster,
    main: import.meta.url,
    namespace: quota.namespace,
    labels: {
      [AWS.SageMaker.KUEUE_QUEUE_NAME_LABEL]: quota.queueName,
      [AWS.SageMaker.KUEUE_PRIORITY_CLASS_LABEL]: "training-priority",
    },
    podTemplate: {
      spec: {
        nodeSelector: hyperpod.instanceGroups.workers.nodeSelector,
      },
    },
  },
  Effect.gen(function* () {
    const putItem = yield* AWS.DynamoDB.PutItem(resultsTable);
    return {
      run: Effect.gen(function* () {
        // evaluation logic; bindings land IAM on the pod-identity role
      }),
    };
  }).pipe(Effect.provide(AWS.DynamoDB.PutItemHttp)),
);

Cluster: Task Governance

// Requires the amazon-sagemaker-hyperpod-taskgovernance EKS add-on.
const policy = yield* AWS.SageMaker.ClusterSchedulerConfig("Scheduler", {
  clusterArn: hyperpod.clusterArn,
  schedulerConfig: {
    PriorityClasses: [{ Name: "training", Weight: 90 }],
    FairShare: "Enabled",
  },
});

// Creates the hyperpod-ns-research namespace + Kueue LocalQueue —
// exposed as `quota.namespace` / `quota.queueName` for governed
// Kubernetes workloads to reference.
const quota = yield* AWS.SageMaker.ComputeQuota("ResearchQuota", {
  clusterArn: hyperpod.clusterArn,
  computeQuotaTarget: { TeamName: "research", FairShareWeight: 10 },
  computeQuotaConfig: {
    ComputeQuotaResources: [
      { InstanceType: "ml.g5.xlarge", Count: 1 },
    ],
  },
});

ClusterSchedulerConfig

Source: src/AWS/SageMaker/ClusterSchedulerConfig.ts

A SageMaker HyperPod cluster policy (task governance) — configures how an EKS-orchestrated HyperPod cluster prioritizes tasks and allocates idle compute across teams via priority classes and fair-share weights.

ClusterSchedulerConfig: Creating Cluster Policies

import * as AWS from "alchemy/AWS";

const policy = yield* AWS.SageMaker.ClusterSchedulerConfig("Scheduler", {
  clusterArn: hyperpod.clusterArn,
  schedulerConfig: {
    PriorityClasses: [
      { Name: "inference", Weight: 100 },
      { Name: "training", Weight: 75 },
    ],
    FairShare: "Enabled",
  },
  description: "Prioritize inference over training",
});

ComputeQuota

Source: src/AWS/SageMaker/ComputeQuota.ts

A SageMaker HyperPod compute allocation (task governance) — reserves instance capacity on an EKS-orchestrated HyperPod cluster for a team, with fair-share weights and borrow/lend rules for idle compute.

ComputeQuota: Creating Compute Allocations

import * as AWS from "alchemy/AWS";

const quota = yield* AWS.SageMaker.ComputeQuota("ResearchQuota", {
  clusterArn: hyperpod.clusterArn,
  computeQuotaTarget: { TeamName: "research", FairShareWeight: 10 },
  computeQuotaConfig: {
    ComputeQuotaResources: [{ InstanceType: "ml.g5.xlarge", Count: 2 }],
    ResourceSharingConfig: { Strategy: "Lend", BorrowLimit: 50 },
  },
});

DeleteRecord

Source: src/AWS/SageMaker/DeleteRecord.ts

Runtime binding for sagemaker:DeleteRecord — delete a record from a FeatureGroup’s online store.

Bind this operation to a FeatureGroup inside a function runtime to get a callable that automatically injects the feature group name. The default SoftDelete mode nulls the feature columns; HardDelete removes the record entirely. EventTime must be later than the stored record’s event time for the deletion to take effect.

DeleteRecord: Deleting Records

// init
const deleteRecord = yield* AWS.SageMaker.DeleteRecord(featureGroup);

// runtime
yield* deleteRecord({
  RecordIdentifierValueAsString: "user-123",
  EventTime: new Date().toISOString(),
});

DescribeEndpoint

Source: src/AWS/SageMaker/DescribeEndpoint.ts

Runtime binding for sagemaker:DescribeEndpoint — read a live endpoint’s status, variants, and deployment state from a function runtime.

Bind this operation to an Endpoint inside a function runtime to get a callable that automatically injects the endpoint name. Use it to check EndpointStatus (e.g. gate invocations while an update is rolling) or to observe per-variant weights and instance counts.

DescribeEndpoint: Observing Endpoints

// init
const describeEndpoint = yield* AWS.SageMaker.DescribeEndpoint(endpoint);

// runtime
const { EndpointStatus, ProductionVariants } = yield* describeEndpoint();

Endpoint

Source: src/AWS/SageMaker/Endpoint.ts

An Amazon SageMaker Endpoint — the live, invocable deployment of an EndpointConfig. Provisioning takes minutes and bills while the endpoint exists (serverless variants bill per request; instance variants bill per instance-hour). Destroy endpoints promptly.

Invoke a deployed endpoint from a function with AWS.SageMakerRuntime.InvokeEndpoint.

Endpoint: Creating Endpoints

import * as AWS from "alchemy/AWS";

const endpoint = yield* AWS.SageMaker.Endpoint("MyEndpoint", {
  endpointConfigName: config.endpointConfigName,
});

Endpoint: Invoking

// init
const invoke = yield* AWS.SageMakerRuntime.InvokeEndpoint(
  endpoint.endpointName,
);

// runtime
const result = yield* invoke({
  ContentType: "application/json",
  Body: JSON.stringify({ instances: [[1, 2, 3, 4]] }),
});

EndpointConfig

Source: src/AWS/SageMaker/EndpointConfig.ts

An Amazon SageMaker EndpointConfig — the deployment recipe that maps one or more Models to hosting resources (provisioned instances or serverless capacity). Pure configuration: it costs nothing until an Endpoint references it.

Endpoint configurations are immutable — any change other than tags replaces the configuration. To roll a live endpoint onto new settings, point the Endpoint at the replacement config (alchemy creates the new config first, updates the endpoint, then deletes the old config).

EndpointConfig: Creating Endpoint Configurations

Serverless Variant

import * as AWS from "alchemy/AWS";

const config = yield* AWS.SageMaker.EndpointConfig("MyConfig", {
  productionVariants: [{
    VariantName: "AllTraffic",
    ModelName: model.modelName,
    ServerlessConfig: { MemorySizeInMB: 2048, MaxConcurrency: 5 },
  }],
});

Provisioned Instances

const config = yield* AWS.SageMaker.EndpointConfig("MyConfig", {
  productionVariants: [{
    VariantName: "AllTraffic",
    ModelName: model.modelName,
    InstanceType: "ml.m5.large",
    InitialInstanceCount: 1,
  }],
});

FeatureGroup

Source: src/AWS/SageMaker/FeatureGroup.ts

An Amazon SageMaker Feature Store FeatureGroup — a typed, versioned table of ML features with an optional low-latency online store (for inference lookups) and an S3-backed offline store (for training).

With the online store enabled, functions read and write records at runtime via the AWS.SageMaker.GetRecord / AWS.SageMaker.PutRecord bindings.

FeatureGroup: Creating Feature Groups

import * as AWS from "alchemy/AWS";

const features = yield* AWS.SageMaker.FeatureGroup("UserFeatures", {
  recordIdentifierFeatureName: "user_id",
  eventTimeFeatureName: "event_time",
  featureDefinitions: [
    { FeatureName: "user_id", FeatureType: "String" },
    { FeatureName: "event_time", FeatureType: "String" },
    { FeatureName: "clicks", FeatureType: "Integral" },
  ],
  onlineStoreConfig: { EnableOnlineStore: true },
});

FeatureGroup: Runtime Access

// init
const putRecord = yield* AWS.SageMaker.PutRecord(features);
const getRecord = yield* AWS.SageMaker.GetRecord(features);

// runtime
yield* putRecord({
  Record: [
    { FeatureName: "user_id", ValueAsString: "user-123" },
    { FeatureName: "event_time", ValueAsString: new Date().toISOString() },
    { FeatureName: "clicks", ValueAsString: "42" },
  ],
});
const { Record } = yield* getRecord({
  RecordIdentifierValueAsString: "user-123",
});

GetRecord

Source: src/AWS/SageMaker/GetRecord.ts

Runtime binding for sagemaker:GetRecord — read the latest record for an identifier from a FeatureGroup’s online store.

Bind this operation to a FeatureGroup inside a function runtime to get a callable that automatically injects the feature group name. If no record exists for the identifier, the response’s Record is empty.

GetRecord: Reading Records

// init
const getRecord = yield* AWS.SageMaker.GetRecord(featureGroup);

// runtime
const { Record } = yield* getRecord({
  RecordIdentifierValueAsString: "user-123",
});

ListRecords

Source: src/AWS/SageMaker/ListRecords.ts

Runtime binding for sagemaker:ListRecords — list the record-identifier values stored in a FeatureGroup’s online store.

Bind this operation to a FeatureGroup inside a function runtime to get a callable that automatically injects the feature group name. Use it to discover which records exist without retrieving the full record data; paginate with NextToken.

ListRecords: Listing Records

// init
const listRecords = yield* AWS.SageMaker.ListRecords(featureGroup);

// runtime
const { RecordIdentifiers } = yield* listRecords({ MaxResults: 100 });

Model

Source: src/AWS/SageMaker/Model.ts

An Amazon SageMaker Model — the immutable pairing of an inference container image (and optional S3 model artifacts) with an execution role. A model is pure configuration: it costs nothing until it is deployed to an endpoint via an EndpointConfig + Endpoint.

SageMaker models are immutable — any change other than tags replaces the model.

Model: Creating Models

Model from an ECR image

import * as AWS from "alchemy/AWS";

const role = yield* AWS.IAM.Role("SageMakerRole", {
  assumeRolePolicyDocument: {
    Version: "2012-10-17",
    Statement: [{
      Effect: "Allow",
      Principal: { Service: "sagemaker.amazonaws.com" },
      Action: ["sts:AssumeRole"],
    }],
  },
  managedPolicyArns: ["arn:aws:iam::aws:policy/AmazonSageMakerFullAccess"],
});

const model = yield* AWS.SageMaker.Model("MyModel", {
  executionRoleArn: role.roleArn,
  primaryContainer: {
    Image: "123456789012.dkr.ecr.us-west-2.amazonaws.com/my-inference:latest",
    ModelDataUrl: "s3://my-bucket/model.tar.gz",
  },
});

Serverless deployment (Model → EndpointConfig → Endpoint)

const config = yield* AWS.SageMaker.EndpointConfig("MyConfig", {
  productionVariants: [{
    VariantName: "AllTraffic",
    ModelName: model.modelName,
    ServerlessConfig: { MemorySizeInMB: 2048, MaxConcurrency: 5 },
  }],
});
const endpoint = yield* AWS.SageMaker.Endpoint("MyEndpoint", {
  endpointConfigName: config.endpointConfigName,
});

PutRecord

Source: src/AWS/SageMaker/PutRecord.ts

Runtime binding for sagemaker:PutRecord — write a record to a FeatureGroup’s online store (and, when configured, its offline store).

Bind this operation to a FeatureGroup inside a function runtime to get a callable that automatically injects the feature group name. Every feature value is passed as a string (ValueAsString) — the feature group’s schema declares the actual types.

PutRecord: Writing Records

// init
const putRecord = yield* AWS.SageMaker.PutRecord(featureGroup);

// runtime
yield* putRecord({
  Record: [
    { FeatureName: "user_id", ValueAsString: "user-123" },
    { FeatureName: "event_time", ValueAsString: new Date().toISOString() },
    { FeatureName: "clicks", ValueAsString: "42" },
  ],
});

UpdateEndpointWeightsAndCapacities

Source: src/AWS/SageMaker/UpdateEndpointWeightsAndCapacities.ts

Runtime binding for sagemaker:UpdateEndpointWeightsAndCapacities — shift traffic between an endpoint’s production variants (or resize one variant) without redeploying.

Bind this operation to an Endpoint inside a function runtime to get a callable that automatically injects the endpoint name. Only applies to instance-based variants (serverless variants have no weights/capacities); the endpoint transitions through Updating back to InService.

UpdateEndpointWeightsAndCapacities: Shifting Traffic

// init
const updateWeights =
  yield* AWS.SageMaker.UpdateEndpointWeightsAndCapacities(endpoint);

// runtime
yield* updateWeights({
  DesiredWeightsAndCapacities: [
    { VariantName: "Blue", DesiredWeight: 9 },
    { VariantName: "Green", DesiredWeight: 1 },
  ],
});