AWS.FinSpace reference

CreateKxChangeset

Source: src/AWS/FinSpace/CreateKxChangeset.ts

Runtime binding for finspace:CreateKxChangeset — ingests data into a kdb database in the bound environment by creating a changeset from staged S3 objects — the Managed kdb data-ingestion primitive. Provide the implementation with Effect.provide(AWS.FinSpace.CreateKxChangesetHttp).

CreateKxChangeset: Ingesting Data

const createChangeset = yield* AWS.FinSpace.CreateKxChangeset(kdb);

const changeset = yield* createChangeset({
  databaseName: "ticks",
  changeRequests: [
    { changeType: "PUT", s3Path: "s3://bucket/2024.01.02/", dbPath: "/2024.01.02/" },
  ],
  clientToken: crypto.randomUUID(),
});

CreateKxDataview

Source: src/AWS/FinSpace/CreateKxDataview.ts

Runtime binding for finspace:CreateKxDataview — creates a dataview — a snapshot of a kdb database at a changeset, materialized onto volumes — in the bound environment. Provide the implementation with Effect.provide(AWS.FinSpace.CreateKxDataviewHttp).

CreateKxDataview: Managing Dataviews

const createDataview = yield* AWS.FinSpace.CreateKxDataview(kdb);

yield* createDataview({
  databaseName: "ticks",
  dataviewName: "latest",
  azMode: "SINGLE",
  availabilityZoneId: "use1-az1",
  autoUpdate: true,
  clientToken: crypto.randomUUID(),
});

CreateKxScalingGroup

Source: src/AWS/FinSpace/CreateKxScalingGroup.ts

Runtime binding for finspace:CreateKxScalingGroup — provisions a shared compute host in the bound environment that multiple kdb clusters can be placed on. Provide the implementation with Effect.provide(AWS.FinSpace.CreateKxScalingGroupHttp).

CreateKxScalingGroup: Managing Scaling Groups

const createScalingGroup = yield* AWS.FinSpace.CreateKxScalingGroup(kdb);

const group = yield* createScalingGroup({
  scalingGroupName: "shared-hosts",
  hostType: "kx.sg.4xlarge",
  availabilityZoneId: "use1-az2",
  clientToken: crypto.randomUUID(),
});

CreateKxUser

Source: src/AWS/FinSpace/CreateKxUser.ts

Runtime binding for finspace:CreateKxUser — creates a kdb user in the bound environment, mapping an IAM role to a kdb identity. Provide the implementation with Effect.provide(AWS.FinSpace.CreateKxUserHttp).

CreateKxUser: Managing kdb Users

const createUser = yield* AWS.FinSpace.CreateKxUser(kdb);

const user = yield* createUser({
  userName: "analyst",
  iamRole: roleArn,
});

CreateKxVolume

Source: src/AWS/FinSpace/CreateKxVolume.ts

Runtime binding for finspace:CreateKxVolume — provisions a NAS_1 storage volume in the bound environment that clusters mount for tickerplant logs and shared savedown space. Provide the implementation with Effect.provide(AWS.FinSpace.CreateKxVolumeHttp).

CreateKxVolume: Managing Volumes

const createVolume = yield* AWS.FinSpace.CreateKxVolume(kdb);

const volume = yield* createVolume({
  volumeName: "tp-logs",
  volumeType: "NAS_1",
  nas1Configuration: { type: "SSD_250", size: 1200 },
  azMode: "SINGLE",
  availabilityZoneIds: ["use1-az2"],
  clientToken: crypto.randomUUID(),
});

DeleteKxClusterNode

Source: src/AWS/FinSpace/DeleteKxClusterNode.ts

Runtime binding for finspace:DeleteKxClusterNode — deletes one node of a kdb cluster in the bound environment — the scale-in primitive for node-level cluster automation. Provide the implementation with Effect.provide(AWS.FinSpace.DeleteKxClusterNodeHttp).

DeleteKxClusterNode: Monitoring Clusters

const deleteNode = yield* AWS.FinSpace.DeleteKxClusterNode(kdb);

yield* deleteNode({ clusterName: "hdb", nodeId });

DeleteKxDataview

Source: src/AWS/FinSpace/DeleteKxDataview.ts

Runtime binding for finspace:DeleteKxDataview — deletes a dataview of a kdb database in the bound environment. Provide the implementation with Effect.provide(AWS.FinSpace.DeleteKxDataviewHttp).

DeleteKxDataview: Managing Dataviews

const deleteDataview = yield* AWS.FinSpace.DeleteKxDataview(kdb);

yield* deleteDataview({
  databaseName: "ticks",
  dataviewName: "latest",
  clientToken: crypto.randomUUID(),
});

DeleteKxScalingGroup

Source: src/AWS/FinSpace/DeleteKxScalingGroup.ts

Runtime binding for finspace:DeleteKxScalingGroup — deletes a scaling group from the bound environment. The group must have no clusters placed on it. Provide the implementation with Effect.provide(AWS.FinSpace.DeleteKxScalingGroupHttp).

DeleteKxScalingGroup: Managing Scaling Groups

const deleteScalingGroup = yield* AWS.FinSpace.DeleteKxScalingGroup(kdb);

yield* deleteScalingGroup({
  scalingGroupName: "shared-hosts",
  clientToken: crypto.randomUUID(),
});

DeleteKxUser

Source: src/AWS/FinSpace/DeleteKxUser.ts

Runtime binding for finspace:DeleteKxUser — deletes a kdb user of the bound environment. Provide the implementation with Effect.provide(AWS.FinSpace.DeleteKxUserHttp).

DeleteKxUser: Managing kdb Users

const deleteUser = yield* AWS.FinSpace.DeleteKxUser(kdb);

yield* deleteUser({ userName: "analyst" });

DeleteKxVolume

Source: src/AWS/FinSpace/DeleteKxVolume.ts

Runtime binding for finspace:DeleteKxVolume — deletes a volume from the bound environment. The volume must not be attached to any cluster. Provide the implementation with Effect.provide(AWS.FinSpace.DeleteKxVolumeHttp).

DeleteKxVolume: Managing Volumes

const deleteVolume = yield* AWS.FinSpace.DeleteKxVolume(kdb);

yield* deleteVolume({
  volumeName: "tp-logs",
  clientToken: crypto.randomUUID(),
});

Environment

Source: src/AWS/FinSpace/Environment.ts

An Amazon FinSpace environment — a managed data management and analytics workspace for the financial services industry.

Caution

FinSpace is closed to new customers and environment provisioning takes ~20 minutes and bills while it exists. Live lifecycle tests are gated behind AWS_TEST_FINSPACE=1.

Environment: Creating Environments

Basic Environment

import * as AWS from "alchemy/AWS";

const env = yield* AWS.FinSpace.Environment("Analytics", {
  description: "research analytics environment",
});

Federated Environment

const env = yield* AWS.FinSpace.Environment("Analytics", {
  federationMode: "FEDERATED",
  federationParameters: {
    samlMetadataURL: "https://idp.example.com/metadata.xml",
    federationProviderName: "idp.example.com",
    applicationCallBackURL: "https://example.com/callback",
  },
});

GetKxChangeset

Source: src/AWS/FinSpace/GetKxChangeset.ts

Runtime binding for finspace:GetKxChangeset — reads one changeset of a kdb database in the bound environment — status, change requests, and error details — so ingestion pipelines can poll a changeset to COMPLETED. Provide the implementation with Effect.provide(AWS.FinSpace.GetKxChangesetHttp).

GetKxChangeset: Ingesting Data

const getChangeset = yield* AWS.FinSpace.GetKxChangeset(kdb);

const { status } = yield* getChangeset({
  databaseName: "ticks",
  changesetId,
});

GetKxCluster

Source: src/AWS/FinSpace/GetKxCluster.ts

Runtime binding for finspace:GetKxCluster — reads one kdb cluster of the bound environment — status, mounted databases, code configuration, and endpoints — for runtime monitoring and orchestration. Provide the implementation with Effect.provide(AWS.FinSpace.GetKxClusterHttp).

GetKxCluster: Monitoring Clusters

const getCluster = yield* AWS.FinSpace.GetKxCluster(kdb);

const { status } = yield* getCluster({ clusterName: "hdb" });

GetKxConnectionString

Source: src/AWS/FinSpace/GetKxConnectionString.ts

Runtime binding for finspace:GetKxConnectionString — retrieves a signed connection string a kdb user can use to connect to a cluster in the bound environment. The returned signedConnectionString embeds a SigV4 signature and is surfaced as Redacted. Provide the implementation with Effect.provide(AWS.FinSpace.GetKxConnectionStringHttp).

GetKxConnectionString: Connecting to Clusters

const getConnectionString = yield* AWS.FinSpace.GetKxConnectionString(kdb);

const { signedConnectionString } = yield* getConnectionString({
  userArn: user.userArn,
  clusterName: "hdb",
});
const value = Redacted.value(signedConnectionString!);

GetKxDatabase

Source: src/AWS/FinSpace/GetKxDatabase.ts

Runtime binding for finspace:GetKxDatabase — reads one kdb database of the bound environment — including lastCompletedChangesetId, the anchor ingestion pipelines use to advance dataviews. Provide the implementation with Effect.provide(AWS.FinSpace.GetKxDatabaseHttp).

GetKxDatabase: Reading Databases

const getDatabase = yield* AWS.FinSpace.GetKxDatabase(kdb);

const { lastCompletedChangesetId } = yield* getDatabase({
  databaseName: "ticks",
});

GetKxDataview

Source: src/AWS/FinSpace/GetKxDataview.ts

Runtime binding for finspace:GetKxDataview — reads one dataview of a kdb database in the bound environment — status, active versions, and segment configurations. Provide the implementation with Effect.provide(AWS.FinSpace.GetKxDataviewHttp).

GetKxDataview: Managing Dataviews

const getDataview = yield* AWS.FinSpace.GetKxDataview(kdb);

const { status } = yield* getDataview({
  databaseName: "ticks",
  dataviewName: "latest",
});

GetKxScalingGroup

Source: src/AWS/FinSpace/GetKxScalingGroup.ts

Runtime binding for finspace:GetKxScalingGroup — reads a scaling group’s status, host type, and the clusters placed on it in the bound environment. Provide the implementation with Effect.provide(AWS.FinSpace.GetKxScalingGroupHttp).

GetKxScalingGroup: Managing Scaling Groups

const getScalingGroup = yield* AWS.FinSpace.GetKxScalingGroup(kdb);

const group = yield* getScalingGroup({ scalingGroupName: "shared-hosts" });
if (group.status === "ACTIVE") {
  yield* Effect.log(`clusters: ${group.clusters?.join(", ")}`);
}

GetKxUser

Source: src/AWS/FinSpace/GetKxUser.ts

Runtime binding for finspace:GetKxUser — reads one kdb user of the bound environment — user ARN and mapped IAM role. Provide the implementation with Effect.provide(AWS.FinSpace.GetKxUserHttp).

GetKxUser: Managing kdb Users

const getUser = yield* AWS.FinSpace.GetKxUser(kdb);

const { userArn, iamRole } = yield* getUser({ userName: "analyst" });

GetKxVolume

Source: src/AWS/FinSpace/GetKxVolume.ts

Runtime binding for finspace:GetKxVolume — reads a volume’s status, NAS configuration, and the clusters attached to it in the bound environment. Provide the implementation with Effect.provide(AWS.FinSpace.GetKxVolumeHttp).

GetKxVolume: Managing Volumes

const getVolume = yield* AWS.FinSpace.GetKxVolume(kdb);

const volume = yield* getVolume({ volumeName: "tp-logs" });
if (volume.status === "ACTIVE") {
  yield* Effect.log(`attached: ${volume.attachedClusters?.length ?? 0}`);
}

KxCluster

Source: src/AWS/FinSpace/KxCluster.ts

A kdb cluster inside an Amazon FinSpace Managed kdb environment — the compute that mounts kdb databases and serves q queries.

Caution

Cluster provisioning is slow (tens of minutes) and bills per node-hour while it exists. Live lifecycle tests are gated behind AWS_TEST_FINSPACE=1.

KxCluster: Creating kdb Clusters

import * as AWS from "alchemy/AWS";

const cluster = yield* AWS.FinSpace.KxCluster("Hdb", {
  environmentId: env.environmentId,
  clusterType: "HDB",
  releaseLabel: "1.0",
  azMode: "SINGLE",
  availabilityZoneId: "use1-az1",
  capacityConfiguration: { nodeType: "kx.s.large", nodeCount: 1 },
  vpcConfiguration: {
    vpcId: vpc.vpcId,
    securityGroupIds: [sg.securityGroupId],
    subnetIds: [subnet.subnetId],
    ipAddressType: "IP_V4",
  },
  databases: [{ databaseName: db.databaseName }],
});

KxDatabase

Source: src/AWS/FinSpace/KxDatabase.ts

A kdb database inside an Amazon FinSpace Managed kdb environment — the versioned, changeset-based store that kdb clusters mount and query.

KxDatabase: Creating kdb Databases

import * as AWS from "alchemy/AWS";

const env = yield* AWS.FinSpace.KxEnvironment("Kdb", { kmsKeyId });
const db = yield* AWS.FinSpace.KxDatabase("Ticks", {
  environmentId: env.environmentId,
  description: "tick data",
});

KxEnvironment

Source: src/AWS/FinSpace/KxEnvironment.ts

An Amazon FinSpace Managed kdb environment — the account-level container that kdb databases, clusters, users, volumes and scaling groups live in.

Caution

kdb environment provisioning is slow (tens of minutes) and the service is gated to onboarded accounts. Live lifecycle tests are gated behind AWS_TEST_FINSPACE=1.

KxEnvironment: Creating kdb Environments

import * as AWS from "alchemy/AWS";

const key = yield* AWS.KMS.Key("KdbKey", {});
const env = yield* AWS.FinSpace.KxEnvironment("Kdb", {
  kmsKeyId: key.keyArn,
  description: "managed kdb environment",
});

KxEnvironment: Connecting to On-Prem Networks

const env = yield* AWS.FinSpace.KxEnvironment("Kdb", {
  kmsKeyId: key.keyArn,
  transitGatewayConfiguration: {
    transitGatewayID: "tgw-0123456789abcdef0",
    routableCIDRSpace: "10.0.0.0/16",
  },
  customDNSConfiguration: [
    { customDNSServerName: "dns.corp.example", customDNSServerIP: "10.0.0.2" },
  ],
});

ListKxChangesets

Source: src/AWS/FinSpace/ListKxChangesets.ts

Runtime binding for finspace:ListKxChangesets — lists the changesets of a kdb database in the bound environment, newest first. Provide the implementation with Effect.provide(AWS.FinSpace.ListKxChangesetsHttp).

ListKxChangesets: Ingesting Data

const listChangesets = yield* AWS.FinSpace.ListKxChangesets(kdb);

const { kxChangesets } = yield* listChangesets({ databaseName: "ticks" });

ListKxClusterNodes

Source: src/AWS/FinSpace/ListKxClusterNodes.ts

Runtime binding for finspace:ListKxClusterNodes — lists the nodes of a kdb cluster in the bound environment — node ids, availability zones, and launch times. Provide the implementation with Effect.provide(AWS.FinSpace.ListKxClusterNodesHttp).

ListKxClusterNodes: Monitoring Clusters

const listNodes = yield* AWS.FinSpace.ListKxClusterNodes(kdb);

const { nodes } = yield* listNodes({ clusterName: "hdb" });

ListKxClusters

Source: src/AWS/FinSpace/ListKxClusters.ts

Runtime binding for finspace:ListKxClusters — lists the kdb clusters of the bound environment, optionally filtered by cluster type. Provide the implementation with Effect.provide(AWS.FinSpace.ListKxClustersHttp).

ListKxClusters: Monitoring Clusters

const listClusters = yield* AWS.FinSpace.ListKxClusters(kdb);

const { kxClusterSummaries } = yield* listClusters({ clusterType: "HDB" });

ListKxDatabases

Source: src/AWS/FinSpace/ListKxDatabases.ts

Runtime binding for finspace:ListKxDatabases — lists the kdb databases of the bound environment. Provide the implementation with Effect.provide(AWS.FinSpace.ListKxDatabasesHttp).

ListKxDatabases: Reading Databases

const listDatabases = yield* AWS.FinSpace.ListKxDatabases(kdb);

const { kxDatabases } = yield* listDatabases();

ListKxDataviews

Source: src/AWS/FinSpace/ListKxDataviews.ts

Runtime binding for finspace:ListKxDataviews — lists the dataviews of a kdb database in the bound environment. Provide the implementation with Effect.provide(AWS.FinSpace.ListKxDataviewsHttp).

ListKxDataviews: Managing Dataviews

const listDataviews = yield* AWS.FinSpace.ListKxDataviews(kdb);

const { kxDataviews } = yield* listDataviews({ databaseName: "ticks" });

ListKxScalingGroups

Source: src/AWS/FinSpace/ListKxScalingGroups.ts

Runtime binding for finspace:ListKxScalingGroups — lists the scaling groups of the bound environment. Provide the implementation with Effect.provide(AWS.FinSpace.ListKxScalingGroupsHttp).

ListKxScalingGroups: Managing Scaling Groups

const listScalingGroups = yield* AWS.FinSpace.ListKxScalingGroups(kdb);

const { scalingGroups } = yield* listScalingGroups();

ListKxUsers

Source: src/AWS/FinSpace/ListKxUsers.ts

Runtime binding for finspace:ListKxUsers — lists the kdb users of the bound environment. Provide the implementation with Effect.provide(AWS.FinSpace.ListKxUsersHttp).

ListKxUsers: Managing kdb Users

const listUsers = yield* AWS.FinSpace.ListKxUsers(kdb);

const { users } = yield* listUsers();

ListKxVolumes

Source: src/AWS/FinSpace/ListKxVolumes.ts

Runtime binding for finspace:ListKxVolumes — lists the volumes of the bound environment, optionally filtered by volume type. Provide the implementation with Effect.provide(AWS.FinSpace.ListKxVolumesHttp).

ListKxVolumes: Managing Volumes

const listVolumes = yield* AWS.FinSpace.ListKxVolumes(kdb);

const { kxVolumeSummaries } = yield* listVolumes({ volumeType: "NAS_1" });

UpdateKxClusterCodeConfiguration

Source: src/AWS/FinSpace/UpdateKxClusterCodeConfiguration.ts

Runtime binding for finspace:UpdateKxClusterCodeConfiguration — deploys new q code (and optionally a new initialization script or command-line arguments) to a running cluster with a rolling, forced, or no-restart strategy. Provide the implementation with Effect.provide(AWS.FinSpace.UpdateKxClusterCodeConfigurationHttp).

UpdateKxClusterCodeConfiguration: Deploying to Clusters

const updateClusterCode = yield* AWS.FinSpace.UpdateKxClusterCodeConfiguration(kdb);

yield* updateClusterCode({
  clusterName: "rdb",
  code: { s3Bucket: "code-bucket", s3Key: "release/v2.zip" },
  deploymentConfiguration: { deploymentStrategy: "ROLLING" },
  clientToken: crypto.randomUUID(),
});

UpdateKxClusterDatabases

Source: src/AWS/FinSpace/UpdateKxClusterDatabases.ts

Runtime binding for finspace:UpdateKxClusterDatabases — remounts a running cluster’s databases at new changesets (with rolling or no-restart deployment) — how ingestion pipelines roll fresh data out to clusters that don’t read through dataviews. Provide the implementation with Effect.provide(AWS.FinSpace.UpdateKxClusterDatabasesHttp).

UpdateKxClusterDatabases: Deploying to Clusters

const updateClusterDatabases = yield* AWS.FinSpace.UpdateKxClusterDatabases(kdb);

yield* updateClusterDatabases({
  clusterName: "hdb",
  databases: [{ databaseName: "ticks", changesetId }],
  deploymentConfiguration: { deploymentStrategy: "NO_RESTART" },
  clientToken: crypto.randomUUID(),
});

UpdateKxDataview

Source: src/AWS/FinSpace/UpdateKxDataview.ts

Runtime binding for finspace:UpdateKxDataview — advances a dataview of a kdb database in the bound environment to a new changeset (or new segment configurations) — how ingestion pipelines roll fresh data out to readers. Provide the implementation with Effect.provide(AWS.FinSpace.UpdateKxDataviewHttp).

UpdateKxDataview: Managing Dataviews

const updateDataview = yield* AWS.FinSpace.UpdateKxDataview(kdb);

yield* updateDataview({
  databaseName: "ticks",
  dataviewName: "latest",
  changesetId,
  clientToken: crypto.randomUUID(),
});

UpdateKxUser

Source: src/AWS/FinSpace/UpdateKxUser.ts

Runtime binding for finspace:UpdateKxUser — re-points a kdb user of the bound environment at a different IAM role. Provide the implementation with Effect.provide(AWS.FinSpace.UpdateKxUserHttp).

UpdateKxUser: Managing kdb Users

const updateUser = yield* AWS.FinSpace.UpdateKxUser(kdb);

yield* updateUser({ userName: "analyst", iamRole: newRoleArn });

UpdateKxVolume

Source: src/AWS/FinSpace/UpdateKxVolume.ts

Runtime binding for finspace:UpdateKxVolume — changes a volume’s description or NAS_1 size/throughput in the bound environment. Provide the implementation with Effect.provide(AWS.FinSpace.UpdateKxVolumeHttp).

UpdateKxVolume: Managing Volumes

const updateVolume = yield* AWS.FinSpace.UpdateKxVolume(kdb);

yield* updateVolume({
  volumeName: "tp-logs",
  nas1Configuration: { type: "SSD_250", size: 2400 },
  clientToken: crypto.randomUUID(),
});