Managing a remote runtime engine for DataStage as a Service Anywhere

DataStage® as a Service Anywhere supports maintenance, updates, and other data considerations with remote runtime engines.

Maintenance

With DataStage-aaS Anywhere, the control plane remains hosted on and managed by IBM Cloud. You are responsible for managing your data plane through the remote engine. Automated scripts are available to update a remote engine, which you can do by downloading the container image to your internal registries and deploying with those internal registries. See https://github.com/IBM/DataStage/blob/main/RemoteEngine/docker/README.md for simple controls including creating, running, cleaning, and upgrading a remote engine.

Scaling

You can add or remove remote engines to scale deployments throughout the month. There is no deployment limit, but you are charged for the maximum amount of VPCs deployed each month whether or not they are used.

Disaster recovery

Deploy additional remote engines to support disaster recovery.

Data observability

You can put an observability solution in place within your container management platform. Databand is integrated with DataStage-aaS Anywhere and can monitor DataStage pipelines.

Storage

The DataStage operator mounts default storage to the remote engine's Kubernetes pods. To add additional storage with persistent volumes, see https://www.ibm.com/docs/en/cloud-paks/cp-data/4.8.x?topic=administering-setting-up-nfs-mount.

Setting proxy information

To set the proxy information to a remote engine for DataStage-SaaS Anywhere, set the following environment variable in the container:

REMOTE_HTTPS_PROXY=http://username:password@host:port