Managing a deployment for a custom foundation model

Last updated: Nov 28, 2024

After you deploy a custom foundation model, you can manage and update the deployment.

You can manage and update a deployed foundation model from a deployment space UI or programmatically.

Managing custom foundation model deployments from deployment space

You can access, update, scale, delete, and monitor the performance of your custom model deployment in your deployment space.

Accessing deployment details from deployment space

Follow these steps to review or update deployment details:

From the Deployments tab of your deployment space, click a deployment name.
Click the Deployment details tab to access information that is related to your custom foundation model deployment.

Note: If your organization is using any of the use cases to track and govern assets, deployment information for a tracked asset is recorded in a factsheet in the associated use case.

Updating deployment details from deployment space

You can update the details for your custom foundation model deployment, such as name, serving name, description, and hardware specifications. For more information, see Updating a deployment.

Scaling a deployment in a deployment space

You can scale your deployment by increasing the number of copies that are created for your deployment. For more information, see Scaling a deployment.

Deleting a deployment from a deployment space

You can delete your custom foundation model deployment when you don't need it anymore, to free up resources. For more information, see Deleting a deployment.

Note:

In workflows where your custom foundation model is used periodically, consider assigning your model the same serving name each time you deploy it. This way, after you delete and then re-deploy the model, you can keep using the same endpoint in your code.

Monitoring deployment performance from a deployment space

You can evaluate your custom foundation model deployment to measure performance and understand model predictions by provisioning a watsonx.governance instance and configuring monitors for fairness, quality, drift, and explainability. For more information, see Evaluating deployments in spaces with watsonx.governance.

Managing a custom foundation model deployment programmatically

Prerequisites

You can access, update, scale, delete, and monitor the performance of your custom model deployment programmatically.

To use the watsonx.ai API, you need a bearer token. For more information, see Credentials for programmatic access.

Updating a deployment programmatically

To update or delete a deployment programmatically, first get the list of deployed models to find the correct metadata for the deployment.

Getting the list of deployed models

Get the list of deployments for the specified project ID. To filter for all deployments that point to custom foundation models, use the type=custom_foundation_model query parameter. Refer to this example code:

curl -X GET "https://<your cloud hostname>/ml/v4/deployments?version=2024-01-29&project_id=<your project ID>&type=custom_foundation_model" \
-H "Authorization: Bearer $TOKEN"

Fields that you can update in a deployed model:

name
description
tags
serving_name
asset
hardware_spec
software_spec
/online/parameters
/online/parameters/serving_name
/online/parameters/foundation_model

For example:

curl -X PATCH "https://<your cloud hostname>//ml/v4/deployments/<your deployment ID>?version=2024-01-29&project_id=<your project ID>" \
-H "Authorization: Bearer $TOKEN" \
-H "content-type: application/json" \
--data '[{
 "op": "replace",
 "path": "/name",
 "value": "<updated deployment name>"
}]'

Deleting your deployment programmatically

If you want to delete the deployment of your custom foundation model, use this code:

curl -vk -X DELETE "https://<your cloud hostname>/ml/v4/deployments/<your deployment ID>?version=2024-01-29&project_id=<your project ID>" -H "Authorization: Bearer $TOKEN"

Monitoring deployment performance from a deployment space

Scaling your deployment programmatically

See Scaling a deployment programmatically.

Next steps

Learn how to prompt a custom foundation model

Parent topic: Deploying custom foundation models