Deploying AI assets
Using IBM Watson Machine Learning, you can deploy machine learning models, scripts, and functions, and prompt templates for generative AI models. After you create deployments, you can test and manage them, and prepare your assets to deploy into pre-production and production environments to generate predictions and insights.
Service The administrator must provision the Watson Machine Learning service on Cloud Pak for Data as a Service platform to use its capabilities.
Deployment process
The typical process for deploying an asset is as follows:
- Choose deployment type: Choose a deployment type for the asset type that you want to deploy.
- Create deployment: Depending on the asset type, you can create an online, batch, application, or detached deployment.
- Test deployment: You can test your deployments by entering test data, providing JSON payload, or creating a batch job.
- Retrieve deployment endpoint: To use your deployment in an application, you must retrieve the endpoint of your deployment. The model endpoint provides access to an interface to invoke and manage model deployments.
The following graphic shows the process for deploying AI assets:
Types of deployments
The most common types of deployments are as follows:
-
Online deployment: Create an online deployment to process input data in real-time. To test the online deployment in real-time, you can submit new customer data to the deployment endpoint to get a prediction in real-time.
-
Batch deployment: Create a batch deployment to process a large batch of input data from a data source and write the output to a selected destination. You can configure the batch deployment job and run the job on a schedule or on demand.
Types of deployable assets
The type of asset that you deploy dictates the type of deployment that you can create. For example, Python functions, scripts, and models, such as AutoAI or Decision Optimization models support online and batch deployments. However, you can create online deployments only for models that are imported from a file. The different types of deployable assets are as follows:
-
Foundation model assets: You can deploy foundation model assets such as tuned model, prompt template assets, or custom foundation models with watsonx.ai.
-
Machine Learning assets: You can deploy machine learning Machine Learning assets such as Python functions, R Shiny applications, NLP models, scripts, and more with Watson Machine Learning.
-
Decision Optimization models: You can deploy Decision Optimization model with Watson Machine Learning.
Ways to deploy assets
You can deploy and manage your assets in the following ways:
-
Deploy and manage assets in a deployment space by using a no-code approach.
-
Deploy and manage assets programmatically by using a custom-code approach.
Deploying and managing assets in deployment spaces
Create a deployment space to collaborate with stakeholders and deploy and manage assets in a deployment space.
To manage your assets within a deployment space, you must promote your assets from a project to your deployment space. You can also import or export assets from your deployment space.
Use the deployments dashboard to get an aggregate view of your deployments and monitor deployment activity.
Deploying and managing assets programmatically
You can deploy and manage assets programmatically by using the watsonx.ai Python client library or Watson Machine Learning API. For additional Cloud Pak for Data as a Service APIs, see Cloud Pak for Data APIs.
Managing frameworks and software specifications for deployments
Software specifications and frameworks contain bundles of packages with corresponding versions of the packages.
You can use predefined software specifications or create custom software specifications by adding new packages to existing frameworks, create new packages, or updating package versions in software specifications.
Managing runtime environments for deployments
Runtime envionments provide the necessary functions that are required to run your deployment.
You can use predefined runtime environments or create custom runtime environments to include more components, depending on your use case. To create a custom runtime environment for your deployment, you must
create a Dockerfile and add a base image. Further, you can add the docker commands to build the runtime environment for your deployment.
Learn more
Parent topic: Deploying and managing AI assets