Tracking models with AI Factsheets

Creating a model use case

Create a model use case to track a model's lifecycle.

1. From the navigation pane, click the Catalog section and then click Model inventory.
2. Click New model use case and enter a name and description.
3. Choose the catalog containing the model use case.
4. After you create the use case, you can specify extra details such as the Model purpose, Business terms, or Supporting documentation.

Changing the status of a model use case

You can change the status of a model use case to reflect a change in the model's lifecycle. For example, you can change the status of a model from Deployed for validation to Validated after you confirm that the model was adequately tested.

To change the status of a model use case:

1. Open the model use case in the model inventory.
2. Switch to the Asset tab.
3. Click the edit icon .
4. Select the new status and click Update to save.

The model use case is displayed in the new status category.

Deleting a model use case

To delete a model use case from the inventory, click Remove from the action item on the model use case tile. The model use case is removed from the inventory and all association with the model in a project or space is deleted.

Searching or filtering to find models

• Use the search bar to search for a model use case by name, tag, classification, or business term.
• Use the filters in the model inventory view to refine the list. You can filter by alert (such as a change in drift, fairness or quality), business terms, catalog, classification, tags, and status.

Viewing alerts

When you evaluate a model, you can set thresholds to trigger an alert when a threshold is met. For example, you can set an alert that is triggered if bias detection for a monitored group changes by 10%. You can view the alerts for a model from the model use case.

If any of the model deployments has alerts, alerts are displayed in the model inventory as a badge on the model use case. Choose Preview alerts from the action menu item on a model use case in the model inventory page to see the details of the alerts.

Viewing the factsheet for a model

From the model tracking page for a model use case in an inventory, click the Asset tab to view the associated factsheet. You can also view the factsheet when you view model details in a project, space, or catalog. For details, see Viewing a model factsheet.

After a model is registered, facts about the model are captured and preserved automatically at each point in the model lifecycle, including whenever a model is updated or changed. For example, events are captured when a model is:

• Created
• Updated in any way, including having a tag or description revised
• Promoted to a space
• Published to a catalog
• Tested with a deployment
• Evaluated lifecycle in OpenScale

No user intervention is required to update the factsheet.

Additionally, facts are preserved about a model in a notebook after the IBM AI Governance Facts Client library (AIGovFactsClient) is started in the notebook. From that point, lifecycle metadata and metrics are automatically preserved.

Viewing the factsheet for a deployment

From the model details page for a model use case in an inventory, click the Asset tab to view the associated factsheet. You can also view the factsheet when you view deployment details in a space. For details, see Viewing a model factsheet.