Last updated: Dec 12, 2024
After your AutoAI RAG experiment completes, view the details of a pattern to understand the composition and performance.
Important: This feature is a beta release. It is not intended for production use.
From the pipeline leaderboard, you can view a table with the following information for each of the patterns the experiment generates:
| Column | Description |
|---|---|
| Rank | Rank of placement according to best performance for the optimized metric |
| Name | Pattern name |
| Model name | Name of the embedding model used to vectorize and index the documents |
| Optimize metric | Performance result for the optimized metric selected for the experiment. A RAG pattern can be optimized for Answer faithfulness and Answer correctness |
| Chunk size | Amount of data retrieved from the indexed documents |
| Retrieval method | Method (Window or Simple) used to retrieve indexed documents |
| Vector store distance metric | Metric (Cosine or Euclidean) used to measure how relevant the stored vectors are to the input vectors |
Click on a pattern name to review the following configuration details for a pattern. Some of the configuration settings are editable when you create the experiment. For details, see Customizing RAG experiment settings.
| Column | Description |
|---|---|
| Pattern | Pattern name |
| Answer correctness (mean) | Correctness of the generated response including both the relevance of the retrieved context and the quality of the generated response. |
| Answer faithfulness (mean) | Accuracy of the generated response to the retrieved text. |
| Context correctness (mean) | Relevancy of the generated response to the input. |
| Composition steps | Steps taken in the experiment. For example, Chunking or Retrieval. |
| Chunk overlap | Number of chunks that share common words or phrases. |
| Chunk size | Amount of data retrieved from the index documents. |
| Chunk augmentation | How relevant text is retrieved for each chunk of the input after splitting the input into multiple chunks. |
| Embedding model | Embedding model used to vectorize and index the document collection. |
| Truncate input tokens | Maximum number of tokens accepted as input. |
| Truncate strategy | Strategy used to determine how to process retrieved documents to optimize the model performance. |
| Context template text | Structured template for the generated response. |
| Model type | Foundation model used in the experiment. Click the preview icon for details about the model. |
| Max new tokens | Maximum number of new tokens that can be generated in addition to the tokens retrieved from indexed documents. |
| Prompt template text | Structure with guidelines for the model to use when retrieving text from indexed documents. |
| Retrieval method | Method used to retrieve indexed documents: Window or Simple. |
| Number of chunks | Number of chunks retrieved from the indexed documents. |
| Window size | Number of adjacent tokens considered by the model when retrieving text from indexed documents. |
| Vector store datasource type | Datasource that connects to the vector store. |
| Vector store distance metric | Metric used to measure how relevant the stored vectors are to the input vector. |
| Vector store index name | Index used as a data structure for storing and retrieving vector stores. |
| Vector store operation | Vector store operation for retrieving data. For example, upsert for inserting or updating rows in the database table |
| Vector store schema fields | Schema for storing the index of vectored documents. |
| Vector store schema ID | autoai_rag_1.0 |
| Vector store schema name | Name of the schema used for structuring the database table |
| Vector store schema type | The schema type struct is used for storing structured data. |
| Sample Q&A | Questions and correct answers provided as test data to measure the performance of the pattern. |
Parent topic: Automating a RAG pattern with AutoAI