diff --git a/site/developer/how-to/feature-overview.qmd b/site/developer/how-to/feature-overview.qmd index 7f9536acfd..cf222ffbdb 100644 --- a/site/developer/how-to/feature-overview.qmd +++ b/site/developer/how-to/feature-overview.qmd @@ -9,6 +9,7 @@ listing: - id: tests type: grid grid-columns: 2 + page-size: 4 image-placeholder: "jupyter-logo-rectangle.svg" max-description-length: 350 image-height: "100%" @@ -17,6 +18,7 @@ listing: - id: data-and-datasets type: grid grid-columns: 2 + page-size: 4 image-placeholder: "jupyter-logo-rectangle.svg" max-description-length: 350 image-height: "100%" @@ -25,6 +27,7 @@ listing: - id: metrics type: grid grid-columns: 2 + page-size: 4 image-placeholder: "jupyter-logo-rectangle.svg" max-description-length: 350 image-height: "100%" @@ -33,6 +36,7 @@ listing: - id: scoring type: grid grid-columns: 2 + page-size: 4 image-placeholder: "jupyter-logo-rectangle.svg" max-description-length: 350 image-height: "100%" diff --git a/site/developer/model-documentation/_code-snippet.qmd b/site/developer/model-documentation/_code-snippet.qmd new file mode 100644 index 0000000000..b98da91b48 --- /dev/null +++ b/site/developer/model-documentation/_code-snippet.qmd @@ -0,0 +1,5 @@ + + +Initialize the {{< var validmind.developer >}} with the _code snippet_ unique to each model per document to connect your environment to the {{< var validmind.platform >}}, ensuring test results are uploaded to the correct model and automatically populated in the right document. \ No newline at end of file diff --git a/site/developer/model-documentation/install-and-initialize-validmind-library.qmd b/site/developer/model-documentation/install-and-initialize-validmind-library.qmd index bd3ce6b84a..8238cf968a 100644 --- a/site/developer/model-documentation/install-and-initialize-validmind-library.qmd +++ b/site/developer/model-documentation/install-and-initialize-validmind-library.qmd @@ -10,46 +10,69 @@ aliases: - /guide/install-and-initialize-validmind-library.html - /guide/install-and-initialize-developer-framework.html - install-and-initialize-client-library.html +listing: + - id: whats-next + type: grid + max-description-length: 250 + sort: false + fields: [title, description] + grid-columns: 2 + contents: + - store-credentials-in-env-file.qmd + - ../how-to/testing-overview.qmd --- -{{< var vm.product >}} generates a unique _code snippet_ for each registered model to connect with your developer environment. You initialize the {{< var validmind.developer >}} with this code snippet, ensuring that your documentation and tests are uploaded to the correct model. +{{< include _code-snippet.qmd >}} ::: {.attn} ## Prerequisites - [x] {{< var link.login >}} -- [x] You are the model owner.[^1] -- [x] You are a [{{< fa code >}} Developer]{.bubble} or assigned another role with sufficient permissions to perform the tasks in this guide.[^2] +- [x] The model you want to connect to is registered in the model inventory.[^1] +- [x] The model document has a template applied.[^2] +- [x] You are the model owner, model validator, or another model stakeholder with write access to the model's documents.[^3] -In order to integrate the {{< var vm.developer >}} and to be able to upload to the {{< var vm.platform >}}, you must provide the following information through a code snippet that you copy from the {{< var validmind.platform >}}: +::: {.callout} +The {{< var validmind.developer >}} requires access to the data sources where relevant datasets and model files are stored in order to help you run tests. + +::: + + +::: + +## Installing the {{< var validmind.developer >}} + +In order to upload test results from the {{< var validmind.developer >}} to the {{< var validmind.platform >}}, provide the following information through a code snippet copied from the {{< var vm.platform >}}: | Argument | Description | |---------------|------------------------------------| | `api_host` | The location of the {{< var vm.product >}} API | | `api_key` | The account API key | | `api_secret` | The account secret key | +| `document` | The model document identifier key | | `model` | The model identifier | -If you do not have a suitable model with a code snippet to use, you can register a model first.[^3] +### Get your code snippet -The {{< var vm.developer >}} also requires access to the data sources where data sets used for training, testing, and trained model files are stored. This access is needed to run model documentation and validation tests, and to upload to the {{< var vm.platform >}} to populate the model documentation and validation reports. +Retrieve your code snippet for your model's selected document from the {{< var validmind.platform >}}: -::: +1. In the left sidebar, click **{{< fa cubes >}} Inventory**. -## Install {{< var vm.product >}} +2. Select a model by clicking on it or find your model by applying a filter or searching for it.[^4] -### Locate the {{< var vm.developer >}} integration instructions +3. In the left sidebar that appears for your model, click **{{< fa rocket >}} Getting Started**. -For existing models, this information can be found in the {{< var validmind.platform >}}: +4. Select the **[document]{.smallcaps}** you want to automatically upload test results to.[^5] -1. In the left sidebar, click **{{< fa cubes >}} Inventory**. +5. Click **{{< fa regular copy >}} Copy snippet to clipboard**. -2. Select a model by clicking on it or find your model by applying a filter or searching for it.[^4] +::: {.callout} +## Selecting a **[document]{.smallcaps}** to connect to requires {{< var validmind.developer >}} version >=2.12.0.[^6] -3. In the left sidebar that appears for your model, click **{{< fa rocket >}} Getting Started**. +A template must already be applied to your selected document to populate test results in the {{< var validmind.platform >}}. Attempting to initialize the {{< var vm.developer >}} with a document that does not have a template applied will result in an error. -4. Locate the code snippet and click **{{< fa regular copy >}} Copy snippet to clipboard**. +::: ### Install the {{< var vm.developer >}} @@ -70,21 +93,27 @@ vm.init( api_host = "{{< var api.host >}}/tracking", api_key = "API_KEY", api_secret = "API_SECRET", + document="document-key", # requires library >=2.12.0 model = "MODEL_IDENTIFIER" ) ``` :::{.callout} -To also enable monitoring, add `monitoring=True` to the `vm.init` method in your code snippet. [^5] +To also enable monitoring, add `monitoring=True` to the `vm.init` method in your code snippet. [^7] ::: -After you have pasted the code snippet into your development source code and run your code, the {{< var validmind.developer >}} will connect and register with the {{< var validmind.platform >}}. You can now use the {{< var vm.developer >}} to document and test your models, and to upload model documentation and test results to the {{< var vm.platform >}}. +::: {.feature} +#### **Automate with {{< var vm.product >}}** + +After you run the code snippet in your environment, the {{< var validmind.developer >}} will connect to your model and selected document in the {{< var validmind.platform >}}. Automate your workflow by using the {{< var vm.developer >}} to run tests, then seamlessly upload your test results to the {{< var vm.platform >}}. + +::: -## Upgrade {{< var vm.product >}} +## Upgrade the {{< var validmind.developer >}} -After installing {{< var vm.product >}},[^6] you'll want to periodically make sure you are on the latest version to access any new features and other enhancements: +After installing the {{< var validmind.developer >}},[^8] you'll want to periodically make sure you are on the latest version to access any new features and other enhancements: -1. In your Jupyter Notebook or developer environment, retrieve the information for the currently installed version of {{< var vm.product >}}: +1. In your Jupyter Notebook or developer environment, retrieve the information for the currently installed version of the {{< var vm.developer >}}: ```python %pip show validmind @@ -94,29 +123,39 @@ After installing {{< var vm.product >}},[^6] you'll want to periodically make su **Example output:** ```bash Name: validmind -Version: 2.5.15 +Version: 2.11.0 ... ``` ::: -2. If the version returned is lower than the version indicated in our production open-source code,[^7] run the following command: +2. If the version returned is lower than the version indicated in our production open-source code,[^9] run the following command: ```python %pip install --upgrade validmind ``` +## What's next + +:::{#whats-next} +::: + + -[^1]: [Manage model stakeholders](/guide/model-inventory/edit-model-inventory-fields.qmd#manage-model-stakeholders) +[^1]: [Register models in the inventory](/guide/model-inventory/register-models-in-inventory.qmd) -[^2]: [Manage permissions](/guide/configuration/manage-permissions.qmd) +[^2]: [Working with templates](/guide/templates/working-with-document-templates.qmd) -[^3]: [Register models in the inventory](/guide/model-inventory/register-models-in-inventory.qmd) +[^3]: [Manage model stakeholders](/guide/model-inventory/edit-model-inventory-fields.qmd#manage-model-stakeholders) [^4]: [Working with the model inventory](/guide/model-inventory/working-with-model-inventory.qmd#search-filter-and-sort-models) -[^5]: [Ongoing monitoring](/guide/monitoring/ongoing-monitoring.qmd) +[^5]: [Working with model documents](/guide/templates/working-with-model-documents.qmd) + +[^6]: [Upgrade the {{< var validmind.developer >}}](#upgrade-the) + +[^7]: [Ongoing monitoring](/guide/monitoring/ongoing-monitoring.qmd) -[^6]: [Install {{< var vm.product >}}](#install-validmind) +[^8]: [Install {{< var vm.product >}}](#install-validmind) -[^7]: **ValidMind GitHub:** [validmind-library/validmind/__version__.py](https://github.com/validmind/validmind-library/blob/prod/validmind/__version__.py) \ No newline at end of file +[^9]: **ValidMind GitHub:** [validmind-library/validmind/__version__.py](https://github.com/validmind/validmind-library/blob/prod/validmind/__version__.py) \ No newline at end of file diff --git a/site/developer/model-documentation/store-credentials-in-env-file.qmd b/site/developer/model-documentation/store-credentials-in-env-file.qmd index a9a7c243fc..48dd9cc500 100644 --- a/site/developer/model-documentation/store-credentials-in-env-file.qmd +++ b/site/developer/model-documentation/store-credentials-in-env-file.qmd @@ -6,16 +6,31 @@ title: "Store model credentials in `.env` files" date: last-modified aliases: - /guide/store-credentials-in-env-file.html +listing: + - id: whats-next + type: grid + max-description-length: 250 + sort: false + fields: [title, description] + grid-columns: 2 + contents: + - ../how-to/testing-overview.qmd + - ../how-to/feature-overview.qmd --- -Learn how to store model identifier credentials in a `.env` file instead of using inline credentials. This topic is relevant for model developers who want to follow best practices for security when running notebooks. +Learn how to store model identifier credentials in an `.env` file instead of using inline credentials, allowing you to follow best practices for security when running Jupyter Notebooks. -## Why is this recommended? +
-Storing credentials in a `.env` file is considered a best practice for security. Embedding credentials directly within the code makes them more susceptible to accidental exposure when sharing code or collaborating on models. +::: {.callout-button .pl4 .nt4} +::: {.callout collapse="true" appearance="minimal"} +### Why is it recommended to store credentials in an `.env` file? + +Storing credentials in a `.env` file is considered a best practice for security. Embedding credentials directly within the code makes them more susceptible to accidental exposure when sharing code or collaborating on models. + +Keeping model credentials in a separate file also allows for precise access control and ensures that sensitive credentials are not publicly accessible. -::: {.callout-important} -Keeing model credentials in a separate file also allows for precise access control and ensures that sensitive credentials are not publically accessible. +::: ::: ::: {.attn} @@ -24,25 +39,41 @@ Keeing model credentials in a separate file also allows for precise access contr - [x] {{< var link.login >}} - [x] The model is registered in the model inventory.[^1] -- [x] You are a [{{< fa code >}} Developer]{.bubble} or assigned another role with sufficient permissions to perform the tasks in this guide.[^2] +- [x] You are the model owner, model validator, or another model stakeholder with sufficient permissions to perform the tasks in this guide.[^2] + +::: {.callout} +Before you follow the steps on this page, we recommend that you first complete our introductory guide on how to [Install and initialize the {{< var validmind.developer >}}](/developer/model-documentation/install-and-initialize-validmind-library.qmd). ::: -## Steps +::: -1. Create a new file in the same folder as your notebook and name it `.env`. +## 1. Store credentials in an `.env` file - This is a hidden file, so you may need to change your settings to view it. +::: {.panel-tabset} + +### a. Get your code snippet + +In the {{< var validmind.platform >}}, retrieve the code snippet for your model: + +i. In the left sidebar, click **{{< fa cubes >}} Inventory**. + +ii. Select a model by clicking on it or find your model by applying a filter or searching for it.[^3] + +iii. In the left sidebar that appears for your model, click **{{< fa rocket >}} Getting Started**. + +iv. Select the **[document]{.smallcaps}** you want to automatically upload test results to.[^4] + +v. Click **{{< fa regular copy >}} Copy snippet to clipboard**. -2. Locate the code snippet for your model: +### b. Create an `.env` file - a. In the left sidebar, click **{{< fa cubes >}} Inventory**. - b. Select a model by clicking on it or find your model by applying a filter or searching for it.[^3] - c. In the left sidebar that appears for your model, click **{{< fa rocket >}} Getting Started**. - d. Locate the code snippet and click **{{< fa regular copy >}} Copy snippet to clipboard**. +i. Create a new file in the same folder as your notebook and name it `.env`. + + This is a hidden file, so you may need to change your settings to view it. + +ii. Use the code snippet from your clipboard to build the credentials in your `.env` file in the following format:[^5] -3. Paste the credentials into your `.env` file in the following format: - ```yaml VM_API_HOST= VM_API_KEY= @@ -50,83 +81,115 @@ Keeing model credentials in a separate file also allows for precise access contr VM_API_MODEL= ``` -::: {.column-margin} -For example, if your credentials look like this: - -```python -import validmind as vm - -vm.init( -api_host = "https://api.prod.validmind.ai/api/v1/tracking", -api_key = "key123", -api_secret = "secret456", -model = "model789" -) -``` - -Then your `.env` file should look like this: +::: {.callout} +To define the document to connect the {{< var validmind.developer >}} to, or to enable monitoring, include these in your `vm.init()` snippet within your Jupyter Notebook.[^6] -```bash -VM_API_HOST=https://api.prod.validmind.ai/api/v1/tracking -VM_API_KEY=key123 -VM_API_SECRET=secret456 -VM_API_MODEL=model789 -``` +::: ::: +## 2. Use credentials in your Jupyter Notebook + +a. After installing the {{< var validmind.developer >}} within your Jupyter Notebook,[^7] insert this code snippet in a code cell above your model identifier credentials: -4. Insert this code snippet above your model identifier credentials: - ```python %load_ext dotenv %dotenv dev.env ``` -5. Remove the inline credentials from `vm.init()`.[^4] +b. Remove the inline credentials from `vm.init()` after importing the {{< var vm.developer >}}: + + ::: {.panel-tabset} + + ### Without ongoing monitoring + + Example updated code cell: - The updated code cell should now look like this: - ```python %load_ext dotenv %dotenv .env - + import validmind as vm - + vm.init( + document="document-key", ) ``` -::: {.column-margin} -To enable monitoring when you are storing credentials in an `.env` file:^[[Enable monitoring](/guide/monitoring/enable-monitoring.qmd)] + ### With ongoing monitoring -```python -%load_ext dotenv -%dotenv .env + Example updated code cell including ongoing monitoring:[^8] -import validmind as vm + ```python + %load_ext dotenv + %dotenv .env -vm.init( - monitoring=True -) -``` + import validmind as vm -::: + vm.init( + document="monitoring", + monitoring=True + ) + ``` + + ::: + + ::: {.callout} + ## A template must already be applied to your selected document to populate test results in the {{< var validmind.platform >}}. + + Selecting a document to connect to requires {{< var validmind.developer >}} version >=2.12.0.[^9] Attempting to initialize the {{< var vm.developer >}} with a document that does not have a template applied will result in an error. -6. Run the cell. Instead of using inline credentials, this cell will now load your model credentials from a `.env` file. + ::: +c. Run the cell. + + Instead of using inline credentials, this code cell will now load your model credentials from a `.env` file. ## What's next -- [Working with model documentation](/guide/model-documentation/working-with-model-documentation.qmd) -- [{{< var validmind.developer >}}](/developer/validmind-library.qmd) +:::{#whats-next} +::: [^1]: [Register models in the inventory](/guide/model-inventory/register-models-in-inventory.qmd) -[^2]: [Manage permissions](/guide/configuration/manage-permissions.qmd) +[^2]: [Manage model stakeholders](/guide/model-inventory/edit-model-inventory-fields.qmd#manage-model-stakeholders) [^3]: [Working with the model inventory](/guide/model-inventory/working-with-model-inventory.qmd#search-filter-and-sort-models) +[^4]: [Working with model documents](/guide/templates/working-with-model-documents.qmd) + +[^5]: + + For example, if your credentials look like this: + + ```python + import validmind as vm + + vm.init( + api_host = "https://api.prod.validmind.ai/api/v1/tracking", + api_key = "key123", + api_secret = "secret456", + document = "document-key", + model = "model789" + ) + ``` + + Then your `.env` file should look like this: + + ```bash + VM_API_HOST=https://api.prod.validmind.ai/api/v1/tracking + VM_API_KEY=key123 + VM_API_SECRET=secret456 + VM_API_MODEL=model789 + ``` + +[^6]: [Use credentials in your Jupyter Notebook](#use-credentials-in-your-jupyter-notebook) + +[^7]: [Install and initialize the {{< var validmind.developer >}}](/developer/model-documentation/install-and-initialize-validmind-library.qmd#install-the) + +[^8]: [Enable monitoring](/guide/monitoring/enable-monitoring.qmd) + +[^9]: [Install and initialize the {{< var validmind.developer >}}](/developer/model-documentation/install-and-initialize-validmind-library.qmd#install-the) diff --git a/site/developer/samples-jupyter-notebooks.qmd b/site/developer/samples-jupyter-notebooks.qmd index 11a5aa88c7..63786d527b 100644 --- a/site/developer/samples-jupyter-notebooks.qmd +++ b/site/developer/samples-jupyter-notebooks.qmd @@ -8,6 +8,7 @@ listing: - id: agents type: grid grid-columns: 2 + page-size: 4 image-placeholder: "jupyter-logo-rectangle.svg" max-description-length: 350 image-height: "100%" @@ -16,6 +17,7 @@ listing: - id: capital-markets type: grid grid-columns: 2 + page-size: 4 image-placeholder: "jupyter-logo-rectangle.svg" max-description-length: 350 image-height: "100%" @@ -24,6 +26,7 @@ listing: - id: code-explainer type: grid grid-columns: 2 + page-size: 4 image-placeholder: "jupyter-logo-rectangle.svg" max-description-length: 350 image-height: "100%" @@ -32,6 +35,7 @@ listing: - id: credit-risk type: grid grid-columns: 2 + page-size: 4 image-placeholder: "jupyter-logo-rectangle.svg" max-description-length: 350 image-height: "100%" @@ -40,6 +44,7 @@ listing: - id: model-validation type: grid grid-columns: 2 + page-size: 4 image-placeholder: "jupyter-logo-rectangle.svg" max-description-length: 350 image-height: "100%" @@ -48,6 +53,7 @@ listing: - id: nlp-and-llm type: grid grid-columns: 2 + page-size: 4 image-placeholder: "jupyter-logo-rectangle.svg" max-description-length: 350 image-height: "100%" @@ -56,6 +62,7 @@ listing: - id: ongoing-monitoring type: grid grid-columns: 2 + page-size: 4 image-placeholder: "jupyter-logo-rectangle.svg" max-description-length: 350 image-height: "100%" @@ -64,6 +71,7 @@ listing: - id: regression type: grid grid-columns: 2 + page-size: 4 image-placeholder: "jupyter-logo-rectangle.svg" max-description-length: 350 image-height: "100%" @@ -72,6 +80,7 @@ listing: - id: time-series type: grid grid-columns: 2 + page-size: 4 image-placeholder: "jupyter-logo-rectangle.svg" max-description-length: 350 image-height: "100%" diff --git a/site/get-started/common-steps/_get-your-code-snippet.qmd b/site/get-started/common-steps/_get-your-code-snippet.qmd index e0bb0138e7..cdcd43ea5a 100644 --- a/site/get-started/common-steps/_get-your-code-snippet.qmd +++ b/site/get-started/common-steps/_get-your-code-snippet.qmd @@ -7,9 +7,11 @@ Before we move on, let's also retrieve the unique *code snippet* for your model a. In the left sidebar that appears for your model, click **{{< fa rocket >}} Getting Started**. -a. Locate the code snippet and click **{{< fa regular copy >}} Copy snippet to clipboard**. +a. Select `Development` from the **[document]{.smallcaps}** drop-down menu. -![{{< fa rocket >}} Getting Started page with a sample code snippet](developer-code-snippet.png){width=90% fig-alt="A screenshot showing the Getting Started page with a sample code snippet" .screenshot} +a. Click **{{< fa regular copy >}} Copy snippet to clipboard**. + +![{{< fa rocket >}} Getting Started page with a sample code snippet](/get-started/developer/developer-code-snippet.png){width=90% fig-alt="A screenshot showing the Getting Started page with a sample code snippet" .screenshot} :::: @@ -19,8 +21,10 @@ Before we move on, let's also retrieve the unique *code snippet* for your model a. In the left sidebar that appears for your model, click **{{< fa rocket >}} Getting Started**. -a. Locate the code snippet and click **{{< fa regular copy >}} Copy snippet to clipboard**. +a. Select `Validation` from the **[document]{.smallcaps}** drop-down menu. + +a. Click **{{< fa regular copy >}} Copy snippet to clipboard**. -![{{< fa rocket >}} Getting Started page with a sample code snippet](validator-code-snippet.png){width=90% fig-alt="A screenshot showing the Getting Started page with a sample code snippet" .screenshot} +![{{< fa rocket >}} Getting Started page with a sample code snippet](/get-started/validator/validator-code-snippet.png){width=90% fig-alt="A screenshot showing the Getting Started page with a sample code snippet" .screenshot} :::: diff --git a/site/get-started/developer/developer-code-snippet.png b/site/get-started/developer/developer-code-snippet.png index 47bc3b6b5a..eea9ac2ef4 100644 Binary files a/site/get-started/developer/developer-code-snippet.png and b/site/get-started/developer/developer-code-snippet.png differ diff --git a/site/get-started/validator/validator-code-snippet.png b/site/get-started/validator/validator-code-snippet.png index cd330fe31b..fe40c0e0cf 100644 Binary files a/site/get-started/validator/validator-code-snippet.png and b/site/get-started/validator/validator-code-snippet.png differ diff --git a/site/guide/monitoring/enable-monitoring.qmd b/site/guide/monitoring/enable-monitoring.qmd index 6bfbfa212f..847c87a67d 100644 --- a/site/guide/monitoring/enable-monitoring.qmd +++ b/site/guide/monitoring/enable-monitoring.qmd @@ -6,12 +6,7 @@ title: "Enable monitoring" date: last-modified --- -To start uploading ongoing monitoring results for a model to {{< var vm.product >}}, you enable monitoring in your code snippet and then select a monitoring template. You can enable monitoring for both new and existing models. - -Enable monitoring with two steps: - -1. [Add `monitoring=True` to your code snippet](#add-monitoringtrue-to-your-code-snippet) -2. [Select a monitoring template](#select-a-monitoring-template) +To start uploading ongoing monitoring results for a model to {{< var vm.product >}}, enable monitoring in your code snippet within the {{< var validmind.platform >}} and then select a monitoring template. You can enable monitoring for both new and existing models. ::: {.callout title="To try out monitoring, check out the code sample for ongoing monitoring of models."} [Quickstart for ongoing monitoring of models with {{< var vm.product >}} {{< fa hand-point-right >}}](/notebooks/use_cases/ongoing_monitoring/quickstart_customer_churn_ongoing_monitoring.ipynb) @@ -27,87 +22,120 @@ Enable monitoring with two steps: ::: -## Add `monitoring=True` to your code snippet +## Steps + +::: {.panel-tabset} + +### 1. Get monitoring code snippet -{{< var vm.product >}} generates a unique _code snippet_ for each registered model to connect with your developer environment. You initialize the {{< var validmind.developer >}} with this code snippet, which ensures that your documentation and tests are uploaded to the correct model when you run the notebook. +{{< include /developer/model-documentation/_code-snippet.qmd >}} -To enable ongoing monitoring for a model, add `monitoring=True` to your code snippet: +To enable ongoing monitoring for a model, add `monitoring=True` to your code snippet: -1. In the {{< var validmind.platform >}}, either register a new model[^2] or locate an existing model: +a. In the {{< var validmind.platform >}}, either register a new model[^2] or locate an existing model: - In the left sidebar, click **{{< fa cubes >}} Inventory**. - Select a model or find your model by applying a filter or searching for it.[^3] -2. Copy the code snippet for the model: +b. Copy the code snippet for the model: - In the left sidebar that appears for your model, click **{{< fa rocket >}} Getting Started**. - - Locate the code snippet and set **[enable ongoing monitoring]{.smallcaps}[?]{.smallercaps}** to true. - - Click **{{< fa regular copy >}} Copy snippet to clipboard**. + - Select `Monitoring` from the **[document]{.smallcaps}** drop-down menu.[^4] + - Set **[enable monitoring]{.smallcaps}[?]{.smallercaps}** to true. + - Click **{{< fa regular copy >}} Copy snippet to clipboard**. + +### 2. Select monitoring template + +Before you can start sending ongoing monitoring data using the {{< var validmind.developer >}} to the {{< var validmind.platform >}}, you must select a monitoring template: + +a. In the {{< var validmind.platform >}}, click **{{< fa cubes >}} Inventory** in the left sidebar. + +b. Select a model or find your model by applying a filter or searching for it.[^6] + +c. In the left sidebar that appears for your model, click **{{< fa file >}} Documents** and select the **Latest** tab.[^7] + +d. Click on a Ongoing Monitoring type file.[^8] + +e. From the **[template]{.smallcaps}** drop-down, select one of the available monitoring templates. + +f. Click **Use Template** to apply your desired template. + + - After you select a template, the initial document where your ongoing monitoring results will be stored becomes available. + - Review this documents structure to confirm that it meets your requirements, then start running your code to populate the template with monitoring results. + +### 3. Run code snippet -3. Paste the snippet into your development source code.[^4] +a. Paste the code snippet into your Jupyter Notebook or other development environment. + +b. Confirm that `monitoring=True` is present in the `vm.init` method, similar to this example: - Confirm that `monitoring=True` is present in the `vm.init` method, similar to this example: - ```python import validmind as vm - + vm.init( api_host="https://api.prod.validmind.ai/api/v1/tracking", api_key="...", api_secret="...", model="...", + document="monitoring", # requires library >=2.12.0 monitoring=True ) ``` +
-::: {.column-margin} -To enable monitoring when you are storing credentials in an `.env` file:^[[Store model credentials in `.env` files](/developer/model-documentation/store-credentials-in-env-file.qmd)] + ::: {.callout-button .pl4 .nt4} + ::: {.callout collapse="true" appearance="minimal"} + ### How do I enable monitoring when I am storing credentials in an `.env` file? -```python -%load_ext dotenv -%dotenv .env + :::: {.flex .flex-wrap .justify-around} -import validmind as vm + ::: {.w-60-ns .pr3} -vm.init( - monitoring=True -) -``` + Load your model credentials from your `.env` file, then remove the inline credentials from `vm.init()` after importing the {{< var vm.developer >}}. -::: + You'll still need to include your document key and the monitoring flag. -## Select a monitoring template + [Store model credentials in `.env` files](/developer/model-documentation/store-credentials-in-env-file.qmd){.button .button-green} -Before you can start sending ongoing monitoring data from your developer environment to {{< var vm.product >}}, you must select a monitoring template: + ::: -1. In the {{< var validmind.platform >}}, click **{{< fa cubes >}} Inventory** in the left sidebar. + ::: {.w-40-ns} -2. Select a model or find your model by applying a filter or searching for it.[^5] + ```python + %load_ext dotenv + %dotenv .env -3. In the left sidebar that appears for your model, click **{{< fa file >}} Documents** and select the **Latest** tab.[^6] + import validmind as vm -4. Click on a Ongoing Monitoring type file.[^7] + vm.init( + document="monitoring", + monitoring=True + ) + ``` + + ::: + + :::: -5. From the **[template]{.smallcaps}** drop-down, select one of the available monitoring templates. -6. Click **Use Template** to apply your desired template. + ::: + ::: - - After you select a template, the initial document where your ongoing monitoring results will be stored becomes available. - - Review this documents structure to confirm that it meets your requirements, then start running your code to populate the template with monitoring results. +c. Run the code snippet to initialize the {{< var validmind.developer >}} and connect to the selected monitoring document for your model in the {{< var validmind.platform >}}.[^5] +::: {.callout} +## A template must already be applied to your selected document to populate monitoring test results in the {{< var validmind.platform >}}. -## Troubleshooting +Attempting to initialize the {{< var vm.developer >}} with a document that does not have a template applied will result in an error. -If you see errors similar to this one when running your code, make sure that you have selected your monitoring template and then re-run your code: +::: -```bash -APIRequestError: Please select an ongoing monitoring template on the {{< var validmind.platform >}} first -``` +::: ## What's next -After you have enabled ongoing monitoring and run your code to generate some output, you can start reviewing the monitoring results.[^8] +After you have enabled ongoing monitoring and run your code to generate some output, you can start reviewing the monitoring results.[^9] @@ -118,12 +146,14 @@ After you have enabled ongoing monitoring and run your code to generate some out [^3]: [Working with the model inventory](/guide/model-inventory/working-with-model-inventory.qmd#search-filter-and-sort-models) -[^4]: [Install and initialize the {{< var validmind.developer >}}](/developer/model-documentation/install-and-initialize-validmind-library.qmd) +[^4]: Selecting a **[document]{.smallcaps}** to connect to requires {{< var validmind.developer >}} version >=2.12.0. + +[^5]: [Install and initialize the {{< var validmind.developer >}}](/developer/model-documentation/install-and-initialize-validmind-library.qmd) -[^5]: [Working with the model inventory](/guide/model-inventory/working-with-model-inventory.qmd#search-filter-and-sort-models) +[^6]: [Working with the model inventory](/guide/model-inventory/working-with-model-inventory.qmd#search-filter-and-sort-models) -[^6]: [Work with document versions](/guide/model-documentation/work-with-document-versions.qmd) +[^7]: [Work with document versions](/guide/model-documentation/work-with-document-versions.qmd) -[^7]: [Working with model documents](/guide/templates/working-with-model-documents.qmd) +[^8]: [Working with model documents](/guide/templates/working-with-model-documents.qmd) -[^8]: [Review monitoring results](review-monitoring-results.qmd) \ No newline at end of file +[^9]: [Review monitoring results](review-monitoring-results.qmd) \ No newline at end of file diff --git a/site/scripts/developer-sidebar/update_how_tos.py b/site/scripts/developer-sidebar/update_how_tos.py index 957e066160..f3a7b729a4 100644 --- a/site/scripts/developer-sidebar/update_how_tos.py +++ b/site/scripts/developer-sidebar/update_how_tos.py @@ -197,6 +197,7 @@ def update_feature_overview(base: Path, subdirs: list) -> None: listing_lines.append(f" - id: {listing_id}") listing_lines.append(f" type: grid") listing_lines.append(f" grid-columns: 2") + listing_lines.append(f" page-size: 4") listing_lines.append(f' image-placeholder: "jupyter-logo-rectangle.svg"') listing_lines.append(f" max-description-length: 350") listing_lines.append(f' image-height: "100%"') diff --git a/site/scripts/developer-sidebar/update_use_cases.py b/site/scripts/developer-sidebar/update_use_cases.py index 0e8713dc8c..b9c39da160 100644 --- a/site/scripts/developer-sidebar/update_use_cases.py +++ b/site/scripts/developer-sidebar/update_use_cases.py @@ -162,6 +162,7 @@ def update_notebooks_page(base: Path, subdirs: list) -> None: listing_lines.append(f" - id: {listing_id}") listing_lines.append(f' type: grid') listing_lines.append(f' grid-columns: 2') + listing_lines.append(f' page-size: 4') listing_lines.append(f' image-placeholder: "jupyter-logo-rectangle.svg"') listing_lines.append(f' max-description-length: 350') listing_lines.append(f' image-height: "100%"') diff --git a/site/training/common-slides/_get-your-code-snippet.qmd b/site/training/common-slides/_get-your-code-snippet.qmd index 0d4c5cd5cc..cc7ca2d838 100644 --- a/site/training/common-slides/_get-your-code-snippet.qmd +++ b/site/training/common-slides/_get-your-code-snippet.qmd @@ -2,6 +2,14 @@ Refer to the LICENSE file in the root of this repository for details. SPDX-License-Identifier: AGPL-3.0 AND ValidMind Commercial --> + + + +:::: {.content-visible when-format="revealjs" when-meta="includes.developer"} ## Get your code snippet :::: {.columns} @@ -31,7 +39,8 @@ SPDX-License-Identifier: AGPL-3.0 AND ValidMind Commercial --> 1. Select the name of your model you registered for this course to open up the model details page. 2. On the left sidebar that appears for your model, click **Getting Started**. -3. Locate the code snippet and click **Copy snippet to clipboard**. +3. Select `Development` from the **[document]{.smallcaps}** drop-down menu. +4. Click **Copy snippet to clipboard**. ::: @@ -46,4 +55,72 @@ When you're done, click [{{< fa chevron-right >}}]() to continue. Make sure you're logged in and have refreshed the page in a Chromium-based web browser. -::: \ No newline at end of file +::: + +:::: + + + +:::: {.content-visible when-format="revealjs" unless-meta="includes.developer"} +## Get your code snippet + +:::: {.columns} + +::: {.column width="80%"} + + + +
+ +
+ +::: + +::: {.column width="20%" .f4} + +::: {.f5} +{{< var vm.product >}} generates a unique *code snippet* for each registered model to connect with your local environment: + +1. Select the name of your model you registered for this course to open up the model details page. +2. On the left sidebar that appears for your model, click **Getting Started**. +3. Select `Validation` from the **[document]{.smallcaps}** drop-down menu. +4. Click **Copy snippet to clipboard**. + +::: + +When you're done, click [{{< fa chevron-right >}}]() to continue. + +::: + +:::: + +:::: {.tc .f6 .embed} +**Can't load the {{< var validmind.platform >}}?** + +Make sure you're logged in and have refreshed the page in a Chromium-based web browser. + +::: + +:::: + + + + + + + + + + diff --git a/site/training/common-slides/_retrieve-your-code-snippet.qmd b/site/training/common-slides/_retrieve-your-code-snippet.qmd index b7d9613d75..8f5bbdd685 100644 --- a/site/training/common-slides/_retrieve-your-code-snippet.qmd +++ b/site/training/common-slides/_retrieve-your-code-snippet.qmd @@ -2,6 +2,62 @@ Refer to the LICENSE file in the root of this repository for details. SPDX-License-Identifier: AGPL-3.0 AND ValidMind Commercial --> + + +:::: {.content-visible when-format="revealjs" when-meta="includes.developer"} +## Retrieve your code snippet + +:::: {.columns} + +::: {.column width="80%"} + + + +
+ +
+ +::: + +::: {.column width="20%"} + +::: {.f4} +As usual, let's connect back up to your model in the {{< var validmind.platform >}}: + +1. Select the name of your model you registered for this course to open up the model details page. +2. On the left sidebar that appears for your model, click **Getting Started**. +3. Select `Development` from the **[document]{.smallcaps}** drop-down menu. +4. Click **Copy snippet to clipboard**. + +::: + +::: + +:::: + +When you're done, click [{{< fa chevron-right >}}]() to continue. + +:::: + + + +:::: {.content-visible when-format="revealjs" unless-meta="includes.developer"} ## Retrieve your code snippet :::: {.columns} @@ -31,7 +87,8 @@ As usual, let's connect back up to your model in the {{< var validmind.platform 1. Select the name of your model you registered for this course to open up the model details page. 2. On the left sidebar that appears for your model, click **Getting Started**. -3. Locate the code snippet and click **Copy snippet to clipboard**. +3. Select `Validation` from the **[document]{.smallcaps}** drop-down menu. +4. Click **Copy snippet to clipboard**. ::: @@ -39,4 +96,15 @@ As usual, let's connect back up to your model in the {{< var validmind.platform :::: -When you're done, click [{{< fa chevron-right >}}]() to continue. \ No newline at end of file +When you're done, click [{{< fa chevron-right >}}]() to continue. + +:::: + + + + + + + + + diff --git a/site/training/developer-fundamentals/implementing-custom-tests.qmd b/site/training/developer-fundamentals/implementing-custom-tests.qmd index bc88dacd23..4707cb6452 100644 --- a/site/training/developer-fundamentals/implementing-custom-tests.qmd +++ b/site/training/developer-fundamentals/implementing-custom-tests.qmd @@ -32,6 +32,8 @@ title-slide-attributes: data-background-color: "#083E44" data-background-image: "../assets/home-hero.svg" skip_preview: true +includes: + developer: true --- # Learning objectives {.center} diff --git a/site/training/developer-fundamentals/learning-to-run-tests.qmd b/site/training/developer-fundamentals/learning-to-run-tests.qmd index 69ea84d76c..f441147c29 100644 --- a/site/training/developer-fundamentals/learning-to-run-tests.qmd +++ b/site/training/developer-fundamentals/learning-to-run-tests.qmd @@ -32,6 +32,8 @@ title-slide-attributes: data-background-color: "#083E44" data-background-image: "../assets/home-hero.svg" skip_preview: true +includes: + developer: true --- # Learning objectives {.center} diff --git a/site/training/training-templates/course-slides.qmd b/site/training/training-templates/course-slides.qmd index e61029eb7c..12d8a7e460 100644 --- a/site/training/training-templates/course-slides.qmd +++ b/site/training/training-templates/course-slides.qmd @@ -34,6 +34,8 @@ title-slide-attributes: data-background-color: "#083E44" data-background-image: "../assets/home-hero.svg" skip_preview: true +includes: + developer: true --- # Learning objectives {.center} @@ -84,7 +86,7 @@ This {{#th}} module is part of a {{#}}-part series: Example embedded `iframe` for interactive copy/paste workaround: -- [Get your code snippet](#get-your-code-snippet) +- [Get your code snippet](#get-your-code-snippet) (Developer ver. preview) Topic summary & next steps: