Merge branch 'develop' into feature/vertex-ai-deployer-model-registry

README.md

@@ -228,22 +228,28 @@ And finally, here are some other examples and use cases for inspiration:
 5. [LLMops](https://github.com/zenml-io/zenml-projects/tree/main/llm-complete-guide): Complete guide to do LLM with ZenML
 
 
+## 📚 Learn from Books
+
+<div align="center">
+  <a href="https://www.amazon.com/LLM-Engineers-Handbook-engineering-production/dp/1836200072">
+    <img src="docs/book/.gitbook/assets/llm_engineering_handbook_cover.jpg" alt="LLM Engineer's Handbook Cover" width="200"/></img>
+  </a>&nbsp;&nbsp;&nbsp;&nbsp;
+  <a href="https://www.amazon.com/-/en/Andrew-McMahon/dp/1837631964">
+    <img src="docs/book/.gitbook/assets/ml_engineering_with_python.jpg" alt="Machine Learning Engineering with Python Cover" width="200"/></img>
+  </a>
+  </br></br>
+</div>
+
+ZenML is featured in these comprehensive guides to modern MLOps and LLM engineering. Learn how to build production-ready machine learning systems with real-world examples and best practices.
+
 ## 🔋 Deploy ZenML
 
 For full functionality ZenML should be deployed on the cloud to
 enable collaborative features as the central MLOps interface for teams.
 
-Currently, there are two main ways to deploy ZenML:
+Read more about various deployment options [here](https://docs.zenml.io/getting-started/deploying-zenml).
 
-- **ZenML Pro**: [ZenML Pro](cloud.zenml.io/?utm_source=readme&utm_medium=referral_link&utm_campaign=cloud_promotion&utm_content=signup_link),
-offers a SaaS version which comes with a control plane to create and manage multiple ZenML servers.
-These servers are managed and maintained by ZenML’s dedicated team, alleviating
-the burden of server management from your end. They also come with additional features like RBAC, a Model Control Plane
-and [much more](https://zenml-io.gitbook.io/zenml-documentation/getting-started/zenml-pro.
-- **Self-hosted deployment**: Alternatively, you have the flexibility to [deploy
-ZenML on your own self-hosted environment](https://docs.zenml.io/getting-started/deploying-zenml#deploying-a-zenml-server).
-This can be achieved through various methods, including using our CLI, Docker,
-Helm, or HuggingFace Spaces.
+Or, sign up for [ZenML Pro to get a fully managed server on a free trial](https://cloud.zenml.io/?utm_source=readme&utm_medium=referral_link&utm_campaign=cloud_promotion&utm_content=signup_link).
 
 ## Use ZenML with VS Code
 
@@ -327,7 +333,7 @@ the Apache License Version 2.0.
     <a href="https://github.com/zenml-io/zenml-projects">Projects Showcase</a>
     <br />
     <br />
-    🎉 Version 0.68.0 is out. Check out the release notes
+    🎉 Version 0.68.1 is out. Check out the release notes
     <a href="https://github.com/zenml-io/zenml/releases">here</a>.
     <br />
     🖥️ Download our VS Code Extension <a href="https://marketplace.visualstudio.com/items?itemName=ZenML.zenml-vscode">here</a>.

RELEASE_NOTES.md

@@ -1,5 +1,14 @@
 <!-- markdown-link-check-disable -->
 
+# 0.68.1
+
+Fixes an issue with some partially cached pipelines running on remote orchestrators.
+
+## What's Changed
+* Remove unavailable upstream steps during cache precomputation by @schustmi in https://github.com/zenml-io/zenml/pull/3146
+
+**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.68.0...0.68.1
+
 # 0.68.0
 
 ## Highlights

docs/book/component-guide/README.md

@@ -1,12 +1,13 @@
 ---
+icon: scroll
 description: Overview of categories of MLOps components and third-party integrations.
 ---
 
-# 📜 Overview
+# Overview
 
 If you are new to the world of MLOps, it is often daunting to be immediately faced with a sea of tools that seemingly all promise and do the same things. It is useful in this case to try to categorize tools in various groups in order to understand their value in your toolchain in a more precise manner.
 
-ZenML tackles this problem by introducing the concept of [Stacks and Stack Components](../user-guide/production-guide/understand-stacks.md). These stack components represent categories, each of which has a particular function in your MLOps pipeline. ZenML realizes these stack components as base abstractions that standardize the entire workflow for your team. In order to then realize the benefit, one can write a concrete implementation of the [abstraction](../how-to/stack-deployment/implement-a-custom-stack-component.md), or use one of the many built-in [integrations](README.md) that implement these abstractions for you.
+ZenML tackles this problem by introducing the concept of [Stacks and Stack Components](../user-guide/production-guide/understand-stacks.md). These stack components represent categories, each of which has a particular function in your MLOps pipeline. ZenML realizes these stack components as base abstractions that standardize the entire workflow for your team. In order to then realize the benefit, one can write a concrete implementation of the [abstraction](../how-to/infrastructure-deployment/stack-deployment/implement-a-custom-stack-component.md), or use one of the many built-in [integrations](README.md) that implement these abstractions for you.
 
 Here is a full list of all stack components currently supported in ZenML, with a description of the role of that component in the MLOps process:
 
@@ -29,7 +30,7 @@ Each pipeline run that you execute with ZenML will require a **stack** and each
 
 ## Writing custom component flavors
 
-You can take control of how ZenML behaves by creating your own components. This is done by writing custom component `flavors`. To learn more, head over to [the general guide on writing component flavors](../how-to/stack-deployment/implement-a-custom-stack-component.md), or read more specialized guides for specific component types (e.g. the [custom orchestrator guide](orchestrators/custom.md)).
+You can take control of how ZenML behaves by creating your own components. This is done by writing custom component `flavors`. To learn more, head over to [the general guide on writing component flavors](../how-to/infrastructure-deployment/stack-deployment/implement-a-custom-stack-component.md), or read more specialized guides for specific component types (e.g. the [custom orchestrator guide](orchestrators/custom.md)).
 
 ## Integrations
 

docs/book/component-guide/alerters/alerters.md

@@ -1,5 +1,6 @@
 ---
 description: Sending automated alerts to chat services.
+icon: message-exclamation
 ---
 
 # Alerters

docs/book/component-guide/alerters/custom.md

@@ -5,7 +5,7 @@ description: Learning how to develop a custom alerter.
 # Develop a Custom Alerter
 
 {% hint style="info" %}
-Before diving into the specifics of this component type, it is beneficial to familiarize yourself with our [general guide to writing custom component flavors in ZenML](../../how-to/stack-deployment/implement-a-custom-stack-component.md). This guide provides an essential understanding of ZenML's component flavor concepts.
+Before diving into the specifics of this component type, it is beneficial to familiarize yourself with our [general guide to writing custom component flavors in ZenML](../../how-to/infrastructure-deployment/stack-deployment/implement-a-custom-stack-component.md). This guide provides an essential understanding of ZenML's component flavor concepts.
 {% endhint %}
 
 ### Base Abstraction
@@ -119,7 +119,7 @@ zenml alerter flavor register flavors.my_flavor.MyAlerterFlavor
 ```
 
 {% hint style="warning" %}
-ZenML resolves the flavor class by taking the path where you initialized zenml (via `zenml init`) as the starting point of resolution. Therefore, please ensure you follow [the best practice](../../how-to/setting-up-a-project-repository/best-practices.md) of initializing zenml at the root of your repository.
+ZenML resolves the flavor class by taking the path where you initialized zenml (via `zenml init`) as the starting point of resolution. Therefore, please ensure you follow [the best practice](../../how-to/project-setup-and-management/setting-up-a-project-repository/set-up-repository.md) of initializing zenml at the root of your repository.
 
 If ZenML does not find an initialized ZenML repository in any parent directory, it will default to the current working directory, but usually, it's better to not have to rely on this mechanism and initialize zenml at the root.
 {% endhint %}

docs/book/component-guide/annotators/annotators.md

@@ -1,4 +1,5 @@
 ---
+icon: expand
 description: Annotating the data in your workflow.
 ---
 

docs/book/component-guide/annotators/custom.md

@@ -5,7 +5,7 @@ description: Learning how to develop a custom annotator.
 # Develop a Custom Annotator
 
 {% hint style="info" %}
-Before diving into the specifics of this component type, it is beneficial to familiarize yourself with our [general guide to writing custom component flavors in ZenML](../../how-to/stack-deployment/implement-a-custom-stack-component.md). This guide provides an essential understanding of ZenML's component flavor concepts.
+Before diving into the specifics of this component type, it is beneficial to familiarize yourself with our [general guide to writing custom component flavors in ZenML](../../how-to/infrastructure-deployment/stack-deployment/implement-a-custom-stack-component.md). This guide provides an essential understanding of ZenML's component flavor concepts.
 {% endhint %}
 
 Annotators are a stack component that enables the use of data annotation as part of your ZenML stack and pipelines. You can use the associated CLI command to launch annotation, configure your datasets and get stats on how many labeled tasks you have ready for use.

docs/book/component-guide/artifact-stores/artifact-stores.md

@@ -1,5 +1,6 @@
 ---
 description: Setting up a persistent storage for your artifacts.
+icon: folder-closed
 ---
 
 # Artifact Stores
@@ -9,17 +10,17 @@ The Artifact Store is a central component in any MLOps stack. As the name sugges
 ZenML automatically serializes and saves the data circulated through your pipelines in the Artifact Store: datasets, models, data profiles, data and model validation reports, and generally any object that is returned by a pipeline step. This is coupled with tracking in ZenML to provide extremely useful features such as caching and provenance/lineage tracking and pipeline reproducibility.
 
 {% hint style="info" %}
-Not all objects returned by pipeline steps are physically stored in the Artifact Store, nor do they have to be. How artifacts are serialized and deserialized and where their contents are stored are determined by the particular implementation of the [Materializer](../../how-to/handle-data-artifacts/handle-custom-data-types.md) associated with the artifact data type. The majority of Materializers shipped with ZenML use the Artifact Store which is part of the active Stack as the location where artifacts are kept.
+Not all objects returned by pipeline steps are physically stored in the Artifact Store, nor do they have to be. How artifacts are serialized and deserialized and where their contents are stored are determined by the particular implementation of the [Materializer](../../how-to/data-artifact-management/handle-data-artifacts/handle-custom-data-types.md) associated with the artifact data type. The majority of Materializers shipped with ZenML use the Artifact Store which is part of the active Stack as the location where artifacts are kept.
 
-If you need to store _a particular type of pipeline artifact_ in a different medium (e.g. use an external model registry to store model artifacts, or an external data lake or data warehouse to store dataset artifacts), you can write your own [Materializer](../../how-to/handle-data-artifacts/handle-custom-data-types.md) to implement the custom logic required for it. In contrast, if you need to use an entirely different storage backend to store artifacts, one that isn't already covered by one of the ZenML integrations, you can [extend the Artifact Store abstraction](custom.md) to provide your own Artifact Store implementation.
+If you need to store _a particular type of pipeline artifact_ in a different medium (e.g. use an external model registry to store model artifacts, or an external data lake or data warehouse to store dataset artifacts), you can write your own [Materializer](../../how-to/data-artifact-management/handle-data-artifacts/handle-custom-data-types.md) to implement the custom logic required for it. In contrast, if you need to use an entirely different storage backend to store artifacts, one that isn't already covered by one of the ZenML integrations, you can [extend the Artifact Store abstraction](custom.md) to provide your own Artifact Store implementation.
 {% endhint %}
 
 In addition to pipeline artifacts, the Artifact Store may also be used as storage backed by other specialized stack components that need to store their data in the form of persistent object storage. The [Great Expectations Data Validator](../data-validators/great-expectations.md) is such an example.
 
 Related concepts:
 
 * the Artifact Store is a type of Stack Component that needs to be registered as part of your ZenML [Stack](../../user-guide/production-guide/understand-stacks.md).
-* the objects circulated through your pipelines are serialized and stored in the Artifact Store using [Materializers](../../how-to/handle-data-artifacts/handle-custom-data-types.md). Materializers implement the logic required to serialize and deserialize the artifact contents and to store them and retrieve their contents to/from the Artifact Store.
+* the objects circulated through your pipelines are serialized and stored in the Artifact Store using [Materializer](../../how-to/data-artifact-management/handle-data-artifacts/handle-custom-data-types.md). Materializers implement the logic required to serialize and deserialize the artifact contents and to store them and retrieve their contents to/from the Artifact Store.
 
 ### When to use it
 
@@ -56,11 +57,11 @@ zenml artifact-store register s3_store -f s3 --path s3://my_bucket
 The Artifact Store provides low-level object storage services for other ZenML mechanisms. When you develop ZenML pipelines, you normally don't even have to be aware of its existence or interact with it directly. ZenML provides higher-level APIs that can be used as an alternative to store and access artifacts:
 
 * return one or more objects from your pipeline steps to have them automatically saved in the active Artifact Store as pipeline artifacts.
-* [retrieve pipeline artifacts](../../how-to/handle-data-artifacts/load-artifacts-into-memory.md) from the active Artifact Store after a pipeline run is complete.
+* [retrieve pipeline artifacts](../../how-to/data-artifact-management/handle-data-artifacts/load-artifacts-into-memory.md) from the active Artifact Store after a pipeline run is complete.
 
 You will probably need to interact with the [low-level Artifact Store API](artifact-stores.md#the-artifact-store-api) directly:
 
-* if you implement custom [Materializers](../../how-to/handle-data-artifacts/handle-custom-data-types.md) for your artifact data types
+* if you implement custom [Materializers](../../how-to/data-artifact-management/handle-data-artifacts/handle-custom-data-types.md) for your artifact data types
 * if you want to store custom objects in the Artifact Store
 
 #### The Artifact Store API
@@ -90,7 +91,7 @@ with fileio.open(artifact_uri, "w") as f:
     f.write(artifact_contents)
 ```
 
-When using the Artifact Store API to write custom Materializers, the base artifact URI path is already provided. See the documentation on [Materializers](../../how-to/handle-data-artifacts/handle-custom-data-types.md) for an example.
+When using the Artifact Store API to write custom Materializers, the base artifact URI path is already provided. See the documentation on [Materializers](../../how-to/data-artifact-management/handle-data-artifacts/handle-custom-data-types.md) for an example.
 {% endhint %}
 
 The following are some code examples showing how to use the Artifact Store API for various operations: