forked from opendatahub-io/notebooks
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request opendatahub-io#479 from atheo89/RHOAIENG-2922
Update Readme.md
- Loading branch information
Showing
1 changed file
with
61 additions
and
190 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,247 +1,118 @@ | ||
# Notebook Images | ||
# OpenDataHub Notebooks | ||
|
||
[![GitHub Tag](https://img.shields.io/github/v/tag/opendatahub-io/notebooks?style=plastic)](https://github.com/opendatahub-io/notebooks/releases) | ||
[![Docker Repository on Quay](https://quay.io/repository/opendatahub/workbench-images/status "Docker Repository on Quay")](https://quay.io/repository/opendatahub/workbench-images?tab=tags) | ||
|
||
These images were created to be used with Open Data Hub (ODH) with the ODH Notebook Controller as the launcher. | ||
|
||
## Container Image Layering | ||
|
||
The different notebooks images available are built in this order: | ||
|
||
**Based on Python 3.8** | ||
|
||
```mermaid | ||
graph TB | ||
subgraph Notebooks | ||
%% Nodes | ||
ubi8-python-3.8(UBI8 Python 3.8); | ||
base-ubi8-python-3.8("Notebooks Base<br/>(base-ubi8-python-3.8)"); | ||
jupyter-minimal-ubi8-python-3.8("Minimal Notebook<br/>(jupyter-minimal-ubi8-python-3.8)"); | ||
jupyter-datascience-ubi8-python-3.8("Data Science Notebook<br/>(jupyter-datascience-ubi8-python-3.8)"); | ||
jupyter-trustyai-ubi8-python-3.8("TrustyAI Notebook<br/>(jupyter-trustyai-ubi8-python-3.8)"); | ||
%% Edges | ||
ubi8-python-3.8 --> base-ubi8-python-3.8; | ||
base-ubi8-python-3.8 --> jupyter-minimal-ubi8-python-3.8; | ||
jupyter-minimal-ubi8-python-3.8 --> jupyter-datascience-ubi8-python-3.8; | ||
jupyter-datascience-ubi8-python-3.8 --> jupyter-pytorch-ubi8-python-3.8; | ||
end | ||
subgraph CUDA | ||
%% Nodes | ||
cuda-ubi8-python-3.8("CUDA Notebooks Base<br/>(cuda-ubi8-python-3.8)"); | ||
cuda-jupyter-minimal-ubi8-python-3.8("CUDA Minimal Notebook<br/>(cuda-jupyter-minimal-ubi8-python-3.8)"); | ||
cuda-jupyter-datascience-ubi8-python-3.8("CUDA Data Science Notebook<br/>(cuda-jupyter-datascience-ubi8-python-3.8)"); | ||
cuda-jupyter-pytorch-ubi8-python-3.8("PyTorch Notebook<br/>(cuda-jupyter-pytorch-ubi8-python-3.8)"); | ||
cuda-jupyter-tensorflow-ubi8-python-3.8("CUDA TensorFlow Notebook<br/>(cuda-jupyter-tensorflow-ubi8-python-3.8)"); | ||
%% Edges | ||
base-ubi8-python-3.8 --> cuda-ubi8-python-3.8; | ||
cuda-ubi8-python-3.8 --> cuda-jupyter-minimal-ubi8-python-3.8; | ||
cuda-jupyter-minimal-ubi8-python-3.8 --> cuda-jupyter-datascience-ubi8-python-3.8; | ||
cuda-jupyter-datascience-ubi8-python-3.8 --> cuda-jupyter-pytorch-ubi8-python-3.8; | ||
cuda-jupyter-datascience-ubi8-python-3.8 --> cuda-jupyter-tensorflow-ubi8-python-3.8; | ||
end | ||
subgraph Habana | ||
%% Nodes | ||
habana-jupyter-1.9.0-ubi8-python-3.8("HabanaAI Data Science Notebook<br/>(habana-jupyter-1.9.0-ubi8-python-3.8)"); | ||
habana-jupyter-1.10.0-ubi8-python-3.8("HabanaAI Data Science Notebook<br/>(habana-jupyter-1.10.0-ubi8-python-3.8)"); | ||
habana-jupyter-1.11.0-ubi8-python-3.8("HabanaAI Data Science Notebook<br/>(habana-jupyter-1.11.0-ubi8-python-3.8)"); | ||
%% Edges | ||
jupyter-datascience-ubi8-python-3.8 --> habana-jupyter-1.9.0-ubi8-python-3.8; | ||
jupyter-datascience-ubi8-python-3.8 --> habana-jupyter-1.10.0-ubi8-python-3.8; | ||
jupyter-datascience-ubi8-python-3.8 --> habana-jupyter-1.11.0-ubi8-python-3.8; | ||
end | ||
subgraph Runtimes | ||
%% Nodes | ||
runtimes-minimal-ubi8-python-3.8("Minimal Runtime<br/>(runtime-minimal-ubi8-python-3.8)"); | ||
runtimes-datascience-ubi8-python-3.8("Data Science Runtime<br/>(runtime-datascience-ubi8-python-3.8)"); | ||
runtimes-pytorch-ubi8-python-3.8("PyTorch Runtime<br/>(runtime-pytorch-ubi8-python-3.8)"); | ||
cuda-runtimes-tensorflow-ubi8-python-3.8("CUDA TensorFlow Runtime<br/>(cuda-runtime-tensorflow-ubi8-python-3.8)"); | ||
%% Edges | ||
base-ubi8-python-3.8 --> runtimes-minimal-ubi8-python-3.8; | ||
base-ubi8-python-3.8 --> runtimes-datascience-ubi8-python-3.8; | ||
base-ubi8-python-3.8 --> runtimes-pytorch-ubi8-python-3.8; | ||
cuda-ubi8-python-3.8 --> cuda-runtimes-tensorflow-ubi8-python-3.8; | ||
end | ||
``` | ||
![GitHub License](https://img.shields.io/github/license/opendatahub-io/notebooks) | ||
|
||
**Based on Python 3.9** | ||
|
||
```mermaid | ||
graph TB | ||
subgraph Notebooks | ||
%% Nodes | ||
ubi9-python-3.9(UBI9 Python 3.9); | ||
base-ubi9-python-3.9("Notebooks Base<br/>(base-ubi9-python-3.9)"); | ||
jupyter-minimal-ubi9-python-3.9("Minimal Notebook<br/>(jupyter-minimal-ubi9-python-3.9)"); | ||
jupyter-datascience-ubi9-python-3.9("Data Science Notebook<br/>(jupyter-datascience-ubi9-python-3.9)"); | ||
jupyter-trustyai-ubi9-python-3.9("TrustyAI Notebook<br/>(jupyter-trustyai-ubi9-python-3.9)"); | ||
%% Edges | ||
ubi9-python-3.9 --> base-ubi9-python-3.9; | ||
base-ubi9-python-3.9 --> jupyter-minimal-ubi9-python-3.9; | ||
jupyter-minimal-ubi9-python-3.9 --> jupyter-datascience-ubi9-python-3.9; | ||
jupyter-datascience-ubi9-python-3.9 --> jupyter-trustyai-ubi9-python-3.9; | ||
end | ||
subgraph CUDA | ||
%% Nodes | ||
cuda-ubi9-python-3.9("CUDA Notebooks Base<br/>(cuda-ubi9-python-3.9)"); | ||
cuda-jupyter-minimal-ubi9-python-3.9("CUDA Minimal Notebook<br/>(cuda-jupyter-minimal-ubi9-python-3.9)"); | ||
cuda-jupyter-datascience-ubi9-python-3.9("CUDA Data Science Notebook<br/>(cuda-jupyter-datascience-ubi9-python-3.9)"); | ||
cuda-jupyter-pytorch-ubi9-python-3.9("CUDA PyTorch Notebook<br/>(cuda-jupyter-pytorch-ubi9-python-3.9)"); | ||
cuda-jupyter-tensorflow-ubi9-python-3.9("CUDA TensorFlow Notebook<br/>(cuda-jupyter-tensorflow-ubi9-python-3.9)"); | ||
%% Edges | ||
base-ubi9-python-3.9 --> cuda-ubi9-python-3.9; | ||
cuda-ubi9-python-3.9 --> cuda-jupyter-minimal-ubi9-python-3.9; | ||
cuda-jupyter-minimal-ubi9-python-3.9 --> cuda-jupyter-datascience-ubi9-python-3.9; | ||
cuda-jupyter-datascience-ubi9-python-3.9 --> cuda-jupyter-pytorch-ubi9-python-3.9; | ||
cuda-jupyter-datascience-ubi9-python-3.9 --> cuda-jupyter-tensorflow-ubi9-python-3.9; | ||
end | ||
subgraph Runtimes | ||
%% Nodes | ||
runtimes-minimal-ubi9-python-3.9("Minimal Runtime<br/>(runtime-minimal-ubi9-python-3.9)"); | ||
runtimes-datascience-ubi9-python-3.9("Data Science Runtime<br/>(runtime-datascience-ubi9-python-3.9)"); | ||
runtimes-pytorch-ubi9-python-3.9("PyTorch Runtime<br/>(runtime-pytorch-ubi9-python-3.9)"); | ||
cuda-runtimes-tensorflow-ubi9-python-3.9("CUDA TensorFlow Runtime<br/>(cuda-runtime-tensorflow-ubi9-python-3.9)"); | ||
%% Edges | ||
base-ubi9-python-3.9 --> runtimes-minimal-ubi9-python-3.9; | ||
base-ubi9-python-3.9 --> runtimes-datascience-ubi9-python-3.9; | ||
base-ubi9-python-3.9 --> runtimes-pytorch-ubi9-python-3.9; | ||
cuda-ubi9-python-3.9 --> cuda-runtimes-tensorflow-ubi9-python-3.9; | ||
end | ||
subgraph Other Notebooks | ||
%% Nodes | ||
c9s-python-3.9("CentOS Stream Base<br/>(c9s-python-3.9)"); | ||
code-server-c9s-python-3.9("code-server <br/>(code-server-c9s-python-3.9)"); | ||
r-studio-c9s-python-3.9("RStudio Server<br/>(r-studio-c9s-python-3.9)"); | ||
%% Edges | ||
c9s-python-3.9 --> code-server-c9s-python-3.9; | ||
c9s-python-3.9 --> r-studio-c9s-python-3.9; | ||
end | ||
|
||
``` | ||
Welcome to the OpenDataHub Notebooks repository! This repository provides a collection of notebooks tailored for data analysis, machine learning, research and coding within the OpenDataHub ecosystem. Designed to streamline data science workflows, these notebooks offer an integrated environment equipped with the latest tools and libraries. These notebooks were created to be used with OpenDataHub ecosystem with the ODH Notebook Controller as the launcher. | ||
|
||
These workbenches are available at: [quay.io/opendatahub/workbench-images](https://quay.io/opendatahub/workbench-images) | ||
|
||
## Getting Started | ||
For a deeper understanding of the architecture underlying this repository, please explore our wiki page [Wiki](https://github.com/opendatahub-io/notebooks/wiki/Workbenches) | ||
|
||
### Prerequisites | ||
Make sure the following tools are installed in your environment: | ||
- podman/docker | ||
- python | ||
- pipenv | ||
- make | ||
- curl | ||
|
||
## Building | ||
|
||
The following workbench images are available: | ||
|
||
### Python 3.8 | ||
- jupyter-minimal-ubi8-python-3.8 | ||
- jupyter-datascience-ubi8-python-3.8 | ||
- jupyter-pytorch-ubi8-python-3.8 | ||
- jupyter-trustyai-ubi8-python-3.8 | ||
- cuda-jupyter-minimal-ubi8-python-3.8 | ||
- cuda-jupyter-datascience-ubi8-python-3.8 | ||
- cuda-jupyter-tensorflow-ubi8-python-3.8 | ||
- habana-jupyter-1.9.0-ubi8-python-3.8 | ||
- habana-jupyter-1.10.0-ubi8-python-3.8 | ||
- habana-jupyter-1.11.0-ubi8-python-3.8 | ||
- runtime-minimal-ubi8-python-3.8 | ||
- runtime-datascience-ubi8-python-3.8 | ||
- runtime-pytorch-ubi8-python-3.8 | ||
- cuda-runtime-tensorflow-ubi8-python-3.8 | ||
|
||
### Python 3.9 | ||
- jupyter-minimal-ubi9-python-3.9 | ||
- jupyter-datascience-ubi9-python-3.9 | ||
- jupyter-pytorch-ubi9-python-3.9 | ||
- jupyter-trustyai-ubi9-python-3.9 | ||
- cuda-jupyter-minimal-ubi9-python-3.9 | ||
- cuda-jupyter-datascience-ubi9-python-3.9 | ||
- cuda-jupyter-tensorflow-ubi9-python-3.9 | ||
- runtime-minimal-ubi9-python-3.9 | ||
- runtime-datascience-ubi9-python-3.9 | ||
- runtime-pytorch-ubi9-python-3.9 | ||
- cuda-runtime-tensorflow-ubi9-python-3.9 | ||
- code-server-c9s-python-3.9 | ||
- r-studio-c9s-python-3.9; | ||
|
||
If you want to manually build a workbench image, you can use the following | ||
command: | ||
### Installation | ||
Clone this repository to your local machine: | ||
|
||
```shell | ||
make ${WORKBENCH_NAME} | ||
git clone https://github.com/opendatahub-io/notebooks.git | ||
cd notebooks | ||
``` | ||
|
||
The image will be built and pushed to the | ||
[quay.io/opendatahub/workbench-images](https://quay.io/opendatahub/workbench-images) | ||
repository. | ||
### Quick Start Guide | ||
|
||
You can overwrite `IMAGE_REGISTRY` and `RELEASE` variables to use a different registry or release tag: | ||
### Build a Notebook | ||
|
||
To build a workbench image, you can execute the following command: | ||
|
||
```shell | ||
make ${WORKBENCH_NAME} -e IMAGE_REGISTRY=quay.io/${YOUR_USER}/workbench-images -e RELEASE=2023x | ||
make ${WORKBENCH_NAME} -e IMAGE_REGISTRY=quay.io/${YOUR_USER}/workbench-images -e RELEASE=2023x | ||
``` | ||
|
||
## Local Execution | ||
Using `IMAGE_REGISTRY` and `RELEASE` variables you can overwrite the default values and use a different registry or release tag | ||
|
||
### Local Execution | ||
|
||
The notebook can be run as container on the local systems. | ||
|
||
Use podman/docker to execute the workbench images as container. | ||
|
||
```shell | ||
podman run -p 8888:8888 quay.io/opendatahub/workbench-images:jupyter-minimal-ubi8-python-3.8-20230808 | ||
podman run -it -p 8888:8888 quay.io/opendatahub/workbench-images:jupyter-minimal-ubi9-python-3.9-2024a-20240317-6f4c36b | ||
``` | ||
|
||
## Testing Notebooks | ||
### Deploy & Test | ||
|
||
#### Notebooks | ||
|
||
Deploy the notebook images in your Kubernetes environment using deploy8-${NOTEBOOK_NAME} for ubi8 or deploy9-${NOTEBOOK_NAME} for ubi9: | ||
Deploy the notebook images in your Kubernetes environment using: | ||
`deploy8-${NOTEBOOK_NAME} for ubi8 or deploy9-${NOTEBOOK_NAME} for ubi9` | ||
|
||
```shell | ||
make deployX-${NOTEBOOK_NAME} | ||
make deployX-${NOTEBOOK_NAME} | ||
``` | ||
|
||
Run the test suite against this notebook: | ||
|
||
```shell | ||
make test-${NOTEBOOK_NAME} | ||
make test-${NOTEBOOK_NAME} | ||
``` | ||
|
||
You can overwrite `NOTEBOOK_REPO_BRANCH_BASE` variable to use a different repository and branch for testing scripts. This is useful when you debug your changes. | ||
|
||
|
||
```shell | ||
make test-${NOTEBOOK_NAME} -e NOTEBOOK_REPO_BRANCH_BASE="https://raw.githubusercontent.com/${YOUR_USER}/notebooks/${YOUR_BRANCH}" | ||
make test-${NOTEBOOK_NAME} -e NOTEBOOK_REPO_BRANCH_BASE="https://raw.githubusercontent.com/${YOUR_USER}/notebooks/${YOUR_BRANCH}" | ||
``` | ||
|
||
Clean up the environment when the tests are finished: | ||
|
||
```shell | ||
make undeployX-${NOTEBOOK_NAME} | ||
make undeployX-${NOTEBOOK_NAME} | ||
``` | ||
|
||
## Validating Runtimes | ||
#### Runtimes | ||
|
||
The runtimes image requires to have curl and python installed, | ||
so that on runtime additional packages can be installed. | ||
The runtimes image requires to have curl and python installed, so that on runtime additional packages can be installed. | ||
|
||
Deploy the runtime images in your Kubernetes environment using deploy8-${WORKBENCH_NAME} for ubi8 or deploy9-${WORKBENCH_NAME} for ubi9: | ||
Deploy the runtime images in your Kubernetes environment using: `deploy8-${WORKBENCH_NAME} for ubi8 or deploy9-${WORKBENCH_NAME} for ubi9` | ||
|
||
```shell | ||
make deployX-${WORKBENCH_NAME} | ||
make deployX-${WORKBENCH_NAME} | ||
``` | ||
|
||
Run the validate test suit for checking compatabilty of runtime images: | ||
|
||
```shell | ||
make validate-runtime-image image=<runtime-image> | ||
make validate-runtime-image image=<runtime-image> | ||
``` | ||
|
||
Clean up the environment when the tests are finished: | ||
|
||
```shell | ||
make undeployX-${WORKBENCH_NAME} | ||
make undeployX-${WORKBENCH_NAME} | ||
``` | ||
|
||
## Contributions | ||
Please refer [CONTRIBUTING.md](CONTRIBUTING.md) file. | ||
## Contributing | ||
|
||
Whether you're fixing bugs, adding new notebooks, or improving documentation, your contributions are welcome. Please refer to our [Contribution Guidlines](CONTRIBUTING.md). | ||
|
||
## Acknowledgments | ||
|
||
A huge thank you to all our contributors and the broader OpenDataHub community! | ||
|
||
## License | ||
|
||
This project is licensed under the Apache License 2.0 - see the [LICENSE](https://github.com/opendatahub-io/notebooks/blob/main/LICENSE) file for details. | ||
|
||
## Contact | ||
|
||
Anything unclear or inaccurate? Please let us know by reporting an issue: [notebooks/issues](https://github.com/opendatahub-io/notebooks/issues/new) |