Skip to content

Commit

Permalink
Merge pull request opendatahub-io#479 from atheo89/RHOAIENG-2922
Browse files Browse the repository at this point in the history
Update Readme.md
  • Loading branch information
ide-developer authored Mar 27, 2024
2 parents aefe060 + 724c6db commit e66dc95
Showing 1 changed file with 61 additions and 190 deletions.
251 changes: 61 additions & 190 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,247 +1,118 @@
# Notebook Images
# OpenDataHub Notebooks

[![GitHub Tag](https://img.shields.io/github/v/tag/opendatahub-io/notebooks?style=plastic)](https://github.com/opendatahub-io/notebooks/releases)
[![Docker Repository on Quay](https://quay.io/repository/opendatahub/workbench-images/status "Docker Repository on Quay")](https://quay.io/repository/opendatahub/workbench-images?tab=tags)

These images were created to be used with Open Data Hub (ODH) with the ODH Notebook Controller as the launcher.

## Container Image Layering

The different notebooks images available are built in this order:

**Based on Python 3.8**

```mermaid
graph TB
subgraph Notebooks
%% Nodes
ubi8-python-3.8(UBI8 Python 3.8);
base-ubi8-python-3.8("Notebooks Base<br/>(base-ubi8-python-3.8)");
jupyter-minimal-ubi8-python-3.8("Minimal Notebook<br/>(jupyter-minimal-ubi8-python-3.8)");
jupyter-datascience-ubi8-python-3.8("Data Science Notebook<br/>(jupyter-datascience-ubi8-python-3.8)");
jupyter-trustyai-ubi8-python-3.8("TrustyAI Notebook<br/>(jupyter-trustyai-ubi8-python-3.8)");
%% Edges
ubi8-python-3.8 --> base-ubi8-python-3.8;
base-ubi8-python-3.8 --> jupyter-minimal-ubi8-python-3.8;
jupyter-minimal-ubi8-python-3.8 --> jupyter-datascience-ubi8-python-3.8;
jupyter-datascience-ubi8-python-3.8 --> jupyter-pytorch-ubi8-python-3.8;
end
subgraph CUDA
%% Nodes
cuda-ubi8-python-3.8("CUDA Notebooks Base<br/>(cuda-ubi8-python-3.8)");
cuda-jupyter-minimal-ubi8-python-3.8("CUDA Minimal Notebook<br/>(cuda-jupyter-minimal-ubi8-python-3.8)");
cuda-jupyter-datascience-ubi8-python-3.8("CUDA Data Science Notebook<br/>(cuda-jupyter-datascience-ubi8-python-3.8)");
cuda-jupyter-pytorch-ubi8-python-3.8("PyTorch Notebook<br/>(cuda-jupyter-pytorch-ubi8-python-3.8)");
cuda-jupyter-tensorflow-ubi8-python-3.8("CUDA TensorFlow Notebook<br/>(cuda-jupyter-tensorflow-ubi8-python-3.8)");
%% Edges
base-ubi8-python-3.8 --> cuda-ubi8-python-3.8;
cuda-ubi8-python-3.8 --> cuda-jupyter-minimal-ubi8-python-3.8;
cuda-jupyter-minimal-ubi8-python-3.8 --> cuda-jupyter-datascience-ubi8-python-3.8;
cuda-jupyter-datascience-ubi8-python-3.8 --> cuda-jupyter-pytorch-ubi8-python-3.8;
cuda-jupyter-datascience-ubi8-python-3.8 --> cuda-jupyter-tensorflow-ubi8-python-3.8;
end
subgraph Habana
%% Nodes
habana-jupyter-1.9.0-ubi8-python-3.8("HabanaAI Data Science Notebook<br/>(habana-jupyter-1.9.0-ubi8-python-3.8)");
habana-jupyter-1.10.0-ubi8-python-3.8("HabanaAI Data Science Notebook<br/>(habana-jupyter-1.10.0-ubi8-python-3.8)");
habana-jupyter-1.11.0-ubi8-python-3.8("HabanaAI Data Science Notebook<br/>(habana-jupyter-1.11.0-ubi8-python-3.8)");
%% Edges
jupyter-datascience-ubi8-python-3.8 --> habana-jupyter-1.9.0-ubi8-python-3.8;
jupyter-datascience-ubi8-python-3.8 --> habana-jupyter-1.10.0-ubi8-python-3.8;
jupyter-datascience-ubi8-python-3.8 --> habana-jupyter-1.11.0-ubi8-python-3.8;
end
subgraph Runtimes
%% Nodes
runtimes-minimal-ubi8-python-3.8("Minimal Runtime<br/>(runtime-minimal-ubi8-python-3.8)");
runtimes-datascience-ubi8-python-3.8("Data Science Runtime<br/>(runtime-datascience-ubi8-python-3.8)");
runtimes-pytorch-ubi8-python-3.8("PyTorch Runtime<br/>(runtime-pytorch-ubi8-python-3.8)");
cuda-runtimes-tensorflow-ubi8-python-3.8("CUDA TensorFlow Runtime<br/>(cuda-runtime-tensorflow-ubi8-python-3.8)");
%% Edges
base-ubi8-python-3.8 --> runtimes-minimal-ubi8-python-3.8;
base-ubi8-python-3.8 --> runtimes-datascience-ubi8-python-3.8;
base-ubi8-python-3.8 --> runtimes-pytorch-ubi8-python-3.8;
cuda-ubi8-python-3.8 --> cuda-runtimes-tensorflow-ubi8-python-3.8;
end
```
![GitHub License](https://img.shields.io/github/license/opendatahub-io/notebooks)

**Based on Python 3.9**

```mermaid
graph TB
subgraph Notebooks
%% Nodes
ubi9-python-3.9(UBI9 Python 3.9);
base-ubi9-python-3.9("Notebooks Base<br/>(base-ubi9-python-3.9)");
jupyter-minimal-ubi9-python-3.9("Minimal Notebook<br/>(jupyter-minimal-ubi9-python-3.9)");
jupyter-datascience-ubi9-python-3.9("Data Science Notebook<br/>(jupyter-datascience-ubi9-python-3.9)");
jupyter-trustyai-ubi9-python-3.9("TrustyAI Notebook<br/>(jupyter-trustyai-ubi9-python-3.9)");
%% Edges
ubi9-python-3.9 --> base-ubi9-python-3.9;
base-ubi9-python-3.9 --> jupyter-minimal-ubi9-python-3.9;
jupyter-minimal-ubi9-python-3.9 --> jupyter-datascience-ubi9-python-3.9;
jupyter-datascience-ubi9-python-3.9 --> jupyter-trustyai-ubi9-python-3.9;
end
subgraph CUDA
%% Nodes
cuda-ubi9-python-3.9("CUDA Notebooks Base<br/>(cuda-ubi9-python-3.9)");
cuda-jupyter-minimal-ubi9-python-3.9("CUDA Minimal Notebook<br/>(cuda-jupyter-minimal-ubi9-python-3.9)");
cuda-jupyter-datascience-ubi9-python-3.9("CUDA Data Science Notebook<br/>(cuda-jupyter-datascience-ubi9-python-3.9)");
cuda-jupyter-pytorch-ubi9-python-3.9("CUDA PyTorch Notebook<br/>(cuda-jupyter-pytorch-ubi9-python-3.9)");
cuda-jupyter-tensorflow-ubi9-python-3.9("CUDA TensorFlow Notebook<br/>(cuda-jupyter-tensorflow-ubi9-python-3.9)");
%% Edges
base-ubi9-python-3.9 --> cuda-ubi9-python-3.9;
cuda-ubi9-python-3.9 --> cuda-jupyter-minimal-ubi9-python-3.9;
cuda-jupyter-minimal-ubi9-python-3.9 --> cuda-jupyter-datascience-ubi9-python-3.9;
cuda-jupyter-datascience-ubi9-python-3.9 --> cuda-jupyter-pytorch-ubi9-python-3.9;
cuda-jupyter-datascience-ubi9-python-3.9 --> cuda-jupyter-tensorflow-ubi9-python-3.9;
end
subgraph Runtimes
%% Nodes
runtimes-minimal-ubi9-python-3.9("Minimal Runtime<br/>(runtime-minimal-ubi9-python-3.9)");
runtimes-datascience-ubi9-python-3.9("Data Science Runtime<br/>(runtime-datascience-ubi9-python-3.9)");
runtimes-pytorch-ubi9-python-3.9("PyTorch Runtime<br/>(runtime-pytorch-ubi9-python-3.9)");
cuda-runtimes-tensorflow-ubi9-python-3.9("CUDA TensorFlow Runtime<br/>(cuda-runtime-tensorflow-ubi9-python-3.9)");
%% Edges
base-ubi9-python-3.9 --> runtimes-minimal-ubi9-python-3.9;
base-ubi9-python-3.9 --> runtimes-datascience-ubi9-python-3.9;
base-ubi9-python-3.9 --> runtimes-pytorch-ubi9-python-3.9;
cuda-ubi9-python-3.9 --> cuda-runtimes-tensorflow-ubi9-python-3.9;
end
subgraph Other Notebooks
%% Nodes
c9s-python-3.9("CentOS Stream Base<br/>(c9s-python-3.9)");
code-server-c9s-python-3.9("code-server <br/>(code-server-c9s-python-3.9)");
r-studio-c9s-python-3.9("RStudio Server<br/>(r-studio-c9s-python-3.9)");
%% Edges
c9s-python-3.9 --> code-server-c9s-python-3.9;
c9s-python-3.9 --> r-studio-c9s-python-3.9;
end

```
Welcome to the OpenDataHub Notebooks repository! This repository provides a collection of notebooks tailored for data analysis, machine learning, research and coding within the OpenDataHub ecosystem. Designed to streamline data science workflows, these notebooks offer an integrated environment equipped with the latest tools and libraries. These notebooks were created to be used with OpenDataHub ecosystem with the ODH Notebook Controller as the launcher.

These workbenches are available at: [quay.io/opendatahub/workbench-images](https://quay.io/opendatahub/workbench-images)

## Getting Started
For a deeper understanding of the architecture underlying this repository, please explore our wiki page [Wiki](https://github.com/opendatahub-io/notebooks/wiki/Workbenches)

### Prerequisites
Make sure the following tools are installed in your environment:
- podman/docker
- python
- pipenv
- make
- curl

## Building

The following workbench images are available:

### Python 3.8
- jupyter-minimal-ubi8-python-3.8
- jupyter-datascience-ubi8-python-3.8
- jupyter-pytorch-ubi8-python-3.8
- jupyter-trustyai-ubi8-python-3.8
- cuda-jupyter-minimal-ubi8-python-3.8
- cuda-jupyter-datascience-ubi8-python-3.8
- cuda-jupyter-tensorflow-ubi8-python-3.8
- habana-jupyter-1.9.0-ubi8-python-3.8
- habana-jupyter-1.10.0-ubi8-python-3.8
- habana-jupyter-1.11.0-ubi8-python-3.8
- runtime-minimal-ubi8-python-3.8
- runtime-datascience-ubi8-python-3.8
- runtime-pytorch-ubi8-python-3.8
- cuda-runtime-tensorflow-ubi8-python-3.8

### Python 3.9
- jupyter-minimal-ubi9-python-3.9
- jupyter-datascience-ubi9-python-3.9
- jupyter-pytorch-ubi9-python-3.9
- jupyter-trustyai-ubi9-python-3.9
- cuda-jupyter-minimal-ubi9-python-3.9
- cuda-jupyter-datascience-ubi9-python-3.9
- cuda-jupyter-tensorflow-ubi9-python-3.9
- runtime-minimal-ubi9-python-3.9
- runtime-datascience-ubi9-python-3.9
- runtime-pytorch-ubi9-python-3.9
- cuda-runtime-tensorflow-ubi9-python-3.9
- code-server-c9s-python-3.9
- r-studio-c9s-python-3.9;

If you want to manually build a workbench image, you can use the following
command:
### Installation
Clone this repository to your local machine:

```shell
make ${WORKBENCH_NAME}
git clone https://github.com/opendatahub-io/notebooks.git
cd notebooks
```

The image will be built and pushed to the
[quay.io/opendatahub/workbench-images](https://quay.io/opendatahub/workbench-images)
repository.
### Quick Start Guide

You can overwrite `IMAGE_REGISTRY` and `RELEASE` variables to use a different registry or release tag:
### Build a Notebook

To build a workbench image, you can execute the following command:

```shell
make ${WORKBENCH_NAME} -e IMAGE_REGISTRY=quay.io/${YOUR_USER}/workbench-images -e RELEASE=2023x
make ${WORKBENCH_NAME} -e IMAGE_REGISTRY=quay.io/${YOUR_USER}/workbench-images -e RELEASE=2023x
```

## Local Execution
Using `IMAGE_REGISTRY` and `RELEASE` variables you can overwrite the default values and use a different registry or release tag

### Local Execution

The notebook can be run as container on the local systems.

Use podman/docker to execute the workbench images as container.

```shell
podman run -p 8888:8888 quay.io/opendatahub/workbench-images:jupyter-minimal-ubi8-python-3.8-20230808
podman run -it -p 8888:8888 quay.io/opendatahub/workbench-images:jupyter-minimal-ubi9-python-3.9-2024a-20240317-6f4c36b
```

## Testing Notebooks
### Deploy & Test

#### Notebooks

Deploy the notebook images in your Kubernetes environment using deploy8-${NOTEBOOK_NAME} for ubi8 or deploy9-${NOTEBOOK_NAME} for ubi9:
Deploy the notebook images in your Kubernetes environment using:
`deploy8-${NOTEBOOK_NAME} for ubi8 or deploy9-${NOTEBOOK_NAME} for ubi9`

```shell
make deployX-${NOTEBOOK_NAME}
make deployX-${NOTEBOOK_NAME}
```

Run the test suite against this notebook:

```shell
make test-${NOTEBOOK_NAME}
make test-${NOTEBOOK_NAME}
```

You can overwrite `NOTEBOOK_REPO_BRANCH_BASE` variable to use a different repository and branch for testing scripts. This is useful when you debug your changes.


```shell
make test-${NOTEBOOK_NAME} -e NOTEBOOK_REPO_BRANCH_BASE="https://raw.githubusercontent.com/${YOUR_USER}/notebooks/${YOUR_BRANCH}"
make test-${NOTEBOOK_NAME} -e NOTEBOOK_REPO_BRANCH_BASE="https://raw.githubusercontent.com/${YOUR_USER}/notebooks/${YOUR_BRANCH}"
```

Clean up the environment when the tests are finished:

```shell
make undeployX-${NOTEBOOK_NAME}
make undeployX-${NOTEBOOK_NAME}
```

## Validating Runtimes
#### Runtimes

The runtimes image requires to have curl and python installed,
so that on runtime additional packages can be installed.
The runtimes image requires to have curl and python installed, so that on runtime additional packages can be installed.

Deploy the runtime images in your Kubernetes environment using deploy8-${WORKBENCH_NAME} for ubi8 or deploy9-${WORKBENCH_NAME} for ubi9:
Deploy the runtime images in your Kubernetes environment using: `deploy8-${WORKBENCH_NAME} for ubi8 or deploy9-${WORKBENCH_NAME} for ubi9`

```shell
make deployX-${WORKBENCH_NAME}
make deployX-${WORKBENCH_NAME}
```

Run the validate test suit for checking compatabilty of runtime images:

```shell
make validate-runtime-image image=<runtime-image>
make validate-runtime-image image=<runtime-image>
```

Clean up the environment when the tests are finished:

```shell
make undeployX-${WORKBENCH_NAME}
make undeployX-${WORKBENCH_NAME}
```

## Contributions
Please refer [CONTRIBUTING.md](CONTRIBUTING.md) file.
## Contributing

Whether you're fixing bugs, adding new notebooks, or improving documentation, your contributions are welcome. Please refer to our [Contribution Guidlines](CONTRIBUTING.md).

## Acknowledgments

A huge thank you to all our contributors and the broader OpenDataHub community!

## License

This project is licensed under the Apache License 2.0 - see the [LICENSE](https://github.com/opendatahub-io/notebooks/blob/main/LICENSE) file for details.

## Contact

Anything unclear or inaccurate? Please let us know by reporting an issue: [notebooks/issues](https://github.com/opendatahub-io/notebooks/issues/new)

0 comments on commit e66dc95

Please sign in to comment.