From 1b9d18de316a5aac26e0e0e07f4db87ff2fd7625 Mon Sep 17 00:00:00 2001 From: Tom Corcoran Date: Tue, 11 Jun 2024 11:00:42 +1000 Subject: [PATCH 01/13] Continued outline for scenario 3 --- data/hackathon/scenario1.mdx | 2 +- data/hackathon/scenario3.mdx | 35 +++++++++++++++++++++++++++++++++-- 2 files changed, 34 insertions(+), 3 deletions(-) diff --git a/data/hackathon/scenario1.mdx b/data/hackathon/scenario1.mdx index 9d910ab..dec26bf 100644 --- a/data/hackathon/scenario1.mdx +++ b/data/hackathon/scenario1.mdx @@ -37,7 +37,7 @@ For this challenge you'll be given two OpenShift clusters - a Single Node OpenShift cluster representing the ACME On Premises environment All challenge tasks must be performed on these clusters so your solutions can be graded successfully. -Some challenges wil require the use of specific clusters +Some challenges will require the use of specific clusters You can and are encouraged to use any supporting documentation or other resources in order to tackle each of the challenge tasks. diff --git a/data/hackathon/scenario3.mdx b/data/hackathon/scenario3.mdx index ba5041d..717c76b 100644 --- a/data/hackathon/scenario3.mdx +++ b/data/hackathon/scenario3.mdx @@ -34,13 +34,44 @@ Install the following opertors (do not install any custom resources) - OpenShift Service Mesh - OpenShift Serverless -## 3.2 - Install Openshift AI +## 3.2 - Install OpenShift AI +Wait until the three operators specified in the previoius section have fully provisioned, before proceeding. +You won't need any Custom Resources for OpenShift Service Mesh and OpenShift Serverless +You will need one for OpenShift AI. A valid strategy would be to open the yaml view and go with all the defaults - the only addition to be to add this knative-serving-cert secret ingressGateway: certificate: - secretName: knative-serving-cert + `secretName: knative-serving-cert` + +## 3.3 - Use OpenShift AI to push Granite model from On Prem object storage to cloud + +Examine your On Prem object storage (Minio). The Granite model files inside the *models* bucket represent an LLM that may have been +- downloaded as is +- downloaded and then fine tuned. +As it's inside object storage, it's ready to push to cloud. + +Now open OpenShift AI and do the following +- create a project +- create a workbench that + - uses Pytorch as a basis + - uses a Persistent Volume of at least 60GB + - uses a Data Connection to your Minio object storage +- for this task, use a + +HINTS +[1] Use this notebook to pull from local object storage and push to cloud object storage + https://github.com/tnscorcoran/rhods-finetunning-demo/blob/main/minio_pull_from_and_push_to.ipynb + TODO - Code the notebook and move it to the correct git repo + (note in a production environment, this would likely be automed using Gitops) + TODO - confirm Gitops would be used + Documentation you may find helpful is: - https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html/installing_and_uninstalling_openshift_ai_self-managed/index +- https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html/getting_started_with_red_hat_openshift_ai_self-managed/creating-a-data-science-project_get-started + +- https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html/getting_started_with_red_hat_openshift_ai_self-managed/creating-a-project-workbench_get-started + + From 91cb5ff1e11f32268ed708c654ccb096d5d5b8b5 Mon Sep 17 00:00:00 2001 From: Tom Corcoran Date: Wed, 12 Jun 2024 09:37:30 +1000 Subject: [PATCH 02/13] Updates to scenarios 3 and 4 --- data/hackathon/scenario3.mdx | 8 +++++++- data/hackathon/scenario4.mdx | 17 +++++++++++++++++ 2 files changed, 24 insertions(+), 1 deletion(-) create mode 100644 data/hackathon/scenario4.mdx diff --git a/data/hackathon/scenario3.mdx b/data/hackathon/scenario3.mdx index 717c76b..8c34e87 100644 --- a/data/hackathon/scenario3.mdx +++ b/data/hackathon/scenario3.mdx @@ -58,7 +58,13 @@ Now open OpenShift AI and do the following - uses Pytorch as a basis - uses a Persistent Volume of at least 60GB - uses a Data Connection to your Minio object storage -- for this task, use a + - uses a Medium sized Container without an accelerator +- for this task, use a Jupyter notebook that pulls the contents of your source bucket to your target bucket on the cloud (Hint available below) + + +## 3.4 - Use your cloud-based OpenShift AI to Serve the model and make it easily consumable by intelligent applications for inference + + HINTS [1] Use this notebook to pull from local object storage and push to cloud object storage diff --git a/data/hackathon/scenario4.mdx b/data/hackathon/scenario4.mdx new file mode 100644 index 0000000..c3266da --- /dev/null +++ b/data/hackathon/scenario4.mdx @@ -0,0 +1,17 @@ +--- +title: Open Innovation - Integrating a new piece of tooling into OpenShift AI Workspace +exercise: 3 +date: '2024-06-05' +tags: ['openshift','ai','kubernetes'] +draft: false +authors: ['default'] +summary: "Let's add some tooling in satndardised way available to all data science users" +--- + +As a sales team you've got an upcoming demo with the Acme Financial Services data science team, who have been training models on their laptops. +Their data scientists are given directives on what tools to use - but in reality, what each one uses has drifted away from what's been directed, resulting in inconsistencies in tooling used across users and more importantly between deveopment and production environments +You are required to show a solution for this problem on OpenShift AI. + +## 4.1 - Test a new library inside a Jupyter Notebook + +For this task, your team are required to \ No newline at end of file From 1751b82b8056b7a9ff3f26e4897666f86ed742d9 Mon Sep 17 00:00:00 2001 From: Tom Corcoran Date: Wed, 12 Jun 2024 09:37:57 +1000 Subject: [PATCH 03/13] Updates to scenarios 3 and 4 --- data/hackathon/scenario3.mdx | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/data/hackathon/scenario3.mdx b/data/hackathon/scenario3.mdx index 8c34e87..85d187f 100644 --- a/data/hackathon/scenario3.mdx +++ b/data/hackathon/scenario3.mdx @@ -64,7 +64,8 @@ Now open OpenShift AI and do the following ## 3.4 - Use your cloud-based OpenShift AI to Serve the model and make it easily consumable by intelligent applications for inference - +First Import a VLLM Server +This is a popular model server format that is co HINTS [1] Use this notebook to pull from local object storage and push to cloud object storage From 9bc0570cb26c5056c5c1c3fa67d6956ccb2dd62f Mon Sep 17 00:00:00 2001 From: Tom Corcoran Date: Thu, 13 Jun 2024 08:45:11 +1000 Subject: [PATCH 04/13] Code for scenario 3 --- data/hackathon/scenario3.mdx | 6 +- .../minio_pull_from_and_push_to.ipynb | 226 ++++++++++++++++++ 2 files changed, 230 insertions(+), 2 deletions(-) create mode 100644 temp/scenario3_hybrid_cloud/minio_pull_from_and_push_to.ipynb diff --git a/data/hackathon/scenario3.mdx b/data/hackathon/scenario3.mdx index 85d187f..3aa5bb5 100644 --- a/data/hackathon/scenario3.mdx +++ b/data/hackathon/scenario3.mdx @@ -64,8 +64,10 @@ Now open OpenShift AI and do the following ## 3.4 - Use your cloud-based OpenShift AI to Serve the model and make it easily consumable by intelligent applications for inference -First Import a VLLM Server -This is a popular model server format that is co +### 3.4.1 - Import a VLLM Server +VLLM is a popular model server format whose APIs are compatible with Open AI (Chat GPT) APIs. This format then lends itself to easy migration of apps already using Open AI - to OpenShift AI. + + HINTS [1] Use this notebook to pull from local object storage and push to cloud object storage diff --git a/temp/scenario3_hybrid_cloud/minio_pull_from_and_push_to.ipynb b/temp/scenario3_hybrid_cloud/minio_pull_from_and_push_to.ipynb new file mode 100644 index 0000000..757f870 --- /dev/null +++ b/temp/scenario3_hybrid_cloud/minio_pull_from_and_push_to.ipynb @@ -0,0 +1,226 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "e0a7a17e", + "metadata": {}, + "source": [ + "# Run this to push vLLM models to Object Storage" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "79714be6-696a-4e62-8c99-ae9bc2ef451a", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "import os\n", + "import boto3" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "37789ea0-8729-4bf5-8ae5-e976fddfc800", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "minio\n", + "minio123\n", + "none\n", + "https://minio-api-minio.apps.rosa-9s9cr.lax9.p1.openshiftapps.com\n", + "models\n" + ] + } + ], + "source": [ + "key_id = os.getenv(\"AWS_ACCESS_KEY_ID\")\n", + "secret_key = os.getenv(\"AWS_SECRET_ACCESS_KEY\")\n", + "region = os.getenv(\"AWS_DEFAULT_REGION\")\n", + "endpoint = os.getenv(\"AWS_S3_ENDPOINT\")\n", + "bucket_name = os.getenv(\"AWS_S3_BUCKET\")\n", + "\n", + "print (key_id)\n", + "print (secret_key)\n", + "print (region)\n", + "print (endpoint)\n", + "print (bucket_name)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "f89ec490-f1c9-4f7a-a2a7-0244c988bb27", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "\n", + "# Uncomment the next 2 lines if you want to use the unmodified (not fine-tuned) Llama-2-7b model we downloaded in llamainfer_demo_caikit.ipynb\n", + "# source_subfolder = \"Llama-2-7b-chat-hf-sharded-bf16-caikit/\"\n", + "# target_subfolder = \"llm-trelis-llama-2-7b-caikit/\"\n", + "\n", + "# Let's default to Llama2 7B\n", + "# source_subfolder = \"Llama-2-7b-chat-hf-sharded-bf16/\"\n", + "# target_subfolder = \"llm-trelis-llama-2-7b/\"\n", + "\n", + "# source_subfolder = \"merlinite-7b-lab-GGUF/\"\n", + "# target_subfolder = \"vLLM/\"\n", + "\n", + "# source_subfolder = \"granite-7b-lab-GGUF/\"\n", + "# target_subfolder = \"granite-7b-lab-GGUF/\"\n", + "\n", + "source_subfolder = \"granite-7b-lab/\"\n", + "target_subfolder = \"granite-7b-lab/\"\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "46070831-9839-4e82-b0ce-5738f133a9d7", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "s3 = boto3.client(\n", + " \"s3\",\n", + " aws_access_key_id=key_id,\n", + " aws_secret_access_key=secret_key,\n", + " endpoint_url=endpoint,\n", + " verify=True)" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "89383bf0-5253-4538-a032-2d9724c851f3", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "tokenizer.model\n", + "README.md\n", + "paper.pdf\n", + "model-00003-of-00003.safetensors\n", + "tokenizer_config.json\n", + "generation_config.json\n", + "tokenizer.json\n", + "added_tokens.json\n", + "model-00001-of-00003.safetensors\n", + "model-00002-of-00003.safetensors\n", + "config.json\n", + ".gitattributes\n", + "model.safetensors.index.json\n", + "special_tokens_map.json\n" + ] + } + ], + "source": [ + "import os\n", + "\n", + "\n", + "Direc = source_subfolder\n", + "# Direc = \".\"\n", + "\n", + "files = os.listdir(Direc)\n", + "\n", + "files = [f for f in files if os.path.isfile(Direc+'/'+f)]\n", + "\n", + " \n", + "for x in files:\n", + " print(x) \n" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "eeb1fdb7-d481-41bf-9b95-2dff8a431d59", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "tokenizer.model\n", + "README.md\n", + "paper.pdf\n", + "model-00003-of-00003.safetensors\n", + "tokenizer_config.json\n", + "generation_config.json\n", + "tokenizer.json\n", + "added_tokens.json\n", + "model-00001-of-00003.safetensors\n", + "model-00002-of-00003.safetensors\n", + "config.json\n", + ".gitattributes\n", + "model.safetensors.index.json\n", + "special_tokens_map.json\n" + ] + } + ], + "source": [ + "for filename in files:\n", + " print(filename)\n", + " s3.upload_file(source_subfolder+filename, bucket_name, target_subfolder+filename)" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "139b96f1-50ca-4ebd-af7a-45098fb9d21b", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Done\n" + ] + } + ], + "source": [ + "print (\"Done\")" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3.9", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.18" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} From 43ce15a19d71db935d88e6ae6402bff4b7d02d8f Mon Sep 17 00:00:00 2001 From: Tom Corcoran Date: Thu, 13 Jun 2024 09:23:06 +1000 Subject: [PATCH 05/13] Code for scenario 3 --- .../vllm-runtime-small-for-granite-7b.yaml | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100644 temp/scenario3_hybrid_cloud/vllm-runtime-small-for-granite-7b.yaml diff --git a/temp/scenario3_hybrid_cloud/vllm-runtime-small-for-granite-7b.yaml b/temp/scenario3_hybrid_cloud/vllm-runtime-small-for-granite-7b.yaml new file mode 100644 index 0000000..48b9a1a --- /dev/null +++ b/temp/scenario3_hybrid_cloud/vllm-runtime-small-for-granite-7b.yaml @@ -0,0 +1,31 @@ +apiVersion: serving.kserve.io/v1alpha1 +kind: ServingRuntime +labels: + opendatahub.io/dashboard: "true" +metadata: + annotations: + openshift.io/display-name: vLLM-SMALL + name: vllm-small +spec: + builtInAdapter: + modelLoadingTimeoutMillis: 90000 + containers: + - args: + - --model + - /mnt/models/ + - --download-dir + - /models-cache + - --port + - "8080" + - --max-model-len + - "2048" + image: quay.io/rh-aiservices-bu/vllm-openai-ubi9:0.4.2 + name: kserve-container + ports: + - containerPort: 8080 + name: http1 + protocol: TCP + multiModel: false + supportedModelFormats: + - autoSelect: true + name: pytorch From 0656d6662827a99d010ae7f23b9ca207147c9f45 Mon Sep 17 00:00:00 2001 From: Tom Corcoran Date: Thu, 13 Jun 2024 09:48:17 +1000 Subject: [PATCH 06/13] Code for scenario 3 --- data/hackathon/scenario3.mdx | 28 ++++++++++++---- .../inference-api-body.json | 32 +++++++++++++++++++ 2 files changed, 54 insertions(+), 6 deletions(-) create mode 100644 temp/scenario3_hybrid_cloud/inference-api-body.json diff --git a/data/hackathon/scenario3.mdx b/data/hackathon/scenario3.mdx index 3aa5bb5..8b31f8f 100644 --- a/data/hackathon/scenario3.mdx +++ b/data/hackathon/scenario3.mdx @@ -8,7 +8,7 @@ authors: ['default'] summary: "Let's deploy the first model across the hybrid cloud." --- -As a sales team you've got an upcoming demo with the Acme Financial Services data science team, who have been training models on their laptops. +As a sales team you've got an upcoming demo with the ACME Financial Services data science team, who have been training models on their laptops. The team have given you access to one of their models in the ACME Financial Services object storage and want to see how this could be deployed to a cluster running in the cloud. @@ -43,6 +43,8 @@ You will need one for OpenShift AI. A valid strategy would be to open the yaml v certificate: `secretName: knative-serving-cert` +Documentation you may find helpful is: +- https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html/installing_and_uninstalling_openshift_ai_self-managed/index ## 3.3 - Use OpenShift AI to push Granite model from On Prem object storage to cloud @@ -61,13 +63,24 @@ Now open OpenShift AI and do the following - uses a Medium sized Container without an accelerator - for this task, use a Jupyter notebook that pulls the contents of your source bucket to your target bucket on the cloud (Hint available below) +Documentation you may find helpful is: + +- https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html/getting_started_with_red_hat_openshift_ai_self-managed/creating-a-data-science-project_get-started + +- https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html/getting_started_with_red_hat_openshift_ai_self-managed/creating-a-project-workbench_get-started + ## 3.4 - Use your cloud-based OpenShift AI to Serve the model and make it easily consumable by intelligent applications for inference -### 3.4.1 - Import a VLLM Server +### 3.4.1 - Import a VLLM Server and enable Single Model Serving VLLM is a popular model server format whose APIs are compatible with Open AI (Chat GPT) APIs. This format then lends itself to easy migration of apps already using Open AI - to OpenShift AI. +Single Model Serving is the preferred mode for serving LLMs +Documentation you may find helpful is: +- https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html-single/serving_models/index#adding-a-custom-model-serving-runtime-for-the-single-model-serving-platform_serving-large-models + +- https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html-single/serving_models/index#deploying-models-using-the-single-model-serving-platform_serving-large-models HINTS [1] Use this notebook to pull from local object storage and push to cloud object storage @@ -76,11 +89,14 @@ HINTS (note in a production environment, this would likely be automed using Gitops) TODO - confirm Gitops would be used -Documentation you may find helpful is: -- https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html/installing_and_uninstalling_openshift_ai_self-managed/index +[2] You can import this yaml to set up your vLLM server + + TODO correct location + https://github.com/tnscorcoran/hackathon/blob/main/temp/scenario3_hybrid_cloud/vllm-runtime-small-for-granite-7b.yaml + + -- https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html/getting_started_with_red_hat_openshift_ai_self-managed/creating-a-data-science-project_get-started -- https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html/getting_started_with_red_hat_openshift_ai_self-managed/creating-a-project-workbench_get-started +- \ No newline at end of file diff --git a/temp/scenario3_hybrid_cloud/inference-api-body.json b/temp/scenario3_hybrid_cloud/inference-api-body.json new file mode 100644 index 0000000..06f9f40 --- /dev/null +++ b/temp/scenario3_hybrid_cloud/inference-api-body.json @@ -0,0 +1,32 @@ +{ + "model": "/mnt/models/", + "prompt": [ + "Give me the history of Arbour hill prison in Dublin" + ], + "max_tokens": 1028, + "temperature": 1, + "top_p": 1, + "n": 1, + "stream": false, + "logprobs": 0, + "echo": false, + "stop": [ + "string" + ], + "presence_penalty": 0, + "frequency_penalty": 0, + "best_of": 2, + "user": "string", + "top_k": -1, + "ignore_eos": false, + "use_beam_search": false, + "stop_token_ids": [ + 0 + ], + "skip_special_tokens": true, + "spaces_between_special_tokens": true, + "repetition_penalty": 1, + "min_p": 0, + "include_stop_str_in_output": false, + "length_penalty": 1 +} From e659aeed10812ba4ca3652fb21793f5b949422cb Mon Sep 17 00:00:00 2001 From: Tom Corcoran Date: Thu, 13 Jun 2024 09:58:35 +1000 Subject: [PATCH 07/13] Updated docs for scenario 3 --- data/hackathon/scenario3.mdx | 56 ++++++++++++++++++++++++------------ 1 file changed, 38 insertions(+), 18 deletions(-) diff --git a/data/hackathon/scenario3.mdx b/data/hackathon/scenario3.mdx index 8b31f8f..5c56ad4 100644 --- a/data/hackathon/scenario3.mdx +++ b/data/hackathon/scenario3.mdx @@ -11,19 +11,7 @@ summary: "Let's deploy the first model across the hybrid cloud." As a sales team you've got an upcoming demo with the ACME Financial Services data science team, who have been training models on their laptops. The team have given you access to one of their models in the ACME Financial Services object storage and want to see how this could be deployed to a cluster running in the cloud. - -## 3.1 - Replicate Model to Cloud Storage - -For this task, your team are required to use the `granite-7b-lab` model available in the object storage running in the ACME Financial Services on prem cluster which is based on Minio. - -After locating the model in on premises object storage, your team need to replicate this model to the ACME Financial Services cloud cluster object storage so that it could be served in future. - -Documentation you may find helpful is: -- https://min.io/docs/minio/linux/index.html - - - -## 3.2 - Install Openshift AI related operators +## 3.1 - Install Openshift AI related operators Now that you've helped the ACME team replicate their chosen model to their cloud OpenShift Cluster, they want to serve the model ASAP. @@ -34,6 +22,7 @@ Install the following opertors (do not install any custom resources) - OpenShift Service Mesh - OpenShift Serverless + ## 3.2 - Install OpenShift AI Wait until the three operators specified in the previoius section have fully provisioned, before proceeding. You won't need any Custom Resources for OpenShift Service Mesh and OpenShift Serverless @@ -47,7 +36,21 @@ Documentation you may find helpful is: - https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html/installing_and_uninstalling_openshift_ai_self-managed/index -## 3.3 - Use OpenShift AI to push Granite model from On Prem object storage to cloud + +## 3.3 - Replicate Model to Cloud Storage + +For this task, your team are required to use the `granite-7b-lab` model available in the object storage running in the ACME Financial Services on prem cluster which is based on Minio. + +After locating the model in on premises object storage, your team need to replicate this model to the ACME Financial Services cloud cluster object storage so that it could be served in future. + +Documentation you may find helpful is: +- https://min.io/docs/minio/linux/index.html + +TODO: @James - I've reversed 3.1 and 3.3 as they may need my Jupyter Notebook + + + +## 3.4.1 - Use OpenShift AI to push Granite model from On Prem object storage to cloud Examine your On Prem object storage (Minio). The Granite model files inside the *models* bucket represent an LLM that may have been - downloaded as is @@ -71,16 +74,29 @@ Documentation you may find helpful is: ## 3.4 - Use your cloud-based OpenShift AI to Serve the model and make it easily consumable by intelligent applications for inference +Single Model Serving is the preferred mode for serving LLMs ### 3.4.1 - Import a VLLM Server and enable Single Model Serving VLLM is a popular model server format whose APIs are compatible with Open AI (Chat GPT) APIs. This format then lends itself to easy migration of apps already using Open AI - to OpenShift AI. -Single Model Serving is the preferred mode for serving LLMs - Documentation you may find helpful is: - https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html-single/serving_models/index#adding-a-custom-model-serving-runtime-for-the-single-model-serving-platform_serving-large-models -- https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html-single/serving_models/index#deploying-models-using-the-single-model-serving-platform_serving-large-models + +### 3.4.2 - Create a Single Model Server on your cloud based OpenShift + +Documentation you may find helpful is: +- https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html-single/serving_models/index#deploying-models-on-the-single-model-serving-platform_serving-large-models + +### 3.4.3 - Make an Inference call to the model. + +Note you should not need to use a token + +Documentation you may find helpful is: +- https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html-single/serving_models/index#accessing-inference-endpoint-for-deployed-model_serving-large-models + + + HINTS [1] Use this notebook to pull from local object storage and push to cloud object storage @@ -89,11 +105,15 @@ HINTS (note in a production environment, this would likely be automed using Gitops) TODO - confirm Gitops would be used -[2] You can import this yaml to set up your vLLM server +[3.4.1] You can import this yaml to set up your vLLM server TODO correct location https://github.com/tnscorcoran/hackathon/blob/main/temp/scenario3_hybrid_cloud/vllm-runtime-small-for-granite-7b.yaml +[3.4.3] You can use this JSON as the API body to make an inference call + + TODO correct location + https://github.com/tnscorcoran/hackathon/blob/main/temp/scenario3_hybrid_cloud/inference-api-body.json From 5036dd537481cc39904120cc388258ddaf3ec5ca Mon Sep 17 00:00:00 2001 From: Tom Corcoran Date: Thu, 13 Jun 2024 10:52:12 +1000 Subject: [PATCH 08/13] Updated docs for scenario 3 --- data/hackathon/scenario3.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data/hackathon/scenario3.mdx b/data/hackathon/scenario3.mdx index 5c56ad4..2601150 100644 --- a/data/hackathon/scenario3.mdx +++ b/data/hackathon/scenario3.mdx @@ -11,7 +11,7 @@ summary: "Let's deploy the first model across the hybrid cloud." As a sales team you've got an upcoming demo with the ACME Financial Services data science team, who have been training models on their laptops. The team have given you access to one of their models in the ACME Financial Services object storage and want to see how this could be deployed to a cluster running in the cloud. -## 3.1 - Install Openshift AI related operators +## 3.1 - Install operators related to Openshift AI Now that you've helped the ACME team replicate their chosen model to their cloud OpenShift Cluster, they want to serve the model ASAP. From 7b18300464917198cc258fbace8ad0d005488761 Mon Sep 17 00:00:00 2001 From: Tom Corcoran Date: Thu, 13 Jun 2024 15:59:51 +1000 Subject: [PATCH 09/13] Updated docs for scenario 3 --- data/hackathon/scenario3.mdx | 65 +++++++++++++++++++++--------------- 1 file changed, 38 insertions(+), 27 deletions(-) diff --git a/data/hackathon/scenario3.mdx b/data/hackathon/scenario3.mdx index 2601150..7348855 100644 --- a/data/hackathon/scenario3.mdx +++ b/data/hackathon/scenario3.mdx @@ -11,11 +11,20 @@ summary: "Let's deploy the first model across the hybrid cloud." As a sales team you've got an upcoming demo with the ACME Financial Services data science team, who have been training models on their laptops. The team have given you access to one of their models in the ACME Financial Services object storage and want to see how this could be deployed to a cluster running in the cloud. -## 3.1 - Install operators related to Openshift AI +## 3.1 - Examine your On-Premises and Cloud based storage -Now that you've helped the ACME team replicate their chosen model to their cloud OpenShift Cluster, they want to serve the model ASAP. +For this task, your team are required to use the `granite-7b-lab` model available in the object storage running in the ACME Financial Services on prem cluster which is based on Minio. + +The `granite-7b-lab` model is available in the object storage running in the ACME Financial Services on prem cluster which is based on Minio. It's located under the `models` bucket. +After locating the model in on premises object storage, your team will need to replicate this model to the ACME Financial Services cloud cluster object storage (to a bucket and sub-folder under the same name) so that it could be served in future. + +Examine both storage locations. + +## 3.2 - Install operators related to Openshift AI -For this challenge your team must demonstrate to ACME how to install OpenShift AI, and serve the existing model called `granite-7b-lab` via OpenShift AI. +Now that you're aware of ACME's replication requirements their chosen model to their cloud OpenShift Cluster, it's time to do that replication and then serve the model ASAP. + +For the first part of this challenge your team must demonstrate to ACME how to install OpenShift AI, replicate to the cloud and serve the existing model called `granite-7b-lab` via OpenShift AI. Install the following opertors (do not install any custom resources) - OpenShift AI @@ -23,7 +32,7 @@ Install the following opertors (do not install any custom resources) - OpenShift Serverless -## 3.2 - Install OpenShift AI +## 3.3 - Install OpenShift AI Wait until the three operators specified in the previoius section have fully provisioned, before proceeding. You won't need any Custom Resources for OpenShift Service Mesh and OpenShift Serverless @@ -37,25 +46,10 @@ Documentation you may find helpful is: -## 3.3 - Replicate Model to Cloud Storage - -For this task, your team are required to use the `granite-7b-lab` model available in the object storage running in the ACME Financial Services on prem cluster which is based on Minio. - -After locating the model in on premises object storage, your team need to replicate this model to the ACME Financial Services cloud cluster object storage so that it could be served in future. - -Documentation you may find helpful is: -- https://min.io/docs/minio/linux/index.html - -TODO: @James - I've reversed 3.1 and 3.3 as they may need my Jupyter Notebook - - +## 3.4 - Set up your OpenShift AI project and workbench -## 3.4.1 - Use OpenShift AI to push Granite model from On Prem object storage to cloud - -Examine your On Prem object storage (Minio). The Granite model files inside the *models* bucket represent an LLM that may have been -- downloaded as is -- downloaded and then fine tuned. -As it's inside object storage, it's ready to push to cloud. +An OpenShift AI project maps directly to an OpenShift project, enjoying the same RBAC and isoltion capailities. +OpenShift AI Workbenches pertain to a base container image, encapsulating a particular toolset used by the data scientists, e.g. Pytorch, Tensorflow etc. Now open OpenShift AI and do the following - create a project @@ -64,7 +58,7 @@ Now open OpenShift AI and do the following - uses a Persistent Volume of at least 60GB - uses a Data Connection to your Minio object storage - uses a Medium sized Container without an accelerator -- for this task, use a Jupyter notebook that pulls the contents of your source bucket to your target bucket on the cloud (Hint available below) + Documentation you may find helpful is: @@ -73,17 +67,31 @@ Documentation you may find helpful is: - https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html/getting_started_with_red_hat_openshift_ai_self-managed/creating-a-project-workbench_get-started -## 3.4 - Use your cloud-based OpenShift AI to Serve the model and make it easily consumable by intelligent applications for inference + + +## 3.5 - Replicate Model to Cloud Storage + +For this task, your team are required to use the `granite-7b-lab` model available in the object storage running in the ACME Financial Services on prem cluster which is based on Minio. + +How you do this replication is up to you. There are options using the Minio CLI and also using hint 3.5 below. + +Documentation you may find helpful is: +- https://min.io/docs/minio/linux/index.html + + + + +## 3.6 - Use your cloud-based OpenShift AI to Serve the model and make it easily consumable by intelligent applications for inference Single Model Serving is the preferred mode for serving LLMs -### 3.4.1 - Import a VLLM Server and enable Single Model Serving +### 3.6.1 - Import a VLLM Server and enable Single Model Serving VLLM is a popular model server format whose APIs are compatible with Open AI (Chat GPT) APIs. This format then lends itself to easy migration of apps already using Open AI - to OpenShift AI. Documentation you may find helpful is: - https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html-single/serving_models/index#adding-a-custom-model-serving-runtime-for-the-single-model-serving-platform_serving-large-models -### 3.4.2 - Create a Single Model Server on your cloud based OpenShift +### 3.6.2 - Create a Single Model Server on your cloud based OpenShift Documentation you may find helpful is: - https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html-single/serving_models/index#deploying-models-on-the-single-model-serving-platform_serving-large-models @@ -97,9 +105,12 @@ Documentation you may find helpful is: +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ HINTS -[1] Use this notebook to pull from local object storage and push to cloud object storage +[3.5] You can use this notebook to pull from local object storage and push to cloud object storage https://github.com/tnscorcoran/rhods-finetunning-demo/blob/main/minio_pull_from_and_push_to.ipynb TODO - Code the notebook and move it to the correct git repo (note in a production environment, this would likely be automed using Gitops) From 253fd052168703440750c505402e7eea7fdb3a6a Mon Sep 17 00:00:00 2001 From: Tom Corcoran Date: Thu, 13 Jun 2024 16:00:33 +1000 Subject: [PATCH 10/13] Updated docs for scenario 3 --- data/hackathon/scenario3.mdx | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/data/hackathon/scenario3.mdx b/data/hackathon/scenario3.mdx index 7348855..225f749 100644 --- a/data/hackathon/scenario3.mdx +++ b/data/hackathon/scenario3.mdx @@ -87,6 +87,8 @@ Single Model Serving is the preferred mode for serving LLMs ### 3.6.1 - Import a VLLM Server and enable Single Model Serving VLLM is a popular model server format whose APIs are compatible with Open AI (Chat GPT) APIs. This format then lends itself to easy migration of apps already using Open AI - to OpenShift AI. +You may use Hint 3.6.1 below + Documentation you may find helpful is: - https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html-single/serving_models/index#adding-a-custom-model-serving-runtime-for-the-single-model-serving-platform_serving-large-models @@ -98,7 +100,10 @@ Documentation you may find helpful is: ### 3.4.3 - Make an Inference call to the model. -Note you should not need to use a token +Note you should not need to use a token. + +You may use Hint 3.6.2 below + Documentation you may find helpful is: - https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html-single/serving_models/index#accessing-inference-endpoint-for-deployed-model_serving-large-models @@ -116,12 +121,12 @@ HINTS (note in a production environment, this would likely be automed using Gitops) TODO - confirm Gitops would be used -[3.4.1] You can import this yaml to set up your vLLM server +[3.6.1] You can import this yaml to set up your vLLM server TODO correct location https://github.com/tnscorcoran/hackathon/blob/main/temp/scenario3_hybrid_cloud/vllm-runtime-small-for-granite-7b.yaml -[3.4.3] You can use this JSON as the API body to make an inference call +[3.6.2] You can use this JSON as the API body to make an inference call TODO correct location https://github.com/tnscorcoran/hackathon/blob/main/temp/scenario3_hybrid_cloud/inference-api-body.json From 398b11d6cf0d737e282480ca54c9bf7a6ebacad7 Mon Sep 17 00:00:00 2001 From: Tom Corcoran Date: Fri, 14 Jun 2024 11:28:25 +1000 Subject: [PATCH 11/13] Fixing Notebook that download the contents of an Object store bucket (source) then push those contents to another Object store bucket (target) --- .../minio_pull_from_and_push_to.ipynb | 234 ++++++++++++------ 1 file changed, 159 insertions(+), 75 deletions(-) diff --git a/temp/scenario3_hybrid_cloud/minio_pull_from_and_push_to.ipynb b/temp/scenario3_hybrid_cloud/minio_pull_from_and_push_to.ipynb index 757f870..89536f3 100644 --- a/temp/scenario3_hybrid_cloud/minio_pull_from_and_push_to.ipynb +++ b/temp/scenario3_hybrid_cloud/minio_pull_from_and_push_to.ipynb @@ -3,9 +3,17 @@ { "cell_type": "markdown", "id": "e0a7a17e", - "metadata": {}, + "metadata": { + "tags": [] + }, "source": [ - "# Run this to push vLLM models to Object Storage" + "# Use this notebook to \n", + " - download the contents of an Object store bucket (source) to a local folder within Jupyter Hub (download_to)\n", + " - then push those contents to another Object store bucket (target)\n", + "\n", + "This *substantially faster* than downloading and uploading manually.\n", + "\n", + "### You will need to make substitutions take actions in the cell marked **MAKE CHANGES HERE** below " ] }, { @@ -21,6 +29,15 @@ "import boto3" ] }, + { + "cell_type": "markdown", + "id": "0a20fc45-7adf-4853-a2f0-4d490d3c4e3f", + "metadata": {}, + "source": [ + "##### The next cell requires you to have created an OpenShift AI Data Connection to Object storage (Minio) \n", + "##### inside your OpenShift AI Project. If you have not, manually override them (*source_region* should be 'none')" + ] + }, { "cell_type": "code", "execution_count": 2, @@ -36,23 +53,40 @@ "minio\n", "minio123\n", "none\n", - "https://minio-api-minio.apps.rosa-9s9cr.lax9.p1.openshiftapps.com\n", + "https://minio-api-minio.apps.rosa-55nsv.lax9.p1.openshiftapps.com\n", "models\n" ] } ], "source": [ - "key_id = os.getenv(\"AWS_ACCESS_KEY_ID\")\n", - "secret_key = os.getenv(\"AWS_SECRET_ACCESS_KEY\")\n", - "region = os.getenv(\"AWS_DEFAULT_REGION\")\n", - "endpoint = os.getenv(\"AWS_S3_ENDPOINT\")\n", - "bucket_name = os.getenv(\"AWS_S3_BUCKET\")\n", - "\n", - "print (key_id)\n", - "print (secret_key)\n", - "print (region)\n", - "print (endpoint)\n", - "print (bucket_name)\n" + "source_key_id = os.getenv(\"AWS_ACCESS_KEY_ID\")\n", + "source_secret_key = os.getenv(\"AWS_SECRET_ACCESS_KEY\")\n", + "source_region = os.getenv(\"AWS_DEFAULT_REGION\")\n", + "source_endpoint = os.getenv(\"AWS_S3_ENDPOINT\")\n", + "source_bucket_name = os.getenv(\"AWS_S3_BUCKET\")\n", + "\n", + "print (source_key_id)\n", + "print (source_secret_key)\n", + "print (source_region)\n", + "print (source_endpoint)\n", + "print (source_bucket_name)\n" + ] + }, + { + "cell_type": "markdown", + "id": "f4a47044-7da0-4465-8d99-53f055e474a9", + "metadata": { + "tags": [] + }, + "source": [ + "# **MAKE CHANGES HERE** \n", + "\n", + "Details of what you need to do: \n", + "- *download_to* is the name of the local directory that will be created here in Jupyter Hub to download to and upload from\n", + "- *source_bucket* and *source_subfolder* are the bucket and subfolder in the Object Storage you will pull from\n", + "- *target_bucket* and *target_subfolder* are the bucket and subfolder in the Object Storage you will push to\n", + "\n", + "## Note - before running the notebook, you should delete the **download_to** if it's already there" ] }, { @@ -64,24 +98,31 @@ }, "outputs": [], "source": [ + "download_to=\"\"\n", "\n", - "# Uncomment the next 2 lines if you want to use the unmodified (not fine-tuned) Llama-2-7b model we downloaded in llamainfer_demo_caikit.ipynb\n", - "# source_subfolder = \"Llama-2-7b-chat-hf-sharded-bf16-caikit/\"\n", - "# target_subfolder = \"llm-trelis-llama-2-7b-caikit/\"\n", + "source_bucket = \"\"\n", + "source_subfolder = \"\"\n", "\n", - "# Let's default to Llama2 7B\n", - "# source_subfolder = \"Llama-2-7b-chat-hf-sharded-bf16/\"\n", - "# target_subfolder = \"llm-trelis-llama-2-7b/\"\n", + "target_bucket = \"\"\n", + "target_subfolder = \"\"\n", + "target_url = \"\"\n", + "target_key_id = \"\"\n", + "target_secret_key = \"\"\n", + "target_endpoint = \"\"\n", "\n", - "# source_subfolder = \"merlinite-7b-lab-GGUF/\"\n", - "# target_subfolder = \"vLLM/\"\n", "\n", - "# source_subfolder = \"granite-7b-lab-GGUF/\"\n", - "# target_subfolder = \"granite-7b-lab-GGUF/\"\n", + "# Example entries (URLs will be invalid for you)\n", + "download_to=\"download_to\"\n", "\n", + "source_bucket = \"models\"\n", "source_subfolder = \"granite-7b-lab/\"\n", + "\n", + "target_bucket = \"models-target\"\n", "target_subfolder = \"granite-7b-lab/\"\n", - "\n" + "target_url = \"https://minio-api-minio.apps.rosa-55nsv.lax9.p1.openshiftapps.com\"\n", + "target_key_id = source_key_id\n", + "target_secret_key = source_secret_key\n", + "target_endpoint = target_url\n" ] }, { @@ -95,16 +136,38 @@ "source": [ "s3 = boto3.client(\n", " \"s3\",\n", - " aws_access_key_id=key_id,\n", - " aws_secret_access_key=secret_key,\n", - " endpoint_url=endpoint,\n", - " verify=True)" + " aws_access_key_id=source_key_id,\n", + " aws_secret_access_key=source_secret_key,\n", + " endpoint_url=source_endpoint,\n", + " verify=True)\n", + "\n" ] }, { "cell_type": "code", "execution_count": 5, - "id": "89383bf0-5253-4538-a032-2d9724c851f3", + "id": "63190c78-c675-4000-8b78-015b35850a16", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "if not os.path.exists(download_to):\n", + " os.mkdir(download_to) " + ] + }, + { + "cell_type": "markdown", + "id": "9732d9fa-1280-40bf-922d-f0c6d61ae75a", + "metadata": {}, + "source": [ + "## Download from source Object Storage" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "2ba297df-bd90-4c4d-9bf6-70b05305a988", "metadata": { "tags": [] }, @@ -113,43 +176,69 @@ "name": "stdout", "output_type": "stream", "text": [ - "tokenizer.model\n", - "README.md\n", - "paper.pdf\n", - "model-00003-of-00003.safetensors\n", - "tokenizer_config.json\n", - "generation_config.json\n", - "tokenizer.json\n", - "added_tokens.json\n", - "model-00001-of-00003.safetensors\n", - "model-00002-of-00003.safetensors\n", - "config.json\n", - ".gitattributes\n", - "model.safetensors.index.json\n", - "special_tokens_map.json\n" + "Downloading granite-7b-lab/.gitattributes to download_to/.gitattributes\n", + "Downloading granite-7b-lab/README.md to download_to/README.md\n", + "Downloading granite-7b-lab/added_tokens.json to download_to/added_tokens.json\n", + "Downloading granite-7b-lab/config.json to download_to/config.json\n", + "Downloading granite-7b-lab/generation_config.json to download_to/generation_config.json\n", + "Downloading granite-7b-lab/model-00001-of-00003.safetensors to download_to/model-00001-of-00003.safetensors\n", + "Downloading granite-7b-lab/model-00002-of-00003.safetensors to download_to/model-00002-of-00003.safetensors\n", + "Downloading granite-7b-lab/model-00003-of-00003.safetensors to download_to/model-00003-of-00003.safetensors\n", + "Downloading granite-7b-lab/model.safetensors.index.json to download_to/model.safetensors.index.json\n", + "Downloading granite-7b-lab/paper.pdf to download_to/paper.pdf\n", + "Downloading granite-7b-lab/special_tokens_map.json to download_to/special_tokens_map.json\n", + "Downloading granite-7b-lab/tokenizer.json to download_to/tokenizer.json\n", + "Downloading granite-7b-lab/tokenizer.model to download_to/tokenizer.model\n", + "Downloading granite-7b-lab/tokenizer_config.json to download_to/tokenizer_config.json\n" ] } ], "source": [ + "import boto3\n", "import os\n", "\n", + "def download_s3_folder(bucket_name, folder_name, local_dir):\n", "\n", - "Direc = source_subfolder\n", - "# Direc = \".\"\n", + " # List all objects in the specified folder\n", + " paginator = s3.get_paginator('list_objects_v2')\n", + " pages = paginator.paginate(Bucket=bucket_name, Prefix=folder_name)\n", "\n", - "files = os.listdir(Direc)\n", + " for page in pages:\n", + " if 'Contents' in page:\n", + " for obj in page['Contents']:\n", + " key = obj['Key']\n", + " if not key.endswith('/'):\n", + " local_file_path = os.path.join(local_dir, key[len(folder_name):])\n", + " local_file_dir = os.path.dirname(local_file_path)\n", "\n", - "files = [f for f in files if os.path.isfile(Direc+'/'+f)]\n", + " if not os.path.exists(local_file_dir):\n", + " os.makedirs(local_file_dir)\n", + " \n", + " print(f\"Downloading {key} to {local_file_path}\")\n", + " s3.download_file(bucket_name, key, local_file_path)\n", + "\n", + "\n", + "\n", + "# Example usage\n", + "bucket_name = source_bucket\n", + "folder_name = source_subfolder # Ensure it ends with a slash\n", + "local_dir = download_to\n", "\n", - " \n", - "for x in files:\n", - " print(x) \n" + "download_s3_folder(bucket_name, folder_name, local_dir)\n" + ] + }, + { + "cell_type": "markdown", + "id": "8fbb46b7-6117-4b53-b043-ab4249fed56d", + "metadata": {}, + "source": [ + "## Upload to target Object Storage" ] }, { "cell_type": "code", - "execution_count": 6, - "id": "eeb1fdb7-d481-41bf-9b95-2dff8a431d59", + "execution_count": null, + "id": "13950de4-9f96-4117-8254-8e37cc5fa48c", "metadata": { "tags": [] }, @@ -158,45 +247,40 @@ "name": "stdout", "output_type": "stream", "text": [ - "tokenizer.model\n", - "README.md\n", - "paper.pdf\n", - "model-00003-of-00003.safetensors\n", - "tokenizer_config.json\n", - "generation_config.json\n", - "tokenizer.json\n", "added_tokens.json\n", - "model-00001-of-00003.safetensors\n", - "model-00002-of-00003.safetensors\n", + "special_tokens_map.json\n", "config.json\n", - ".gitattributes\n", "model.safetensors.index.json\n", - "special_tokens_map.json\n" + "model-00002-of-00003.safetensors\n" ] } ], "source": [ + "target_s3 = boto3.client(\n", + " \"s3\",\n", + " aws_access_key_id=target_key_id,\n", + " aws_secret_access_key=target_secret_key,\n", + " endpoint_url=target_endpoint,\n", + " verify=True)\n", + "\n", + "Direc = download_to\n", + "\n", + "files = os.listdir(Direc)\n", + "files = [f for f in files if os.path.isfile(Direc+'/'+f)]\n", + "\n", "for filename in files:\n", " print(filename)\n", - " s3.upload_file(source_subfolder+filename, bucket_name, target_subfolder+filename)" + " target_s3.upload_file(download_to+\"/\"+filename, target_bucket, target_subfolder+filename)" ] }, { "cell_type": "code", - "execution_count": 7, + "execution_count": null, "id": "139b96f1-50ca-4ebd-af7a-45098fb9d21b", "metadata": { "tags": [] }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Done\n" - ] - } - ], + "outputs": [], "source": [ "print (\"Done\")" ] From bec4737fda50bee111ac1957052432d995f723f1 Mon Sep 17 00:00:00 2001 From: Tom Corcoran Date: Wed, 19 Jun 2024 11:35:27 +1000 Subject: [PATCH 12/13] Updates to docs --- data/hackathon/scenario4.mdx | 28 +++++++++++++++++++++++----- 1 file changed, 23 insertions(+), 5 deletions(-) diff --git a/data/hackathon/scenario4.mdx b/data/hackathon/scenario4.mdx index c3266da..c367b12 100644 --- a/data/hackathon/scenario4.mdx +++ b/data/hackathon/scenario4.mdx @@ -8,10 +8,28 @@ authors: ['default'] summary: "Let's add some tooling in satndardised way available to all data science users" --- -As a sales team you've got an upcoming demo with the Acme Financial Services data science team, who have been training models on their laptops. -Their data scientists are given directives on what tools to use - but in reality, what each one uses has drifted away from what's been directed, resulting in inconsistencies in tooling used across users and more importantly between deveopment and production environments -You are required to show a solution for this problem on OpenShift AI. +The ACME Financial Services team are underway with the OpenShift AI Proof of Concept and are now stuck with a workbench problem. +The ACME has been using the `Standard Data Science` workbench image with Python 3.9 and are now trying to enable additional libraries for their entire team. +While experimenting they were completing this with a `pip install` insde their Jupyter Notebooks. They now need these libraries to be baiked into their workbench images, so they can standardise on tooling across the team. -## 4.1 - Test a new library inside a Jupyter Notebook +As the pre-sales team, you've offered to show how this is done. -For this task, your team are required to \ No newline at end of file + +## 4.1 - Review notebook requiring custom Python library with in an existing workbench +Fire up an OpenShift AI workbench, using the `Standard Data Science` base image and launch/run the Notebook https://github.com/odh-labs/hackathon/blob/main/temp/scenario4_open_innovation/custom-image.ipynb +You see it fails due to missing mibrary pypdf + + +## 4.2 - Build custom workbench image +Your first task after reviewing the failing scenario to build a custom image and deploy it as am Image Stream into the cluster (it should go in the redhat-ods-applications) + +Documentation you may find helpful is: +- https://github.com/red-hat-data-services/notebooks +- https://docs.openshift.com/container-platform/4.15/cicd/builds/understanding-buildconfigs.html +- https://docs.openshift.com/container-platform/4.15/openshift_images/image-streams-manage.html + + + + +HINTS +[4.2] SOlution to From c96542bd2f2c79e5f747cb4a207089427b0c05d1 Mon Sep 17 00:00:00 2001 From: Tom Corcoran Date: Thu, 20 Jun 2024 09:47:37 +1000 Subject: [PATCH 13/13] Updates to docs --- data/hackathon/scenario2.mdx | 3 ++- data/hackathon/scenario4.mdx | 42 +++++++++++++++++++++++++++++++++--- 2 files changed, 41 insertions(+), 4 deletions(-) diff --git a/data/hackathon/scenario2.mdx b/data/hackathon/scenario2.mdx index 4093dc8..7a0b0bf 100644 --- a/data/hackathon/scenario2.mdx +++ b/data/hackathon/scenario2.mdx @@ -41,7 +41,8 @@ You'll know this is complete using the following command oc get node -l nvidia.com/gpu.present ``` - +Documentation you may find helpful is: +- ## 2.3 - Check your work diff --git a/data/hackathon/scenario4.mdx b/data/hackathon/scenario4.mdx index c367b12..b744bea 100644 --- a/data/hackathon/scenario4.mdx +++ b/data/hackathon/scenario4.mdx @@ -17,19 +17,55 @@ As the pre-sales team, you've offered to show how this is done. ## 4.1 - Review notebook requiring custom Python library with in an existing workbench Fire up an OpenShift AI workbench, using the `Standard Data Science` base image and launch/run the Notebook https://github.com/odh-labs/hackathon/blob/main/temp/scenario4_open_innovation/custom-image.ipynb -You see it fails due to missing mibrary pypdf +You see it fails due to missing library pypdf ## 4.2 - Build custom workbench image -Your first task after reviewing the failing scenario to build a custom image and deploy it as am Image Stream into the cluster (it should go in the redhat-ods-applications) +Your first task after reviewing the failing scenario to build a custom workbench image on the ACME Financial Services On Premises cluster via a `BuildConfig`. + +You have 2 options +1) Use a Build Config and push the image as an Image Stream into the redhat-ods-applications namespace +2) Use Podman to build a container locally and push to Internal OpenShift Container Registry + +Your custom image name should be `acme-workbench-ai-custom` Documentation you may find helpful is: - https://github.com/red-hat-data-services/notebooks - https://docs.openshift.com/container-platform/4.15/cicd/builds/understanding-buildconfigs.html +- https://github.com/opendatahub-io-contrib/workbench-images + + +## 4.3 - Ensure Image is present in the Internal OpenShift Container Registry +After building an image, you'll need to ensure the image is present in the Internal OpenShift Container Registry. + +Documentation you may find helpful is: +- https://docs.openshift.com/container-platform/4.15/registry/index.html - https://docs.openshift.com/container-platform/4.15/openshift_images/image-streams-manage.html +TODO Add screenshots + +## 4.4 - Verify image is imported into OpenShift AI +Your newly imported image should be visible in the `Image selection` dropdown within the Workbench tab inside your OpenShift AI + +TODO Add screenshots +## 4.5 - Check your work +If your ACME Financial Services custom image `acme-workbench-ai-custom` is available in the dropdownplease post a screenshot of it to #event-anz-ai-hackathon with the message: + +Please review [team name] solution for exercise 4. + +This exercise is worth 25 points. The event team will reply in slack to confirm your updated team total score. + + +TODO PULL all Check your work sections from https://github.com/jmhbnz/workshops/blob/main/data/workshop/scenario1.mdx HINTS -[4.2] SOlution to +[4.2.1] TODO - move the contents of this repo to the final github location: +Solution via Helm Chart is available here: https://github.com/butler54/rhoai-custom-image + + TODO - MOVE TO DOCS: + oc project redhat-ods-applications + helm install myimage ./helm + +