Progress on exercises (#9)

* Continued outline for scenario 3

* Updates to scenarios 3 and 4

* Updates to scenarios 3 and 4

* Code for scenario 3

* Code for scenario 3

* Code for scenario 3

* Updated docs for scenario 3

* Updated docs for scenario 3

* Updated docs for scenario 3

* Updated docs for scenario 3

* Fixing Notebook that download the contents of an Object store bucket (source) then push those contents to another Object store bucket (target)

data/hackathon/scenario1.mdx

@@ -37,7 +37,7 @@ For this challenge you'll be given two OpenShift clusters
 - a Single Node OpenShift cluster representing the ACME On Premises environment
 
 All challenge tasks must be performed on these clusters so your solutions can be graded successfully.
-Some challenges wil require the use of specific clusters
+Some challenges will require the use of specific clusters
 You can and are encouraged to use any supporting documentation or other resources in order to tackle each of the challenge tasks.
 
 <Zoom>

data/hackathon/scenario3.mdx

@@ -8,39 +8,131 @@ authors: ['default']
 summary: "Let's deploy the first model across the hybrid cloud."
 ---
 
-As a sales team you've got an upcoming demo with the Acme Financial Services data science team, who have been training models on their laptops.
+As a sales team you've got an upcoming demo with the ACME Financial Services data science team, who have been training models on their laptops.
 The team have given you access to one of their models in the ACME Financial Services object storage and want to see how this could be deployed to a cluster running in the cloud.
 
-
-## 3.1 - Replicate Model to Cloud Storage
+## 3.1 - Examine your On-Premises and Cloud based storage
 
 For this task, your team are required to use the `granite-7b-lab` model available in the object storage running in the ACME Financial Services on prem cluster which is based on Minio.
 
-After locating the model in on premises object storage, your team need to replicate this model to the ACME Financial Services cloud cluster object storage so that it could be served in future.
-
-Documentation you may find helpful is:
-- https://min.io/docs/minio/linux/index.html
-
+The `granite-7b-lab` model is available in the object storage running in the ACME Financial Services on prem cluster which is based on Minio. It's located under the `models` bucket.
+After locating the model in on premises object storage, your team will need to replicate this model to the ACME Financial Services cloud cluster object storage (to a bucket and sub-folder under the same name) so that it could be served in future.
 
+Examine both storage locations.
 
-## 3.2 - Install Openshift AI related operators
+## 3.2 - Install  operators related to Openshift AI 
 
-Now that you've helped the ACME team replicate their chosen model to their cloud OpenShift Cluster, they want to serve the model ASAP.
+Now that you're aware of ACME's replication requirements their chosen model to their cloud OpenShift Cluster, it's time to do that replication and then serve the model ASAP.
 
-For this challenge your team must demonstrate to ACME how to install OpenShift AI, and serve the existing model called `granite-7b-lab` via OpenShift AI.
+For the first part of this challenge your team must demonstrate to ACME how to install OpenShift AI, replicate to the cloud and serve the existing model called `granite-7b-lab` via OpenShift AI.
 
 Install the following opertors (do not install any custom resources)
 - OpenShift AI
 - OpenShift Service Mesh
 - OpenShift Serverless
 
-## 3.2 - Install Openshift AI
 
+## 3.3 - Install OpenShift AI
+Wait until the three operators specified in the previoius section have fully provisioned, before proceeding.
+You won't need any Custom Resources for OpenShift Service Mesh and OpenShift Serverless
+
+You will need one for OpenShift AI. A valid strategy would be to open the yaml view and go with all the defaults - the only addition to be to add this knative-serving-cert secret
         ingressGateway:
           certificate:
-            secretName: knative-serving-cert
-
+            `secretName: knative-serving-cert`
 
 Documentation you may find helpful is:
 - https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html/installing_and_uninstalling_openshift_ai_self-managed/index
 
+
+
+## 3.4 - Set up your OpenShift AI project and workbench
+
+An OpenShift AI project maps directly to an OpenShift project, enjoying the same RBAC and isoltion capailities.
+OpenShift AI Workbenches pertain to a base container image, encapsulating a particular toolset used by the data scientists, e.g. Pytorch, Tensorflow etc.
+
+Now open OpenShift AI and do the following
+- create a project
+- create a workbench that 
+  - uses Pytorch as a basis
+  - uses a Persistent Volume of at least 60GB 
+  - uses a Data Connection to your Minio object storage
+  - uses a Medium sized Container without an accelerator
+
+
+Documentation you may find helpful is:
+
+- https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html/getting_started_with_red_hat_openshift_ai_self-managed/creating-a-data-science-project_get-started
+
+- https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html/getting_started_with_red_hat_openshift_ai_self-managed/creating-a-project-workbench_get-started
+
+
+
+
+## 3.5 - Replicate Model to Cloud Storage
+
+For this task, your team are required to use the `granite-7b-lab` model available in the object storage running in the ACME Financial Services on prem cluster which is based on Minio.
+
+How you do this replication is up to you. There are options using the Minio CLI and also using hint 3.5 below.
+
+Documentation you may find helpful is:
+- https://min.io/docs/minio/linux/index.html
+
+
+
+
+## 3.6 - Use your cloud-based OpenShift AI to Serve the model and make it easily consumable by intelligent applications for inference
+Single Model Serving is the preferred mode for serving LLMs
+
+### 3.6.1 - Import a VLLM Server and enable Single Model Serving
+VLLM is a popular model server format whose APIs are compatible with Open AI (Chat GPT) APIs. This format then lends itself to easy migration of apps already using Open AI - to OpenShift AI.
+
+You may use Hint 3.6.1 below
+
+Documentation you may find helpful is:
+- https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html-single/serving_models/index#adding-a-custom-model-serving-runtime-for-the-single-model-serving-platform_serving-large-models
+
+
+### 3.6.2 - Create a Single Model Server on your cloud based OpenShift
+
+Documentation you may find helpful is:
+- https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html-single/serving_models/index#deploying-models-on-the-single-model-serving-platform_serving-large-models
+
+### 3.4.3 - Make an Inference call to the model.
+
+Note you should not need to use a token.
+
+You may use Hint 3.6.2 below
+
+
+Documentation you may find helpful is:
+- https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.9/html-single/serving_models/index#accessing-inference-endpoint-for-deployed-model_serving-large-models
+
+
+
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+HINTS
+[3.5] You can use this notebook to pull from local object storage and push to cloud object storage
+    https://github.com/tnscorcoran/rhods-finetunning-demo/blob/main/minio_pull_from_and_push_to.ipynb
+    TODO - Code the notebook and move it to the correct git repo
+    (note in a production environment, this would likely be automed using Gitops)
+    TODO - confirm Gitops would be used
+
+[3.6.1] You can import this yaml to set up your vLLM server
+
+  TODO correct location
+  https://github.com/tnscorcoran/hackathon/blob/main/temp/scenario3_hybrid_cloud/vllm-runtime-small-for-granite-7b.yaml
+
+[3.6.2] You can use this JSON as the API body to make an inference call
+
+  TODO correct location
+  https://github.com/tnscorcoran/hackathon/blob/main/temp/scenario3_hybrid_cloud/inference-api-body.json
+
+
+
+
+
+- 

data/hackathon/scenario4.mdx

@@ -0,0 +1,17 @@
+---
+title: Open Innovation - Integrating a new piece of tooling into OpenShift AI Workspace
+exercise: 3
+date: '2024-06-05'
+tags: ['openshift','ai','kubernetes']
+draft: false
+authors: ['default']
+summary: "Let's add some tooling in satndardised way available to all data science users"
+---
+
+As a sales team you've got an upcoming demo with the Acme Financial Services data science team, who have been training models on their laptops.
+Their data scientists are given directives on what tools to use - but in reality, what each one uses has drifted away from what's been directed, resulting in inconsistencies in tooling used across users and more importantly between deveopment and production environments
+You are required to show a solution for this problem on OpenShift AI.
+
+## 4.1 - Test a new library inside a Jupyter Notebook
+
+For this task, your team are required to 

temp/scenario3_hybrid_cloud/inference-api-body.json

@@ -0,0 +1,32 @@
+{
+  "model": "/mnt/models/",
+  "prompt": [
+    "Give me the history of Arbour hill prison in Dublin"
+  ],
+  "max_tokens": 1028,
+  "temperature": 1,
+  "top_p": 1,
+  "n": 1,
+  "stream": false,
+  "logprobs": 0,
+  "echo": false,
+  "stop": [
+    "string"
+  ],
+  "presence_penalty": 0,
+  "frequency_penalty": 0,
+  "best_of": 2,
+  "user": "string",
+  "top_k": -1,
+  "ignore_eos": false,
+  "use_beam_search": false,
+  "stop_token_ids": [
+    0
+  ],
+  "skip_special_tokens": true,
+  "spaces_between_special_tokens": true,
+  "repetition_penalty": 1,
+  "min_p": 0,
+  "include_stop_str_in_output": false,
+  "length_penalty": 1
+}