Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wip][ui] Fix model mesh tests and update runtime images #2060

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
2 changes: 1 addition & 1 deletion ods_ci/tests/Resources/CLI/DSProjects/DSProjects.resource
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,6 @@ Delete All DS Projects With Name Like
Log List of DS Projects to be deleted: @{dsp_list} console=yes
FOR ${dsp_name} IN @{dsp_list}
${return_code}= Run And Return Rc
... oc delete project ${dsp_name}
... oc delete project ${dsp_name} --force
Should Be Equal As Integers ${return_code} 0 msg=Error deleting DS Project ${dsp_name}
END
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ spec:
name: caikit
containers:
- name: kserve-container
image: quay.io/modh/text-generation-inference@sha256:28a15bf3dce43753a78e7b1b89ac40ef54dd4e2f1c103667daee009725cc2562
image: quay.io/modh/text-generation-inference@sha256:bb36bb41cc744a8ff94d537f74c228e8b4e17c2468c50ccd89fc21ecc3940a70
command: ["text-generation-launcher"]
args: ["--model-name=/mnt/models/artifacts/"]
env:
Expand All @@ -23,7 +23,7 @@ spec:
## Note: cannot add readiness/liveness probes to this container because knative will refuse them.
# multi-container probing will be available after https://github.com/knative/serving/pull/14853 is merged
- name: transformer-container
image: quay.io/modh/caikit-tgis-serving@sha256:5350ac5f5883b7072296edf8583279ffe2b37c737d59f999c26f5f03bb7889bf
image: quay.io/modh/caikit-tgis-serving@sha256:fe0d1f1233d0b056ca7c690f765b20611e20837465674998e3d293df9b95e838
env:
- name: RUNTIME_LOCAL_MODELS_DIR
value: /mnt/models
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ spec:
name: caikit
containers:
- name: kserve-container
image: quay.io/modh/text-generation-inference@sha256:28a15bf3dce43753a78e7b1b89ac40ef54dd4e2f1c103667daee009725cc2562
image: quay.io/modh/text-generation-inference@sha256:bb36bb41cc744a8ff94d537f74c228e8b4e17c2468c50ccd89fc21ecc3940a70
command: ["text-generation-launcher"]
args: ["--model-name=/mnt/models/artifacts/"]
env:
Expand All @@ -21,14 +21,12 @@ spec:
# cpu: 8
# memory: 16Gi
- name: transformer-container
image: quay.io/modh/caikit-tgis-serving@sha256:5350ac5f5883b7072296edf8583279ffe2b37c737d59f999c26f5f03bb7889bf
image: quay.io/modh/caikit-tgis-serving@sha256:fe0d1f1233d0b056ca7c690f765b20611e20837465674998e3d293df9b95e838
env:
- name: TRANSFORMERS_CACHE
value: /tmp/transformers_cache
- name: RUNTIME_LOCAL_MODELS_DIR
value: /mnt/models
- name: TRANSFORMERS_CACHE
value: /tmp/transformers_cache
- name: RUNTIME_GRPC_ENABLED
value: "false"
- name: RUNTIME_HTTP_ENABLED
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ spec:
- --rest_bind_address=0.0.0.0
- --target_device=AUTO
- --metrics_enable
image: quay.io/modh/openvino_model_server@sha256:9086c1ba1ba30d358194c534f0563923aab02d03954e43e9f3647136b44a5daf
image: quay.io/modh/openvino_model_server@sha256:f1140e9d987580d1aab1ccc62519b48b1d2673308b2db496e9e505e3be788d9f
name: kserve-container
ports:
- containerPort: 8888
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ spec:
name: pytorch
containers:
- name: kserve-container
image: quay.io/modh/text-generation-inference@sha256:28a15bf3dce43753a78e7b1b89ac40ef54dd4e2f1c103667daee009725cc2562
image: quay.io/modh/text-generation-inference@sha256:bb36bb41cc744a8ff94d537f74c228e8b4e17c2468c50ccd89fc21ecc3940a70
command: ["text-generation-launcher"]
args:
- "--model-name=/mnt/models/"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ spec:
- '--served-model-name={{.Name}}'
- '--distributed-executor-backend=mp'
- '--chat-template=/app/data/template/template_chatml.jinja'
image: quay.io/modh/vllm@sha256:167aa05982214c3941afbdde7c5aff6f81040193a0db1dc06a5ad16af0a8780e
image: quay.io/modh/vllm@sha256:c86ff1e89c86bc9821b75d7f2bbc170b3c13e3ccf538bf543b1110f23e056316
name: kserve-container
command:
- python3
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ spec:
containers:
- name: kserve-container
#image: kserve/vllmserver:latest
image: quay.io/modh/vllm@sha256:a2593489ee20b8e5f01358a9aa984fc90618c6335f4c8e138e94ce635ffb112a
image: quay.io/modh/vllm@sha256:c86ff1e89c86bc9821b75d7f2bbc170b3c13e3ccf538bf543b1110f23e056316
startupProbe:
httpGet:
port: 8080
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,9 @@ ${SERVING_MODEL_SERVERS_SIDE_MENU}= xpath=//span[text()='Models and model ser
${TOKEN_AUTH_CHECKBOX_XP}= xpath://input[@id="alt-form-checkbox-auth"]
${ADD_SERVICE_ACCOUNT_BUTTON}= xpath://button[text()='Add a service account']
${SERVICE_ACCOUNT_INPUT}= xpath://input[@data-testid='service-account-form-name']
${REPLICAS_COUNT_XP}= xpath=//input[@id='model-server-replicas']
${REPLICAS_COUNT_XP}= xpath=//input[@aria-label='model server replicas number input']
${PROJECT_SELECTOR_XP}= xpath://main[contains(@id, 'dashboard-page-main')]//*[@data-testid="project-selector-toggle"]
${DEPLOY_MULTI_MODEL_BTN}= //button[contains(@data-testid,"add-server-button")]


*** Keywords ***
Expand All @@ -39,8 +40,11 @@ Create Model Server
${existing_server}= Run Keyword And Return Status Wait Until Page Contains Element //button[.="${server_name}"]
IF ${existing_server} Run Keyword And Return
... Log Model Server '${server_name}' already exists, reusing server console=True
ELSE
SeleniumLibrary.Click Button //button[@data-testid="multi-serving-select-button"]
END
SeleniumLibrary.Click Button Add model server
SeleniumLibrary.Wait Until Page Contains Element ${DEPLOY_MULTI_MODEL_BTN}
SeleniumLibrary.Click Button ${DEPLOY_MULTI_MODEL_BTN}
SeleniumLibrary.Wait Until Page Contains Element //span[.="Add model server"]
Set Model Server Name ${server_name}
Set Replicas Number With Buttons ${no_replicas}
Expand Down Expand Up @@ -221,7 +225,7 @@ Get Model Serving Access Token via UI
SeleniumLibrary.Wait Until Page Contains Element xpath://td[@data-label="Tokens"]/button
SeleniumLibrary.Click Element xpath://td[@data-label="Tokens"]/button
${token}= SeleniumLibrary.Get Element Attribute
... xpath://div[.="${service_account_name} "]/../../td[@data-label="Token Secret"]//span/input value
... xpath://div[.="${service_account_name}"]/../../td[@data-label="Token Secret"]//span/input value
END
RETURN ${token}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@
Delete Model Via UI
[Documentation] Deletes a deployed model from the Model Serving page in the dashboard.
[Arguments] ${model_name}
${model_row_xpath}= Set Variable xpath://tr[starts-with(., '${model_name} ')]
${model_row_xpath}= Set Variable xpath://tr[starts-with(., '${model_name}')]
Wait Until Element Is Visible ${model_row_xpath}
ODHDashboard.Click Action From Actions Menu item_title=${model_name} action=Delete
Wait Until Page Contains Element xpath://input[@id="delete-modal-input"]
Expand All @@ -226,7 +226,7 @@
[Arguments] ${model_name}
# TODO: Open model serving home page if needed?
# Click on Inference Endpoints link
${endpoint_link}= Set Variable //a[@data-testid="metrics-link-test-model" and text()="${model_name}"]/ancestor::tr//td//button[@data-testid="internal-external-service-button"]
${endpoint_link}= Set Variable //a[contains(@data-testid, 'metrics-link')][text() = '${model_name}']/ancestor::tr//td//button[@data-testid="internal-external-service-button"]

Check warning

Code scanning / Robocop

Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test

Line is too long (184/120)
SeleniumLibrary.Wait Until Page Contains Element ${endpoint_link}
SeleniumLibrary.Click Button ${endpoint_link}
# Get the external URL
Expand Down Expand Up @@ -515,10 +515,27 @@
Set Model Server Runtime
[Documentation] Opens the Serving runtime dropdown in the deploy model modal window for models
... and select the given runtime
[Arguments] ${runtime}=Caikit TGIS
Page Should Contain Element ${KSERVE_RUNTIME_DROPDOWN}
Click Element ${KSERVE_RUNTIME_DROPDOWN}
Click Element //span[contains(text(),"${runtime}")]
[Arguments] ${runtime}=Caikit TGIS ${retries}=1
TRY
${is_enabled}= Run Keyword And Return Status
... Element Should Be Enabled xpath://button[@id="serving-runtime-template-selection"]

IF ${is_enabled}
FOR ${retry_idx} IN RANGE 0 1+${retries}
Click Element xpath://button[@id="serving-runtime-template-selection"]
Page Should Contain Element xpath://span[contains(., "${runtime}")]
${selected}= Run Keyword And Return Status
... Click Element xpath://span[contains(., "${runtime}")]
IF ${selected}==${TRUE} BREAK

Check notice

Code scanning / Robocop

'{{ block_name }}' condition can be simplified Note test

'INLINE IF' condition can be simplified
END
ELSE
Element Should Be Disabled id:serving-runtime-template-selection
${text}= Get Text xpath://button[@id="serving-runtime-template-selection"]/span
Should Be True ${runtime} in ${text}
END
EXCEPT
Log framework ${runtime} does not appear to be supported by the chosen model server
END

Get Kserve Inference Host Via UI
[Documentation] Fetches the host of the model's URL from the Data Science Project UI
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -69,26 +69,27 @@
[Tags] Sanity
... ODS-2268
Open Data Science Projects Home Page
Create Data Science Project title=${PRJ_TITLE}-2268 description=${PRJ_DESCRIPTION}
Recreate S3 Data Connection project_title=${PRJ_TITLE}-2268 dc_name=model-serving-connection
${namespace}= Set Variable ${PRJ_TITLE}-2268
Create Data Science Project title=${namespace} description=${PRJ_DESCRIPTION}
Recreate S3 Data Connection project_title=${namespace} dc_name=model-serving-connection
... aws_access_key=${S3.AWS_ACCESS_KEY_ID} aws_secret_access=${S3.AWS_SECRET_ACCESS_KEY}
... aws_bucket_name=ods-ci-s3
Create Model Server token=${FALSE} server_name=${RUNTIME_NAME} existing_server=${TRUE}
Serve Model project_name=${PRJ_TITLE}-2268 model_name=${MODEL_NAME} framework=tensorflow
Create Model Server token=${FALSE} server_name=${RUNTIME_NAME} existing_server=${FALSE}
Serve Model project_name=${namespace} model_name=${MODEL_NAME} framework=tensorflow
... existing_data_connection=${TRUE} data_connection_name=model-serving-connection
... model_path=inception_resnet_v2.pb
${runtime_pod_name}= Replace String Using Regexp string=${RUNTIME_NAME} pattern=\\s replace_with=-
${runtime_pod_name}= Convert To Lower Case ${runtime_pod_name}
Wait Until Keyword Succeeds 5 min 10 sec Verify Openvino Deployment runtime_name=${RUNTIME_POD_NAME}
Wait Until Keyword Succeeds 5 min 10 sec Verify Serving Service
Wait Until Keyword Succeeds 5 min 10 sec Verify Openvino Deployment runtime_name=${RUNTIME_POD_NAME} project_name=${namespace}

Check warning

Code scanning / Robocop

Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test

Line is too long (141/120)

Check warning

Code scanning / Robocop

Variable '{{ name }}' has inconsistent naming. First used as '{{ first_use }}' Warning test

Variable '${RUNTIME_POD_NAME}' has inconsistent naming. First used as '${runtime_pod_name}'
Wait Until Keyword Succeeds 5 min 10 sec Verify Serving Service project_name=${namespace}
Verify Model Status ${MODEL_NAME} success
Set Suite Variable ${MODEL_CREATED} ${TRUE}
${url}= Get Model Route Via UI ${MODEL_NAME}
${status_code} ${response_text}= Send Random Inference Request endpoint=${url} name=input
${status_code} ${response_text}= Send Random Inference Request endpoint=${url} name=input:0

Check notice

Code scanning / Robocop

Variable '{{ name }}' is assigned but not used Note test

Variable '${response_text}' is assigned but not used
... shape={"B": 1, "H": 299, "W": 299, "C": 3} no_requests=1
Should Be Equal As Strings ${status_code} 200
[Teardown] Run Keywords Run Keyword If Test Failed Get Modelmesh Events And Logs
... server_name=${RUNTIME_NAME} project_title=${PRJ_TITLE}-2869
... server_name=${RUNTIME_NAME} project_title=${namespace}
... AND
... Model Serving Test Teardown

Expand All @@ -102,7 +103,7 @@
Recreate S3 Data Connection project_title=${PRJ_TITLE} dc_name=model-serving-connection
... aws_access_key=${S3.AWS_ACCESS_KEY_ID} aws_secret_access=${S3.AWS_SECRET_ACCESS_KEY}
... aws_bucket_name=ods-ci-s3
Create Model Server token=${TRUE} server_name=${SECURED_RUNTIME} existing_server=${TRUE}
Create Model Server token=${TRUE} server_name=${SECURED_RUNTIME} existing_server=${FALSE}
Serve Model project_name=${PRJ_TITLE} model_name=${SECURED_MODEL} model_server=${SECURED_RUNTIME}
... existing_data_connection=${TRUE} data_connection_name=model-serving-connection existing_model=${TRUE}
... framework=onnx model_path=mnist-8.onnx
Expand All @@ -113,7 +114,7 @@
Verify Model Status ${SECURED_MODEL} success
Set Suite Variable ${MODEL_CREATED} ${TRUE}
[Teardown] Run Keywords Run Keyword If Test Failed Get Modelmesh Events And Logs
... server_name=${RUNTIME_NAME} project_title=${PRJ_TITLE}-2869
... server_name=${RUNTIME_NAME} project_title=${PRJ_TITLE}
... AND
... Model Serving Test Teardown

Expand All @@ -127,7 +128,7 @@
Recreate S3 Data Connection project_title=${SECOND_PROJECT} dc_name=model-serving-connection
... aws_access_key=${S3.AWS_ACCESS_KEY_ID} aws_secret_access=${S3.AWS_SECRET_ACCESS_KEY}
... aws_bucket_name=ods-ci-s3
Create Model Server token=${TRUE} server_name=${SECURED_RUNTIME} existing_server=${TRUE}
Create Model Server token=${TRUE} server_name=${SECURED_RUNTIME} existing_server=${FALSE}
Serve Model project_name=${SECOND_PROJECT} model_name=${SECURED_MODEL} model_server=${SECURED_RUNTIME}
... existing_data_connection=${TRUE} data_connection_name=model-serving-connection existing_model=${TRUE}
... framework=onnx model_path=mnist-8.onnx
Expand All @@ -139,7 +140,7 @@
${out}= Get Model Inference ${SECURED_MODEL} ${INFERENCE_INPUT} token_auth=${FALSE}
Should Contain ${out} <button type="submit" class="btn btn-lg btn-primary">Log in with OpenShift</button>
[Teardown] Run Keywords Run Keyword If Test Failed Get Modelmesh Events And Logs
... server_name=${RUNTIME_NAME} project_title=${PRJ_TITLE}-2869
... server_name=${RUNTIME_NAME} project_title=${SECOND_PROJECT}
... AND
... Model Serving Test Teardown

Expand All @@ -157,37 +158,38 @@
[Tags] Tier1
... RHOAIENG-2869
Open Data Science Projects Home Page
Create Data Science Project title=${PRJ_TITLE}-2869 description=${PRJ_DESCRIPTION}
Recreate S3 Data Connection project_title=${PRJ_TITLE}-2869 dc_name=model-serving-connection
${namespace}= Set Variable ${PRJ_TITLE}-2869
Create Data Science Project title=${namespace} description=${PRJ_DESCRIPTION}
Recreate S3 Data Connection project_title=${namespace} dc_name=model-serving-connection
... aws_access_key=${S3.AWS_ACCESS_KEY_ID} aws_secret_access=${S3.AWS_SECRET_ACCESS_KEY}
... aws_bucket_name=ods-ci-s3
Create Model Server token=${FALSE} server_name=${RUNTIME_NAME} existing_server=${TRUE}
Serve Model project_name=${PRJ_TITLE}-2869 model_name=${MODEL_NAME} framework=tensorflow
Serve Model project_name=${namespace} model_name=${MODEL_NAME} framework=tensorflow
... existing_data_connection=${TRUE} data_connection_name=model-serving-connection
... model_path=inception_resnet_v2.pb
${runtime_pod_name}= Replace String Using Regexp string=${RUNTIME_NAME} pattern=\\s replace_with=-
${runtime_pod_name}= Convert To Lower Case ${runtime_pod_name}
Wait Until Keyword Succeeds 5 min 10 sec Verify Openvino Deployment runtime_name=${RUNTIME_POD_NAME}
Wait Until Keyword Succeeds 5 min 10 sec Verify Serving Service
Wait Until Keyword Succeeds 5 min 10 sec Verify Openvino Deployment runtime_name=${RUNTIME_POD_NAME} project_name=${namespace}

Check warning

Code scanning / Robocop

Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test

Line is too long (141/120)

Check warning

Code scanning / Robocop

Variable '{{ name }}' has inconsistent naming. First used as '{{ first_use }}' Warning test

Variable '${RUNTIME_POD_NAME}' has inconsistent naming. First used as '${runtime_pod_name}'
Wait Until Keyword Succeeds 5 min 10 sec Verify Serving Service project_name=${namespace}
Verify Model Status ${MODEL_NAME} success
Set Suite Variable ${MODEL_CREATED} ${TRUE}
${url}= Get Model Route Via UI ${MODEL_NAME}
${status_code} ${response_text}= Send Random Inference Request endpoint=${url} name=input
... shape={"B": 1, "H": 299, "W": 299, "C": 3} no_requests=1
Should Be Equal As Strings ${status_code} 200
Serve Model project_name=${PRJ_TITLE}-2869 model_name=${MODEL_NAME} framework=openvino_ir
Serve Model project_name=${namespace} model_name=${MODEL_NAME} framework=openvino_ir
... existing_data_connection=${TRUE} data_connection_name=model-serving-connection
... model_path=openvino-example-model existing_model=${TRUE}
${runtime_pod_name}= Replace String Using Regexp string=${RUNTIME_NAME} pattern=\\s replace_with=-
${runtime_pod_name}= Convert To Lower Case ${runtime_pod_name}
Wait Until Keyword Succeeds 5 min 10 sec Verify Openvino Deployment runtime_name=${runtime_pod_name}
... project_name=${PRJ_TITLE}-2869
Wait Until Keyword Succeeds 5 min 10 sec Verify Serving Service ${PRJ_TITLE}-2869
... project_name=${namespace}
Wait Until Keyword Succeeds 5 min 10 sec Verify Serving Service ${namespace}
Verify Model Status ${MODEL_NAME} success
Run Keyword And Continue On Failure Verify Model Inference ${MODEL_NAME} ${INFERENCE_INPUT_OPENVINO}
... ${EXPECTED_INFERENCE_OUTPUT_OPENVINO} token_auth=${FALSE}
[Teardown] Run Keywords Run Keyword If Test Failed Get Modelmesh Events And Logs
... server_name=${RUNTIME_NAME} project_title=${PRJ_TITLE}-2869
... server_name=${RUNTIME_NAME} project_title=${namespace}
... AND
... Model Serving Test Teardown

Expand Down Expand Up @@ -218,7 +220,7 @@
Recreate S3 Data Connection project_title=${new_project} dc_name=model-serving-connection
... aws_access_key=${S3.AWS_ACCESS_KEY_ID} aws_secret_access=${S3.AWS_SECRET_ACCESS_KEY}
... aws_bucket_name=ods-ci-s3
Create Model Server token=${FALSE} server_name=${server_name} existing_server=${TRUE}
Create Model Server token=${FALSE} server_name=${server_name} existing_server=${FALSE}
Serve Model project_name=${new_project} model_name=${model_name} framework=openvino_ir
... existing_data_connection=${TRUE} data_connection_name=model-serving-connection
... model_path=openvino-example-model existing_model=${TRUE}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@
Verify Model Status ${MODEL_NAME} success
Set Suite Variable ${MODEL_CREATED} ${TRUE}
${url}= Get Model Route Via UI ${MODEL_NAME}
${status_code} ${response_text}= Send Random Inference Request endpoint=${url} name=input
${status_code} ${response_text}= Send Random Inference Request endpoint=${url} name=input:0

Check notice

Code scanning / Robocop

Variable '{{ name }}' is assigned but not used Note test

Variable '${response_text}' is assigned but not used
... shape={"B": 1, "H": 299, "W": 299, "C": 3} no_requests=1
Should Be Equal As Strings ${status_code} 200
[Teardown] Run Keywords Clean All Models Of Current User AND
Expand Down
Loading
Loading