Skip to content

Commit

Permalink
[pull] main from kserve:main (#295)
Browse files Browse the repository at this point in the history 
* docs: changed every instance of default_bucket to bucket (kserve#507)

#### Motivation

Replacing default_bucket -> bucket everywhere in this repo to ensure
it's consistent with KServe.

#### Modifications

replaced every instance of `default_bucket` to `bucket`

#### Result

Tested the [quickstart
install](https://github.com/kserve/modelmesh-serving/blob/main/docs/quickstart.md)
after modifying
[quickstart.yaml](https://github.com/kserve/modelmesh-serving/blob/6c86da9473d50de63f9ea3af8a4d7c223849547e/config/dependencies/quickstart.yaml#L127)

pods up and running - 

```
kubectl get pods
NAME                                              READY   STATUS    RESTARTS   AGE
etcd-6fdc487479-m9pkx                             1/1     Running   0          32m
minio-6b5c846587-8bwdv                            1/1     Running   0          32m
modelmesh-controller-5cd8d68bc-9ls9p              1/1     Running   0          31m
modelmesh-serving-mlserver-1.x-66bb94dcf6-hvgzj   4/4     Running   0          26m
modelmesh-serving-mlserver-1.x-66bb94dcf6-qtdzw   4/4     Running   0          26m
```

Model deployed and InferenceService is Ready - 

```
kubectl get isvc
NAME                   URL                                               READY   PREV   LATEST   PREVROLLEDOUTREVISION   LATESTREADYREVISION   AGE
example-sklearn-isvc   grpc://modelmesh-serving.modelmesh-serving:8033   True  
```

```
kubectl describe isvc example-sklearn-isvc
Name:         example-sklearn-isvc
Namespace:    modelmesh-serving
Labels:       <none>
Annotations:  serving.kserve.io/deploymentMode: ModelMesh
API Version:  serving.kserve.io/v1beta1
Kind:         InferenceService
Metadata:
  Creation Timestamp:  2024-05-28T07:19:00Z
  Generation:          1
  Resource Version:    5950
  UID:                 db71cf11-7842-4bc1-af97-647282e6b9b9
Spec:
  Predictor:
    Model:
      Model Format:
        Name:  sklearn
      Storage:
        Key:   localMinIO
        Path:  sklearn/mnist-svm.joblib
Status:
  Components:
    Predictor:
      Grpc URL:  grpc://modelmesh-serving.modelmesh-serving:8033
      Rest URL:  http://modelmesh-serving.modelmesh-serving:8008
      URL:       grpc://modelmesh-serving.modelmesh-serving:8033
  Conditions:
    Last Transition Time:  2024-05-28T07:25:07Z
    Status:                True
    Type:                  PredictorReady
    Last Transition Time:  2024-05-28T07:25:07Z
    Status:                True
    Type:                  Ready
  Model Status:
    Copies:
      Failed Copies:  0
      Total Copies:   1
    States:
      Active Model State:  Loaded
      Target Model State:  
    Transition Status:     UpToDate
  URL:                     grpc://modelmesh-serving.modelmesh-serving:8033
Events:                    <none>
```

Inference Request successful - 

```
MODEL_NAME=example-sklearn-isvc
grpcurl \
  -plaintext \
  -proto fvt/proto/kfs_inference_v2.proto \
  -d '{ "model_name": "'"${MODEL_NAME}"'", "inputs": [{ "name": "predict", "shape": [1, 64], "datatype": "FP32", "contents": { "fp32_contents": [0.0, 0.0, 1.0, 11.0, 14.0, 15.0, 3.0, 0.0, 0.0, 1.0, 13.0, 16.0, 12.0, 16.0, 8.0, 0.0, 0.0, 8.0, 16.0, 4.0, 6.0, 16.0, 5.0, 0.0, 0.0, 5.0, 15.0, 11.0, 13.0, 14.0, 0.0, 0.0, 0.0, 0.0, 2.0, 12.0, 16.0, 13.0, 0.0, 0.0, 0.0, 0.0, 0.0, 13.0, 16.0, 16.0, 6.0, 0.0, 0.0, 0.0, 0.0, 16.0, 16.0, 16.0, 7.0, 0.0, 0.0, 0.0, 0.0, 11.0, 13.0, 12.0, 1.0, 0.0] }}]}' \
  localhost:8033 \
  inference.GRPCInferenceService.ModelInfer

Handling connection for 8033
{
  "modelName": "example-sklearn-isvc__isvc-6b2eb0b8bf",
  "outputs": [
    {
      "name": "predict",
      "datatype": "INT64",
      "shape": [
        "1",
        "1"
      ],
      "contents": {
        "int64Contents": [
          "8"
        ]
      }
    }
  ]
}
```


This issue closes kserve#456

---------

Signed-off-by: Aayush Subramaniam <[email protected]>

* ci: Add nightly build twice a week (kserve#513)

Signed-off-by: Christian Kadner <[email protected]>

* chore: Use ubi8/go-toolset:1.21 for dev image (kserve#515)

Signed-off-by: Spolti <[email protected]>

---------

Signed-off-by: Aayush Subramaniam <[email protected]>
Signed-off-by: Christian Kadner <[email protected]>
Signed-off-by: Spolti <[email protected]>
Co-authored-by: Aayush Subramaniam <[email protected]>
Co-authored-by: Christian Kadner <[email protected]>
Co-authored-by: Filippe Spolti <[email protected]>
  • Loading branch information
@pull @aayushsss1
@ckadner @spolti
4 people authored Aug 20, 2024
1 parent 589dbac commit 4909708
Show file tree
Hide file tree
Showing 9 changed files with 15 additions and 56 deletions.
2 changes: 2 additions & 0 deletions .github/workflows/build.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ name: Build

on:
workflow_dispatch:
schedule:
- cron: '0 8 * * 1,4' # midnight US/Pacific on Sundays and Wednesdays
push:
branches: [master]
tags:
Expand Down
51 changes: 4 additions & 47 deletions Dockerfile.develop
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,53 +21,9 @@
###############################################################################
# Create the develop, test, and build environment
###############################################################################
ARG GOLANG_VERSION=1.21
FROM registry.access.redhat.com/ubi8/go-toolset:$GOLANG_VERSION

# TODO: replace the "go_toolset" build stage once ubi8/go-toolset:1.21 is available
# the go-toolset 1.21 is based on ubi9, we need to update it in the base image as well.
FROM registry.access.redhat.com/ubi8/ubi-minimal:latest as go-toolset

# https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope
# We need TARGETOS and TARGETARCH (not BUILDOS and BUILDARCH) since the developer
# image should be built for the OS of the developer using it (this is not a "builder image")
ARG TARGETOS
ARG TARGETARCH

ARG GOLANG_VERSION=1.21.6

USER root

ENV HOME=/root \
PATH=/usr/local/go/bin:$PATH:

WORKDIR /workspace

# install necessary tools that are included in the ubi/go-toolset image that we are temporarily replacing
RUN --mount=type=cache,target=/root/.cache/microdnf:rw \
microdnf --setopt=cachedir=/root/.cache/microdnf --nodocs install \
diffutils \
gcc-c++ \
make \
wget \
tar \
git \
which \
&& microdnf update --nodocs \
&& true

# install go
RUN true \
&& wget -qO go.tgz "https://golang.org/dl/go${GOLANG_VERSION}.${TARGETOS:-linux}-${TARGETARCH:-amd64}.tar.gz" \
&& tar -C /usr/local -xzf go.tgz \
&& go version \
&& rm go.tgz \
&& true


####################################################################################
# TODO: replace "go-toolset" build stage with ubi/go-toolset:1.21 once available #
# and swap `microdnf` commands for `dnf` #
####################################################################################
FROM go-toolset

# https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope
# don't provide "default" values (e.g. 'ARG TARGETARCH=amd64') for non-buildx environments,
Expand All @@ -90,7 +46,7 @@ WORKDIR /workspace
# Install build and dev tools
# NOTE: Require python38 to install pre-commit
RUN --mount=type=cache,target=/root/.cache/dnf:rw \
microdnf install --setopt=cachedir=/root/.cache/dnf -y --nodocs \
dnf install --setopt=cachedir=/root/.cache/dnf -y --nodocs \
nodejs \
jq \
python38 \
Expand Down Expand Up @@ -166,6 +122,7 @@ RUN true \
&& true

# Use setup-envtest for kubebuilder to use K8s version 1.23+ for autoscaling/v2 (HPA)
# Pin envtest to specific version until update to go 1.22
RUN true \
&& go install sigs.k8s.io/controller-runtime/tools/[email protected] \
&& setup-envtest use 1.26 \
Expand Down
2 changes: 1 addition & 1 deletion config/default/storage-secret.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,5 @@ metadata:
# "secret_access_key": "abcdff6a11223344aabbcc66ee231e6dd0c1122ff1234567",
# "endpoint_url": "https://s3.us-south.cloud-object-storage.appdomain.cloud",
# "region": "us-south",
# "default_bucket": "modelmesh-example-public"
# "bucket": "modelmesh-example-public"
# }
2 changes: 1 addition & 1 deletion config/dependencies/fvt.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ stringData:
"access_key_id": "AKIAIOSFODNN7EXAMPLE",
"secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"endpoint_url": "http://minio:9000",
"default_bucket": "modelmesh-example-models",
"bucket": "modelmesh-example-models",
"region": "us-south"
}
---
Expand Down
2 changes: 1 addition & 1 deletion config/dependencies/minio-storage-secret.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,6 @@ stringData:
"access_key_id": "AKIAIOSFODNN7EXAMPLE",
"secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"endpoint_url": "http://minio.controller_namespace:9000",
"default_bucket": "modelmesh-example-models",
"bucket": "modelmesh-example-models",
"region": "us-south"
}
2 changes: 1 addition & 1 deletion config/dependencies/quickstart.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,6 @@ stringData:
"access_key_id": "AKIAIOSFODNN7EXAMPLE",
"secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"endpoint_url": "http://minio:9000",
"default_bucket": "modelmesh-example-models",
"bucket": "modelmesh-example-models",
"region": "us-south"
}
2 changes: 1 addition & 1 deletion docs/predictors/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ There should be secret key called `localMinIO` that looks like:
"access_key": "AKIAIOSFODNN7EXAMPLE",
"secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"endpoint_url": "http://minio:9000",
"default_bucket": "modelmesh-example-models"
"bucket": "modelmesh-example-models"
}
```

Expand Down
6 changes: 3 additions & 3 deletions docs/predictors/setup-storage.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ $ mc ls myminio/models/onnx

### 3. Add a storage entry to the `storage-config` secret

Ensure there is a key defined in the common `storage-config` secret corresponding to the S3-based storage instance holding your model. The value of this secret key should be JSON like the following, `default_bucket` is optional.
Ensure there is a key defined in the common `storage-config` secret corresponding to the S3-based storage instance holding your model. The value of this secret key should be JSON like the following, `bucket` is optional.

Users can specify use of a custom certificate via the storage config `certificate` parameter. The custom certificate should be in the form of an embedded Certificate Authority (CA) bundle in PEM format.

Expand All @@ -72,7 +72,7 @@ Using MinIO the JSON contents look like:
"access_key_id": "minioadmin",
"secret_access_key": "minioadmin/K7JTCMP/EXAMPLEKEY",
"endpoint_url": "http://127.0.0.1:9000:9000",
"default_bucket": "",
"bucket": "",
"region": "us-east"
}
```
Expand Down Expand Up @@ -207,7 +207,7 @@ stringData:
# "access_key_id": "AKIAIOSFODNN7EXAMPLE",
# "secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
# "endpoint_url": "http://minio:9000",
# "default_bucket": "modelmesh-example-models",
# "bucket": "modelmesh-example-models",
# "region": "us-south"
# }
pvc1: |
Expand Down
2 changes: 1 addition & 1 deletion fvt/globals.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ var StorageConfigDataMinio = map[string]interface{}{
"access_key_id": "AKIAIOSFODNN7EXAMPLE",
"secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"endpoint_url": "http://minio:9000",
"default_bucket": "modelmesh-example-models",
"bucket": "modelmesh-example-models",
"region": "us-south",
},
}
Expand Down

0 comments on commit 4909708

Please sign in to comment.