Create a minimally functional fulfillment webhook for dialogflow.

* The webhook handles requests asking who owns a specific area. THe fulfillment server maps the are to owners using the Kubeflow area label owners file. The fulfillment server then responds with a list of names. * cmd/jwt is a simple go program using jose-util used to generate JWTs. These JWTs are used as secrets to authorize Dialgoflow webhooks. * Related to kubeflow#142 * Start ops-log.md to keep track of infrastructure changes.
jlewi · May 24, 2020 · 3defd11 · 3defd11
1 parent 63d09d5
commit 3defd11
Show file tree

Hide file tree

Showing 35 changed files with 831 additions and 45 deletions.
diff --git a/.gitignore b/.gitignore
@@ -21,3 +21,4 @@ py/code_intelligence/.data/**
 **/.ipynb_checkpoints/
 # TODO(jlewi): Is this a remote module? Why is the fairing src getting cloned here?
 Label_Microservice/src/**
+chatbot/cmd/jwt/jwt
diff --git a/chatbot/Makefile b/chatbot/Makefile
@@ -0,0 +1,9 @@
+# Trigger a build and push.
+.PHONY: push
+push:
+	skaffold render --output=../kubeflow_clusters/code-intelligence/acm-repo/dialogflow-webhook.yaml -v info
+	git add ..
+	git commit -m "Latest code"
+	# TODO(jlewi): How can we avoid hardcoding the remote name
+	git push jlewi
+
diff --git a/chatbot/README.md b/chatbot/README.md
@@ -6,18 +6,35 @@ This is a Dialogflow fulfillment server.
 
 It is currently running in
 
-* **Full cluster name**: gke_issue-label-bot-dev_us-east1-d_issue-label-bot
 * **project**: issue-label-bot-dev
-* **cluster**: issue-label-bot
+* **cluster**: code-intelligence
+
+It is deployed through ACM. To deploy it
+
+```
+skaffold render --output=../kubeflow_clusters/code-intelligence/acm-repo/dialogflow-webhook.yaml
+```
+
+Then commit and push the updated manifests to ACM
 
 ## Notes.
 
 To expose the webhook we need to bypass IAP. To do this we create a second K8s service to create a second GCP Backend Service
 but with IAP enabled.
 
-```
-kubectl --context=issue-label-bot-dev  -n istio-system create -f istio-ingressgateway.yaml
-```
+This requires the following modifications
+
+* `kubeflow_clusters/code-intelligence/dialogflow-ingress.yaml` this file defines the Kubernetes service and backendconfig
+* `kubeflow_clusters/code-intelligence/extensions_v1beta1_ingress_envoy-ingress.yaml` - We modify our existing
+   gateway to add a second path which won't have IAP enabled.
+
+
+ We need to manually modify the GCP health check associated with this backend
+
+ * Set the path to "/healthz/ready"
+ * Set the target port to the node port mapped to the istio status-port
+
+### Authorization using manual JWTs
 
 We need to modify the security policy applied at the ingress gateway so that it won't reject requests without a valid
 JWT.
@@ -27,45 +44,76 @@ This traffic can't be routed through IAP. We will still use a JWT to restrict tr
 
 So we need to add a second JWT origin rule to match this traffic to the policy.
 
-We can do this as
+
+#### Creating a JWT
+
+To generate JWTs we use the [jose-util](https://github.com/square/go-jose/tree/master/jose-util).
+
+First we need to generate a public-private key pair to sign JWTs.
+
 
 ```
-kubectl --context=issue-label-bot-dev -n istio-system patch policy  ingress-jwt -p "$(cat ingress-jwt.patch.yaml)" --type=merge
+git clone [email protected]:square/go-jose.git git_go-jose
+cd git_go-jose/jose-uitl
+go build.
 ```
 
-To verify that is working we can port-forward to the service.
+Generate a key pair
+
 
 ```
-kubectl --context=issue-label-bot-dev  -n istio-system port-forward service/chatbot-istio-ingressgateway 9080:80
+./jose-util generate-key --alg=ES256 --use sig --kid=chatbot
 ```
 
-Send a request with a JWT this should fail with "Origin Authentication Failure" since there is no JWT.
+This will generate a json file.
+
+Convert this to a JWK file. A JWK file is just a json file which has a list of keys; the keys being the contents of the
+the json file outputted by jose-util. 
+
+
+Upload the public bit to a public GCS bucket. The following is our current public key.
+
 
 ```
-curl localhost:9080/chatbot/dev/  -d '{}' -H "Content-Type: application/json" 
+https://storage.cloud.google.com/issue-label-bot-dev_public/chatbot/keys/jwk-sig-chatbot-pub.json
 ```
 
 
+JWT's are bearer tokens so be sure to keep it secret. These JWTs also currently don't expire. 
+
+We modify the ISTIO policy to accept this JWT for paths prefixed by /chatbot; see `kubeflow_clusters/code-intelligence/acm-repo/authentication.istio.io_v1alpha1_policy_ingress-jwt.yaml`
 
-To authorize Dialogflow webhook we will use a JWT.  We use the jose-util to generate a public private key pair
+To verify that is working try sending a request without a JWT
 
 ```
-git clone [email protected]:square/go-jose.git git_go-jose
-cd git_go-jose/jose-uitl
-go build.
+curl https://code-intelligence.endpoints.issue-label-bot-dev.cloud.goog/chatbot/
 ```
 
-Generate a key pair
+* This should fail with error "Origin Authentication Failure"
+
+Now generate a JWT using the binary 
 
+We can do this using the binary `cmd/jwt`
 
 ```
-./jose-util generate-key --alg=ES256 --use sig --kid=chatbot
+cd cmd/jwt
+go build .
+./jwt
 ```
 
-Upload the public bit to a public GCS bucket
+Send a request using this JWT in the header
 
 ```
-https://storage.cloud.google.com/issue-label-bot-dev_public/chatbot/keys/jwk-sig-chatbot-pub.json
+curl https://code-intelligence.endpoints.issue-label-bot-dev.cloud.goog/chatbot/ -H "Authorization: Bearer ${CHATBOTJWT}"
+
+```
+
+* This request should succeed.
+
+To test the webhook path
+
+```
+curl https://code-intelligence.endpoints.issue-label-bot-dev.cloud.goog/chatbot/dev/dialogflow/webhook -H "Authorization: Bearer ${CHATBOTJWT}" -d '{}' -H "Content-Type: application/json" 
 ```
 
 ## Referencess

diff --git a/chatbot/cmd/jwt/jwt b/chatbot/cmd/jwt/jwt
diff --git a/chatbot/cmd/jwt/main.go b/chatbot/cmd/jwt/main.go
@@ -0,0 +1,54 @@
+package main
+
+import (
+	"flag"
+	"fmt"
+	"github.com/square/go-jose/v3"
+	"github.com/square/go-jose/v3/jwt"
+	"io/ioutil"
+	"log"
+)
+
+type options struct {
+	PrivateKey string
+	Audience string
+}
+func main() {
+	o := &options{}
+
+	cl := jwt.Claims{}
+
+	flag.StringVar(&o.PrivateKey, "private-key", "/home/jlewi/secrets/chatbot/jwk-sig-chatbot-priv.json", "Path to the private key.")
+	flag.StringVar(&cl.Subject, "subject", "dialogflow-webhook", "Subject for the JWT.")
+	flag.StringVar(&cl.Issuer, "issuer", "[email protected]", "Issuer of the JWT.")
+	flag.StringVar(&o.Audience, "audience", "kubeflow-chatbot", "Audience.")
+
+	cl.Audience = [] string{
+		o.Audience,
+	}
+	keyBytes, err := ioutil.ReadFile(o.PrivateKey)
+
+	if err != nil {
+		log.Fatalf("Error reading signing key; error %v", err)
+	}
+
+	key := &jose.JSONWebKey{}
+	err = key.UnmarshalJSON(keyBytes)
+
+	if err !=  nil {
+		log.Fatalf("Error reading signing key; error unmarshaling the key %v", err)
+	}
+
+	sig, err := jose.NewSigner(jose.SigningKey{Algorithm: jose.ES256, Key: key}, (&jose.SignerOptions{}).WithType("JWT"))
+	if err != nil {
+		panic(err)
+	}
+
+
+	raw, err := jwt.Signed(sig).Claims(cl).CompactSerialize()
+	if err != nil {
+		panic(err)
+	}
+
+	fmt.Println(raw)
+}
diff --git a/chatbot/go.mod b/chatbot/go.mod
@@ -3,6 +3,7 @@ module github.com/kubeflow/code-intelligence/chatbot
 go 1.13
 
 require (
+	github.com/aws/aws-sdk-go v1.15.78
 	github.com/ghodss/yaml v1.0.0
 	github.com/go-kit/kit v0.9.0
 	github.com/hashicorp/go-getter v1.4.1

diff --git a/chatbot/manifest/chatbot/deployment.yaml b/chatbot/manifest/chatbot/deployment.yaml
@@ -0,0 +1,22 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: server
+spec:
+  # We only need and want a single replica.
+  replicas: 1
+  selector:
+    matchLabels:
+      app: chatbot
+  template:
+    metadata:
+      labels:
+        app: chatbot
+    spec:
+      containers:
+      - name: management-context
+        image: gcr.io/issue-label-bot-dev/chatbot/server
+        command:
+          - /server
+          - --area-config-path=https://raw.githubusercontent.com/kubeflow/community/master/labels-owners.yaml
+          - --port=80
diff --git a/chatbot/manifest/chatbot/kustomization.yaml b/chatbot/manifest/chatbot/kustomization.yaml
@@ -0,0 +1,6 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+- deployment.yaml
+- service.yaml
+namespace: chatbot
diff --git a/chatbot/manifest/chatbot/service.yaml b/chatbot/manifest/chatbot/service.yaml
@@ -0,0 +1,13 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: server  
+spec:
+  ports:
+  - name: http
+    port: 80
+    protocol: TCP
+    targetPort: 80
+  selector:
+    app: chatbot
+  type: ClusterIP
diff --git a/chatbot/manifest/chatbot/virtual_service.yaml b/chatbot/manifest/chatbot/virtual_service.yaml
@@ -0,0 +1,22 @@
+apiVersion: networking.istio.io/v1alpha3
+kind: VirtualService
+metadata:
+  name: server
+spec:
+  gateways:
+  - kubeflow/kubeflow-gateway
+  hosts:
+  - '*'
+  http:
+  - match:
+    - uri:
+        prefix: /chatbot/
+    rewrite:
+      uri: /
+    route:
+    - destination:
+        # This needs to be updated manually to be in sync with kustomize
+        # generated values (i.e. namespace and prefix)
+        host: server.chatbot.svc.cluster.local
+        port:
+          number: 80
diff --git a/chatbot/manifest/dev/kustomization.yaml b/chatbot/manifest/dev/kustomization.yaml
@@ -0,0 +1,6 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+namespace: chatbot-dev
+resources:
+- ../chatbot
+- virtual_service.yaml
diff --git a/chatbot/manifest/dev/virtual_service.yaml b/chatbot/manifest/dev/virtual_service.yaml
@@ -0,0 +1,23 @@
+# There needs to be a different virtual service for every namespace because
+# the destination host will depend on the namespace.
+# We can't use an overlay because http is a list so the entire item gets overwritten.
+apiVersion: networking.istio.io/v1alpha3
+kind: VirtualService
+metadata:
+  name: server
+spec:
+  gateways:
+  - istio-system/ingressgateway
+  hosts:
+  - '*'
+  http:
+  - match:
+    - uri:
+        prefix: /chatbot/dev/
+    rewrite:
+      uri: /
+    route:
+    - destination:        
+        host: server.chatbot-dev.svc.cluster.local
+        port:
+          number: 80
diff --git a/chatbot/pkg/dialogflow/test_data/request.json b/chatbot/pkg/dialogflow/test_data/request.json
@@ -0,0 +1,86 @@
+{
+  "responseId": "071103df-35c0-43f1-8bfd-898ef58ec0bf-0f0e27e1",
+  "queryResult": {
+    "queryText": "Who owns jupyter?",
+    "parameters": {
+      "area": "jupyter",
+      "platform": ""
+    },
+    "allRequiredParamsPresent": true,
+    "fulfillmentText": "Sorry I don\u0027t know the answer",
+    "fulfillmentMessages": [{
+      "text": {
+        "text": ["Sorry I don\u0027t know the answer"]
+      }
+    }],
+    "outputContexts": [{
+      "name": "projects/kf-chat-bot/agent/sessions/90653e34-d9ed-3dbc-8ea5-cad6971952be/contexts/__system_counters__",
+      "parameters": {
+        "no-input": 0.0,
+        "no-match": 0.0,
+        "area": "jupyter",
+        "area.original": "jupyter",
+        "platform": "",
+        "platform.original": ""
+      }
+    }],
+    "intent": {
+      "name": "projects/kf-chat-bot/agent/intents/aff4b470-617c-47c5-833d-9e1e269c6e24",
+      "displayName": "find-owner"
+    },
+    "intentDetectionConfidence": 1.0,
+    "languageCode": "en"
+  },
+  "originalDetectIntentRequest": {
+    "source": "slack",
+    "payload": {
+      "data": {
+        "type": "event_callback",
+        "api_app_id": "A0144MM9WCS",
+        "token": "abcefj",
+        "event": {
+          "blocks": [{
+            "type": "rich_text",
+            "elements": [{
+              "elements": [{
+                "text": "Who owns jupyter?",
+                "type": "text"
+              }],
+              "type": "rich_text_section"
+            }],
+            "block_id": "FEa"
+          }],
+          "text": "Who owns jupyter?",
+          "channel_type": "im",
+          "event_ts": "1590278953.003400",
+          "team": "T7QLHSH6U",
+          "user": "U7QLP8YKA",
+          "client_msg_id": "7e223c77-9489-41fa-aed2-6195e56f2bde",
+          "channel": "D013ZF4GP0W",
+          "ts": "1590278953.003400",
+          "type": "message"
+        },
+        "event_id": "Ev013RFBK6TZ",
+        "authed_users": ["U014W2FSTT2"],
+        "event_time": "1590278953",
+        "team_id": "T7QLHSH6U"
+      }
+    }
+  },
+  "session": "projects/kf-chat-bot/agent/sessions/90653e34-d9ed-3dbc-8ea5-cad6971952be",
+  "alternativeQueryResults": [{
+    "queryText": "Who owns jupyter?",
+    "outputContexts": [{
+      "name": "projects/kf-chat-bot/agent/sessions/90653e34-d9ed-3dbc-8ea5-cad6971952be/contexts/__system_counters__",
+      "parameters": {
+        "no-input": 0.0,
+        "no-match": 0.0,
+        "area": "jupyter",
+        "area.original": "jupyter",
+        "platform": "",
+        "platform.original": ""
+      }
+    }],
+    "languageCode": "en"
+  }]
+}