Skip to content

Commit

Permalink
Add better tools for demo (#13)
Browse files Browse the repository at this point in the history
* wip: add tools for more comprehensive demo

Signed-off-by: Calum Murray <[email protected]>

* fix: don't crash with KeyError when not in k8s

Signed-off-by: Calum Murray <[email protected]>

* fix: pin python version so that it doesn't break randomly

Signed-off-by: Calum Murray <[email protected]>

* feat: add memory into chat

Signed-off-by: Calum Murray <[email protected]>

* feat: the resource cost calculations work now

Signed-off-by: Calum Murray <[email protected]>

* 🧹 Adding new tools to installer (#1)

Signed-off-by: Matthias Wessendorf <[email protected]>

---------

Signed-off-by: Calum Murray <[email protected]>
Signed-off-by: Matthias Wessendorf <[email protected]>
Co-authored-by: Matthias Wessendorf <[email protected]>
  • Loading branch information
Cali0707 and matzew authored Jun 27, 2024
1 parent 742f426 commit 4bb05c8
Show file tree
Hide file tree
Showing 22 changed files with 469 additions and 10 deletions.
2 changes: 1 addition & 1 deletion core/chat-app/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM python:3.12-slim
FROM python:3.12.3-slim

WORKDIR /app

Expand Down
14 changes: 11 additions & 3 deletions core/chat-app/chat.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,15 @@
from dotenv import load_dotenv

from langchain_openai import ChatOpenAI
from langchain.memory import ConversationBufferMemory
from langchain.tools.render import format_tool_to_openai_function
from langchain_core.messages import BaseMessage, HumanMessage
from langchain_core.messages import FunctionMessage
from langchain.globals import set_debug

from langgraph.prebuilt import ToolExecutor, ToolInvocation
from langgraph.graph import StateGraph, END
from langgraph.checkpoint.memory import MemorySaver

import chainlit as cl

Expand All @@ -24,9 +26,13 @@

class AgentState(TypedDict):
messages: Annotated[Sequence[BaseMessage], operator.add]
chat_history: list[BaseMessage]

@cl.on_chat_start
async def on_chat_start():
chat_history = ConversationBufferMemory(return_messages=True)
cl.user_session.set("chat_history", chat_history)

model = ChatOpenAI(temperature=0.1, streaming=True, max_retries=5, timeout=60.)

tools = [HumanInput()]
Expand All @@ -49,7 +55,7 @@ def should_continue(state: AgentState) -> str:
async def call_model(state: AgentState):
print("calling model...")
messages = state["messages"]
print(messages)
print(state)
response = await model.ainvoke(messages)
return {"messages": [response]}

Expand Down Expand Up @@ -101,7 +107,9 @@ async def call_tool(state: AgentState):

graph.add_edge("action", "agent")

runner = graph.compile()
memory = MemorySaver()

runner = graph.compile(checkpointer=memory)

cl.user_session.set("runner", runner)

Expand All @@ -125,7 +133,7 @@ async def main(message: cl.Message):

msg = cl.Message(content="")

async for event in runner.astream_events(inputs, version="v1"):
async for event in runner.astream_events(inputs, {"configurable": {"thread_id": "thread-1"}}, version="v1"):
kind = event["event"]
if kind == "on_chat_model_stream":
content = event["data"]["chunk"].content
Expand Down
21 changes: 15 additions & 6 deletions core/chat-app/cloudevent_tool.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,14 @@
from kubernetes.dynamic.exceptions import ResourceNotFoundError
from kubernetes.client import api_client

if env["IN_KUBERNETES"] is not None:
client = dynamic.DynamicClient(
api_client.ApiClient(configuration=config.load_incluster_config())
)
try:
if env["IN_KUBERNETES"] is not None:
client = dynamic.DynamicClient(
api_client.ApiClient(configuration=config.load_incluster_config())
)
except KeyError:
print("not in k8s, not loading client config")


def make_request_maker(eventtype, request_structure):
attributes = {
Expand Down Expand Up @@ -65,7 +69,7 @@ async def arun(self, **kwargs) -> str:

return arun

types = {"string": str, "int": int, "list:string": List[str], "list:int": List[int]}
types = {"string": str, "int": int, "float": float, "list:string": List[str], "list:int": List[int], "list:float": List[float]}

def make_input_class(eventtype, request_structure: Dict[str, Dict[str, Any]]):
d = {}
Expand Down Expand Up @@ -114,7 +118,12 @@ def process_eventtype_to_request_structure(eventtype):

def create_cloudevents_tools() -> List:
result = []
if env["IN_KUBERNETES"] is not None:
try:
in_k8s = env["IN_KUBERNETES"] is not None
except KeyError:
in_k8s = False

if in_k8s:
print("getting eventtypes")
for et in get_eventtypes():
request_structure = process_eventtype_to_request_structure(et)
Expand Down
11 changes: 11 additions & 0 deletions core/chat-app/config/eventtypes/average-resource-consumption.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
apiVersion: eventing.knative.dev/v1beta2
kind: EventType
metadata:
name: average.resource.consumption
spec:
reference:
apiVersion: serving.knative.dev/v1
kind: Service
name: average-resource-consumption
description: "This tells you the average resource consumption for recent months"
type: "average.resource.consumption"
12 changes: 12 additions & 0 deletions core/chat-app/config/eventtypes/resource-cost.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
apiVersion: eventing.knative.dev/v1beta2
kind: EventType
metadata:
name: resource.cost.calculator
spec:
reference:
apiVersion: serving.knative.dev/v1
kind: Service
name: resource-cost-calculator
description: "This calculates the cost for a given resource kind and average usage over a month. Use this tool whenever there is a question about the cost of any cloud resources"
schemaData: '{"resourceKind":{"type":"string","description":"The type of resource you would like to calculate the cost of"},"usage":{"type":"float","description":"The average usage of the resource"},"unit":{"type":"string","description":"The unit of the resource usage"}}'
type: "resource.cost.calculator"
2 changes: 2 additions & 0 deletions hack/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ if [[ $FULL_INSTALL ]]; then
wait_for_deployments "knative-eventing"
fi

(cd tools/resource-cost-calculator && func deploy)
(cd tools/average-resource-consumption && func deploy)
(cd tools/knative-text-responder && func deploy)
(cd tools/word-length && func deploy)

Expand Down
5 changes: 5 additions & 0 deletions tools/average-resource-consumption/.funcignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

# Use the .funcignore file to exclude files which should not be
# tracked in the image build. To instruct the system not to track
# files in the image build, add the regex pattern or file information
# to this file.
5 changes: 5 additions & 0 deletions tools/average-resource-consumption/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

# Functions use the .func directory for local runtime data which should
# generally not be tracked in source control. To instruct the system to track
# .func in source control, comment the following line (prefix it with '# ').
/.func
23 changes: 23 additions & 0 deletions tools/average-resource-consumption/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Go Cloud Events Function

Welcome to your new Go Function! The boilerplate function code can be found in [`handle.go`](handle.go). This Function is meant to respond exclusively to [Cloud Events](https://cloudevents.io/), but you can remove the check for this in the function and it will respond just fine to plain vanilla incoming HTTP requests.

## Development

Develop new features by adding a test to [`handle_test.go`](handle_test.go) for each feature, and confirm it works with `go test`.

Update the running analog of the function using the `func` CLI or client library, and it can be invoked using a manually-created CloudEvent:

```console
curl -v -X POST -d '{"message": "hello"}' \
-H'Content-type: application/json' \
-H'Ce-id: 1' \
-H'Ce-source: cloud-event-example' \
-H'Ce-subject: Echo content' \
-H'Ce-type: MyEvent' \
-H'Ce-specversion: 1.0' \
http://localhost:8080/
```

For more, see [the complete documentation]('https://github.com/knative/func/tree/main/docs')

16 changes: 16 additions & 0 deletions tools/average-resource-consumption/func.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
specVersion: 0.36.0
name: average-resource-consumption
runtime: go
registry: quay.io/cali0707
namespace: default
created: 2024-06-25T14:08:48.57074338-04:00
invoke: cloudevent
build:
builder: pack
deploy:
namespace: default
image: quay.io/cali0707/average-resource-consumption@sha256:c18f42d15ec1436982b9e7c5faf5c99716ab1360645296ac86d1e7c45703f3d2
subscriptions:
- source: chat-broker
filters:
type: average.resource.consumption
14 changes: 14 additions & 0 deletions tools/average-resource-consumption/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
module function

go 1.21

require (
github.com/cloudevents/sdk-go/v2 v2.5.0
github.com/google/uuid v1.1.1
)

require (
github.com/json-iterator/go v1.1.10 // indirect
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421 // indirect
github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742 // indirect
)
34 changes: 34 additions & 0 deletions tools/average-resource-consumption/go.sum
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
github.com/cloudevents/sdk-go/v2 v2.5.0 h1:Ts6aLHbBUJfcNcZ4ouAfJ4+Np7SE1Yf2w4ADKRCd7Fo=
github.com/cloudevents/sdk-go/v2 v2.5.0/go.mod h1:nlXhgFkf0uTopxmRXalyMwS2LG70cRGPrxzmjJgSG0U=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/uuid v1.1.1 h1:Gkbcsh/GbpXz7lPftLA3P6TYMwjCLYm83jiFQZF/3gY=
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/json-iterator/go v1.1.10 h1:Kz6Cvnvv2wGdaG/V8yMvfkmNiXq9Ya2KUv4rouJJr68=
github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421 h1:ZqeYNhU3OHLH3mGKHDcjJRFFRrJa6eAM5H+CtDdOsPc=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742 h1:Esafd1046DLDQ0W1YjYsBW+p8U2u7vzgW2SQVmlNazg=
github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0=
go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
61 changes: 61 additions & 0 deletions tools/average-resource-consumption/handle.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
package function

import (
"context"
"fmt"

"github.com/cloudevents/sdk-go/v2/event"
"github.com/google/uuid"
)

type ResourceConsumptionMetric struct {
Value float64 `json:"value"`
Unit string `json:"unit"`
}

type AverageResourceConsumption struct {
CPU ResourceConsumptionMetric `json:"cpu"`
Memory ResourceConsumptionMetric `json:"memory"`
Month string `json:"month"`
}

// Handle an event.
func Handle(ctx context.Context, e event.Event) (*event.Event, error) {
fmt.Printf("Received a new event\n%s", e.String())
mockData := []AverageResourceConsumption{
{CPU: ResourceConsumptionMetric{Value: 2, Unit: "Cores"}, Memory: ResourceConsumptionMetric{Value: 8200, Unit: "MiB"}, Month: "March"},
{CPU: ResourceConsumptionMetric{Value: 2.5, Unit: "Cores"}, Memory: ResourceConsumptionMetric{Value: 8117, Unit: "MiB"}, Month: "April"},
{CPU: ResourceConsumptionMetric{Value: 3.5, Unit: "Cores"}, Memory: ResourceConsumptionMetric{Value: 9217, Unit: "MiB"}, Month: "May"},
{CPU: ResourceConsumptionMetric{Value: 4.5, Unit: "Cores"}, Memory: ResourceConsumptionMetric{Value: 10117, Unit: "MiB"}, Month: "June"},
}

response := event.New()
response.SetID(uuid.New().String())
response.SetType("average.resource.consumption.response")
response.SetSource("/average-resource-consumption")
response.SetExtension("responseid", e.ID())
err := response.SetData(event.TextJSON, mockData)
if err != nil {
fmt.Printf("failed to set data on event: %s", err.Error())
return nil, err
}
return &response, nil // echo to caller
}

/*
Other supported function signatures:
Handle()
Handle() error
Handle(context.Context)
Handle(context.Context) error
Handle(event.Event)
Handle(event.Event) error
Handle(context.Context, event.Event)
Handle(context.Context, event.Event) error
Handle(event.Event) *event.Event
Handle(event.Event) (*event.Event, error)
Handle(context.Context, event.Event) *event.Event
Handle(context.Context, event.Event) (*event.Event, error)
*/
32 changes: 32 additions & 0 deletions tools/average-resource-consumption/handle_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
package function

import (
"context"
"testing"

"github.com/cloudevents/sdk-go/v2/event"
)

// TestHandle ensures that Handle accepts a valid CloudEvent without error.
func TestHandle(t *testing.T) {
// Assemble
e := event.New()
e.SetID("id")
e.SetType("type")
e.SetSource("source")
e.SetData("text/plain", "data")

// Act
echo, err := Handle(context.Background(), e)
if err != nil {
t.Fatal(err)
}

// Assert
if echo == nil {
t.Errorf("received nil event") // fail on nil
}
if string(echo.Data()) != "data" {
t.Errorf("the received event expected data to be 'data', got '%s'", echo.Data())
}
}
5 changes: 5 additions & 0 deletions tools/resource-cost-calculator/.funcignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

# Use the .funcignore file to exclude files which should not be
# tracked in the image build. To instruct the system not to track
# files in the image build, add the regex pattern or file information
# to this file.
5 changes: 5 additions & 0 deletions tools/resource-cost-calculator/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

# Functions use the .func directory for local runtime data which should
# generally not be tracked in source control. To instruct the system to track
# .func in source control, comment the following line (prefix it with '# ').
/.func
23 changes: 23 additions & 0 deletions tools/resource-cost-calculator/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Go Cloud Events Function

Welcome to your new Go Function! The boilerplate function code can be found in [`handle.go`](handle.go). This Function is meant to respond exclusively to [Cloud Events](https://cloudevents.io/), but you can remove the check for this in the function and it will respond just fine to plain vanilla incoming HTTP requests.

## Development

Develop new features by adding a test to [`handle_test.go`](handle_test.go) for each feature, and confirm it works with `go test`.

Update the running analog of the function using the `func` CLI or client library, and it can be invoked using a manually-created CloudEvent:

```console
curl -v -X POST -d '{"message": "hello"}' \
-H'Content-type: application/json' \
-H'Ce-id: 1' \
-H'Ce-source: cloud-event-example' \
-H'Ce-subject: Echo content' \
-H'Ce-type: MyEvent' \
-H'Ce-specversion: 1.0' \
http://localhost:8080/
```

For more, see [the complete documentation]('https://github.com/knative/func/tree/main/docs')

16 changes: 16 additions & 0 deletions tools/resource-cost-calculator/func.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
specVersion: 0.36.0
name: resource-cost-calculator
runtime: go
registry: quay.io/cali0707
namespace: default
created: 2024-06-25T14:54:34.256280245-04:00
invoke: cloudevent
build:
builder: pack
deploy:
namespace: default
image: quay.io/cali0707/resource-cost-calculator@sha256:d0c71347d3df508320ddb96355ce61e58645797febd8f9f9324f9638ffbd3205
subscriptions:
- source: chat-broker
filters:
type: resource.cost.calculator
Loading

0 comments on commit 4bb05c8

Please sign in to comment.