logging_events.mdx

title	description	icon
Logging Events		notes

Wyvern has a standardized way of logging events that’s useful either for debugging, observability and training/improving your models. Here are the events wyvern records:

Event Type	Explanation	Data Being Logged
Candidate event	Wyvern records each candidate passed into Wyvern from it’s caller.	1. The entity id (ie product_id) 2. The entity score (from your search, OpenSearch or algolia) 3. The candidate order (from from your search)
Feature event	Wyvern records each feature generated for the purposes of evaluating model	1. The entity id (ie product_id) 2. The feature identifier (ie what entity the feature corresponds to.. for example the entity’s brand, or query:product) 3. feature name 4. feature value
Model event	Wyvern records the model scores for each candidate. This includes the actual predictions for each model that’s evaluated, including the ensembling of multiple models	1. The entity id 2. The model source 3. The model generated score
Business logic event	Wyvern records each candidate which had business logic applied to it. whether the ML model results was modified or not and what was change was)	1. The entity id (ie product_id) 2. The business logic application source 3. The original candidate score 4. The adjusted candidate score
Impression event	Wyvern records the final scores for each candidate, after it has gone through Model Scoring and Business logic pipelines	1. Impression id 2. Impression type 3. Impression order (this will be 0 for non-ranking situations) 4. Impression score
Custom event	Wyvern records your custom event data. See Custom Logging to see how to build a custom event log	1. The custom entity id 2. The custom entity identifier type 3. custom data

Custom logging

Here’s an example of how to log custom event data within Wyvern:

from wyvern.event_logging import event_logger
from wyvern.components.events.events import EntityEventData

# define your custom event data
class EmailEventData(EntityEventData):
    owner_name: str
    title: str

# log the event
event_logger.log_custom_events(
    events=[
        EmailEventData(
            entity_identifier="[email protected]",
            entity_identifier_type="email",
            owner_name="Jim",
            title="Check-in",
            ),
        EmailEventData(
            entity_identifier="[email protected]",
            entity_identifier_type="email",
            owner_name="Shu",
            title="Hello World",
        ),
    ]
)

1. import the EntityEventData and build your custom entity event data like the EmailEventData in this exampl

   e

2. import event_logger from wyvern.components.events.events and call event_logger.log_custom_events to log your custom event

   s

Integration

By default, event logging is always enabled. This might cause error if you have not set up your event logging integration. To disable Wyvern event logging, set the EVENT_LOGGING_ENABLED=false in your environment.

Raw Event Logging Integration

Currently wyvern is only integrated with AWS kinesis, which sends all the events to your destination.

Steps to set up kinesis with Wyvern:

1. Check out Creating an Amazon Kinesis Data Firehose Delivery Stream to set up kinesis and send the data to s3.
2. Once you set up your kinesis delivery system, set up an IAM role that has access to the firehose delivery stream.
3. Set up your AWS configurations in the environment for running wyvern:

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_REGION_NAME

Data Warehouse Integration

Snowflake

Use snowpipe to auto ingest wyvern's events to your data warehouse.

Event Transformation (Coming soon)

Wyvern provides a built-in transformer to transform the raw event logs into event tables show in this page.