Azure OpenAI Insights

Azure OpenAI Insights

The 'Azure OpenAI Insights' workbook offers deep insights into Azure OpenAI usage, helping you manage costs, optimize performance, and make strategic decisions for a robust AI infrastructure.

Introduction

In the ever-evolving world of Artificial Intelligence, organizations and entities across various sectors are on a quest to leverage advanced technologies efficiently. Azure OpenAI opens a realm of possibilities, offering both challenges and excitement, particularly for those at the early stages of AI adoption.

Read more in depth in this Tech Community blog: Azure OpenAI Insights: Monitoring AI with Confidence

This workbook offers deep insights into Azure OpenAI resources and usage (Platform Metrics and Logs) and can be powerful tool in analyzing & monitoring your AI initiatives.

Structure and Views

Structure

• Overview - Holistic view of Azure OpenAI resources
• Monitor - Holistic view of Azure OpenAI resources Metrics
• Insights - Holistic view of Azure OpenAI resources Logs

Views

• Overview

  • Azure OpenAI Resources by
    • SubscriptionId
    • Resource Group
    • Location
    • Kind
    • Public Network Access
    • Private Network Access
  • All Azure OpenAI Resources

• Monitor

  • Overview
    • Requests
    • Processed Inference Tokens
    • Processed Prompt Tokens
    • Generated Completions Tokens
    • Processed FineTuned Training Hours
    • Provisioned-managed Utilization
    • Active Tokens
    • Prompt Token Cache Match Rate
    • Time to Response
  • HTTP Requests
    • Requests
      • by Model Name
      • by Model Version
      • by Model Deployment Name
      • by Status Code
      • by StreamType
      • by Operation Name
      • by API Name
      • by Region
    • Time to Response
      • by Model Name
      • by Model Deployment Name
    • Prompt Token Cache Match Rate
      • by Model Name
      • by Model Deployment Name
  • Token-Based Usage
    • Processed Inference Tokens
      • by Model Name
      • by Model Deployment Name
    • Processed Prompt Tokens
      • by Model Name
      • by Model Deployment Name
    • Generate Completion Tokens
      • by Model Name
      • by Model Deployment Name
    • Active Tokens
      • by Model Name
      • by Model Deployment Name
  • PTU Utilization
    • Provisioned-managed Utilization
      • by Model Name
      • Model Version
      • by Model Deployment Name
      • by StreamType
      • by Region
  • Fine-tuning
    • Processed FineTuned Training Hours
      • by Model Name
      • by Model Deployment Name

• Insights

  • Overview
    • Requests
      • by Resource
      • by Location
      • by StreamType
      • by Api Version
      • by Model Deployment Name + Operation Name
      • by Model Deployment Name
      • by Model Name + Operation Name
      • by Model Name
      • by Operation Name
      • by Avg Duration (ms)
      • by Avg Request Length (bytes)
      • by Avg Response Length (bytes)
  • By CallerIP
    • Requests
    • Operation Name
    • Model Deployment Name + Operation Name
    • Model Name + Operation Name
    • Avg Duration (ms)
    • Avg Request Length (bytes)
    • Avg Response Length (bytes)
  • All Logs
    • Successful requests
  • Failures
    • Failed requests
      • by Resources
      • by Api Version
      • by Operation name
      • by Stream Type

Filters

Filter supported:

• Model Deployment Name
• Model Name
• Model Version
• Api Version
• Operation Name
• Stream Type
• Location

All the filters are related to each other to allow a granular view and simplify the tracking of the logs.

Average Duration (ms)

Average Request / Response Length (bytes)