diff --git a/sdk/evaluation/azure-ai-evaluation/CHANGELOG.md b/sdk/evaluation/azure-ai-evaluation/CHANGELOG.md index 758e9c980987..dfb1ba9caaa3 100644 --- a/sdk/evaluation/azure-ai-evaluation/CHANGELOG.md +++ b/sdk/evaluation/azure-ai-evaluation/CHANGELOG.md @@ -15,8 +15,50 @@ ## 1.1.0 (2024-12-12) +### Features Added +- Added image support in `ContentSafetyEvaluator`, `ViolenceEvaluator`, `SexualEvaluator`, `SelfHarmEvaluator`, `HateUnfairnessEvaluator` and `ProtectedMaterialEvaluator`. Provide image URLs or base64 encoded images in `conversation` input for image evaluation. See below for an example: + +```python +evaluator = ContentSafetyEvaluator(credential=azure_cred, azure_ai_project=project_scope) +conversation = { + "messages": [ + { + "role": "system", + "content": [ + {"type": "text", "text": "You are an AI assistant that understands images."} + ], + }, + { + "role": "user", + "content": [ + {"type": "text", "text": "Can you describe this image?"}, + { + "type": "image_url", + "image_url": { + "url": "https://cdn.britannica.com/68/178268-050-5B4E7FB6/Tom-Cruise-2013.jpg" + }, + }, + ], + }, + { + "role": "assistant", + "content": [ + { + "type": "text", + "text": "The image shows a man with short brown hair smiling, wearing a dark-colored shirt.", + } + ], + }, + ] +} +print("Calling Content Safety Evaluator for multi-modal") +score = evaluator(conversation=conversation) +``` + +- Please switch to generic evaluators for image evaluations as mentioned above. `ContentSafetyMultimodalEvaluator`, `ContentSafetyMultimodalEvaluatorBase`, `ViolenceMultimodalEvaluator`, `SexualMultimodalEvaluator`, `SelfHarmMultimodalEvaluator`, `HateUnfairnessMultimodalEvaluator` and `ProtectedMaterialMultimodalEvaluator` will be deprecated in the next release. + ### Bugs Fixed -- Removed `[remote]` extra. This is no longer needed when tracking results in Azure AI Studio. +- Removed `[remote]` extra. This is no longer needed when tracking results in Azure AI Foundry portal. - Fixed `AttributeError: 'NoneType' object has no attribute 'get'` while running simulator with 1000+ results ## 1.0.1 (2024-11-15)