Added changelog for image evaluation (Azure#38907)

* Added changelog for image evaluation * Update sdk/evaluation/azure-ai-evaluation/CHANGELOG.md Co-authored-by: Minsoo Thigpen <[email protected]> * Added evaluator names * Polish * Update sdk/evaluation/azure-ai-evaluation/CHANGELOG.md Co-authored-by: Minsoo Thigpen <[email protected]> --------- Co-authored-by: Minsoo Thigpen <[email protected]>
azure-sdk · Dec 20, 2024 · 10033b8 · 10033b8
1 parent 0959b98
commit 10033b8
Showing 1 changed file with 43 additions and 1 deletion.
diff --git a/sdk/evaluation/azure-ai-evaluation/CHANGELOG.md b/sdk/evaluation/azure-ai-evaluation/CHANGELOG.md
@@ -15,8 +15,50 @@
 
 ## 1.1.0 (2024-12-12)
 
+### Features Added
+- Added image support in `ContentSafetyEvaluator`, `ViolenceEvaluator`, `SexualEvaluator`, `SelfHarmEvaluator`, `HateUnfairnessEvaluator` and `ProtectedMaterialEvaluator`. Provide image URLs or base64 encoded images in `conversation` input for image evaluation. See below for an example:
+
+```python
+evaluator = ContentSafetyEvaluator(credential=azure_cred, azure_ai_project=project_scope)
+conversation = {
+    "messages": [
+        {
+            "role": "system",
+            "content": [
+                {"type": "text", "text": "You are an AI assistant that understands images."}
+            ],
+        },
+        {
+            "role": "user",
+            "content": [
+                {"type": "text", "text": "Can you describe this image?"},
+                {
+                    "type": "image_url",
+                    "image_url": {
+                        "url": "https://cdn.britannica.com/68/178268-050-5B4E7FB6/Tom-Cruise-2013.jpg"
+                    },
+                },
+            ],
+        },
+        {
+            "role": "assistant",
+            "content": [
+                {
+                    "type": "text",
+                    "text": "The image shows a man with short brown hair smiling, wearing a dark-colored shirt.",
+                }
+            ],
+        },
+    ]
+}
+print("Calling Content Safety Evaluator for multi-modal")
+score = evaluator(conversation=conversation)
+```
+
+- Please switch to generic evaluators for image evaluations as mentioned above. `ContentSafetyMultimodalEvaluator`, `ContentSafetyMultimodalEvaluatorBase`, `ViolenceMultimodalEvaluator`, `SexualMultimodalEvaluator`, `SelfHarmMultimodalEvaluator`, `HateUnfairnessMultimodalEvaluator` and `ProtectedMaterialMultimodalEvaluator` will be deprecated in the next release.
+
 ### Bugs Fixed
-- Removed `[remote]` extra. This is no longer needed when tracking results in Azure AI Studio.
+- Removed `[remote]` extra. This is no longer needed when tracking results in Azure AI Foundry portal.
 - Fixed `AttributeError: 'NoneType' object has no attribute 'get'` while running simulator with 1000+ results
 
 ## 1.0.1 (2024-11-15)