Extract pbi sensitivity label information (#658)

* Extract pbi sensitivity label information

* Add dependency

* Refactor

* Add tests

* Bump version

* Update readme

* Update readme

* Address reviewer comments

* Address reviewer comments

* Update readme

metaphor/common/api_request.py

@@ -5,7 +5,7 @@
 from urllib.parse import urlparse
 
 import requests
-from pydantic import ValidationError, parse_obj_as
+from pydantic import TypeAdapter, ValidationError
 
 from metaphor.common.logger import debug_files, get_logger
 
@@ -46,7 +46,7 @@ def get_request(
             json.dump(result.json(), fp, indent=2)
         debug_files.append(out_file)
         try:
-            return parse_obj_as(type_, transform_response(result))
+            return TypeAdapter(type_).validate_python(transform_response(result))
         except ValidationError as error:
             logger.error(
                 f"url: {url}, result: {json.dumps(result.json())}, error: {error}"

metaphor/power_bi/README.md

@@ -8,11 +8,20 @@ We recommend creating a dedicated Azure AD Application and a dedicated security
 
 1. Follow [Step 1 of this doc](https://docs.microsoft.com/en-us/power-bi/developer/embedded/embed-service-principal#step-1---create-an-azure-ad-app) to create an Azure AD app and a client secret.
 
-> Note: Make sure to NOT add any Power BI Service permissions to the app. Doing so will lead to authentication errors when calling the APIs. See [this notice](https://docs.microsoft.com/en-us/power-bi/enterprise/read-only-apis-service-principal-authentication#:~:text=Make%20sure%20there%20are%20no%20Power%20BI%20admin%2Dconsent%2Drequired%20permissions%20set%20on%20this%20application.%20For%20more%20information%2C%20see%20Managing%20consent%20to%20applications%20and%20evaluating%20consent%20requests.) for more details.
+    > Note: Make sure to NOT add any Power BI Service permissions to the app. Doing so will lead to authentication errors when calling the APIs. See [this notice](https://docs.microsoft.com/en-us/power-bi/enterprise/read-only-apis-service-principal-authentication#:~:text=Make%20sure%20there%20are%20no%20Power%20BI%20admin%2Dconsent%2Drequired%20permissions%20set%20on%20this%20application.%20For%20more%20information%2C%20see%20Managing%20consent%20to%20applications%20and%20evaluating%20consent%20requests.) for more details.
 
-2. Follow [this doc](https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/active-directory-groups-create-azure-portal#create-a-basic-group-and-add-members) to create a security group and add the app's service principal as a member.
+2. (Optional) To get the sensitivity labe information. Follow the steps to ddd `InformationProtectionPolicy.Read.All` to the dedicated Azure AD application.
+    1. Sign into the Azure portal
+    2. Select Azure Active Directory, then Enterprise applications.
+    3. Select the application you created
+    4. Select `Permissions`, and `Add a permission`
+    5. Select `Microsoft Graph` -> `Application permissions`
+    6. Find `InformationProtectionPolicy.Read.All` and click add button below.
+    7. Grant the permission you select.
 
-3. Log into [Power BI Admin Portal](https://app.powerbi.com/admin-portal/tenantSettings) as a Power BI admin, enable the following settings under **Admin API settings** for the security group created in the previous step:
+3. Follow [this doc](https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/active-directory-groups-create-azure-portal#create-a-basic-group-and-add-members) to create a security group and add the app's service principal as a member.
+
+4. Log into [Power BI Admin Portal](https://app.powerbi.com/admin-portal/tenantSettings) as a Power BI admin, enable the following settings under **Admin API settings** for the security group created in the previous step:
     - Allow service principals to use Power BI APIs
     - Allow service principals to use read-only Power BI admin APIs
     - Enhance admin APIs responses with detailed metadata
@@ -58,6 +67,7 @@ When extracting lineages, the connector parses the snowflake account directly fr
 snowflake_account: <snowflake_account>
 ```
 
+
 ## Testing
 
 Follow the [Installation](../../README.md) instructions to install `metaphor-connectors` in your environment (or virtualenv). Make sure to include either `all` or `power_bi` extra.