Skip to content

Commit

Permalink
Merge pull request #64 from marvinbuss/marvinbuss/microsoft_purview
Browse files Browse the repository at this point in the history
Change Docs to Microsoft Purview
  • Loading branch information
marvinbuss authored Nov 1, 2022
2 parents 452c5ce + 14dbf5b commit 1356fcc
Show file tree
Hide file tree
Showing 3 changed files with 8 additions and 8 deletions.
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Purview Event-driven Automation
# Microsoft Purview Event-driven Automation

---

_This solution will simplify data governance tasks in an organization that uses Purview as its core data catalog._
_This solution will simplify data governance tasks in an organization that uses Microsoft Purview as its core data catalog._

---

Expand All @@ -11,7 +11,7 @@ In a decentralized Data Platform like the [Data Management & Analytics Scenario]
The following automation is being taken care of by this solution:

1. Discovery of Data Sources within the Data Platform.
2. Onboarding of Data Sources within the respective Purview Collection.
2. Onboarding of Data Sources within the respective Microsoft Purview Collection.
3. Setup of Managed Private Endpoints onto the managed virtual network inside Purview.
4. Setup of Scanning and Triggers for Data Sources.
5. Setup of Lineage for supported Data Sources (Synapse, Data Factory, etc.).
Expand All @@ -20,7 +20,7 @@ Below you will find some of the core capabilities:

## Event-driven onboarding of Data Sources

The solution onboards data sources automatically to a Purview collection when a new data source gets created within an Azure subscription. The a collection structure will be automatically created within your Purview account.
The solution onboards data sources automatically to a Microsoft Purview collection when a new data source gets created within an Azure subscription. The a collection structure will be automatically created within your Purview account.

![Event-driven onboarding of Data Sources](/docs/images/PurviewOnboarding.gif)

Expand Down
6 changes: 3 additions & 3 deletions docs/Architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@

![Purview Automation Architecture](./images/Architecture.png)

The architecture is built on top of [Cloud Scale Analytics](https://github.com/Azure/data-management-zone) and can be deployed as natural extension to onboard data services at scale to your Azure Purview Account. It is recommended to deploy the solution into the [Data Management Zone](https://github.com/Azure/data-management-zone) as centralized solution and deploy a System Event Grid Topic for the event driven automation in each and every [Data Landing Zone](https://github.com/Azure/data-landing-zone).
The architecture is built on top of [Cloud Scale Analytics](https://github.com/Azure/data-management-zone) and can be deployed as natural extension to onboard data services at scale to your Microsoft Purview Account. It is recommended to deploy the solution into the [Data Management Zone](https://github.com/Azure/data-management-zone) as centralized solution and deploy a System Event Grid Topic for the event driven automation in each and every [Data Landing Zone](https://github.com/Azure/data-landing-zone).

## Prerequisites

Before deploying the solution, the following resources must exist: Azure Purview, Azure Virtual Network with two subnets (one subnet being delegated to `Microsoft.Web/serverFarms` and a second subnet with `privateEndpointNetworkPolicies` set to `Disabled`) and Azure Private DNS Zones for Azure Blob Storage (`privatelink.blob.core.windows.net`), Azure File Storage (`privatelink.file.core.windows.net`) and Azure Key Vault (`privatelink.vaultcore.azure.net`).
Before deploying the solution, the following resources must exist: Microsoft Purview, Azure Virtual Network with two subnets (one subnet being delegated to `Microsoft.Web/serverFarms` and a second subnet with `privateEndpointNetworkPolicies` set to `Disabled`) and Azure Private DNS Zones for Azure Blob Storage (`privatelink.blob.core.windows.net`), Azure File Storage (`privatelink.file.core.windows.net`) and Azure Key Vault (`privatelink.vaultcore.azure.net`).

All of these services can be created with the templates provided in the [prerequisites folder](/docs/reference/prerequisites/). Just modify the parameters in the [parameter file](/docs/reference/prerequisites/params.json) and then start deploying the template (Azure CLI: `az deployment sub create -l "<your-location>" -f "main.bicep" -p "params.json"`).

Expand All @@ -16,7 +16,7 @@ The architecture is using Private Endpoints and Vnet injection across the board.

* *Azure Event Grid System Topics*: The Azure Event Grid System Topics must be deployed per [Azure Data Landing Zone](https://github.com/Azure/data-landing-zone). This requirement arises from the fact that an Event Grid System Topics can only subscribe to events coming from the subscription it was deployed in. The Event Grid System Topic receives events about successfully deployed and deleted Azure resources to which consumers can subscribe.

* *Azure Function running on App Service Environment*: The Azure Function is running the [.NET 6 application in this repository](/code/PurviewAutomation/) and subscribes to the events delivered by each and every Azure Event Grid System Topic in the respective Data Landing Zones. When a new event is received the application code parses the event information about the successfully deployed Azure resource and onboards the service to the Azure Purview account to which is is connected to. When an event about a deleted service is received, the function will try to remove the data service from the respective collection in Purview.
* *Azure Function running on App Service Environment*: The Azure Function is running the [.NET 6 application in this repository](/code/PurviewAutomation/) and subscribes to the events delivered by each and every Azure Event Grid System Topic in the respective Data Landing Zones. When a new event is received the application code parses the event information about the successfully deployed Azure resource and onboards the service to the Microsoft Purview account to which is is connected to. When an event about a deleted service is received, the function will try to remove the data service from the respective collection in Purview.

* *Storage Account*: The storage account is used by the Azure Function and is required for it to properly function.

Expand Down
2 changes: 1 addition & 1 deletion docs/Deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ The following must be available to deploy the solution inside your Azure environ
- For the deployment, please choose one of the **Supported Regions**.
- It is also expected that a few resources have been deployed upfront. This includes:
- A Vnet with two subnets, whereas one needs to have `privateEndpointNetworkPolicies` disabled and the other one needs to be delegated to `Microsoft.Web/serverFarms`.
- An Azure Purview account in one of the supported regions.
- A Microsoft Purview account in one of the supported regions.
- Private DNS Zones for blob storage, file storage, and key vault.

### (Optional) Deployment of required services
Expand Down

0 comments on commit 1356fcc

Please sign in to comment.