This solution will simplify data governance tasks in an organization that uses Microsoft Purview as its core data catalog.
In a decentralized Data Platform like the Data Management & Analytics Scenario reference architecture, it becomes increasingly difficult for data governance personas to govern the data estate. Data Product teams can create their data services in a self-service way, which makes it increasingly difficult for data governance personas to onboard data sources, scan the content, classify the data and govern the quality of the data. This solution aims at automating these tasks.
The following automation is being taken care of by this solution:
- Discovery of Data Sources within the Data Platform.
- Onboarding of Data Sources within the respective Microsoft Purview Collection.
- Setup of Managed Private Endpoints onto the managed virtual network inside Purview.
- Setup of Scanning and Triggers for Data Sources.
- Setup of Lineage for supported Data Sources (Synapse, Data Factory, etc.).
Below you will find some of the core capabilities:
The solution onboards data sources automatically to a Microsoft Purview collection when a new data source gets created within an Azure subscription. The a collection structure will be automatically created within your Purview account.
The solution can remove data sources when they get deleted in Azure. Scaned data assets will not get removed when data sources get deleted from a Purview collection. This feature can be disabled.
The solution can automatically create and trigger scans for data services. This feature can be disabled.
You have two options, to deploy this solution to your Azure tenant:
- GitHub Actions and
- Deploy to Azure Button.
For more details, please visit the deployment documentation page.