Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support data collection rule #331

Merged
merged 9 commits into from
Aug 17, 2022
Merged

Support data collection rule #331

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
fda88b3
Support for data collection rule with Azure Monitor Logs
SenthuranSivananthan Aug 10, 2022
309cbb9
Validate for correct log levels
SenthuranSivananthan Aug 10, 2022
bf26188
Reduce log levels to warning and higher
SenthuranSivananthan Aug 10, 2022
0811037
Update onboarding document sample
SenthuranSivananthan Aug 10, 2022
e17fae7
Update logging archetype
SenthuranSivananthan Aug 10, 2022
426395b
Test: When DCR is disabled
SenthuranSivananthan Aug 10, 2022
3eb2d9a
Update schema changelog
SenthuranSivananthan Aug 10, 2022
515758f
Update schema changelog
SenthuranSivananthan Aug 10, 2022
728cf1c
Test: When DCR is enabled
SenthuranSivananthan Aug 10, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions azresources/monitor/dcr-azure-monitor-logs.bicep
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
@description('Location for the deployment.')
param location string = resourceGroup().location

@description('Name of the data collection rule')
param name string

@description('Windows Event Logs data source configuration.')
param windowsEventLogs array

@description('syslog data source configuration.')
param syslog array

@description('Log Analytics Workspace Id')
param logAnalyticsWorkspaceId string

resource dcr 'Microsoft.Insights/dataCollectionRules@2021-09-01-preview' = {
name: name
location: location
properties: {
dataSources: {
windowsEventLogs: windowsEventLogs
syslog: syslog
}
destinations: {
logAnalytics: [
{
name: 'logAnalytics'
workspaceResourceId: logAnalyticsWorkspaceId
}
]
}
dataFlows: [
{
streams: [
'Microsoft-Event'
'Microsoft-Syslog'
]
destinations: [
'logAnalytics'
]
}
]
}
}

output dcrId string = dcr.id
56 changes: 56 additions & 0 deletions config/logging/CanadaESLZ-main/logging.parameters.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,62 @@
},
"logAnalyticsAutomationAccountName": {
"value": "automation-account"
},
"dataCollectionRule": {
"value": {
"enabled": true,
"name": "DCR-AzureMonitorLogs",
"windowsEventLogs": [
{
"streams": [
"Microsoft-Event"
],
"xPathQueries": [
"Application!*[System[(Level=1 or Level=2 or Level=3)]]",
"Security!*[System[(band(Keywords,13510798882111488))]]",
"System!*[System[(Level=1 or Level=2 or Level=3)]]"
],
"name": "eventLogsDataSource"
}
],
"syslog": [
{
"streams": [
"Microsoft-Syslog"
],
"facilityNames": [
"auth",
"authpriv",
"cron",
"daemon",
"mark",
"kern",
"local0",
"local1",
"local2",
"local3",
"local4",
"local5",
"local6",
"local7",
"lpr",
"mail",
"news",
"syslog",
"user",
"uucp"
],
"logLevels": [
"Warning",
"Error",
"Critical",
"Alert",
"Emergency"
],
"name": "sysLogsDataSource"
}
]
}
}
}
}
60 changes: 59 additions & 1 deletion docs/archetypes/logging.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ Subscription can be moved to a target Management Group through Azure ARM Templat
| Microsoft Defender for Cloud | Configures security contact information (email and phone). |
| Subscription Role Assignments | Configures subscription scoped role assignments. Roles can be built-in or custom. |
| Subscription Budget | Configures monthly subscription budget with email notification. Budget is configured by default for 10 years and the amount. |
| Log Analytics | Configures Automation Account, Log Analytics Workspace and Log Analytics Solutions (AgentHealthAssessment, AntiMalware, AzureActivity, ChangeTracking, Security, SecurityInsights, ServiceMap, SQLAdvancedThreatProtection, SQLAssessment, SQLVulnerabilityAssessment, Updates, VMInsights). **SecurityInsights** solution pack will enable Microsoft Sentinel. |
| Log Analytics | Configures Automation Account, Log Analytics Workspace and Log Analytics Solutions (AgentHealthAssessment, AntiMalware, AzureActivity, ChangeTracking, Security, SecurityInsights, ServiceMap, SQLAdvancedThreatProtection, SQLAssessment, SQLVulnerabilityAssessment, Updates, VMInsights). **SecurityInsights** solution pack will enable Microsoft Sentinel. |
| Data Collection Rule | Configures one data collection rule with Windows Event Logs & syslog data sources. |
| Subscription Tags | A set of tags that are assigned to the subscription. |
| Resource Tags | A set of tags that are assigned to the resource group and resources. These tags must include all required tags as defined the Tag Governance policy. |

Expand Down Expand Up @@ -94,6 +95,7 @@ This example configures:
6. Resource Tags (aligned to the default tags defined in [Policies](../../policy/custom/definitions/policyset/Tags.parameters.json))
7. Automation Account
8. Log Analytics Workspace
9. Data Collection Rule

```json
{
Expand Down Expand Up @@ -171,6 +173,62 @@ This example configures:
},
"logAnalyticsAutomationAccountName": {
"value": "automation-account"
},
"dataCollectionRule": {
"value": {
"enabled": true,
"name": "DCR-AzureMonitorLogs",
"windowsEventLogs": [
{
"streams": [
"Microsoft-Event"
],
"xPathQueries": [
"Application!*[System[(Level=1 or Level=2 or Level=3)]]",
"Security!*[System[(band(Keywords,13510798882111488))]]",
"System!*[System[(Level=1 or Level=2 or Level=3)]]"
],
"name": "eventLogsDataSource"
}
],
"syslog": [
{
"streams": [
"Microsoft-Syslog"
],
"facilityNames": [
"auth",
"authpriv",
"cron",
"daemon",
"mark",
"kern",
"local0",
"local1",
"local2",
"local3",
"local4",
"local5",
"local6",
"local7",
"lpr",
"mail",
"news",
"syslog",
"user",
"uucp"
],
"logLevels": [
"Warning",
"Error",
"Critical",
"Alert",
"Emergency"
],
"name": "sysLogsDataSource"
}
]
}
}
}
}
Expand Down
Binary file modified docs/media/architecture/archetype-logging.jpg
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
112 changes: 112 additions & 0 deletions docs/onboarding/azure-devops-pipelines.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -565,6 +565,62 @@ This role assignment is used to grant users access to the logging subscription b
},
"logAnalyticsAutomationAccountName": {
"value": "automation-account"
},
"dataCollectionRule": {
"value": {
"enabled": true,
"name": "DCR-AzureMonitorLogs",
"windowsEventLogs": [
{
"streams": [
"Microsoft-Event"
],
"xPathQueries": [
"Application!*[System[(Level=1 or Level=2 or Level=3)]]",
"Security!*[System[(band(Keywords,13510798882111488))]]",
"System!*[System[(Level=1 or Level=2 or Level=3)]]"
],
"name": "eventLogsDataSource"
}
],
"syslog": [
{
"streams": [
"Microsoft-Syslog"
],
"facilityNames": [
"auth",
"authpriv",
"cron",
"daemon",
"mark",
"kern",
"local0",
"local1",
"local2",
"local3",
"local4",
"local5",
"local6",
"local7",
"lpr",
"mail",
"news",
"syslog",
"user",
"uucp"
],
"logLevels": [
"Warning",
"Error",
"Critical",
"Alert",
"Emergency"
],
"name": "sysLogsDataSource"
}
]
}
}
}
}
Expand Down Expand Up @@ -1683,6 +1739,62 @@ Migration process:
},
"logAnalyticsAutomationAccountName": {
"value": "< value from var-logging-logAnalyticsAutomationAccountName >"
},
"dataCollectionRule": {
"value": {
"enabled": true,
"name": "DCR-AzureMonitorLogs",
"windowsEventLogs": [
{
"streams": [
"Microsoft-Event"
],
"xPathQueries": [
"Application!*[System[(Level=1 or Level=2 or Level=3)]]",
"Security!*[System[(band(Keywords,13510798882111488))]]",
"System!*[System[(Level=1 or Level=2 or Level=3)]]"
],
"name": "eventLogsDataSource"
}
],
"syslog": [
{
"streams": [
"Microsoft-Syslog"
],
"facilityNames": [
"auth",
"authpriv",
"cron",
"daemon",
"mark",
"kern",
"local0",
"local1",
"local2",
"local3",
"local4",
"local5",
"local6",
"local7",
"lpr",
"mail",
"news",
"syslog",
"user",
"uucp"
],
"logLevels": [
"Warning",
"Error",
"Critical",
"Alert",
"Emergency"
],
"name": "sysLogsDataSource"
}
]
}
}
}
}
Expand Down
Binary file modified docs/visio/05 - Archetypes.vsdx
View file
Open in desktop
Binary file not shown.
18 changes: 18 additions & 0 deletions landingzones/lz-platform-logging/main.bicep
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,9 @@ param logAnalyticsAutomationAccountName string
@description('Log Analytics Workspace Data Retention in days.')
param logAnalyticsRetentionInDays int

@description('Data Collection Rule configuration.')
param dataCollectionRule object

@description('Flag to determine whether delete lock should be created on resource group(s). Default: true')
param enableDeleteLockOnResourceGroup bool = true

Expand Down Expand Up @@ -227,6 +230,21 @@ module logAnalytics '../../azresources/monitor/log-analytics.bicep' = {
}
}

// Create Data Collection Rule
module dcr '../../azresources/monitor/dcr-azure-monitor-logs.bicep' = if (dataCollectionRule.enabled) {
name: 'deploy-dcr'
scope: rgLogging
params: {
location: location

name: dataCollectionRule.name
windowsEventLogs: dataCollectionRule.windowsEventLogs
syslog: dataCollectionRule.syslog

logAnalyticsWorkspaceId: logAnalytics.outputs.workspaceResourceId
}
}

/*
Scaffold the subscription which includes:
* Microsoft Defender for Cloud - Enable Azure Defender (all available options)
Expand Down
Loading