-
Notifications
You must be signed in to change notification settings - Fork 10
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* added missing -r in CLI USAGE pip installation for ETP THX JSR ;) * enhanced JSR feedback to SPLUNK.md * enhanced JSR feedback to SPLUNK.md * Fix minor typo * fixed a bug in the installation instructions etp -> eaa * API credential docs + version upgrades * references to api cred creation added * minor changelog addition * added filter code incl. err handling * fixed edgerc - sample file * fixed minor rawcmd issue, amended documentation for filtering * added doc link for ETP API filtering * amended new features to README.md * added cpu / mem warning to readme * fixed typo * Minor edits on API credentials doc * Minor change (screenshot dimension etc...) * Minor doc update * new mfa version * bumped ULS Version * fixed a bug in the re-fork counter to handover to docker after x-attempts * bumped to a higher version 0.9.0 * bumped EAA version + amended stuff to changelog * added FAQ document * amended changelog Co-authored-by: Antoine Drochon <[email protected]>
- Loading branch information
1 parent
a073d1c
commit 2d20b50
Showing
23 changed files
with
404 additions
and
34 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,4 @@ | ||
| FROM python:3.9.5-slim-buster | ||
| FROM python:3.9.6-slim-buster | ||
| LABEL maintainer="Mike Schiessl - [email protected]" | ||
| LABEL APP="Akamai Universal Log Streamer" | ||
|
|
||
|
|
@@ -8,8 +8,8 @@ ARG ULS_DIR="$HOMEDIR/uls" | |
| ARG EXT_DIR="$ULS_DIR/ext" | ||
|
|
||
| ARG ETP_CLI_VERSION="0.3.5" | ||
| ARG EAA_CLI_VERSION="0.3.9" | ||
| ARG MFA_CLI_VERSION="0.0.5" | ||
| ARG EAA_CLI_VERSION="0.4.1" | ||
| ARG MFA_CLI_VERSION="0.0.6" | ||
|
|
||
| # ENV VARS | ||
| ENV ULS_DIR=$ULS_DIR | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| # Additional ULS Features | ||
| This document handles the usage of features not explained anywhere else. | ||
|
|
||
| ## FILTER (--filter) Feature | ||
| This feature got introduced in ULS v0.0.5. | ||
| It allows to decrease the number of log lines sent to SIEM for cost, performance or security reasons. | ||
|
|
||
| The filter is **regex** based so it is capable of filtering "json" AND raw text events. | ||
| Only events **matching the filter pattern will be sent towards the selected SIEM** output. | ||
| Filtering can lead to higher CPU / MEMORY consumption in production environments. | ||
| For ETP there is also an option to [filter directly on the API request side](AKAMAI_API_CREDENTIALS.md#etp-api-event-filters). | ||
|
|
||
| ### Usage examples: | ||
| - Filter for "geo_country": "Germany" (in EAA Access logs) | ||
| ```bash | ||
| python3 bin/uls.py -i eaa -f access -o raw <additional params> --filter '.*"geo_country": "Germany".*' | ||
| ``` | ||
| - Filter for "actionName": "Block - Error Page" (in ETP Threat logs) | ||
| ```bash | ||
| python3 bin/uls.py -i etp -f threat -o raw <additional params> --filter '.*"actionName": "Block - Error Page".*' | ||
| ``` | ||
| Always test your filter with the "RAW" console output on the command line before you send the data towards a SIEM | ||
| ```bash | ||
| python3 bin/uls.py -i eaa --feed access --filter '.*"geo_country": "Germany".*' -o raw | ||
| ``` | ||
|
|
||
| ## RAWCMD (--rawcmd) Feature | ||
| This feature got introduced in ULS v0.0.3. | ||
| Attention: This is a pretty critical setting, which can break ULS behaviour. | ||
|
|
||
| Raw commands within ULS can be used to trigger cli calls, that have not been integrated into ULS (yet). | ||
| This allows a more flexible implementation to solve some edge cases. | ||
| RAWCMD just requires the input to be selected. | ||
|
|
||
| Example: | ||
| ```bash | ||
| python3 bin/uls.py -i etp --rawcmd 'event threat -f' -l debug -o raw | ||
| ``` | ||
| This will also run the etp threat feed in "tail -f" mode | ||
|
|
||
| Please be aware: Not all output from the cli will be redirected to ULS by default. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,173 @@ | ||
| # Configure Akamai API credentials for ULS<!-- omit in toc --> | ||
|
|
||
| This document describes how to create Akamai API credentials and configure them in Unified Log Streamer (ULS) to access the different Akamai products and data feeds. | ||
|
|
||
| ## Table of contents<!-- omit in toc --> | ||
|
|
||
| - [Feeds / API overview](#feeds--api-overview) | ||
| - [Setting up API credentials for ULS](#setting-up-api-credentials-for-uls) | ||
| - [Enterprise Application Access (EAA)](#enterprise-application-access-eaa) | ||
| - [EAA Legacy API (for Access and Admin Audit feeds)](#eaa-legacy-api-for-access-and-admin-audit-feeds) | ||
| - [EAA {OPEN} API (for Connector Health feed)](#eaa-open-api-for-connector-health-feed) | ||
| - [Enterprise Threat Protector (ETP)](#enterprise-threat-protector-etp) | ||
| - [ETP {OPEN} API Reporting](#etp-open-api-reporting) | ||
| - [Akamai MFA](#akamai-mfa) | ||
| - [MFA Integration for logging](#mfa-integration-for-logging) | ||
| - [Advanced .edgerc usage](#advanced-edgerc-usage) | ||
| - [Multiple customer contracts](#multiple-customer-contracts) | ||
| - [Partner & employee enhancement](#partner--employee-enhancement) | ||
| - [ETP API EVENT Filters](#etp-api-event-filters) | ||
|
|
||
| ## Feeds / API overview | ||
|
|
||
| |Product long name|Acronym|Feed|API| | ||
| |---|---|---|---| | ||
| |Enterprise Application Access|EAA|ACCESS|[EAA Legacy API](#eaa-legacy-api-for-access-and-admin-audit-feeds)| | ||
| |Enterprise Application Access|EAA|ADMIN|[EAA Legacy API](#eaa-legacy-api-for-access-and-admin-audit-feeds)| | ||
| |Enterprise Application Access|EAA|HEALTH|[{OPEN} API / Enterprise Application Access](#eaa-open-api-for-connector-health-feed)| | ||
| |Enterprise Threat Protector|ETP|THREAT|[{OPEN} API / ETP Report](#etp-open-api-reporting)| | ||
| |Enterprise Threat Protector|ETP|AUP|[{OPEN} API / ETP Report](#etp-open-api-reporting)| | ||
| |Akamai MFA|MFA|AUTH|[MFA Integration](#mfa-integration-for-logging)| | ||
| |Akamai MFA|MFA|POLICY|[MFA Integration](#mfa-integration-for-logging)| | ||
|
|
||
| ## Setting up API credentials for ULS | ||
|
|
||
| ULS will read the API credentials from a text file, by default named `.edgerc` and stored in the home directory of the current user. The credentials configuration file can have multiple sections allowing to use multiple tenants (in case of multi-contract structure, or Akamai Partner). | ||
|
|
||
| Some basic information around `.edgerc` can be found [here](https://developer.akamai.com/legacy/introduction/Conf_Client.html). | ||
| This repo also provides a [.edgerc sample file](examples/.edgerc-sample) with all config sections added and explained. | ||
|
|
||
| Feel free to use the file as a template and comment out the sections not needed with `;`. | ||
|
|
||
| You'll find below all the details how to create the credentials based on the Akamai Data you plan to use with ULS. | ||
|
|
||
| ### Enterprise Application Access (EAA) | ||
|
|
||
| #### EAA Legacy API (for Access and Admin Audit feeds) | ||
|
|
||
| To create **EAA Legacy API** credentials, connect to [Akamai Control Center](https://control.akamai.com) | ||
|
|
||
| - Select **Enterprise Center** from the main navigation menu on the left | ||
| - Navigate to **General Settings** > **Settings** | ||
| - Select the **API Keys** tab | ||
| - Click **Generate new API Key** top right button | ||
| - Enter a name and a description | ||
| - On the confirmation screen, copy the **Key** and the **Secret**: | ||
| - <img src="images/uls_apicreds_eaa_ec.png" width="846" /> | ||
| - Add/replace/amend the following section to your `.edgerc` file and replace the data accordingly, example in the default section: | ||
|
|
||
| ```INI | ||
| [default] | ||
| ; API credentials for EAA access and admin logs | ||
| eaa_api_host = manage.akamai-access.com | ||
| eaa_api_key = XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXXX | ||
| eaa_api_secret = XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXXX | ||
| ``` | ||
|
|
||
| #### EAA {OPEN} API (for Connector Health feed) | ||
|
|
||
| To create **Akamai {OPEN} API** credentials, please follow [these instructions](https://developer.akamai.com/legacy/introduction/Prov_Creds.html). | ||
|
|
||
| Make sure the API user has `READ-WRITE` permission on the **Enterprise Application Access** API. For ULS usage, it is safe to provide all required API permission (such as EAA, ETP) to a single API user. | ||
|
|
||
| <img src="images/uls_apicreds_eaa_openapi.png" width="1251" alt="Edit API client screenshot in Akamai Control Center" /> | ||
|
|
||
| Please add/replace/amend the following section to your `.edgerc` file and replace the data accordingly, example in the default section: | ||
|
|
||
| ```INI | ||
| [default] | ||
| ; Akamai {OPEN} API credentials | ||
| host = akaa-xxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx.luna.akamaiapis.net | ||
| client_token = akab-xxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx | ||
| client_secret = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx | ||
| access_token = akab-xxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx | ||
| ``` | ||
|
|
||
| ### Enterprise Threat Protector (ETP) | ||
|
|
||
| #### ETP {OPEN} API Reporting | ||
|
|
||
| To create **AKAMAI {OPEN} API** credentials, please follow [these instructions](https://developer.akamai.com/legacy/introduction/Prov_Creds.html). | ||
|
|
||
| Make sure the API user has **READ-WRITE** permission to the **etp-config** API | ||
| For ULS usage, it is safe to provide all required roles (such as EAA, ETP) to a single api user. | ||
|
|
||
| For ETP usage, an additional config value (**etp_config_id**) is required. | ||
| The `etp_config_id` value can be obtained as follows: | ||
| - Connect to [Akamai Control Center](https://control.akamai.com) | ||
| - Select **Enterprise Center** | ||
| - Select **Locations** > **Locations** (or any other ETP specific page) | ||
| - Check out the URL bar of your browser, locate your **ETP configuration identifier** between `/etp/` and `/location/`: | ||
|  | ||
|
|
||
| Please add/replace/amend the following section to your `.edgerc` file and replace the data accordingly: | ||
| ```INI | ||
| [default] | ||
| ; Akamai {OPEN} API credentials | ||
| host = akaa-xxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx.luna.akamaiapis.net | ||
| client_token = akab-xxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx | ||
| client_secret = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx | ||
| access_token = akab-xxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx | ||
|
|
||
| ; ETP Config ID (required for ETP usage, can be obtained from the Akamai Web Interface) | ||
| etp_config_id = your-ETP-config-ID | ||
| ``` | ||
|
|
||
| ### Akamai MFA | ||
|
|
||
| #### MFA Integration for logging | ||
|
|
||
| To create **MFA Integration** credentials, connect to [Akamai Control Center](https://control.akamai.com). | ||
|
|
||
| - Select **Enterprise Center** from the main navigation menu on the left | ||
| - Navigate to **MFA** > **Integrations** | ||
| - Click on (+) to add a new MFA integration: | ||
| <img src="images/uls_apicreds_mfa_create.png" width="778" /> | ||
| - Confirm by clicking the **Save & Deploy** button | ||
| - **Copy** the credentials as shown below: | ||
| <img src="images/uls_apicreds_mfa_creds.png" width="366" /> | ||
| - Add/replace/amend the following section to your `.edgerc` file and replace the data accordingly: | ||
|
|
||
| ```INI | ||
| [default] | ||
| ; Akamai MFA logging integration credentials | ||
| mfa_integration_id = app_xxxxxxxxxxxxxxxxxxxxx | ||
| mfa_signing_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx | ||
| ``` | ||
|
|
||
| ## Advanced .edgerc usage | ||
|
|
||
| ### Multiple customer contracts | ||
|
|
||
| If your organization has multiple contracts, please add the following "contract_id" line to your `.edgerc` file in order select the proper contract. | ||
| If ETP and EAA are on different contracts, we recommend the creation of two different `.edgerc` files. | ||
|
|
||
| ```INI | ||
| [default] | ||
| ; If your organization have multiple contracts with EAA service | ||
| ; please add it below. Contact your Akamai representative to obtain it | ||
| contract_id = A-B-1CD2E34 | ||
| ``` | ||
|
|
||
| ### Partner & employee enhancement | ||
|
|
||
| For Partners or AKAMAI employees please add the "extra_qs" line to your `.edgerc` file in order to switch towards the desired tenant. Please replace "TENANT-SWITCH-KEY" with the provided switch key. | ||
|
|
||
| ```INI | ||
| [default] | ||
| ; If you are a partner managing multiple customers, you can use the switchkey | ||
| ; For more information, see: | ||
| ; https://learn.akamai.com/en-us/learn_akamai/getting_started_with_akamai_developers/developer_tools/accountSwitch.html | ||
| extra_qs = accountSwitchKey=TENANT-SWITCH-KEY | ||
| ``` | ||
|
|
||
| ### ETP API EVENT Filters | ||
|
|
||
| For Enterprise Threat protector (ETP), events can already be filtered at API level, so they won't even be transferred towards ULS. | ||
| This can be used for performance / scaling as well for cost saving reasons. | ||
| Please find more information around filtering on ETP API in the [ETP APIv3 documentation](https://developer.akamai.com/api/enterprise_security/enterprise_threat_protector_reporting/v3.html#filter) | ||
|
|
||
| ```INI | ||
| [default] | ||
| etp_event_filters = {"list":{"nin":["12345"]}} | ||
| ``` |
Oops, something went wrong.