Skip to content

Commit

Permalink
docs(ingest/glue): add permissions for glue (#12290)
Browse files Browse the repository at this point in the history
  • Loading branch information
anshbansal authored Jan 10, 2025
1 parent efc5d31 commit a92a107
Showing 1 changed file with 14 additions and 1 deletion.
15 changes: 14 additions & 1 deletion metadata-ingestion/src/datahub/ingestion/source/aws/glue.py
Original file line number Diff line number Diff line change
Expand Up @@ -248,6 +248,9 @@ def report_table_dropped(self, table: str) -> None:
"Enabled by default when stateful ingestion is turned on.",
)
@capability(SourceCapability.LINEAGE_COARSE, "Enabled by default")
@capability(
SourceCapability.LINEAGE_FINE, "Support via the `emit_s3_lineage` config field"
)
class GlueSource(StatefulIngestionSourceBase):
"""
Note: if you also have files in S3 that you'd like to ingest, we recommend you use Glue's built-in data catalog. See [here](../../../../docs/generated/ingestion/sources/s3.md) for a quick guide on how to set up a crawler on Glue and ingest the outputs with DataHub.
Expand Down Expand Up @@ -284,12 +287,22 @@ class GlueSource(StatefulIngestionSourceBase):
"Action": [
"glue:GetDataflowGraph",
"glue:GetJobs",
"s3:GetObject",
],
"Resource": "*"
}
```
plus `s3:GetObject` for the job script locations.
For profiling datasets, the following additional permissions are required:
```json
{
"Effect": "Allow",
"Action": [
"glue:GetPartitions",
],
"Resource": "*"
}
```
"""

Expand Down

0 comments on commit a92a107

Please sign in to comment.