Skip to content

Commit

Permalink
hotfix JSON to Parquet Trigger resource
Browse files Browse the repository at this point in the history
If you create two resources with the same output name (JsonToParquetTrigger) where one is created if {condition} and the other is created if {not condition}, then neither resource is created. Super intuitive! This fixes that by using the condition in a much cleaner way.
  • Loading branch information
philerooski authored Oct 13, 2023
1 parent 70875df commit f8aa14d
Showing 1 changed file with 1 addition and 15 deletions.
16 changes: 1 addition & 15 deletions templates/glue-workflow.j2
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,6 @@ Resources:
Name: !Sub ${Namespace}-JsonToParquetWorkflow

JsontoParquetTrigger:
Condition: IsMainNamespace
Type: AWS::Glue::Trigger
Properties:
Name: !Sub "${Namespace}-JsontoParquetTrigger"
Expand All @@ -132,20 +131,7 @@ Resources:
Description: This trigger starts the JSON to Parquet workflow.
Type: SCHEDULED
Schedule: !Ref JsontoParquetTriggerSchedule
StartOnCreation: true
WorkflowName: !Ref JsonToParquetWorkflow

JsontoParquetTrigger:
Condition: IsDevelopmentNamespace
Type: AWS::Glue::Trigger
Properties:
Name: !Sub "${Namespace}-JsontoParquetTrigger"
Actions:
- CrawlerName: !Ref StandardCrawler
Description: This trigger starts the JSON to Parquet workflow.
Type: SCHEDULED
Schedule: !Ref JsontoParquetTriggerSchedule
StartOnCreation: false
StartOnCreation: !If [IsMainNamespace, true, false]
WorkflowName: !Ref JsonToParquetWorkflow

StandardCrawler:
Expand Down

0 comments on commit f8aa14d

Please sign in to comment.