Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for preconditions in ShellCommandActivity, allow custom s3 paths in preconditions #245

Open
p5k6 opened this issue May 18, 2016 · 1 comment

Comments

@p5k6
Copy link
Contributor

p5k6 commented May 18, 2016

reference PR #244
Background: we're setting up an event-driven pipeline, where our Ops team delivers a file to a set s3 location, and should be run when delivered.
Adding support for preconditions in ShellCommandActivity allows us to build a custom step which begins execution when a path (using S3KeyExists) is available in s3.

I'll post up an example (probably tomorrow) which hopefully will explain/show a bit better :)

@p5k6
Copy link
Contributor Author

p5k6 commented May 19, 2016

here's the custom step and an example script that show how we've set up an event driven pipeline. When our ops team delivers to s3://my-s3-bucket-name/app/qa/full/#{format(@scheduledStartTime, 'YYYY-MM-dd')}/, then the pipeline kicks off.

This does only work where dataduct has pr #244 built (our staging branch has merged it for us)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant