add sm custom kernel module #106
Conversation
Signed-off-by: Anton Kukushkin <[email protected]>
| - "sagemaker:DescribeDomain" | ||
| - "sagemaker:AddTags" | ||
| - "iam:PassRole" | ||
| Resource: "*" |
There was a problem hiding this comment.
too wide open....lets discuss
There was a problem hiding this comment.
thanks good catch, especially pass role, will scope down
There was a problem hiding this comment.
Scoped down iam:PassRole.
One interesting caveat regarding ECR deployment IAM permissions - ECRDeployment construct always adds a policy that contains wildcard permissions, even if you provide it with a custom role. Filed an issue for that.
Signed-off-by: Anton Kukushkin <[email protected]>
Signed-off-by: Anton Kukushkin <[email protected]>
| cache_image=$IMAGE_URI | ||
| docker pull ${cache_image} 2>&1 > /dev/null || true | ||
| cd docker/$SEEDFARMER_PARAMETER_SAGEMAKER_IMAGE_NAME && docker build --progress plain --cache-from=${cache_image} -t $IMAGE_URI . | ||
| docker push $IMAGE_URI |
There was a problem hiding this comment.
Instead of going with the traditional way of building docker images via CLI, you could use CDK way of building docker images. Ref: https://github.com/srinivasreddych/demo-with-mps3-reinvent23/blob/main/modules/pre-processing/image-extraction/stack.py#L68
This will look for a dockerfile, build and push it to ECR and you can avoid the build script and build commands in the deployspec.yaml
There was a problem hiding this comment.
Ah, nice, didn't know about this. Thanks!
There was a problem hiding this comment.
Refactored to use this. Thanks Srini!
| return sm_client.describe_image_version(ImageName=image_name) | ||
|
|
||
|
|
||
| def check_image_version_exists(): |
There was a problem hiding this comment.
I am not an SM expert, but is there a CDK way of doing this, like using an L3 construct?
There was a problem hiding this comment.
There is an L1 construct for image version, but I will need to check whether it does everything we need it to do.
There was a problem hiding this comment.
Refactored to use CDK constructs where possible - Image, ImageVersion, and AppImageConfig - check the latest commit please. Dealing with L1 is a bit annoying but arguably still easier than maintaining a custom script.
…loy image Signed-off-by: Anton Kukushkin <[email protected]>
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
Signed-off-by: Anton Kukushkin <[email protected]>
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
|
@dgraeber @srinivasreddych should I reopen this against https://github.com/awslabs/mlops-modules? |
|
Closed as this is moving to https://github.com/awslabs/mlops-modules |
Issue #, if available:
N/A
Description of changes:
Added SageMaker custom kernel module to IDF.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.