Trying the Spot + Ondemand in Managed Autoscaling configuration, but there is ParamValidation Exception

[Pram098]

Describe the bug

Hi, recently i am testing the functionality where i was thinking of using few cores as on-demand and few core nodes as spot, but there seems to be validation exception, when i manually tried the same config in EMR, it is allowing those values.

{
"ComputeLimits": {
"UnitType": "Instances",
"MinimumCapacityUnits": 2,
"MaximumCapacityUnits": 26,
"MaximumOnDemandCapacityUnits": 12,
"MaximumCoreCapacityUnits": 15
}
}

my expectation was:
12 core on-demand, 3 core spot
11 spot task nodes
1 master node

But my boto version gives exception.

Expected Behavior

it shouldn't throw validation exception

Current Behavior

it is throwing
botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the RunJobFlow operation: Invalid configuration for managed scaling. If you select On-Demand market type for core nodes in a cluster with instance groups, MaximumCoreCapacityUnits must be less than or equal to MaximumOnDemandCapacityUnits.

Reproduction Steps

same as description

Possible Solution

same as description

Additional Information/Context

No response

SDK version used

1.18.59

Environment details (OS name and version, etc.)

amazon linux

[tim-finnigan]

Hi @Pram098 thanks for reaching out. The version 1.18.59 is quite old - the latest version of boto3/botocore is 1.33.7. (Here is the CHANGELOG for reference.)

Can you try updating to a more recent version to see if that resolves the issue?

[Pram098]

I checked again, i noticed that for Instance Fleet there isn't any problem but Instance Group there is restriction by AWS, i guess it's good to close it.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.