StateFunc applied twice for resources under PRC

[flostadler]

What happened?

Resources under PRC have the StateFunc applied twice to their resources.
This leads to mangled data in state and perma diffs.

ApplyResourceChange applies the StateFunc, but the preceding Diff also applies it.
Often this is fine because StateFunc can be applied multiple times (e.g. normalizing json). But for some operations like calculating a hash it's not.

This causes: pulumi/pulumi-aws#4446

Example

• Perma diff in EC2 LaunchConfiguration userData causes replaces pulumi-aws#4446
• Repro: Pass PlannedPrivate from PlanResourceChange to ApplyResourceChange #2407

Output of pulumi about

n/a

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[t0yv0]

That's unfortunate. Does TF not apply it twice e.g not once in PlanResourceChange and once in ApplyResourceChange? Or is it that Pulumi specifically recomputes the plan and that is the problem?

[flostadler]

This seems to be caused by not setting NewExtra in the ResourceAttrDiff under PRC in the diff function. Without PRC this is set to the original value. The field readers return that value instead if it's set.

So right now under PRC the resource CRUD methods retrieve the value that's already been processed by StateFunc instead of the raw value and then process it again.

[flostadler]

We're loosing NewExtra here:

pulumi-terraform-bridge/pkg/tfshim/sdk-v2/provider2.go

Lines 284 to 287 in 69fe479

	diff := p.unpackDiff(ty, d)
	cfg, st, pl := diff.config, state.stateValue, diff.plannedState
	priv := diff.v2InstanceDiff.tf.Meta
	resp, err := p.server.ApplyResourceChange(ctx, t, ty, cfg, st, pl, priv, meta)

It's set in the diff.v2InstanceDiff.tf.Attributes, but we're not passing that down to ARC. Instead we're looking for NewExtra Fields in PlannedPrivate of the request. PlannedPrivate gets constructed from diff.v2InstanceDiff.tf.Meta, but that's nil in the cases listed in the issue.

Will investigate if we're loosing diff.v2InstanceDiff.tf.Meta somewhere or PlannedPrivate needs to be set to some other value.

[flostadler]

We're loosing PlannedMeta here:

pulumi-terraform-bridge/pkg/tfshim/sdk-v2/provider2.go

Lines 257 to 267 in 69fe479

	plan, err := p.server.PlanResourceChange(ctx, t, ty, cfg, st, prop, priv, meta, ic)
	if err != nil {
	return nil, err
	}
	return &v2InstanceDiff2{
	v2InstanceDiff: v2InstanceDiff{
	tf: plan.PlannedDiff,
	},
	config: cfg,
	plannedState: plan.PlannedState,
	}, nil

PRC returns it, but we're dropping it because the type v2InstanceDiff2 doesn't include it. We'll need to fix that

[pulumi-bot]

This issue has been addressed in PR #2407 and shipped in release v3.91.0.