Performance regression in profile parser

[krousey]

Describe the bug

I have an AWS config file with 1,000+ profiles (generated from a just-in-time access tool). The latest version of the SDK cannot parse this file in a reasonable amount of time (I've never actually waited long enough for it to finish). This has not been a problem in previous versions of the SDK.

Expected Behavior

aws.LoadDefaultConfig(context.Background()) to return in less than 10 seconds.

Here's a benchmark run against github.com/aws/aws-sdk-go-v2/[email protected]

$ rm go.mod go.sum; go mod init awsprofilebench; go get github.com/aws/aws-sdk-go-v2/[email protected]; go test -bench=.
go: creating new go.mod: module awsprofilebench
go: to add module requirements and sums:
        go mod tidy
go: added github.com/aws/aws-sdk-go-v2 v1.9.0
go: added github.com/aws/aws-sdk-go-v2/config v1.7.0
go: added github.com/aws/aws-sdk-go-v2/credentials v1.4.0
go: added github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.5.0
go: added github.com/aws/aws-sdk-go-v2/internal/ini v1.2.2
go: added github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.3.0
go: added github.com/aws/aws-sdk-go-v2/service/sso v1.4.0
go: added github.com/aws/aws-sdk-go-v2/service/sts v1.7.0
go: added github.com/aws/smithy-go v1.8.0
goos: linux
goarch: amd64
pkg: awsprofilebench
cpu: 12th Gen Intel(R) Core(TM) i7-12700H
BenchmarkLoadProfile1-20             441           2649360 ns/op
BenchmarkLoadProfile10-20            448           2743787 ns/op
BenchmarkLoadProfile100-20           298           4191916 ns/op
BenchmarkLoadProfile1000-20           73          17328670 ns/op
PASS
ok      awsprofilebench 5.894s

Current Behavior

aws.LoadDefaultConfig(context.Background()) takes over a minute.

Here's a benchmark run against github.com/aws/aws-sdk-go-v2/[email protected]

$ rm go.mod go.sum; go mod init awsprofilebench; go get github.com/aws/aws-sdk-go-v2/[email protected]; go test -bench=.
go: creating new go.mod: module awsprofilebench
go: to add module requirements and sums:
        go mod tidy
go: added github.com/aws/aws-sdk-go-v2 v1.22.1
go: added github.com/aws/aws-sdk-go-v2/config v1.22.0
go: added github.com/aws/aws-sdk-go-v2/credentials v1.15.1
go: added github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.2
go: added github.com/aws/aws-sdk-go-v2/internal/configsources v1.2.1
go: added github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.5.1
go: added github.com/aws/aws-sdk-go-v2/internal/ini v1.5.0
go: added github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.10.1
go: added github.com/aws/aws-sdk-go-v2/service/sso v1.17.0
go: added github.com/aws/aws-sdk-go-v2/service/ssooidc v1.19.0
go: added github.com/aws/aws-sdk-go-v2/service/sts v1.25.0
go: added github.com/aws/smithy-go v1.16.0
goos: linux
goarch: amd64
pkg: awsprofilebench
cpu: 12th Gen Intel(R) Core(TM) i7-12700H
BenchmarkLoadProfile1-20             396           2738498 ns/op
BenchmarkLoadProfile10-20             82          14214374 ns/op
BenchmarkLoadProfile100-20             1        1061156978 ns/op
BenchmarkLoadProfile1000-20            1        113478498265 ns/op
PASS
ok      awsprofilebench 117.123s

Reproduction Steps

You can run the following benchmarks against the latest packages to see the performance. I have pasted 2 different runs in the behaviors section.

bench_test.go:

package awsprofilebench

import (
	"context"
	"fmt"
	"os"
	"testing"

	"github.com/aws/aws-sdk-go-v2/config"
)

func BenchmarkLoadProfile1(b *testing.B) {
	benchConfigLoad(b, 1)
}

func BenchmarkLoadProfile10(b *testing.B) {
	benchConfigLoad(b, 10)
}

func BenchmarkLoadProfile100(b *testing.B) {
	benchConfigLoad(b, 100)
}

func BenchmarkLoadProfile1000(b *testing.B) {
	benchConfigLoad(b, 1000)
}

func benchConfigLoad(b *testing.B, numProfiles int) {
	ctx := context.Background()
	profileFile, err := generateProfiles(numProfiles)
	if err != nil {
		b.Fatal(err)
	}
	defer os.Remove(profileFile)
	opts := config.WithSharedConfigFiles([]string{profileFile})

	b.ResetTimer()
	for n := 0; n < b.N; n++ {
		config.LoadDefaultConfig(ctx, opts)
	}
}

const profileTemplate = `
[profile role%d]
tool_sso_start_url  = https://example.awsapps.com/start
tool_sso_region     = us-west-2
tool_sso_account_id = 12345678901234
tool_sso_role_name  = some_role_name
tool_generated_from = some_tool
credential_process  = some_tool credential-process
`

func generateProfiles(n int) (string, error) {
	f, err := os.CreateTemp("", fmt.Sprintf("aws-bench-config-%d-*", n))
	if err != nil {
		return "", err
	}

	for i := 0; i < n; i++ {
		if _, err := fmt.Fprintf(f, profileTemplate, n); err != nil {
			f.Close()
			os.Remove(f.Name())
			return "", err
		}
	}

	if err := f.Close(); err != nil {
		os.Remove(f.Name())
		return "", err
	}

	return f.Name(), nil
}

Possible Solution

No response

Additional Information/Context

No response

SDK version used

1.22.1

Environment details (Version of Go (go version)? OS name and version, etc.)

1.20

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.