Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Panic on protobuf schema conversion #1068

Closed
alecthomas opened this issue Mar 12, 2024 · 2 comments
Closed

Panic on protobuf schema conversion #1068

alecthomas opened this issue Mar 12, 2024 · 2 comments
Labels
bug Something isn't working next Work that will be be picked up next

Comments

@alecthomas
Copy link
Collaborator

I only saw this once and wasn't able to reproduce it. It might have been caused by a corrupted .pb file?

panic: unhandled type: <nil>

goroutine 1 [running]:
github.com/TBD54566975/ftl/backend/schema.typeToSchema(0x0?)
        /Users/alec/dev/ftl/backend/schema/protobuf_dec.go:68 +0x89c
github.com/TBD54566975/ftl/backend/schema.fieldToSchema(0x140001d8700)
        /Users/alec/dev/ftl/backend/schema/field.go:75 +0x64
github.com/TBD54566975/ftl/backend/schema.fieldListToSchema(...)
        /Users/alec/dev/ftl/backend/schema/field.go:65
github.com/TBD54566975/ftl/backend/schema.DataFromProto(0x140001b6fa0)
        /Users/alec/dev/ftl/backend/schema/data.go:175 +0x250
github.com/TBD54566975/ftl/backend/schema.declListToSchema({0x140003a8920, 0x3, 0x1014ea3f0?})
        /Users/alec/dev/ftl/backend/schema/protobuf_dec.go:27 +0xa4
github.com/TBD54566975/ftl/backend/schema.ModuleFromProto(0x14000328280)
        /Users/alec/dev/ftl/backend/schema/module.go:209 +0x68
github.com/TBD54566975/ftl/backend/schema.moduleListToSchema({0x14000333140, 0x5, 0x1014ea3f0?})
        /Users/alec/dev/ftl/backend/schema/module.go:229 +0x7c
github.com/TBD54566975/ftl/backend/schema.FromProto(0xc0?)
        /Users/alec/dev/ftl/backend/schema/schema.go:148 +0x2c
github.com/TBD54566975/ftl/buildengine.(*Engine).startSchemaSync(0x140001fd280, {0x1014ea3f0, 0x14000191bd0})
        /Users/alec/dev/ftl/buildengine/engine.go:76 +0x110
github.com/TBD54566975/ftl/buildengine.New({0x1014ea3b8?, 0x140003f6f00?}, {0x1014fae08, 0x140000f2bd0}, {0x14000059120, 0x1, 0x1})
        /Users/alec/dev/ftl/buildengine/engine.go:64 +0x2d8
main.(*buildCmd).Run(0x10208e078, {0x1014ea3b8, 0x140003f6f00})
        /Users/alec/dev/ftl/cmd/ftl/cmd_build.go:17 +0x54
reflect.Value.call({0x1012bd600?, 0x10208e078?, 0x14000587928?}, {0x100f2446f, 0x4}, {0x14000688b40, 0x1, 0x100a894e8?})
        /Users/alec/Library/Caches/hermit/pkg/go-1.22.1/src/reflect/value.go:596 +0x980
reflect.Value.Call({0x1012bd600?, 0x10208e078?, 0x14000587a28?}, {0x14000688b40?, 0x1014ca4e0?, 0x14000587960?})
        /Users/alec/Library/Caches/hermit/pkg/go-1.22.1/src/reflect/value.go:380 +0x94
github.com/alecthomas/kong.callFunction({0x1012bd600?, 0x10208e078?, 0x14000587bb0?}, 0x140003f70e0)
        /Users/alec/go/pkg/mod/github.com/alecthomas/[email protected]/callbacks.go:98 +0x390
github.com/alecthomas/kong.(*Context).RunNode(0x14000218800, 0x140004a5770, {0x14000587de8, 0x1, 0x101507990?})
        /Users/alec/go/pkg/mod/github.com/alecthomas/[email protected]/context.go:774 +0x630
github.com/alecthomas/kong.(*Context).Run(0x14000337680?, {0x14000587de8?, 0x140003f6f00?, 0x101237740?})
        /Users/alec/go/pkg/mod/github.com/alecthomas/[email protected]/context.go:799 +0x138
main.main()
        /Users/alec/dev/ftl/cmd/ftl/main.go:109 +0x8b0
@alecthomas alecthomas added the bug Something isn't working label Mar 12, 2024
@github-actions github-actions bot added the triage Issue needs triaging label Mar 12, 2024
@alecthomas alecthomas mentioned this issue Mar 12, 2024
Open
@alecthomas alecthomas added next Work that will be be picked up next and removed triage Issue needs triaging labels Mar 13, 2024
@alecthomas
Copy link
Collaborator Author

Saw this again:

info: Starting FTL with 1 controller(s)
2024/03/14 10:53:56 http2: panic serving 127.0.0.1:65335: unhandled type: <nil>
goroutine 102 [running]:
golang.org/x/net/http2.(*serverConn).runHandler.func1()
	/Users/alec/go/pkg/mod/golang.org/x/[email protected]/http2/server.go:2361 +0x13c
panic({0x1038cfb80?, 0x140002f74e0?})
	/Users/alec/Library/Caches/hermit/pkg/go-1.22.1/src/runtime/panic.go:770 +0x124
github.com/TBD54566975/ftl/backend/schema.typeToSchema(0x140004702f8?)
	/Users/alec/dev/ftl/backend/schema/protobuf_dec.go:68 +0x788
github.com/TBD54566975/ftl/backend/schema.fieldToSchema(0x14000292c80)
	/Users/alec/dev/ftl/backend/schema/field.go:75 +0x64
github.com/TBD54566975/ftl/backend/schema.fieldListToSchema(...)
	/Users/alec/dev/ftl/backend/schema/field.go:65
github.com/TBD54566975/ftl/backend/schema.DataFromProto(0x1400036a000)
	/Users/alec/dev/ftl/backend/schema/data.go:189 +0x250
github.com/TBD54566975/ftl/backend/schema.declListToSchema({0x1400033f2a0, 0x4, 0x14000000101?})
	/Users/alec/dev/ftl/backend/schema/protobuf_dec.go:27 +0x14c
github.com/TBD54566975/ftl/backend/schema.ModuleFromProto(0x14000292300)
	/Users/alec/dev/ftl/backend/schema/module.go:217 +0x68
github.com/TBD54566975/ftl/backend/schema.ModuleFromBytes({0x14000050e00, 0x691, 0x691})
	/Users/alec/dev/ftl/backend/schema/module.go:227 +0x74
github.com/TBD54566975/ftl/backend/schema.(*Module).Scan(0x14000132ee0, {0x1038b9c80?, 0x1400059bc68?})
	/Users/alec/dev/ftl/backend/schema/module.go:35 +0x5c
github.com/jackc/pgx/v5/pgtype.(*scanPlanCodecSQLScanner).Scan(0x103abb0a0?, {0x1400001c50b?, 0x14000470628?, 0x103abb0a0?}, {0x103abb0a0, 0x14000132ee0})
	/Users/alec/go/pkg/mod/github.com/jackc/pgx/[email protected]/pgtype/pgtype.go:370 +0xfc
github.com/jackc/pgx/v5/pgtype.(*pointerPointerScanPlan).Scan(0x1400033e140, {0x1400001c50b, 0x691, 0x691}, {0x1038b27e0?, 0x140002922b8?})
	/Users/alec/go/pkg/mod/github.com/jackc/pgx/[email protected]/pgtype/pgtype.go:489 +0x134
github.com/jackc/pgx/v5.(*baseRows).Scan(0x14000152120, {0x140002f53b0, 0x9, 0x1038ffa40?})
	/Users/alec/go/pkg/mod/github.com/jackc/pgx/[email protected]/rows.go:272 +0x588
github.com/jackc/pgx/v5/pgxpool.(*poolRows).Scan(0x14000020968, {0x140002f53b0?, 0x1400036c300?, 0x1035cf1b5?})
	/Users/alec/go/pkg/mod/github.com/jackc/pgx/[email protected]/pgxpool/rows.go:70 +0x34
github.com/TBD54566975/ftl/backend/controller/sql.(*Queries).GetActiveDeployments(0x1400051a570, {0x103b4eff8, 0x1400036c300}, 0x0)
	/Users/alec/dev/ftl/backend/controller/sql/queries.sql.go:189 +0x28c
github.com/TBD54566975/ftl/backend/controller/dal.(*DAL).GetStatus(0x1400051a580, {0x103b4eff8, 0x1400036c300}, 0x10?, 0x0, 0x0, 0x0)
	/Users/alec/dev/ftl/backend/controller/dal/dal.go:273 +0x250
github.com/TBD54566975/ftl/backend/controller.(*Service).Status(0x14000206a00, {0x103b4eff8?, 0x1400036c300?}, 0x1400036c240?)
	/Users/alec/dev/ftl/backend/controller/controller.go:244 +0x4c
connectrpc.com/connect.NewUnaryHandler[...].func1({0x103b57840, 0x14000402100})
	/Users/alec/go/pkg/mod/connectrpc.com/[email protected]/handler.go:52 +0x13c
connectrpc.com/otelconnect.(*Interceptor).WrapUnary.func1({0x103b4eff8, 0x1400036c2a0}, {0x103b57840, 0x14000402100})
	/Users/alec/go/pkg/mod/connectrpc.com/[email protected]/interceptor.go:152 +0x10b8
github.com/TBD54566975/ftl/internal/rpc.(*metadataInterceptor).WrapUnary.func1({0x103b4f0a0, 0x140001f0070}, {0x103b57840, 0x14000402100})
	/Users/alec/dev/ftl/internal/rpc/context.go:155 +0x188
connectrpc.com/connect.NewUnaryHandler[...].func2({0x12baa1b18, 0x1400033e060})
	/Users/alec/go/pkg/mod/connectrpc.com/[email protected]/handler.go:84 +0x394
connectrpc.com/connect.(*Handler).ServeHTTP(0x140001e4930, {0x103b4c160, 0x140001e2008}, 0x140000005a0)
	/Users/alec/go/pkg/mod/connectrpc.com/[email protected]/handler.go:265 +0x7f0
github.com/TBD54566975/ftl/backend/protos/xyz/block/ftl/v1/ftlv1connect.NewControllerServiceHandler.func1({0x103b4c160, 0x140001e2008}, 0x140000005a0)
	/Users/alec/dev/ftl/backend/protos/xyz/block/ftl/v1/ftlv1connect/ftl.connect.go:533 +0x110
net/http.HandlerFunc.ServeHTTP(0x1400034f520?, {0x103b4c160?, 0x140001e2008?}, 0x1030bac44?)
	/Users/alec/Library/Caches/hermit/pkg/go-1.22.1/src/net/http/server.go:2166 +0x38
net/http.(*ServeMux).ServeHTTP(0x14000422b40?, {0x103b4c160, 0x140001e2008}, 0x140000005a0)
	/Users/alec/Library/Caches/hermit/pkg/go-1.22.1/src/net/http/server.go:2683 +0x1a4
github.com/TBD54566975/ftl/internal/rpc.NewServer.ContextValuesMiddleware.func3({0x103b4c160, 0x140001e2008}, 0x14000422b40)
	/Users/alec/dev/ftl/internal/rpc/rpc.go:90 +0xb4
net/http.HandlerFunc.ServeHTTP(0x1400016c0a0?, {0x103b4c160?, 0x140001e2008?}, 0x1400048afa8?)
	/Users/alec/Library/Caches/hermit/pkg/go-1.22.1/src/net/http/server.go:2166 +0x38
golang.org/x/net/http2.(*serverConn).runHandler(0x14000041680?, 0x0?, 0x14000041140?, 0x0?)
	/Users/alec/go/pkg/mod/golang.org/x/[email protected]/http2/server.go:2368 +0xc0
created by golang.org/x/net/http2.(*serverConn).scheduleHandler in goroutine 133
	/Users/alec/go/pkg/mod/golang.org/x/[email protected]/http2/server.go:2303 +0x208
2024/03/14 10:53:57 http2: panic serving 127.0.0.1:65335: unhandled type: <nil>
goroutine 138 [running]:
golang.org/x/net/http2.(*serverConn).runHandler.func1()
	/Users/alec/go/pkg/mod/golang.org/x/[email protected]/http2/server.go:2361 +0x13c
panic({0x1038cfb80?, 0x14000594dd0?})
	/Users/alec/Library/Caches/hermit/pkg/go-1.22.1/src/runtime/panic.go:770 +0x124
github.com/TBD54566975/ftl/backend/schema.typeToSchema(0x140000e82f8?)
	/Users/alec/dev/ftl/backend/schema/protobuf_dec.go:68 +0x788
github.com/TBD54566975/ftl/backend/schema.fieldToSchema(0x1400039a400)
	/Users/alec/dev/ftl/backend/schema/field.go:75 +0x64
github.com/TBD54566975/ftl/backend/schema.fieldListToSchema(...)
	/Users/alec/dev/ftl/backend/schema/field.go:65
github.com/TBD54566975/ftl/backend/schema.DataFromProto(0x140003377c0)
	/Users/alec/dev/ftl/backend/schema/data.go:189 +0x250
github.com/TBD54566975/ftl/backend/schema.declListToSchema({0x14000134b80, 0x4, 0x14000000101?})
	/Users/alec/dev/ftl/backend/schema/protobuf_dec.go:27 +0x14c
github.com/TBD54566975/ftl/backend/schema.ModuleFromProto(0x1400039a180)
	/Users/alec/dev/ftl/backend/schema/module.go:217 +0x68
github.com/TBD54566975/ftl/backend/schema.ModuleFromBytes({0x140000ea000, 0x691, 0x691})
	/Users/alec/dev/ftl/backend/schema/module.go:227 +0x74
github.com/TBD54566975/ftl/backend/schema.(*Module).Scan(0x140001e5ce0, {0x1038b9c80?, 0x140004728a0?})
	/Users/alec/dev/ftl/backend/schema/module.go:35 +0x5c
github.com/jackc/pgx/v5/pgtype.(*scanPlanCodecSQLScanner).Scan(0x103abb0a0?, {0x1400001c50b?, 0x140000e8628?, 0x103abb0a0?}, {0x103abb0a0, 0x140001e5ce0})
	/Users/alec/go/pkg/mod/github.com/jackc/pgx/[email protected]/pgtype/pgtype.go:370 +0xfc
github.com/jackc/pgx/v5/pgtype.(*pointerPointerScanPlan).Scan(0x1400033e140, {0x1400001c50b, 0x691, 0x691}, {0x1038b27e0?, 0x1400039a138?})
	/Users/alec/go/pkg/mod/github.com/jackc/pgx/[email protected]/pgtype/pgtype.go:489 +0x134
github.com/jackc/pgx/v5.(*baseRows).Scan(0x14000152ea0, {0x1400002d680, 0x9, 0x1038ffa40?})
	/Users/alec/go/pkg/mod/github.com/jackc/pgx/[email protected]/rows.go:272 +0x588
github.com/jackc/pgx/v5/pgxpool.(*poolRows).Scan(0x140000208c8, {0x1400002d680?, 0x1400035c4e0?, 0x1035cf1b5?})
	/Users/alec/go/pkg/mod/github.com/jackc/pgx/[email protected]/pgxpool/rows.go:70 +0x34
github.com/TBD54566975/ftl/backend/controller/sql.(*Queries).GetActiveDeployments(0x1400051a570, {0x103b4eff8, 0x1400035c4e0}, 0x0)
	/Users/alec/dev/ftl/backend/controller/sql/queries.sql.go:189 +0x28c
github.com/TBD54566975/ftl/backend/controller/dal.(*DAL).GetStatus(0x1400051a580, {0x103b4eff8, 0x1400035c4e0}, 0x10?, 0x0, 0x0, 0x0)
	/Users/alec/dev/ftl/backend/controller/dal/dal.go:273 +0x250
github.com/TBD54566975/ftl/backend/controller.(*Service).Status(0x14000206a00, {0x103b4eff8?, 0x1400035c4e0?}, 0x1400035c450?)
	/Users/alec/dev/ftl/backend/controller/controller.go:244 +0x4c
connectrpc.com/connect.NewUnaryHandler[...].func1({0x103b57840, 0x14000318080})
	/Users/alec/go/pkg/mod/connectrpc.com/[email protected]/handler.go:52 +0x13c
connectrpc.com/otelconnect.(*Interceptor).WrapUnary.func1({0x103b4eff8, 0x1400035c4b0}, {0x103b57840, 0x14000318080})
	/Users/alec/go/pkg/mod/connectrpc.com/[email protected]/interceptor.go:152 +0x10b8
github.com/TBD54566975/ftl/internal/rpc.(*metadataInterceptor).WrapUnary.func1({0x103b4f0a0, 0x1400052c380}, {0x103b57840, 0x14000318080})
	/Users/alec/dev/ftl/internal/rpc/context.go:155 +0x188
connectrpc.com/connect.NewUnaryHandler[...].func2({0x12baa1b18, 0x140005441e0})
	/Users/alec/go/pkg/mod/connectrpc.com/[email protected]/handler.go:84 +0x394
connectrpc.com/connect.(*Handler).ServeHTTP(0x140001e4930, {0x103b4c160, 0x1400007c1b8}, 0x14000152a20)
	/Users/alec/go/pkg/mod/connectrpc.com/[email protected]/handler.go:265 +0x7f0
github.com/TBD54566975/ftl/backend/protos/xyz/block/ftl/v1/ftlv1connect.NewControllerServiceHandler.func1({0x103b4c160, 0x1400007c1b8}, 0x14000152a20)
	/Users/alec/dev/ftl/backend/protos/xyz/block/ftl/v1/ftlv1connect/ftl.connect.go:533 +0x110
net/http.HandlerFunc.ServeHTTP(0x1400034f520?, {0x103b4c160?, 0x1400007c1b8?}, 0x1030bac44?)
	/Users/alec/Library/Caches/hermit/pkg/go-1.22.1/src/net/http/server.go:2166 +0x38
net/http.(*ServeMux).ServeHTTP(0x140006966c0?, {0x103b4c160, 0x1400007c1b8}, 0x14000152a20)
	/Users/alec/Library/Caches/hermit/pkg/go-1.22.1/src/net/http/server.go:2683 +0x1a4
github.com/TBD54566975/ftl/internal/rpc.NewServer.ContextValuesMiddleware.func3({0x103b4c160, 0x1400007c1b8}, 0x140006966c0)
	/Users/alec/dev/ftl/internal/rpc/rpc.go:90 +0xb4
net/http.HandlerFunc.ServeHTTP(0x14000012050?, {0x103b4c160?, 0x1400007c1b8?}, 0x14000309fa8?)
	/Users/alec/Library/Caches/hermit/pkg/go-1.22.1/src/net/http/server.go:2166 +0x38
golang.org/x/net/http2.(*serverConn).runHandler(0x0?, 0x1029566a4?, 0x140000dbe00?, 0x0?)
	/Users/alec/go/pkg/mod/golang.org/x/[email protected]/http2/server.go:2368 +0xc0
created by golang.org/x/net/http2.(*serverConn).scheduleHandler in goroutine 133
	/Users/alec/go/pkg/mod/golang.org/x/[email protected]/http2/server.go:2303 +0x208
panic: unhandled type: <nil>

goroutine 158 [running]:
github.com/TBD54566975/ftl/backend/schema.typeToSchema(0x14000377618?)
	/Users/alec/dev/ftl/backend/schema/protobuf_dec.go:68 +0x788
github.com/TBD54566975/ftl/backend/schema.fieldToSchema(0x140003e2d00)
	/Users/alec/dev/ftl/backend/schema/field.go:75 +0x64
github.com/TBD54566975/ftl/backend/schema.fieldListToSchema(...)
	/Users/alec/dev/ftl/backend/schema/field.go:65
github.com/TBD54566975/ftl/backend/schema.DataFromProto(0x14000322be0)
	/Users/alec/dev/ftl/backend/schema/data.go:189 +0x250
github.com/TBD54566975/ftl/backend/schema.declListToSchema({0x14000544640, 0x4, 0x14000000101?})
	/Users/alec/dev/ftl/backend/schema/protobuf_dec.go:27 +0x14c
github.com/TBD54566975/ftl/backend/schema.ModuleFromProto(0x14000319c00)
	/Users/alec/dev/ftl/backend/schema/module.go:217 +0x68
github.com/TBD54566975/ftl/backend/schema.ModuleFromBytes({0x140000e6000, 0x691, 0x691})
	/Users/alec/dev/ftl/backend/schema/module.go:227 +0x74
github.com/TBD54566975/ftl/backend/schema.(*Module).Scan(0x1400052cbd0, {0x1038b9c80?, 0x14000526ed0?})
	/Users/alec/dev/ftl/backend/schema/module.go:35 +0x5c
github.com/jackc/pgx/v5/pgtype.(*scanPlanCodecSQLScanner).Scan(0x103abb0a0?, {0x1400001c50b?, 0x14000377948?, 0x103abb0a0?}, {0x103abb0a0, 0x1400052cbd0})
	/Users/alec/go/pkg/mod/github.com/jackc/pgx/[email protected]/pgtype/pgtype.go:370 +0xfc
github.com/jackc/pgx/v5/pgtype.(*pointerPointerScanPlan).Scan(0x1400033e140, {0x1400001c50b, 0x691, 0x691}, {0x1038b27e0?, 0x14000319bb8?})
	/Users/alec/go/pkg/mod/github.com/jackc/pgx/[email protected]/pgtype/pgtype.go:489 +0x134
github.com/jackc/pgx/v5.(*baseRows).Scan(0x140001530e0, {0x14000344480, 0x9, 0x1038ffa40?})
	/Users/alec/go/pkg/mod/github.com/jackc/pgx/[email protected]/rows.go:272 +0x588
github.com/jackc/pgx/v5/pgxpool.(*poolRows).Scan(0x140000208a0, {0x14000344480?, 0x1400035da70?, 0x1035cf1b5?})
	/Users/alec/go/pkg/mod/github.com/jackc/pgx/[email protected]/pgxpool/rows.go:70 +0x34
github.com/TBD54566975/ftl/backend/controller/sql.(*Queries).GetActiveDeployments(0x1400051a570, {0x103b4eff8, 0x1400035da70}, 0x0)
	/Users/alec/dev/ftl/backend/controller/sql/queries.sql.go:189 +0x28c
github.com/TBD54566975/ftl/backend/controller/dal.(*DAL).GetActiveDeployments(0x0?, {0x103b4eff8?, 0x1400035da70?})
	/Users/alec/dev/ftl/backend/controller/dal/dal.go:717 +0x30
github.com/TBD54566975/ftl/backend/controller.(*Service).reconcileRunners(0x14000206a00, {0x103b4eff8, 0x1400035da70})
	/Users/alec/dev/ftl/backend/controller/controller.go:859 +0x34
github.com/TBD54566975/ftl/backend/controller/scheduledtask.(*Scheduler).run.func2()
	/Users/alec/dev/ftl/backend/controller/scheduledtask/scheduledtask.go:153 +0x50
created by github.com/TBD54566975/ftl/backend/controller/scheduledtask.(*Scheduler).run in goroutine 93
	/Users/alec/dev/ftl/backend/controller/scheduledtask/scheduledtask.go:152 +0x9f0

@alecthomas
Copy link
Collaborator Author

Have not seen this since.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working next Work that will be be picked up next
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@alecthomas