Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing error handling when writing to unknown Clickhouse columns #55

Open
fschoell opened this issue Mar 18, 2024 · 0 comments
Open

Missing error handling when writing to unknown Clickhouse columns #55

fschoell opened this issue Mar 18, 2024 · 0 comments

Comments

@fschoell
Copy link
Contributor

When the sink receives a database entry for a column that doesn't exist in Clickhouse, the sink crashes with a nil pointer exception:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0xf99e6e8]

goroutine 137 [running]:
github.com/streamingfast/substreams-sink-sql/db.convertOpToClickhouseValues(0xc000e12330)
	/Users/work/GoLand/substreams-sink-sql/db/dialect_clickhouse.go:172 +0x208
github.com/streamingfast/substreams-sink-sql/db.clickhouseDialect.Flush({}, {0x15?, 0x1?}, {0x107da570, 0xc000e12870}, 0xc00073a700, {0x107da570?, 0x106b9400?}, 0xc0005ab840?)
	/Users/work/GoLand/substreams-sink-sql/db/dialect_clickhouse.go:64 +0x9c5
github.com/streamingfast/substreams-sink-sql/db.(*Loader).Flush(0xc00073a700, {0xfb36cf3?, 0xbe?}, {0xc000d7b530, 0x28}, 0xc001002130, 0xbe)
	/Users/work/GoLand/substreams-sink-sql/db/flush.go:29 +0x279
github.com/streamingfast/substreams-sink-sql/sinker.(*SQLSinker).HandleBlockScopedData(0xc0006be600, {0x107da5a8, 0xc000836dc0}, 0xc00093b100, 0x119a51aa, 0xc001002130)
	/Users/work/GoLand/substreams-sink-sql/sinker/sinker.go:125 +0x5c5
github.com/streamingfast/substreams-sink.(*Sinker).doRequest(0xc0006e44d0, {0x107da5a8, 0xc000836dc0}, 0xc000820dd0, 0xc00073ab00, {0x107b78e0, 0xc000066270}, {0xc0000661a0, 0x1, 0x1}, ...)
	/Users/work/.go/pkg/mod/github.com/streamingfast/[email protected]/sinker.go:472 +0x1974
github.com/streamingfast/substreams-sink.(*Sinker).run(0xc0006e44d0, {0x107da5a8, 0xc000836dc0}, 0xc000820dd0, {0x107cf668, 0xc0006be600})
	/Users/work/.go/pkg/mod/github.com/streamingfast/[email protected]/sinker.go:274 +0x867
github.com/streamingfast/substreams-sink.(*Sinker).Run(0xc0006e44d0, {0x107da5a8, 0xc000836dc0}, 0xc000820dd0, {0x107cf668, 0xc0006be600})
	/Users/work/.go/pkg/mod/github.com/streamingfast/[email protected]/sinker.go:199 +0x488
github.com/streamingfast/substreams-sink-sql/sinker.(*SQLSinker).Run(0xc0006be600, {0x107da5a8, 0xc000836dc0})
	/Users/work/GoLand/substreams-sink-sql/sinker/sinker.go:94 +0x7c5
created by github.com/streamingfast/cli.(*Application).SuperviseAndStart in goroutine 1
	/Users/work/.go/pkg/mod/github.com/streamingfast/[email protected]/application.go:102 +0x37a

Seems like there is no check if the column actually exists before accessing it in here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant