Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Comment on pre-existing table crashes target-postgres #197

Open
laurentS opened this issue Feb 19, 2021 · 0 comments
Open

Comment on pre-existing table crashes target-postgres #197

laurentS opened this issue Feb 19, 2021 · 0 comments

Comments

@laurentS
Copy link
Collaborator

I tried running this target (via meltano) with postgres_schema set to an existing schema which already contained tables (with data). Some of the tables had comments on them, as created by:

COMMENT ON TABLE public.mytable IS ''my custom table with important data';

When running the target, it crashed with the following stacktrace:

target-postgres | ERROR Exception writing records
target-postgres | Traceback (most recent call last):
target-postgres |   File "~/myproject/.meltano/loaders/target-postgres/venv/lib/python3.8/site-packages/target_postgres/postgres.py", line 237, in write_batch
target-postgres |     self.setup_table_mapping_cache(cur)
target-postgres |   File "~/myproject/.meltano/loaders/target-postgres/venv/lib/python3.8/site-packages/target_postgres/postgres.py", line 224, in setup_table_mapping_cache
target-postgres |     table_path = json.loads(raw_json).get('path', None)
target-postgres |   File "/home/laurent/.pyenv/versions/3.8.7/lib/python3.8/json/__init__.py", line 357, in loads
target-postgres |     return _default_decoder.decode(s)
target-postgres |   File "/home/laurent/.pyenv/versions/3.8.7/lib/python3.8/json/decoder.py", line 337, in decode
target-postgres |     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
target-postgres |   File "/home/laurent/.pyenv/versions/3.8.7/lib/python3.8/json/decoder.py", line 355, in raw_decode
target-postgres |     raise JSONDecodeError("Expecting value", s, err.value) from None
target-postgres | json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
target-postgres | CRITICAL ('Exception writing records', JSONDecodeError('Expecting value: line 1 column 1 (char 0)'))

With some extra logging, raw_json actually contains the value of the comment mentioned above, which clearly isn't JSON.
This is the line of code that breaks

table_path = json.loads(raw_json).get('path', None)

Is this a bug, or does target-postgres expect the schema to be empty when it starts?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant