Skip to content
This repository has been archived by the owner on Jan 8, 2020. It is now read-only.

"enclose.by.quotes" not functioning #53

Open
Hypnotec opened this issue Mar 9, 2018 · 1 comment
Open

"enclose.by.quotes" not functioning #53

Hypnotec opened this issue Mar 9, 2018 · 1 comment

Comments

@Hypnotec
Copy link

Hypnotec commented Mar 9, 2018

E> It appears that 'enclose.by.quotes' does not function when using 'custom.query'

E> I might advocate that anyone who comes across this project consider switching to StreamSets, as Flume doesn't support RDBMS and StreamSets provides support for a large array of sources and targets without relying on third-party support.

Does the following property, "enclose.by.quotes", not function when using PSQL? I am not able to get it to work while using the following Flume properties:

#
# SOURCE
agent.sources.source-sql.type = org.keedio.flume.source.SQLSource
agent.sources.source-sql.channels = channel-file

agent.sources.source-sql.status.file.path = /home/admin/.tmp/.flume
agent.sources.source-sql.status.file.name = source-sql.messages.status

agent.sources.source-sql.hibernate.connection.url = jdbc:postgresql://localhost:5432/test
agent.sources.source-sql.hibernate.connection.user = postgres
agent.sources.source-sql.hibernate.connection.password = ''
agent.sources.source-sql.hibernate.connection.driver_class = org.postgresql.Driver

# NOTE: 'message_time' is integer, 'message' is text
agent.sources.source-sql.custom.query = select message_time, message from messages where message_time > $@$ order by message_time
agent.sources.source-sql.enclose.by.quotes = false

agent.sources.source-sql.run.query.delay = 5000
agent.sources.source-sql.batch.size = 500
agent.sources.source-sql.max.rows = 10000

Currently, my work around is to remove all instances of quotes through an interceptor, but I am not a huge fan of this approach as users may input double-quotes - which the 'message' field would capture.

@lazaromedina
Copy link
Contributor

Hi Hypnotec,
when trying to reproduce like issue #44, we get values with double quotes.

  • can you please , version of flume-ng-sql
  • as i see in you config, you are using default delimiter "," can you try a different one?

best, Luis

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants