Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UTF-16 #21

Open
netmiller opened this issue Sep 6, 2018 · 3 comments
Open

UTF-16 #21

netmiller opened this issue Sep 6, 2018 · 3 comments

Comments

@netmiller
Copy link

Any advise how to read utf-16 (LittleEndian) file info buffer.
Tried normal way :

while (row = reader.next()) {
    line = row.toString('utf16le');
    ........
}

but seems to be incorrect ?

@tracker1
Copy link

would probably be best to fork and update the search for the newline pairing. just curious why not utf8?

@jforge
Copy link

jforge commented Nov 9, 2020

would probably be best to fork and update the search for the newline pairing. just curious why not utf8?

The data source could be UTF-16LE encoded, and one wants to handle raw lines of data.
Example: sync, read/compare windows files from a machine with fixed encoding (UTF-16LE)

Actually the project should support any source encoding.

@quoininc-huudatran
Copy link

Actually the project should support any source encoding.

I'm not sure if this is still an issue, but I believe it's working? I have a dynamic encoding detection with chardet, and I use that value with row.toString(encoding) and I correctly get the string I want.. from my code samples.

@netmiller The exact encoding I use is UTF-16LE. Maybe you need to pass that instead of utf16le.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants