Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use cross-platform friendly character encoding in get_table.py #168

Open
coatsd opened this issue Jan 9, 2021 · 2 comments
Open

Use cross-platform friendly character encoding in get_table.py #168

coatsd opened this issue Jan 9, 2021 · 2 comments
Assignees

Comments

@coatsd
Copy link
Contributor

coatsd commented Jan 9, 2021

image
The current code in get_table.py uses a platform specific character encoding method that fails on other operating systems. Ideally, we would use a standardized encoding that would allow us to output it to a file on any operating system. This stackoverflow issue discusses this issue:
https://stackoverflow.com/questions/27092833/unicodeencodeerror-charmap-codec-cant-encode-characters

@colindavey
Copy link
Contributor

I discovered this problem when putting the python code into Docker. It worked fine on my computer, so I didn't know there was a problem until trying under Docker. I tried a number of fixes that didn't work, including adding encoding="utf-8" as suggested in the stackoverflow article.

One thing I discovered is that python 3.7 has a fix for this, and my computer was running 3.7, which is why never saw the problem. On the Docker side, what I did, in conjunction with MVP studio, was to upgrade the base Linux image to a newer one that has python 3.7.

Before that, I found that I could fix the problem in Docker with this environment variable: export LC_CTYPE="C.UTF-8". Oliver Dain suggested doing this: https://www.tecmint.com/set-system-locales-in-linux/, but we didn't pursue it after deciding to upgrade the Docker image.

@colindavey
Copy link
Contributor

colindavey commented Jan 12, 2021

One decision we have is whether this code will continue to live in the lhb web repo, and be run by developers in their terminal, or whether it will be moved to the lhb py repo, where it will be primarily run as a web app. At this point, both repos have it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants