Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mapping errors due to already-existing file, but no duplicate ids in batch #69

Open
kspurgin opened this issue Apr 25, 2024 · 0 comments

Comments

@kspurgin
Copy link
Collaborator

kspurgin commented Apr 25, 2024

Record id "CC2004.52.1f.001" mapped fine, and written to: Q0MyMDA0LjUyLjFmLjAwMQ==.xml

Subsequent record id "CC2004.52.1L.001" maps ok, but fails to write XML file, with error:

An XML record with filename Q0MyMDA0LjUyLjFMLjAwMQ==.xml has already been written. Check for duplicates.

Compare:

Q0MyMDA0LjUyLjFmLjAwMQ==.xml
Q0MyMDA0LjUyLjFMLjAwMQ==.xml
---------------^------------

The filenames produced by CMT::Xml::FileNamer are unique, but the filesystem ignores case differences. Hooray for being "helpful," MacOS.

Options for this:

  • Ignore it. It happens fairly rarely and only in gigantic batches (so far). Run the error row from the orig batch as a subsequent fixup batch after the initial batch is done.
  • Create a case-sensitive disk partition and move all data work to that (not likely for me any time soon, but remember this when you get a new laptop!)
  • Update CMT to add some qualifier to the file name if the File.exist? check in CMT::Xml::FileWriter returns true. S3 keys are case sensitive, and the S3 key creation does not depend on the file name. Thanks, past-me.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant