Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot open same S3File instance using different modes #1474

Open
devmonkey22 opened this issue Nov 20, 2024 · 0 comments · May be fixed by #1475
Open

Cannot open same S3File instance using different modes #1474

devmonkey22 opened this issue Nov 20, 2024 · 0 comments · May be fixed by #1475

Comments

@devmonkey22
Copy link

devmonkey22 commented Nov 20, 2024

I'm using S3Storage backend with a Django model with a FileField field. With a model instance with a FileField tied to this storage, I need to be able to open (read) the same file as text and later binary (i.e. to compute hash) from the same model instance. However, Django FileFields (really FieldFiles) keep a reference to the underlying File (aka S3File) even after closing which makes it require being able to reopen with a different mode. When closed, the S3File._file is cleared out, but S3File._mode remains, thus preventing a future open from changing modes before it opens the file again.

Currently I receive the S3File.open error "Cannot reopen file with a new mode." which was added in #1321 to fix #1319 and released in version 1.14.2.

Example Code:

data = "abcdefghijklmnopqrstuvwxyz\n"
model = SomeModel.objects.create()
model.file_contents.save("data.txt", ContentFile(data*2))

# Simulate reading binary (to compute SHA256)
sha_hash = hashlib.new('sha256')
with model.file_contents.open('rb') as f:
	for chunk in f.chunks():
		sha_hash.update(chunk)

# This fails with ValueError: Cannot reopen file with a new mode.
with model.file_contents.open('r') as f:
	line_count = 0
	for line in f.readlines():
		self.assertEqual(line, data)
		line_count += 1
	self.assertEqual(line_count, 2)

This raises:

Traceback (most recent call last):
  File ".../test_models.py", line 233, in test_s3_reopen
    with msg.data.open('r') as f:
  File ".../python3.10/site-packages/django/db/models/fields/files.py", line 81, in open
    self.file.open(mode)
  File ".../python3.10/site-packages/storages/backends/s3.py", line 152, in open
    raise ValueError("Cannot reopen file with a new mode.")
ValueError: Cannot reopen file with a new mode.

I'll push a PR shortly that should resolve this. Let me know if it causes unintended issues allowing S3File to be reopened that I didn't notice.

devmonkey22 added a commit to devmonkey22/django-storages that referenced this issue Nov 20, 2024
devmonkey22 added a commit to devmonkey22/django-storages that referenced this issue Nov 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant