-
-
Notifications
You must be signed in to change notification settings - Fork 56
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add generic uploader #25
Open
gerbyzation
wants to merge
37
commits into
TkTech:master
Choose a base branch
from
thedataplace:static-uploader
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 24 commits
Commits
Show all changes
37 commits
Select commit
Hold shift + click to select a range
a2bd0f2
add google cloud signed url support
gerbyzation 66a8433
allow for both explicit sa credentials and implicit authentication
gerbyzation b55b528
call generate signed url on blob
gerbyzation 9fe3dc9
add docker setup w/ local mounted volume
gerbyzation 42acc13
tests run, prob still misconfigured
gerbyzation 1a0b647
adding s3filestore for testing purps
gerbyzation b3da887
add first test borrowed from s3filestore
gerbyzation b3b2692
fix typo in unpacking
gerbyzation 726929f
fix test dependencies and add more tests from s3filestore
gerbyzation d173b64
pin working dependencies
gerbyzation ae7ba0e
add tests for storage.py
gerbyzation 668119d
Merge branch 'tests' of https://github.com/thedataplace/ckanext-cloud…
gerbyzation ab9c2ec
add first tests, inspired by s3filestore
gerbyzation 59ac7fd
start work on uploader
gerbyzation f0c737a
get uploading & viewing generic files working (warning code is ☢))
gerbyzation 2a56bf1
refactor upload methods
gerbyzation cd97074
Merge remote-tracking branch 'origin/gcloud-support' into static-uplo…
gerbyzation 0545a92
refactor get_url_from_filename to get_url_from_path
gerbyzation 30d4986
add plugin tests file
gerbyzation 352f597
replace filepath joins with method
gerbyzation f976efd
fix tests
gerbyzation 0aeed2c
fix google secure url, refactor to raise exception if no secure url a…
gerbyzation e23940c
ignore all .egg-info things
gerbyzation 9eab142
remove docker setup from repo
gerbyzation f3896af
remove print in plugin.py
gerbyzation 6c16b68
fix site-url in test
gerbyzation b9677cc
formatting fixed in storage.py
gerbyzation e32b28a
use secure_urls only for resources
gerbyzation a52c607
fix disabled tests
gerbyzation e34bbac
add cover report
gerbyzation f48b24b
test file uploads are always public
gerbyzation ffa8e1c
update cover report
gerbyzation 18d9bb4
remove cover report
gerbyzation 98747be
add use_secure_url_for_generics option
gerbyzation da87c3d
Set google ACL if needed, add permanent redirect
gerbyzation 3919ea5
add google storage option and use_secure_urls_for_generics to readme
gerbyzation 850498f
update note #2
gerbyzation File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
# Variables in this file will be substituted into docker-compose.yml | ||
# Save a copy of this file as .env and insert your own values. | ||
# Verify correct substitution with "docker-compose config" | ||
# If variables are newly added or enabled, please delete and rebuild the images to pull in changes: | ||
# docker-compose down | ||
# docker rmi -f docker_ckan docker_db | ||
# docker rmi $(docker images -f dangling=true -q) | ||
# docker-compose build | ||
# docker-compose up -d | ||
# docker-compose restart ckan # give the db service time to initialize the db cluster on first run | ||
|
||
# Image: ckan | ||
CKAN_SITE_ID=default | ||
# | ||
# On AWS, your CKAN_SITE_URL is the output of: | ||
# curl -s http://169.254.169.254/latest/meta-data/public-hostname | ||
# CKAN_SITE_URL=http://ec2-xxx-xxx-xxx-xxx.ap-southeast-2.compute.amazonaws.com | ||
# When running locally, CKAN_SITE_URL must contain the port | ||
CKAN_SITE_URL=http://localhost:5000 | ||
# | ||
# CKAN_PORT must be available on the host: sudo netstat -na | ||
# To apply change: docker-compose down && docker rmi docker_ckan && docker-compose build ckan | ||
CKAN_PORT=5000 | ||
# | ||
# Email settings | ||
CKAN_SMTP_SERVER=smtp.corporateict.domain:25 | ||
CKAN_SMTP_STARTTLS=True | ||
CKAN_SMTP_USER=user | ||
CKAN_SMTP_PASSWORD=pass | ||
CKAN_SMTP_MAIL_FROM=ckan@localhost | ||
# | ||
# Image: db | ||
POSTGRES_PASSWORD=ckan | ||
# | ||
# POSTGRES_PORT must be available on the host: sudo netstat -na | grep 5432 | ||
# To apply change: docker-compose down && docker rmi docker_db docker_ckan && docker-compose build | ||
POSTGRES_PORT=5432 | ||
# | ||
# The datastore database will be created in the db container as docs | ||
# Readwrite user/pass will be ckan:POSTGRES_PASSWORD | ||
# Readonly user/pass will be datastore_ro:DATASTORE_READONLY_PASSWORD | ||
DATASTORE_READONLY_PASSWORD=datastore cloudstorage | ||
|
||
CKAN_SITE_TITLE='Testing subject' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove debugging print