Releases: monash-merc/mydata
Releases · monash-merc/mydata
v0.3.4
v0.3.3
v0.3.2
v0.3.2 Release Notes
====================
1. Fixed an issue where verifications could fail due to missing trailing slash in
an API call.
2. Fixed a Mac OS X issue where MyData failed to clean up SSH subprocesses.
3. Fixed a folder scanning issue where an unknown user group resulted in a critical error.
- Now the data is still uploaded and the experiment metadata is used to record the user
group which MyData would have granted access to if it could.
4. Fixed an issue with the User Group folder structure where MyData could fail to
determine whether an experiment had already been created for the current user,
group and MyData instance.
5. Fixed an issue with the User Group folder structure where MyData failed to populate
the content_type field when created an ObjectACL for a user group.
6. The Uploads view is now cleared when re-running the folder scans and uploads.
v0.3.1
v0.3.1 Release Notes
====================
1. Reduced memory usage
- On Windows, no longer using dd.exe to extract chunks when uploading files.
- The "dd.exe" command in the previous version was using up to 256 MB of RAM
while extracting a chunk from a file to upload.
- A new pure Python method implemented to extract chunks from files to upload
is more efficient.
- Smaller chunks are used when calculating MD5 sums
2. Fixed "Exit MyData" system tray menu item on Windows (when running in background mode)
- This menu item requests privilege elevation.
- The code to do the privilege elevation was broken on Windows. Now fixed.
3. Fixed "Quit MyData" menu item on Mac OS X, which said "Quit Run", matching run.py
- Now it says "Quit MyData".
v0.3.1-beta1 Release Notes
==========================
NB: v0.3.0 was never released (except for v0.3.0-beta1), because some server-side
changes were required, which meant incrementing the version number to v0.3.1
(instead of just removing the "beta" status of v0.3.0), so that we could tag the
server-side code as "MyData_v0.3.1".
1. Removed MyData's restriction on the maximum length of the main data
directory path. The maximum length is still restricted for Store.Monash
(256 characters now, instead of 64), but it is done on the server side
only, not in MyData.
2. MyData now requests verification of each datafile when it finishes
uploading, instead of waiting for MyTardis to schedule its verification.
3. When approving staging uploads from MyData, the MyTardis administrator
can now assign a temporary storage box specific to one facility, e.g.
"TestFacility-staging" and MyData will use that temporary storage box for
staging uploads, instead of MyTardis's default temporary storage box.
4. A new "Stop" button on MyData's toolbar allows data scans and uploads
to be canceled irrespective of which view is open. (Previously this could
only be done from the Uploads view.) The status of the "Stop" button
(enabled or disabled) indicates whether MyData's data scan and upload
threads are currently running.
5. Fixed a missing import which could result in an "Undefined symbol
Unauthorized" error for MyTardis accounts without permission to create
ObjectACLs.
6. Fixed a bug where the progress dialog failed to close when the user
pressed "Cancel". The current progress dialog is only for scanning user
folders to compile a list of datasets and datafiles. Further work is needed
to provide a central progress dialog to summarize the verifications and
uploads (as displayed in the Verifications and Uploads tabs).
v0.3.1-beta1
v0.3.1-beta1 Release Notes
=====================
NB: v0.3.0 was never released (except for v0.3.0-beta1), because some server-side
changes were required, which meant incrementing the version number to v0.3.1
(instead of just removing the "beta" status of v0.3.0), so that we could tag the
server-side code as "MyData_v0.3.1".
1. Removed MyData's restriction on the maximum length of the main data
directory path. The maximum length is still restricted for Store.Monash
(256 characters now, instead of 64), but it is done on the server side
only, not in MyData.
2. MyData now requests verification of each datafile when it finishes
uploading, instead of waiting for MyTardis to schedule its verification.
3. When approving staging uploads from MyData, the MyTardis administrator
can now assign a temporary storage box specific to one facility, e.g.
"TestFacility-staging" and MyData will use that temporary storage box for
staging uploads, instead of MyTardis's default temporary storage box.
4. A new "Stop" button on MyData's toolbar allows data scans and uploads
to be canceled irrespective of which view is open. (Previously this could
only be done from the Uploads view.) The status of the "Stop" button
(enabled or disabled) indicates whether MyData's data scan and upload
threads are currently running.
5. Fixed a missing import which could result in an "Undefined symbol
Unauthorized" error for MyTardis accounts without permission to create
ObjectACLs.
6. Fixed a bug where the progress dialog failed to close when the user
pressed "Cancel". The current progress dialog is only for scanning user
folders to compile a list of datasets and datafiles. Further work is needed
to provide a central progress dialog to summarize the verifications and
uploads (as displayed in the Verifications and Uploads tabs).
v0.3.0-beta1
Updating download links.
v0.3.0-alpha1
RELEASE NOTES
=============
Contents
1. Overview
2. For end users
2.1 New features and bug fixes
2.2 Backwards incompatibility warnings
2.3 Features in development, not included in this release
3. For developers and MyTardis sys admins
1. Overview
This is not a release packed with new functionality for end users. The vast
majority of changes are "under the hood". This has been done to enable MyData
to upload data to MyTardis servers running an officially tagged version of
MyTardis, instead of the unofficial fork of MyTardis which was used previously.
See: See: http://mydata.readthedocs.org/en/latest/mytardis-prerequisites.html
2. For end users
2.1 New features and bug fixes
* MyData v0.3.0 only needs to request approval once for uploads to MyTardis via
staging, even if you change network interfaces. MyData now uses UUIDs
instead of MAC addresses to uniquely identify MyData instances.
* MyData v0.3.0 will be compatible with the soon-to-be-released Store.Monash
service.
* Mac OS X builds are now being done on a newer OS X version (10.9.5), so
hopefully MyData.app's certificate should work for OS X 10.10 users (so you
won't get errors about MyData being from an unidentified developer).
* Fixed bug where email matching was case sensitive, so MyData failed to
match a user folder (named with an email address) with the corresponding
MyTardis user record.
* Fixed an issue where MyData's progress dialog (displayed while scanning user
folders) sometimes lingered (and appeared blank) on Mac OS X.
* Logging level can now be specified as a command-line argument. So if your
~/.MyData_debug_log.txt is growing too large too quickly, you can change your
MyData launcher / shortcut to use "--loglevel INFO", instead of the default
log level, which is currently "DEBUG". A small number of DEBUG-level log
messages will still appear early on in MyData's startup before MyData has
processed its command-line arguments.
* Additional user-friendly error messages for incorrectly configured MyTardis
backends.
2.2 Backwards incompatibility warnings
* Pointing MyData v0.3.0 at a data directory previously managed by an earlier
version of MyData could result in old datasets being re-uploaded, due to a
change in the MyTardis experiment metadata used by MyData.
* MyData instances which were previously approved for uploads via staging
will need to be reapproved, because a different method (UUID) is now
used to identify each MyData instance instead of using a MAC address.
This doesn't necessarily require pasting a public key into
~mydata/.ssh/authorized_keys on the staging server, because it might
already be there. In that case, it is just a matter of updating the
UploaderRegistrationRequest record (which is now part of the "MyData"
Tardis app) in the Django admin interface.
* Partial uploads started with MyData v0.2 cannot be resumed with MyData
v0.3 without manual intervention from a MyTardis administrator.
2.3 Features still in development, not included in this release
The following planned features are particularly important to some of our
stakeholders:
* Overall progress indicator (https://github.com/monash-merc/mydata/issues/33)
- Various MyData tab views (along with the status bar) show indications of
progress, but there is no overall progress indicator yet.
* Scheduling / auto-refresh (https://github.com/monash-merc/mydata/issues/35)
- MyData v0.3.0 can either be run manually (interactively), or a shortcut to
MyData (run in --background mode) can be put in a "Startup Items" folder or
can be managed with any suitable task scheduler. But this version of
MyData doesn't have built-in scheduling functionality. Scheduling will be
included in a future MyData release, and is currently in development:
http://mydata.readthedocs.org/en/schedule/settings.html#schedule
3. For developers and MyTardis sys admins
* MyData.py and its sibling modules have been moved into a "mydata" package,
and MyData is now launched from the command-line with "run.py" (in the
repository's root directory), instead of "MyData.py" (which is now in the
"mydata" package directory).
- This has various advantages. For example, MyData has custom logging
functionality to log to the GUI, to ~/.MyData_debug_log.txt, and to a string
in memory for submitting debug logs to MyData developers. This functionality
was previously imported with "from logger.Logger import logger", where the
first "logger" was the package subdirectory, "Logger" was the module within
that package, and "logger" was a singleton instance of the Logger class
within the Logger module. One problem with this type of import is that it
is not clear from the import statement that we are importing MyData-specific
logging functionality. Now the "logger" package directory has been renamed
to "mydata/logs" and the "Logger.py" module has been renamed to
"__init__.py", so the logging import statement becomes
"from mydata.logs import logger".
* Rather than having most Python modules in the same directory, now
subdirectories (Python packages) are used to separate models, views,
controllers etc.
* Previous MyData versions used some Java-style conventions of making module
names (e.g. SettingsDialog.py) match the class name ("SettingsDialog") defined
within that module. Module names are being replaced with more Pythonic names,
e.g. "views/settings.py". But class names are still UpperCamelCase, which
matches wxPython's built in classes, e.g. "MessageDialog".
* The various platform-specific build scripts have been combined into a
single setup.py file.
- This isn't equivalent to as a traditional Python setup.py file, because
MyData is a desktop application, not a Python module.
- "python setup.py build" creates MyData.exe or MyData.app
- "python setup.py bdist" creates a setup wizard (Windows) or DMG
(Mac OS X).
- "python setup.py install" runs the setup wizard (Windows) or opens the
DMG (Mac OS X).
* Looking up paths to icon files can be tricky, because the path depends on
whether you are running MyData as a Python script from the command-line, or
whether you are running it as a standalone application (in which case the
path to icons is platform-dependent). Now the paths to icon files are
determined in a single place: mydata/media/__init__.py, instead of manually
building icon paths in various places in the code.
* The license info has been updated to clarify that the icons included with
MyData are not intended to be distributed under GPL and should not be used
outside of MyData without permission from the owners:
http://mydata.readthedocs.org/en/latest/license.html
https://raw.githubusercontent.com/monash-merc/mydata/master/mydata/media/Aha-Soft/LICENSE.txt
* The MyTardis Uploader models used by MyData's "mydata/models/uploader.py"
(formerly "UploaderModel.py") are no longer assumed to be accessible via the
core MyTardis API ("/api/v1/uploader..."). They should now be installed as
part of a separate MyTardis app:
"https://github.com/wettenhj/mytardis-app-mydata", as described here:
http://mydata.readthedocs.org/en/latest/mytardis-prerequisites.html
MyData now uses additions to the MyTardis API provided by the app, using its
namespace, i.e. "/api/v1/mydata_uploader" instead of "/api/v1/uploader".
* Removed a bunch of unused files from the repository.
- Removed "icon_048.png" (an auto-generated "MT" icon).
- Removed markdown docs, obsoleted by RST docs in docs/source/
- Removed "Exit MyData.exe" which was used to test privilege elevation. Now
"we just try to run "MyData.exe" with --version as an administrator to
test privilege elevation.
- Removed AddFolderDialog.py which was used when we were trying to support
adding data folders by drag and drop instead of automatic scanning.
- Removed DragAndDrop.py. Drag and drop is now only supported in
SettingsDialog, so the drag-and-drop stuff is in mydata/views/settings.py
* When saving settings (control-s or command-s from SettingsDialog), MyData no
longer remembers the saved path. It makes sense for MyData to always default
to the same location for saving settings, because we need to ensure that we
are consistently using the same UUID for uniquely identifying each MyData
uploader instance.
* When creating ObjectACLs, MyData was failing to specify content_type. Fixed.
* Whilst MyData still expects to get an absolute path (a.k.a. temp_url) back
from MyTardis when first requesting a staging upload, for resuming partial
uploads, MyData now expects to find a relative path e.g.
"dataset-5/subdir/file.txt" in the URI field of the DataFileObject retreived
from the MyTardis API. This means that to be able to know the absolute path
for the resumed upload, MyData needs to retrieve the base path (e.g.
"/var/lib/receiving/") from the Uploader record. To achieve this, the
Uploader record in the "mytardis-app-mydata" app now includes a StorageBox
foreign key, and the UploaderStagingHost foreign key has been removed. The
fields which were previously in the UploaderStagingHost model (username and
hostname / IP address for SCP uploads) are now expected to be added to the
staging storage box as StorageBoxAttributes (with keys "scp_username" and
"scp_hostname"), see:
http://mydata.readthedocs.org/en/latest/mytardis-prerequisites.html
* MyData displays the latest git commit hash next to its version number in its
About Dialog and when running "python run.py --version". Previously this
could give an out-of-date commit hash if you forgot to run
"CreateCommitDef.py" before launching MyData with "python run.py" (formerly
"python MyData.py"). Now the commit hash is updated every time the "mydata"
package is loaded, i.e. in "mydata/__init__.py". The "CreateCommitDef.py"
script has been rem...
v0.2.1
v0.2.0
v0.2.0 beta1
v0.2.0-beta1 Updating version number.