[ENH] Add-on installation with Conda #2561

astaric · 2017-09-05T20:48:20Z

Issue

Some add-ons have dependencies that are a pain to install on windows. Conda makes this easier

Description of changes

Add-on installation/upgrading/uninstallation calls conda commands before pip. If conda command succeeds, pip says that everything is up to date/uninstalled and does nothing.

Needs testing on win/linux, with different configurations (orange installed in conda root/conda environment, orange installed beside conda, ...)

Includes

Code changes
Tests
Documentation

kernc · 2017-09-05T22:44:15Z

Orange/canvas/application/addons.py

+            if command == Install:
+                self.setStatusMessage(
+                    "Installing {}".format(pkg.installable.name))
+                if self.conda:


If self.conda is an instance of CondaInstaller(), shouldn't this check be self.conda.conda?

Technically yes, but since CondaInstaller reimplements __bool__, this is the same thing

codecov-io · 2017-09-06T04:54:08Z

Codecov Report

Merging #2561 into master will increase coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #2561      +/-   ##
==========================================
+ Coverage   75.83%   75.83%   +<.01%     
==========================================
  Files         338      338              
  Lines       59541    59541              
==========================================
+ Hits        45150    45152       +2     
+ Misses      14391    14389       -2

ajdapretnar · 2017-09-06T07:54:01Z

Not sure I tested it right, but I've made a new environment and got the branch from Github. Ran Orange and tried to install the add-ons via the dialog. Got this:

Traceback (most recent call last):
  File "c:\orange\orange3\Orange\canvas\application\addons.py", line 780, in _next
    self.pip.install(pkg.installable)
  File "c:\orange\orange3\Orange\canvas\application\addons.py", line 816, in install
    run_command(cmd)
  File "c:\orange\orange3\Orange\canvas\application\addons.py", line 917, in run_command
    raise CommandFailed(command, process.returncode, output)
  File "c:\orange\orange3\Orange\canvas\application\addons.py", line 742, in __init__
    cmd = " ".join(map(shlex.quote, cmd.cmd))
AttributeError: 'list' object has no attribute 'cmd'

Windows10.

astaric · 2017-09-06T08:02:51Z

Fixed, can you try again?

ajdapretnar · 2017-09-06T08:32:07Z

Um, now I don't get this error, but another issue occurs. Potentially not related to this PR, but I tried to install all the add-ons and some crash normally (Geo, Recommendation, Timeseries), while the rest seem to get installed. However, when I re-run Orange, the entire program crashes. Ran with -l 4, I see that the issue is in Spark and Textable add-ons, most likely.

markotoplak · 2017-09-06T14:44:46Z

Work on linux without conda. Works just as well as installing with pip directly (succeeds if pip succeeds, fails with an error of it).

kernc · 2017-09-06T14:54:09Z

Even if a single addon is selected and the installation fails, the information dialog encouraging a restart still shows.

astaric · 2017-09-24T20:44:22Z

Blocked by #2616

Until we install conda command and conda-forge channel with our miniconda installer, this will not work

astaric · 2017-09-26T08:46:29Z

@kernc, @ajdapretnar, @markotoplak

I think I am done. Could you try it out again?

kernc · 2017-09-26T12:47:22Z

Orange/canvas/application/addons.py

+        else:
+            files = [os.path.join(
+                USER_SITE, 'orangecontrib',
+                dist.project_name.split('-')[-1].lower()), ]


This assumes all user-installed add-ons are in orangecontrib package, but this is just an internal convention, with the real add-on being anything that provides an orange3.addon entry point.

So I should wrap this in try: except: pass?

This is the same code we are using at the moment, I have just moved it to a separate class.

I know, and I wrote it, but it's wrong in its assumptions.

No idea how to make it correct.

HTH

kernc · 2017-09-26T14:17:26Z

Installing e.g. Orange3-Timeseries

Running conda install --yes Orange3-Timeseries
Fetching package metadata ...............
Solving package specifications: .

Package plan for installation in environment /home/jk/Downloads/miniconda3:

The following NEW packages will be INSTALLED:

    anyqt:              0.0.8-py35_1 conda-forge
    bottleneck:         1.2.1-py35_1 conda-forge
    dill:               0.2.6-py35_0 conda-forge
    keyring:            9.0-py35_0   conda-forge
    orange3:            3.4.1-py35_0 anaconda   
    orange3-timeseries: 0.3.1-py35_0 conda-forge

Orange3 (lower version) gets reinstalled.

astaric · 2017-09-26T14:21:50Z

@kernc what are your conda channel priorities?

kernc · 2017-09-26T14:30:12Z

The order of channels in .condarc:

channels:
  - conda-forge
  - anaconda
  - anaconda-fusion
  - defaults

ajdapretnar · 2017-09-27T07:42:01Z

When installing Orange3-Network, installation seems to stall forever. It reaches 'Orange3-Network successfully installed' message, but it never exits the process. I also can't quit the dialog window and have to force quit in the console.

Also, installations prints every increment in the process, e.g. orange3-networks 3%, orange3-networks 4%, orange3-networks 5%... etc. Is there a silent mode we can turn on? :)

Works on windows.

astaric · 2017-10-17T07:00:34Z

I have added the -q flag to conda calls. The stalling is not related to conda, since it already happens sometimes with the master version of Orange.

Otherwise, orange tries to downgrade to the (outdated) version from defaults.

astaric · 2017-10-20T14:30:33Z

I have disabled the conda installation of add-ons by default (it can be enabled with a checkbox in settings), hidden the download progressbar (--quiet option is passed to conda).

I have updated the order of channels in the environment created by our miniconda installer, as having defaults on top downgrades orange whenever you install an add-on. Pip uninstall now only runs when conda fails to uninstall. And all package names passed to conda are now lowercase (heuristic, but works :)).

If there is anything that needs to be fixed, please let me know. Otherwise I consider this PR complete.

As some commands are case sensitive and other not, always normalize package names to lowercase. When uninstalling, run pip uninstall only when that package could not be uninstalled with conda.

kernc reviewed Sep 5, 2017

View reviewed changes

astaric force-pushed the conda-install-addons branch from e0a88cb to e95028b Compare September 6, 2017 04:54

astaric force-pushed the conda-install-addons branch from e95028b to c356dc9 Compare September 6, 2017 08:01

astaric force-pushed the conda-install-addons branch from bc7b8ce to 1278e52 Compare September 6, 2017 12:24

astaric force-pushed the conda-install-addons branch from 1278e52 to 4e502cb Compare September 8, 2017 07:33

ajdapretnar mentioned this pull request Sep 8, 2017

Re-run add-on installation when failing on a package #2407

Closed

astaric force-pushed the conda-install-addons branch from 4e502cb to 0ab4f25 Compare September 21, 2017 12:19

astaric changed the title ~~Conda install addons~~ [ENH] Add-on installation with Conda Sep 21, 2017

astaric force-pushed the conda-install-addons branch from 0ab4f25 to 700dd68 Compare September 25, 2017 15:02

kernc reviewed Sep 26, 2017

View reviewed changes

astaric added 4 commits October 16, 2017 09:54

addon: Refactor installation code

e674f93

addon: Install with conda (if possible)

c93de13

CondaInstaller: Separate path for conda on windows

7f273f5

add-ons: Hide subprocess console windows

6da6958

Works on windows.

astaric force-pushed the conda-install-addons branch from 700dd68 to 6da6958 Compare October 16, 2017 07:56

astaric added this to the 3.7 milestone Oct 17, 2017

add-ons: Opt-in installation with conda

c05681f

conda: hide download progressbars

c0aeb7b

astaric force-pushed the conda-install-addons branch from b73c8d1 to c0aeb7b Compare October 20, 2017 11:54

conda installer: put conda-forge channel on top

0ed37c2

Otherwise, orange tries to downgrade to the (outdated) version from defaults.

add-ons: Fix conda commands

7fc1f91

As some commands are case sensitive and other not, always normalize package names to lowercase. When uninstalling, run pip uninstall only when that package could not be uninstalled with conda.

astaric force-pushed the conda-install-addons branch from 348ebad to 7fc1f91 Compare October 21, 2017 11:50

lanzagar merged commit 89059d5 into biolab:master Oct 24, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ENH] Add-on installation with Conda #2561

[ENH] Add-on installation with Conda #2561

astaric commented Sep 5, 2017

kernc Sep 5, 2017

astaric Sep 6, 2017

codecov-io commented Sep 6, 2017 •

edited

Loading

ajdapretnar commented Sep 6, 2017 •

edited by kernc

Loading

astaric commented Sep 6, 2017

ajdapretnar commented Sep 6, 2017 •

edited

Loading

markotoplak commented Sep 6, 2017

kernc commented Sep 6, 2017

astaric commented Sep 24, 2017

astaric commented Sep 26, 2017

kernc Sep 26, 2017

astaric Sep 26, 2017 •

edited

Loading

kernc Sep 26, 2017

kernc commented Sep 26, 2017

astaric commented Sep 26, 2017

kernc commented Sep 26, 2017

ajdapretnar commented Sep 27, 2017

astaric commented Oct 17, 2017

astaric commented Oct 20, 2017

[ENH] Add-on installation with Conda #2561

[ENH] Add-on installation with Conda #2561

Conversation

astaric commented Sep 5, 2017

Issue

Description of changes

Includes

kernc Sep 5, 2017

Choose a reason for hiding this comment

astaric Sep 6, 2017

Choose a reason for hiding this comment

codecov-io commented Sep 6, 2017 • edited Loading

Codecov Report

ajdapretnar commented Sep 6, 2017 • edited by kernc Loading

astaric commented Sep 6, 2017

ajdapretnar commented Sep 6, 2017 • edited Loading

markotoplak commented Sep 6, 2017

kernc commented Sep 6, 2017

astaric commented Sep 24, 2017

astaric commented Sep 26, 2017

kernc Sep 26, 2017

Choose a reason for hiding this comment

astaric Sep 26, 2017 • edited Loading

Choose a reason for hiding this comment

kernc Sep 26, 2017

Choose a reason for hiding this comment

kernc commented Sep 26, 2017

astaric commented Sep 26, 2017

kernc commented Sep 26, 2017

ajdapretnar commented Sep 27, 2017

astaric commented Oct 17, 2017

astaric commented Oct 20, 2017

codecov-io commented Sep 6, 2017 •

edited

Loading

ajdapretnar commented Sep 6, 2017 •

edited by kernc

Loading

ajdapretnar commented Sep 6, 2017 •

edited

Loading

astaric Sep 26, 2017 •

edited

Loading