Skip to content

UtilizingExternalTools

Jump to bottom
Gregory V. Bard edited this page Jul 19, 2016 · 16 revisions

Utilizing External Tools (and Jupyter) from SageMathCloud

This FAQ is all about connecting SageMathCloud to external resources. A highly related page is the FAQ for Programming Software in SageMathCloud, with information about C, R, Fortran, Java, Octave, et cetera.

Remember: if you don't find what you need, or if you'd like to ask a question, then please email [email protected] at any time. We'd love to hear from you! Please include a link (the URL address in your browser) to any relevant project or document, as part of your email.

List of Questions:

Question: I would like to ssh into my project.

This is explained on the page "All about Projects." Click here to jump there

Question: How can I sync my files with Dropbox or Google Drive?

It is currently not possible to sync files in a SageMathCloud project with either Dropbox or Google Drive.

Question: I would like to push and pull from github using ssh.

First, despite the fact that you are accessing SageMathCloud through the internet, you are actually working in a highly restricted environment. Processes running inside a free project are not allowed to directly access the internet. (We do not allow such access for free users, since when we did, malicious users launched attacks on other computers from SageMathCloud.) Enable internet access by adding the "internet access" quota. See https://cloud.sagemath.com/policies/pricing.html

Once you have enabled network access for your project, to use ssh from SMC to connect to github you need to create an ssh key and explicitly allow that account/key access to github.

  1. Make a terminal: "+New --> Terminal"

  2. Create an ssh key public/private key pair by typing the command ssh-keygen.

  3. Copy .ssh/id_rsa.pub to github.

Question: I would like to edit files (with sync) or manipulate data files that are on some remote server that I have ssh access to

WARNING: The following will only work if your project has outgoing network access, i.e., it says "Network access: true" under project settings. (Outgoing network access is blocked by default due to people using SageMathCloud as a platform to attack other computers. Please sign up for a membership in order to get network access.)

Suppose you have an account on some computer that you can ssh to, e.g, so this works for you:

ssh [email protected]

To temporarily make the files at [email protected] available in a cloud project, open a Terminal and type

mkdir foo   # make any directory you want
sshfs [email protected]:path/to/files/ foo/

and type your password to login to [email protected]. You'll find that you can now use the files at path/to/files in [email protected] as if they were in the directory foo in your project. You can edit files (with sync), open and output data files, etc. When you're done, type

fusermount -u foo

to unmount foo. Whenever the project server is restarted, foo/ will also be unmounted. So you'll have to type sshfs [email protected]: foo/ to mount foo every once in a while.

(Alternatively, you can type crontab -e in a terminal, and add a line like this */2 * * * * sshfs [email protected]: foo/, but that would require setting up ssh keys for passwordless connections to [email protected], which you may or may not want to do.)

NOTE: The files in foo/ will not be included in snapshots, because they are part of another filesystem, and the snapshot system is configured to not cross filesystem boundaries.

Question: I would like to use Emacs in a Terminal on Chrome. Help!

Chrome absolutely doesn't allow normal web applications to intercept certain keystrokes, which makes Emacs-in-a-terminal painful on some operating systems (esp. Linux/Windows). E.g., Ctrl-N brings up a new window, instead of going to the next line! (It is possible to eventually get around this by creating a Chrome extension, but I haven't done this yet.) In the meantime, you can install the SageMathCloud chrome app from the app store or on Linux, type google-chrome --app=https://cloud.sagemath.com to local the website as an app (if you use chromium instead, type chromium-browser --app=https://cloud.sagemath.com).

This is not a completely solution since in some cases, control-shift-minus and control-shift-plus still zoom in and out (on ChromeOS they zoom the entire desktop -- every window, all icons, the time, etc., in and out!). This is a major problem, because control-shift-minus is "undo" in emacs. The workaround I currently use is to put this in my .emacs file, and instead type "alt-u" for undo:

(define-key esc-map "u" 'undo)

Note that undo is also available by default through other key bindings, like C-x u and C-/. (Execute "M-x where-is undo".) So there are other options available.

< a name="convert-R-markdown-to-PDF"/> How do I convert R markdown to a PDF file in SageMathCloud?

Run knit in R and pandoc as a shell command.

This is easier explained via an example:

Suppose you have an R markdown file "myproject.Rmd". In a .sagews worksheet, do

% r
library(markdown)
library(knitr)
knit("myproject.Rmd")

The last step above produces an ordinary markdown (.md) file. The following shell command converts it to a PDF file.

%sh
pandoc myproject.md -o myproject.pdf

Note: make sure that the R and the sh cells have the same working directory.

I created an HTML form in a .sagews file using HTML, CSS and JS. How can I connect my form with my Python code?

To create a connection between your HTML form in a .sagews file created using HTML, CSS and JS, you need to use the worksheet.execute_code() function in your JS code.

Because worksheet.execute_code isn't a standard JS function, but special SageMathCloud function, you need to load your JS code with worksheet.execute_code(). In particular do NOT use

load('path/to/js/code.js')

but instead use

salvus.javascript(open('path/to/js/code.js').read())

For example in .sagews file suppose you created a div with id='myApp' as follows:

%html
<div id='myApp'>
....
    <div id="msgLog"></div>
    <div id="msgErr"></div>
</div>

Let's say your Python function will double x:

def myfunc(x):
    print(x*2)

In your JS code type:

worksheet.execute_code({
    code: 'myfunc(n)',
    data: {n: 2},
    preparse: true,
    cb: function(msg){
            if(msg.stdout){$('#myApp #msgLog').html(msg.stdout);}
            if(msg.stderr){$('#myApp #msgErr').html(msg.stderr);}
    }
});

Please note that myfunc() doesn't return anything. On the contrary, it uses print() to send output. This is because JS and python are different languages, and you can't just use return in your Python function to return some answer. stdout in JS code means standard output stream. That is, the print function in your Python code places the result of myfunc() in the output stream. That's why you need to use print() but not return() in your Python code.

Also, if your Python code will raise some exception, then it will result in output to stderr -- the standard error stream. If you JS code (as in the example above) catches stderr, you can get any error message from your Python code.

Question: I would like to use a customized version of FriCAS in a worksheet

The following Sage/Python command modifies the PATH variable to include $HOME/bin. Sage looks for the fricas executable in this PATH:

os.environ['PATH'] = '%s/bin:%s'%(os.environ['HOME'],os.environ['PATH'])

Restart the worksheet to make sure the new version of FriCAS is started.

Question: I want to use a Jupyter Notebook (with Sage support)

Click "+New", then enter a filename and click Jupyter Notebook. This is Jupyter, but enhanced with realtime synchronization. You can also just run a standard Jupyter notebook server (no sync, not integrated into cloud) by (1) finding your project's id in project settings, then (2) visiting https://cloud.sagemath.com/[project_id]/port/jupyter (you will possibly have to refresh your browser if this takes too long the first time). Any collaborator on your project can securely use the Jupyter notebook server by visiting this link, but nobody else can.

If you want to make Jupyter Sage-friendly, so the Sage preparser and library are available, type

 %load_ext sage

Note: nobody has added support for displaying plots yet to this mode (but Volker Braun just did a lot of work on this and it is coming soon!). As a temporary workaround, this function will display a Sage plot:

def showplot(g):
    import IPython.display
    save(g, 'a.png')
    display(IPython.display.Image('a.png'))

For example, this will now work:

g = plot(sin,(0,5)) + plot(cos, (0,5), color='red')
showplot(g)

Question: I want to use my own copy of Jupyter, which is part of my own Python stack

You can install any version of Python you want into your own project -- just download and build it locally, and install Jupyter (and whatever you want) into it. Then, make a directory "bin" in the home directory of your project and put a symlink to your ipython there, e.g.,

cd ~/bin/
ln -s /path/to/your/ipython .

In project settings, restart your project server (just to be sure), then make sure that when you type "ipython" on the command line, you get your Jupyter. Then clicking on existing Jupyter notebooks or creating new ones (through the gui) should use your copy of Jupyter. It just uses the "Jupyter" that is in the path, since ipython-notebook just sets up some options and runs "ipython notebook" (look at ~/.smc/ipython-notebook).

Question: I want to install the cite2c Jupyter extension, so I can use Zotero

(WARNING: Due to people launching attacks from SMC, you must pay to enable network access for your project in order to use github.)

Do the following in a terminal:

git clone https://github.com/takluyver/cite2c.git && cd cite2c && sage install.py

Now when you open a Jupyter notebook, there will be two new buttons at the end of the button bar, which will let you insert citations.

How do I use a Jupyter kernel from a Sage worksheet?

See Sage Jupyter

Question: How can I add my own Jupyter Kernel?

Jupyter Kernels are small JSON files, telling the Jupyter Notebook what program to start to communicate with the actual kernel that holds your working session and executes your code. To install your own, they need to go into the directory:

$HOME/.local/share/jupyter/kernels/[name_of_your_kernel]/kernel.json

For example, this is the content of kernel.json file is running Anaconda Python 3:

{
 "display_name": "My Anaconda (Python 3)",
 "argv": [
  "/projects/anaconda3/bin/python3",
  "-E",
  "-m",
  "ipykernel",
  "--matplotlib=inline",
  "-f",
  "{connection_file}"
 ],
 "language": "python",
 "env":{
    "LD_LIBRARY_PATH" : "/projects/anaconda3/lib",
    "PYTHONPATH" : "/projects/anaconda3/lib/python3.5:/projects/anaconda3/lib/python3.5/site-packages",
    "PYTHONHOME" : "/projects/anaconda3/lib/python3.5"
 }
}

Once that file is there, click the reload button (top right of the jupyter notebook) to let it search for kernels and find your configuration.

Question: Is Scilab broken?

No, the Java stacktrace only happens because there is no graphical interface available due to the fact that there is no graphical interface ;-)

Type

scilab-adv-cli

or

scilab-cli

or

scilab -nw

Question: How can I use the fish shell with correctly working Home/End keys?

You can automatically span a new process instead of the default bash and you have to set the TERM env variable correctly to use xterm! I.e.

  1. Click on the rocket-icon in a terminal

  2. Add this to the setup file:

    source ~/.bashrc
export TERM=xterm-256color
exec fish

Next time you launch this terminal, or restart it via exit and pressing the return-key, this setup file will be evaluated and you'll end up in the fish shell with the correct TERM env variable set.

Question: I'm using the vim pentadactyl plugin, and the editor isn't working.

The Pentadactyl plugin is not compatible with the Codemirror editor in SageMathCloud. You can disable Pentadactyl for a particular page, perhaps by pressing Control+Z. SagemMathCloud has it's own Vim bindings for worksheets and editing files; to enable it, go to setting (click the gear by your name in the upper right), then select Vim next to Editor --> Bindings.

Question: My vim key bindings aren't working in SageMathCloud.

If you have a browser extension that uses the vim keys it may intercept the 'Esc' character, preventing you from switching to command mode in SageMath Cloud. Either remap 'Esc' to a different key for the browser extension or disable the extension on cloud.sagemath.com.

Question: I want to XXX, but I don't see XXX above.

Do not hesitate to email THE LINK TO YOUR PROJECT TO [email protected] or https://groups.google.com/forum/?fromgroups#!forum/sage-cloud

This Wiki is for CoCalc.com.

A more structured documentation is the CoCalc User Manual.

For further questions, please contact us.

Clone this wiki locally