GPU's on https://cocalc.com... coming soon?

[williamstein]

#gpu #woot I just ran a GPU notebook on cocalc.com ! It successfully trained a model on MNIST using pytorch in 69 seconds; doing the same with the CPU in my project took 3x as long:

I did this by spinning up this little machine with an Nvideo T4 attached (it's about $250/month) in a different region, and installed the Ubuntu 22.04 server Lambda stack Ubuntu ppa:

I also installed nodejs version 18:

apt-get install -y gnupg2 vim libreadline-dev libz-dev libssl-dev libpq5 libpq-dev      postgresql-client rsync ssh curl wget git dpkg-dev make python python3-pip flex g++ libxml2-dev libxslt-dev
 curl -sL https://deb.nodesource.com/setup_18.x | bash -
apt-get install -y nodejs
npm install -g npm pnpm

Then I installed the latest version of cocalc:

git clone https://github.com/sagemathinc/cocalc
cd cocalc/src
pnpm build-dev

This builds far more than what is needed for just using this machine to run code.

I then started a Jupyter notebook with the python3 systemwide kernel in one of my notebooks a.ipynb, and added a project-specific api key in project settings. On my GPU machine, I did:

cd cocalc/src/packages/compute
API_SERVER=https://cocalc.com API_KEY=sk-7....XD000004 DEBUG_CONSOLE=yes DEBUG=cocalc:* node
...
> x = require('@cocalc/compute/jupyter').jupyter({path:'a.ipynb',project_id:"4a5f0542-5873-4eed-a85c-a18c706e8bcd"});0;

Suddenly the notebook a.ipynb on cocalc.com was running on this GPU machine. I still get all the realtime collab, timetravel, etc. features of cocalc -- it's just that the kernel runs on this GPU machine. That machine can be anywhere that has an outgoing network connection to https://cocalc.com via HTTP, since it uses only a single websocket.

This is just the very first steps in the architecture that we're build on cocalc to enable much more flexible and powerful compute for our notebooks. I'm not at all suggesting anybody directly do what is outlined above... though in theory you could today if you wanted to. Instead, we're going to use the functionality to build easy pay-as-you-go use of arbitrary compute resources from many different clouds...