diff --git a/notebooks/data/metadata.star b/notebooks/data/metadata.star index f5f6b4dd..038b239d 100644 --- a/notebooks/data/metadata.star +++ b/notebooks/data/metadata.star @@ -1,4 +1,4 @@ -# Created by the starfile Python package (version 0.4.11) at 11:16:57 on 09/03/2022 +# Created by the starfile Python package (version 0.4.11) at 17:31:20 on 02/05/2022 data_ diff --git a/notebooks/particle_metadata.ipynb b/notebooks/particle_metadata.ipynb index ac28be92..b8199c62 100644 --- a/notebooks/particle_metadata.ipynb +++ b/notebooks/particle_metadata.ipynb @@ -9,7 +9,7 @@ }, "source": [ "# Particle Metadata\n", - "``ioSPI`` library provides functionalities to work with cryo-EM data. To store the data, ``ioSPI`` uses the STAR (Self-defining Text Archiving and Retrieval) format (Hall, Allen and Brown, 1991) which is used by RELION for the storage of label-value pairs for all kinds of input and output metadata. In ``ioSPI``, the module `particle_metatdata` is used to create a STAR file `.star`. This module formats and writes particle metadata as `.star` files, following RELION conventions. This tutorial shows you how to create a `.star` file using `particle_detadata`." + "``ioSPI`` library provides functionalities to work with cryo-EM data. To play with the data, we first need to store the data in a way that `ioSPI`can understand. To do that, ``ioSPI`` uses the STAR (Self-defining Text Archiving and Retrieval) format (Hall, Allen and Brown, 1991) which is used by RELION for the storage of label-value pairs for all kinds of input and output metadata. In ``ioSPI``, the module `particle_metatdata` is used to create a STAR file `.star`. This module formats and writes particle metadata as `.star` files, following RELION conventions. This tutorial shows you how to create a `.star` file using `particle_detadata`." ] }, { diff --git a/notebooks/tutorial.ipynb b/notebooks/tutorial.ipynb index 52aaa5fb..4a07454b 100644 --- a/notebooks/tutorial.ipynb +++ b/notebooks/tutorial.ipynb @@ -1,431 +1,544 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# ioSPI\n", - "In this tutorial we will explain the basic pipeline of ``ioSPI``. Library which provides functionalities to work with cryo-EM data." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Particle Metadata\n", - "\n", - "In the first part of the tutorial we show how to create a `.star` file using the module `particle_metadata`. This module basically format and write particle metadata as `.star` files, following RELION conventions." - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [], - "source": [ - "import os\n", - "import sys\n", - "import warnings\n", - "\n", - "sys.path.append(os.path.dirname(os.getcwd()))\n", - "warnings.filterwarnings('ignore')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In order to create the `.star` file is necessary to provide information about the experiment, such as the image pixel size and image center shift. This information is passed in the form of a list and a `Config` object." - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": {}, - "outputs": [], - "source": [ - "from ioSPI import particle_metadata\n", - "\n", - "class Config:\n", - " \"\"\"Class to instantiate the config object.\"\"\"\n", - " def __init__(self, ctf, shift):\n", - " self.ctf = ctf\n", - " self.shift = shift" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [], - "source": [ - "data_list = [[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]]\n", - "config = Config(ctf=True, shift=True)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The names of the metadata for the star file (in RELION convention) can be accessed using the function `get_starfile_metadata_names` passing a `Config` object." - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "['__rlnImageName', '__rlnAngleRot', '__rlnAngleTilt', '__rlnAnglePsi', '__rlnOriginX', '__rlnOriginY', '__rlnDefocusU', '__rlnDefocusV', '__rlnDefocusAngle', '__rlnVoltage', '__rlnImagePixelSize', '__rlnSphericalAberration', '__rlnAmplitudeContrast', '__rlnCtfBfactor']\n" - ] - } - ], - "source": [ - "variable_names = particle_metadata.get_starfile_metadata_names(config)\n", - "print(variable_names)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Using the list of values and the `Config` object it is possible to format the `.star` file that will be later saved with the `format_metadata_for_writing_cryoem_convention` function, which creates a dataframe with the data." - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "
\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
__rlnImageName__rlnAngleRot__rlnAngleTilt__rlnAnglePsi__rlnOriginX__rlnOriginY__rlnDefocusU__rlnDefocusV__rlnDefocusAngle__rlnVoltage__rlnImagePixelSize__rlnSphericalAberration__rlnAmplitudeContrast__rlnCtfBfactor
01234567891011121314
\n", - "
" - ], - "text/plain": [ - " __rlnImageName __rlnAngleRot __rlnAngleTilt __rlnAnglePsi __rlnOriginX \\\n", - "0 1 2 3 4 5 \n", - "\n", - " __rlnOriginY __rlnDefocusU __rlnDefocusV __rlnDefocusAngle \\\n", - "0 6 7 8 9 \n", - "\n", - " __rlnVoltage __rlnImagePixelSize __rlnSphericalAberration \\\n", - "0 10 11 12 \n", - "\n", - " __rlnAmplitudeContrast __rlnCtfBfactor \n", - "0 13 14 " - ] - }, - "execution_count": 8, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "metadata_df = particle_metadata.format_metadata_for_writing_cryoem_convention(data_list=data_list, config=config)\n", - "metadata_df" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "After formatting the data we can use the function `write_metadata_to_starfile` providing the metadata, the path and name of the star file. " - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": {}, - "outputs": [], - "source": [ - "metadata_path = os.path.join(os.getcwd(), \"data\")\n", - "filename = \"metadata.star\"\n", - "particle_metadata.write_metadata_to_starfile(path=metadata_path, metadata=metadata_df, filename=filename)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Finally, we check whether a `.star` file with the name `metadata.star` was created or not, using the function `check_star` function which will raise an exception if the file is not found." - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": {}, - "outputs": [], - "source": [ - "particle_metadata.check_star_file(os.path.join(metadata_path, filename))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The file was successfully created as shown by printing its content." - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "# Created by the starfile Python package (version 0.4.11) at 11:16:57 on 09/03/2022\n", - "\n", - "data_\n", - "\n", - "loop_\n", - "___rlnImageName #1\n", - "___rlnAngleRot #2\n", - "___rlnAngleTilt #3\n", - "___rlnAnglePsi #4\n", - "___rlnOriginX #5\n", - "___rlnOriginY #6\n", - "___rlnDefocusU #7\n", - "___rlnDefocusV #8\n", - "___rlnDefocusAngle #9\n", - "___rlnVoltage #10\n", - "___rlnImagePixelSize #11\n", - "___rlnSphericalAberration #12\n", - "___rlnAmplitudeContrast #13\n", - "___rlnCtfBfactor #14\n", - "1\t2\t3\t4\t5\t6\t7\t8\t9\t10\t11\t12\t13\t14\n", - "\n", - "\n", - "\n" - ] - } - ], - "source": [ - "with open(os.path.join(metadata_path, filename)) as star_file:\n", - " print(star_file.read())\n", - " star_file.close()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Datasets\n", - "\n", - "In the second part of the tutorial we will show how to manage cryo-EM datasets using the `datasets` module from `ioSPI`. This module uses the Open Science Foundation (OSF) framework, which is an initiative that aims to increase the openness, reproducibility and integrity of scientific research. Among other functionalities, it is possible to upload scientific data which can be accessed by an Application Programming Interface (API). ``ioSPI`` offers functionalities that allow uploading and accessing cryo-EM data using the OSF APIv2. This notebook is by no means a tutorial for the OSF API, but we will introduce some basic concepts used here. For more information you can access . The two main components from the OSF that we use here are:\n", - "\n", - "* GUID: Every file, project, and component on the OSF gets a Globally Unique ID (GUID). The GUID is the five characters after the in the web address. For instance, we provide cryo-EM Datasets with GUID 24htr. There you will find cryo-EM datasets for different proteins.\n", - "* Nodes: On OSF files, projects, and components are called nodes, they can be either public or private and it is where the data is contained. For example, is a node containing cryo-EM data." - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": {}, - "outputs": [], - "source": [ - "from ioSPI import datasets" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Before following this tutorial you will have to create your own node for upload cryo-EM data and set an access token. The access token is responsible to manage which actions can be performed by the user who possesses the token. After this step it is possible to instantiate an `OSFUpload` object, to access this OSF node using the OSF API by informing the GUID of the node and the access token. " - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\n" - ] - } - ], - "source": [ - "token = 's9XHmPaLzRoJnNFSLngb2pH6KPtmCOxEgdJitRiQuRROq7omjX8SKyBIgfZYTiC1X0154o'\n", - "\n", - "guid = '8e5j9'\n", - "\n", - "osf = datasets.OSFUpload(token = token, data_node_guid= guid)\n", - "print(osf.headers)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now we will create a child node inside the parent node, for representing the dataset with 80s ribosome data. We will use the pdb id as the name for this new node, and the function will return its GUID. Since the child is also a node, it can accessed separately from the parent node." - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "8yg32\n" - ] - } - ], - "source": [ - "pdb_id = '4v6x'\n", - "child_guid = osf.write_child_node(parent_guid = guid, title = pdb_id)\n", - "print(child_guid)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Having created a child node for putting the files related to the 80s ribosome, we can upload files to it by using the function `write_files`." - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Uploaded c:\\Users\\Luis\\Documents\\GitHub\\ioSPI\\notebooks\\4v6x.pdb \n" - ] - }, - { - "data": { - "text/plain": [ - "True" - ] - }, - "execution_count": 16, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#Create a list of the files to be uploaded\n", - "file_paths = [os.path.join(os.getcwd(), 'data', pdb_id + '.pdb')]\n", - "\n", - "#Write files at the child node\n", - "osf.write_files(dataset_guid = child_guid, file_paths = file_paths)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "We can now check if it was uploaded correctly by checking if the function `read_structure_guid` will return a GUID corresponding to the pdb id passed as parameter." - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "'8yg32'" - ] - }, - "execution_count": 17, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "osf.read_structure_guid(pdb_id)" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.8.12" - } - }, - "nbformat": 4, - "nbformat_minor": 2 -} +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "outputs": [], + "source": [ + "# ioSPI\n", + "In this tutorial we will explain the basic pipeline of ``ioSPI``. Library which provides functionalities to work with cryo-EM data." + ], + "metadata": { + "collapsed": false, + "pycharm": { + "name": "#%%\n" + } + } + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "## Particle Metadata\n", + "\n", + "In the first part of the tutorial we show how to create a `.star` file using the module `particle_metadata`. This module basically format and write particle metadata as `.star` files, following RELION conventions." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "import os\n", + "import sys\n", + "import warnings\n", + "\n", + "sys.path.append(os.path.dirname(os.getcwd()))\n", + "warnings.filterwarnings('ignore')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "In order to create the `.star` file is necessary to provide information about the experiment, such as the image pixel size and image center shift. This information is passed in the form of a list and a `Config` object." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "from ioSPI import particle_metadata\n", + "\n", + "class Config:\n", + " \"\"\"Class to instantiate the config object.\"\"\"\n", + " def __init__(self, ctf, shift):\n", + " self.ctf = ctf\n", + " self.shift = shift" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "data_list = [[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]]\n", + "config = Config(ctf=True, shift=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "The names of the metadata for the star file (in RELION convention) can be accessed using the function `get_starfile_metadata_names` passing a `Config` object." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['__rlnImageName', '__rlnAngleRot', '__rlnAngleTilt', '__rlnAnglePsi', '__rlnOriginX', '__rlnOriginY', '__rlnDefocusU', '__rlnDefocusV', '__rlnDefocusAngle', '__rlnVoltage', '__rlnImagePixelSize', '__rlnSphericalAberration', '__rlnAmplitudeContrast', '__rlnCtfBfactor']\n" + ] + } + ], + "source": [ + "variable_names = particle_metadata.get_starfile_metadata_names(config)\n", + "print(variable_names)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "Using the list of values and the `Config` object it is possible to format the `.star` file that will be later saved with the `format_metadata_for_writing_cryoem_convention` function, which creates a dataframe with the data." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
__rlnImageName__rlnAngleRot__rlnAngleTilt__rlnAnglePsi__rlnOriginX__rlnOriginY__rlnDefocusU__rlnDefocusV__rlnDefocusAngle__rlnVoltage__rlnImagePixelSize__rlnSphericalAberration__rlnAmplitudeContrast__rlnCtfBfactor
01234567891011121314
\n", + "
" + ], + "text/plain": [ + " __rlnImageName __rlnAngleRot __rlnAngleTilt __rlnAnglePsi __rlnOriginX \\\n", + "0 1 2 3 4 5 \n", + "\n", + " __rlnOriginY __rlnDefocusU __rlnDefocusV __rlnDefocusAngle \\\n", + "0 6 7 8 9 \n", + "\n", + " __rlnVoltage __rlnImagePixelSize __rlnSphericalAberration \\\n", + "0 10 11 12 \n", + "\n", + " __rlnAmplitudeContrast __rlnCtfBfactor \n", + "0 13 14 " + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "metadata_df = particle_metadata.format_metadata_for_writing_cryoem_convention(data_list=data_list, config=config)\n", + "metadata_df" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "After formatting the data we can use the function `write_metadata_to_starfile` providing the metadata, the path and name of the star file. " + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "metadata_path = os.path.join(os.getcwd(), \"data\")\n", + "filename = \"metadata.star\"\n", + "particle_metadata.write_metadata_to_starfile(path=metadata_path, metadata=metadata_df, filename=filename)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "Finally, we check whether a `.star` file with the name `metadata.star` was created or not, using the function `check_star` function which will raise an exception if the file is not found." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "particle_metadata.check_star_file(os.path.join(metadata_path, filename))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "The file was successfully created as shown by printing its content." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "# Created by the starfile Python package (version 0.4.11) at 11:16:57 on 09/03/2022\n", + "\n", + "data_\n", + "\n", + "loop_\n", + "___rlnImageName #1\n", + "___rlnAngleRot #2\n", + "___rlnAngleTilt #3\n", + "___rlnAnglePsi #4\n", + "___rlnOriginX #5\n", + "___rlnOriginY #6\n", + "___rlnDefocusU #7\n", + "___rlnDefocusV #8\n", + "___rlnDefocusAngle #9\n", + "___rlnVoltage #10\n", + "___rlnImagePixelSize #11\n", + "___rlnSphericalAberration #12\n", + "___rlnAmplitudeContrast #13\n", + "___rlnCtfBfactor #14\n", + "1\t2\t3\t4\t5\t6\t7\t8\t9\t10\t11\t12\t13\t14\n", + "\n", + "\n", + "\n" + ] + } + ], + "source": [ + "with open(os.path.join(metadata_path, filename)) as star_file:\n", + " print(star_file.read())\n", + " star_file.close()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "# Datasets\n", + "\n", + "In the second part of the tutorial we will show how to manage cryo-EM datasets using the `datasets` module from `ioSPI`. This module uses the Open Science Foundation (OSF) framework, which is an initiative that aims to increase the openness, reproducibility and integrity of scientific research. Among other functionalities, it is possible to upload scientific data which can be accessed by an Application Programming Interface (API). ``ioSPI`` offers functionalities that allow uploading and accessing cryo-EM data using the OSF APIv2. This notebook is by no means a tutorial for the OSF API, but we will introduce some basic concepts used here. For more information you can access . The two main components from the OSF that we use here are:\n", + "\n", + "* GUID: Every file, project, and component on the OSF gets a Globally Unique ID (GUID). The GUID is the five characters after the in the web address. For instance, we provide cryo-EM Datasets with GUID 24htr. There you will find cryo-EM datasets for different proteins.\n", + "* Nodes: On OSF files, projects, and components are called nodes, they can be either public or private and it is where the data is contained. For example, is a node containing cryo-EM data." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "from ioSPI import datasets" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "Before following this tutorial you will have to create your own node for upload cryo-EM data and set an access token. The access token is responsible to manage which actions can be performed by the user who possesses the token. After this step it is possible to instantiate an `OSFUpload` object, to access this OSF node using the OSF API by informing the GUID of the node and the access token. " + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [ + { + "ename": "HTTPError", + "evalue": "401 Client Error: Unauthorized for url: https://api.osf.io/v2/", + "output_type": "error", + "traceback": [ + "\u001B[0;31m---------------------------------------------------------------------------\u001B[0m", + "\u001B[0;31mHTTPError\u001B[0m Traceback (most recent call last)", + "Input \u001B[0;32mIn [8]\u001B[0m, in \u001B[0;36m\u001B[0;34m()\u001B[0m\n\u001B[1;32m 1\u001B[0m token \u001B[38;5;241m=\u001B[39m \u001B[38;5;124m'\u001B[39m\u001B[38;5;124ms9XHmPaLzRoJnNFSLngb2pH6KPtmCOxEgdJitRiQuRROq7omjX8SKyBIgfZYTiC1X0154o\u001B[39m\u001B[38;5;124m'\u001B[39m\n\u001B[1;32m 3\u001B[0m guid \u001B[38;5;241m=\u001B[39m \u001B[38;5;124m'\u001B[39m\u001B[38;5;124m8e5j9\u001B[39m\u001B[38;5;124m'\u001B[39m\n\u001B[0;32m----> 5\u001B[0m osf \u001B[38;5;241m=\u001B[39m \u001B[43mdatasets\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mOSFUpload\u001B[49m\u001B[43m(\u001B[49m\u001B[43mtoken\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m=\u001B[39;49m\u001B[43m \u001B[49m\u001B[43mtoken\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mdata_node_guid\u001B[49m\u001B[38;5;241;43m=\u001B[39;49m\u001B[43m \u001B[49m\u001B[43mguid\u001B[49m\u001B[43m)\u001B[49m\n\u001B[1;32m 6\u001B[0m \u001B[38;5;28mprint\u001B[39m(osf\u001B[38;5;241m.\u001B[39mheaders)\n", + "File \u001B[0;32m~/ucsb/projects/compSPI/ioSPI/notebooks/../../ioSPI/ioSPI/datasets.py:37\u001B[0m, in \u001B[0;36mOSFUpload.__init__\u001B[0;34m(self, token, data_node_guid)\u001B[0m\n\u001B[1;32m 34\u001B[0m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mheaders \u001B[38;5;241m=\u001B[39m {\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mAuthorization\u001B[39m\u001B[38;5;124m\"\u001B[39m: \u001B[38;5;124mf\u001B[39m\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mBearer \u001B[39m\u001B[38;5;132;01m{\u001B[39;00mtoken\u001B[38;5;132;01m}\u001B[39;00m\u001B[38;5;124m\"\u001B[39m}\n\u001B[1;32m 35\u001B[0m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mbase_url \u001B[38;5;241m=\u001B[39m \u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mhttps://api.osf.io/v2/\u001B[39m\u001B[38;5;124m\"\u001B[39m\n\u001B[0;32m---> 37\u001B[0m \u001B[43mrequests\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mget\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mbase_url\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mheaders\u001B[49m\u001B[38;5;241;43m=\u001B[39;49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mheaders\u001B[49m\u001B[43m)\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mraise_for_status\u001B[49m\u001B[43m(\u001B[49m\u001B[43m)\u001B[49m\n\u001B[1;32m 39\u001B[0m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mdata_node_guid \u001B[38;5;241m=\u001B[39m data_node_guid\n", + "File \u001B[0;32m~/ucsb/projects/compSPI/ioSPI/venv_iospi/lib/python3.8/site-packages/requests/models.py:960\u001B[0m, in \u001B[0;36mResponse.raise_for_status\u001B[0;34m(self)\u001B[0m\n\u001B[1;32m 957\u001B[0m http_error_msg \u001B[38;5;241m=\u001B[39m \u001B[38;5;124mu\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;132;01m%s\u001B[39;00m\u001B[38;5;124m Server Error: \u001B[39m\u001B[38;5;132;01m%s\u001B[39;00m\u001B[38;5;124m for url: \u001B[39m\u001B[38;5;132;01m%s\u001B[39;00m\u001B[38;5;124m'\u001B[39m \u001B[38;5;241m%\u001B[39m (\u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mstatus_code, reason, \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39murl)\n\u001B[1;32m 959\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m http_error_msg:\n\u001B[0;32m--> 960\u001B[0m \u001B[38;5;28;01mraise\u001B[39;00m HTTPError(http_error_msg, response\u001B[38;5;241m=\u001B[39m\u001B[38;5;28mself\u001B[39m)\n", + "\u001B[0;31mHTTPError\u001B[0m: 401 Client Error: Unauthorized for url: https://api.osf.io/v2/" + ] + } + ], + "source": [ + "token = 's9XHmPaLzRoJnNFSLngb2pH6KPtmCOxEgdJitRiQuRROq7omjX8SKyBIgfZYTiC1X0154o'\n", + "\n", + "guid = '8e5j9'\n", + "\n", + "osf = datasets.OSFUpload(token = token, data_node_guid= guid)\n", + "print(osf.headers)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "Now we will create a child node inside the parent node, for representing the dataset with 80s ribosome data. We will use the pdb id as the name for this new node, and the function will return its GUID. Since the child is also a node, it can accessed separately from the parent node." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "8yg32\n" + ] + } + ], + "source": [ + "pdb_id = '4v6x'\n", + "child_guid = osf.write_child_node(parent_guid = guid, title = pdb_id)\n", + "print(child_guid)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "Having created a child node for putting the files related to the 80s ribosome, we can upload files to it by using the function `write_files`." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Uploaded c:\\Users\\Luis\\Documents\\GitHub\\ioSPI\\notebooks\\4v6x.pdb \n" + ] + }, + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#Create a list of the files to be uploaded\n", + "file_paths = [os.path.join(os.getcwd(), 'data', pdb_id + '.pdb')]\n", + "\n", + "#Write files at the child node\n", + "osf.write_files(dataset_guid = child_guid, file_paths = file_paths)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "We can now check if it was uploaded correctly by checking if the function `read_structure_guid` will return a GUID corresponding to the pdb id passed as parameter." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "'8yg32'" + ] + }, + "execution_count": 17, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "osf.read_structure_guid(pdb_id)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.12" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} \ No newline at end of file