TK9_Reprocessamento_de_DOIs.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Analize e processamento do TK #9 do `scieloorg/doi_request`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Para mais informações sobre o problema e motivações olhar o link: \n",
    "https://github.com/scieloorg/doi_request/issues/9"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Arquivo CSV gerado com os dados dos documentos da coleção SciELO Brasil que não possuem registro no Crossref."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>collection</th>\n",
       "      <th>issn_scielo</th>\n",
       "      <th>journal</th>\n",
       "      <th>title</th>\n",
       "      <th>doc_type</th>\n",
       "      <th>pid</th>\n",
       "      <th>doi</th>\n",
       "      <th>doi_prefix</th>\n",
       "      <th>version</th>\n",
       "      <th>pub_year</th>\n",
       "      <th>has_doi</th>\n",
       "      <th>no_doi</th>\n",
       "      <th>doi_found</th>\n",
       "      <th>doi_not_found</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>scl</td>\n",
       "      <td>0044-5967</td>\n",
       "      <td>Acta Amazonica</td>\n",
       "      <td>O uso do solo na Amazônia</td>\n",
       "      <td>editorial</td>\n",
       "      <td>S0044-59671973000100003</td>\n",
       "      <td>10.1590/1809-43921973031003</td>\n",
       "      <td>10.159</td>\n",
       "      <td>xml</td>\n",
       "      <td>1973</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>scl</td>\n",
       "      <td>0044-5967</td>\n",
       "      <td>Acta Amazonica</td>\n",
       "      <td>An evolutionary and ecological perspective of ...</td>\n",
       "      <td>undefined</td>\n",
       "      <td>S0044-59671973000100005</td>\n",
       "      <td>10.1590/1809-43921973031005</td>\n",
       "      <td>10.159</td>\n",
       "      <td>xml</td>\n",
       "      <td>1973</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>scl</td>\n",
       "      <td>0044-5967</td>\n",
       "      <td>Acta Amazonica</td>\n",
       "      <td>Anatomia de Anacardium spruceanum Bth, Ex Engl...</td>\n",
       "      <td>undefined</td>\n",
       "      <td>S0044-59671973000100039</td>\n",
       "      <td>10.1590/1809-43921973031039</td>\n",
       "      <td>10.159</td>\n",
       "      <td>xml</td>\n",
       "      <td>1973</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>scl</td>\n",
       "      <td>0044-5967</td>\n",
       "      <td>Acta Amazonica</td>\n",
       "      <td>The effect of slash and burn agriculture on pl...</td>\n",
       "      <td>undefined</td>\n",
       "      <td>S0044-59671973000100055</td>\n",
       "      <td>10.1590/1809-43921973031055</td>\n",
       "      <td>10.159</td>\n",
       "      <td>xml</td>\n",
       "      <td>1973</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>scl</td>\n",
       "      <td>0044-5967</td>\n",
       "      <td>Acta Amazonica</td>\n",
       "      <td>The chemical composition of Amazonian plants ()</td>\n",
       "      <td>undefined</td>\n",
       "      <td>S0044-59671973000100063</td>\n",
       "      <td>10.1590/1809-43921973031063</td>\n",
       "      <td>10.159</td>\n",
       "      <td>xml</td>\n",
       "      <td>1973</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  collection issn_scielo         journal  \\\n",
       "0        scl   0044-5967  Acta Amazonica   \n",
       "1        scl   0044-5967  Acta Amazonica   \n",
       "2        scl   0044-5967  Acta Amazonica   \n",
       "3        scl   0044-5967  Acta Amazonica   \n",
       "4        scl   0044-5967  Acta Amazonica   \n",
       "\n",
       "                                               title   doc_type  \\\n",
       "0                          O uso do solo na Amazônia  editorial   \n",
       "1  An evolutionary and ecological perspective of ...  undefined   \n",
       "2  Anatomia de Anacardium spruceanum Bth, Ex Engl...  undefined   \n",
       "3  The effect of slash and burn agriculture on pl...  undefined   \n",
       "4    The chemical composition of Amazonian plants ()  undefined   \n",
       "\n",
       "                       pid                          doi  doi_prefix version  \\\n",
       "0  S0044-59671973000100003  10.1590/1809-43921973031003      10.159     xml   \n",
       "1  S0044-59671973000100005  10.1590/1809-43921973031005      10.159     xml   \n",
       "2  S0044-59671973000100039  10.1590/1809-43921973031039      10.159     xml   \n",
       "3  S0044-59671973000100055  10.1590/1809-43921973031055      10.159     xml   \n",
       "4  S0044-59671973000100063  10.1590/1809-43921973031063      10.159     xml   \n",
       "\n",
       "   pub_year  has_doi  no_doi  doi_found  doi_not_found  \n",
       "0      1973        1       0          0              1  \n",
       "1      1973        1       0          0              1  \n",
       "2      1973        1       0          0              1  \n",
       "3      1973        1       0          0              1  \n",
       "4      1973        1       0          0              1  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "doi_files = pd.read_csv('./SciELO_Brazil_DOI.csv', delimiter=\";\", low_memory=False)\n",
    "doi_files.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1626, 14)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "doi_files.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Testar se o doi ainda não esta registrado no `https://www.doi.org`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Para testar se os dois ainda estao indisponiveis, foi construido o script `generate_doi_not_found.py` que consulta toda a lista dos dois no proprio site do `www.doi.org`\n",
    "\n",
    "Esse script deve ser executado num virtualenv de `python>=3.7`e que tenha cido instalados as dependencias atraves do compando `pip install -r requirements.generate_doi_not_found.txt`\n",
    "\n",
    "```shell\n",
    "$ python generate_doi_not_found.py\n",
    "```\n",
    "\n",
    "Ao termido desse processamento teremos o arquivo `df_doi_not_found.csv` e podemos continuar a analize.\n",
    "Nessa etapa devemos pesquizar no *SciELO - DOI Manager* se esse PID ja foi processados ou não, para isso foi construido o script `consult_doi_request_and_extract_data.py` que o ira consultar o banco de dados da aplicação e constroi os arquivos `df_doi_not_processed.csv`, `df_doi_processed.csv` com os dados de dois processado e nao processado. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```shell\n",
    "$ python consult_doi_request_and_extract_data.py \\\n",
    "    --host HOSTDB \\\n",
    "    --port 5432 \\\n",
    "    --user doi_user \n",
    "    --password XXXXXXX \n",
    "    --database doi_manager\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Carregando dados dos arquivos gerados para facilitar processo\n",
    "df_doi_not_processed = pd.read_csv(\"./df_doi_not_processed.csv\")\n",
    "df_doi_processed = pd.read_csv(\"./df_doi_processed.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DOIs nunca processados 102\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>collection</th>\n",
       "      <th>issn_scielo</th>\n",
       "      <th>journal</th>\n",
       "      <th>title</th>\n",
       "      <th>doc_type</th>\n",
       "      <th>pid</th>\n",
       "      <th>doi</th>\n",
       "      <th>doi_prefix</th>\n",
       "      <th>version</th>\n",
       "      <th>pub_year</th>\n",
       "      <th>has_doi</th>\n",
       "      <th>no_doi</th>\n",
       "      <th>doi_found</th>\n",
       "      <th>doi_not_found</th>\n",
       "      <th>request</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>scl</td>\n",
       "      <td>0044-5967</td>\n",
       "      <td>Acta Amazonica</td>\n",
       "      <td>O uso do solo na Amazônia</td>\n",
       "      <td>editorial</td>\n",
       "      <td>S0044-59671973000100003</td>\n",
       "      <td>10.1590/1809-43921973031003</td>\n",
       "      <td>10.159</td>\n",
       "      <td>xml</td>\n",
       "      <td>1973</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>404 Client Error: Not Found for url: https://w...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>scl</td>\n",
       "      <td>0044-5967</td>\n",
       "      <td>Acta Amazonica</td>\n",
       "      <td>An evolutionary and ecological perspective of ...</td>\n",
       "      <td>undefined</td>\n",
       "      <td>S0044-59671973000100005</td>\n",
       "      <td>10.1590/1809-43921973031005</td>\n",
       "      <td>10.159</td>\n",
       "      <td>xml</td>\n",
       "      <td>1973</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>404 Client Error: Not Found for url: https://w...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>scl</td>\n",
       "      <td>0044-5967</td>\n",
       "      <td>Acta Amazonica</td>\n",
       "      <td>Anatomia de Anacardium spruceanum Bth, Ex Engl...</td>\n",
       "      <td>undefined</td>\n",
       "      <td>S0044-59671973000100039</td>\n",
       "      <td>10.1590/1809-43921973031039</td>\n",
       "      <td>10.159</td>\n",
       "      <td>xml</td>\n",
       "      <td>1973</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>404 Client Error: Not Found for url: https://w...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>scl</td>\n",
       "      <td>0044-5967</td>\n",
       "      <td>Acta Amazonica</td>\n",
       "      <td>The effect of slash and burn agriculture on pl...</td>\n",
       "      <td>undefined</td>\n",
       "      <td>S0044-59671973000100055</td>\n",
       "      <td>10.1590/1809-43921973031055</td>\n",
       "      <td>10.159</td>\n",
       "      <td>xml</td>\n",
       "      <td>1973</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>404 Client Error: Not Found for url: https://w...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>scl</td>\n",
       "      <td>0044-5967</td>\n",
       "      <td>Acta Amazonica</td>\n",
       "      <td>The chemical composition of Amazonian plants ()</td>\n",
       "      <td>undefined</td>\n",
       "      <td>S0044-59671973000100063</td>\n",
       "      <td>10.1590/1809-43921973031063</td>\n",
       "      <td>10.159</td>\n",
       "      <td>xml</td>\n",
       "      <td>1973</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>404 Client Error: Not Found for url: https://w...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 collection issn_scielo         journal  \\\n",
       "0           0             0        scl   0044-5967  Acta Amazonica   \n",
       "1           1             1        scl   0044-5967  Acta Amazonica   \n",
       "2           2             2        scl   0044-5967  Acta Amazonica   \n",
       "3           3             3        scl   0044-5967  Acta Amazonica   \n",
       "4           4             4        scl   0044-5967  Acta Amazonica   \n",
       "\n",
       "                                               title   doc_type  \\\n",
       "0                          O uso do solo na Amazônia  editorial   \n",
       "1  An evolutionary and ecological perspective of ...  undefined   \n",
       "2  Anatomia de Anacardium spruceanum Bth, Ex Engl...  undefined   \n",
       "3  The effect of slash and burn agriculture on pl...  undefined   \n",
       "4    The chemical composition of Amazonian plants ()  undefined   \n",
       "\n",
       "                       pid                          doi  doi_prefix version  \\\n",
       "0  S0044-59671973000100003  10.1590/1809-43921973031003      10.159     xml   \n",
       "1  S0044-59671973000100005  10.1590/1809-43921973031005      10.159     xml   \n",
       "2  S0044-59671973000100039  10.1590/1809-43921973031039      10.159     xml   \n",
       "3  S0044-59671973000100055  10.1590/1809-43921973031055      10.159     xml   \n",
       "4  S0044-59671973000100063  10.1590/1809-43921973031063      10.159     xml   \n",
       "\n",
       "   pub_year  has_doi  no_doi  doi_found  doi_not_found  \\\n",
       "0      1973        1       0          0              1   \n",
       "1      1973        1       0          0              1   \n",
       "2      1973        1       0          0              1   \n",
       "3      1973        1       0          0              1   \n",
       "4      1973        1       0          0              1   \n",
       "\n",
       "                                             request  \n",
       "0  404 Client Error: Not Found for url: https://w...  \n",
       "1  404 Client Error: Not Found for url: https://w...  \n",
       "2  404 Client Error: Not Found for url: https://w...  \n",
       "3  404 Client Error: Not Found for url: https://w...  \n",
       "4  404 Client Error: Not Found for url: https://w...  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(\"DOIs nunca processados\", len(df_doi_not_processed))\n",
    "df_doi_not_processed.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DOIs ja processados 1525\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>journal</th>\n",
       "      <th>pid</th>\n",
       "      <th>doi</th>\n",
       "      <th>submission_status</th>\n",
       "      <th>feedback_status</th>\n",
       "      <th>feedback_xml</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>Anais da Academia Brasileira de Ciências</td>\n",
       "      <td>S0001-37652019000400516</td>\n",
       "      <td>10.1590/0001-3765201920180614</td>\n",
       "      <td>success</td>\n",
       "      <td>success</td>\n",
       "      <td>&lt;doi_batch_diagnostic status=\"completed\" sp=\"d...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Anais da Academia Brasileira de Ciências</td>\n",
       "      <td>S0001-37652019000600402</td>\n",
       "      <td>10.1590/0001-3765201920190208</td>\n",
       "      <td>success</td>\n",
       "      <td>success</td>\n",
       "      <td>&lt;doi_batch_diagnostic status=\"completed\" sp=\"d...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>Anais da Academia Brasileira de Ciências</td>\n",
       "      <td>S0001-37652019000600611</td>\n",
       "      <td>10.1590/0001-3765201920190218</td>\n",
       "      <td>success</td>\n",
       "      <td>success</td>\n",
       "      <td>&lt;doi_batch_diagnostic status=\"completed\" sp=\"d...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>Bragantia</td>\n",
       "      <td>S0006-87052019005013101</td>\n",
       "      <td>10.1590/1678-4499.20180178</td>\n",
       "      <td>success</td>\n",
       "      <td>success</td>\n",
       "      <td>&lt;doi_batch_diagnostic status=\"completed\" sp=\"d...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>Bragantia</td>\n",
       "      <td>S0006-87052019005013102</td>\n",
       "      <td>10.1590/1678-4499.20180251</td>\n",
       "      <td>success</td>\n",
       "      <td>success</td>\n",
       "      <td>&lt;doi_batch_diagnostic status=\"completed\" sp=\"d...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0                                   journal  \\\n",
       "0           0  Anais da Academia Brasileira de Ciências   \n",
       "1           1  Anais da Academia Brasileira de Ciências   \n",
       "2           2  Anais da Academia Brasileira de Ciências   \n",
       "3           3                                 Bragantia   \n",
       "4           4                                 Bragantia   \n",
       "\n",
       "                       pid                            doi submission_status  \\\n",
       "0  S0001-37652019000400516  10.1590/0001-3765201920180614           success   \n",
       "1  S0001-37652019000600402  10.1590/0001-3765201920190208           success   \n",
       "2  S0001-37652019000600611  10.1590/0001-3765201920190218           success   \n",
       "3  S0006-87052019005013101     10.1590/1678-4499.20180178           success   \n",
       "4  S0006-87052019005013102     10.1590/1678-4499.20180251           success   \n",
       "\n",
       "  feedback_status                                       feedback_xml  \n",
       "0         success  <doi_batch_diagnostic status=\"completed\" sp=\"d...  \n",
       "1         success  <doi_batch_diagnostic status=\"completed\" sp=\"d...  \n",
       "2         success  <doi_batch_diagnostic status=\"completed\" sp=\"d...  \n",
       "3         success  <doi_batch_diagnostic status=\"completed\" sp=\"d...  \n",
       "4         success  <doi_batch_diagnostic status=\"completed\" sp=\"d...  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(\"DOIs ja processados\", len(df_doi_processed))\n",
    "df_doi_processed.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Analizando erros do DOI não pocessados"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Esses casos nessecitam ser Processados adicionados pelo SciELO - DOI Manager"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "S0044-59671973000100003\n",
      "S0044-59671973000100005\n",
      "S0044-59671973000100039\n",
      "S0044-59671973000100055\n",
      "S0044-59671973000100063\n",
      "S0044-59671973000100065\n",
      "S0044-59671973000100071\n",
      "S0044-59671973000200003\n",
      "S0044-59671973000200005\n",
      "S0044-59671973000200007\n",
      "S0044-59671973000200017\n",
      "S0044-59671973000200033\n",
      "S0044-59671973000200041\n",
      "S0044-59671973000200043\n",
      "S0044-59671973000200047\n",
      "S0044-59671973000300003\n",
      "S0044-59671973000300005\n",
      "S0044-59671973000300029\n",
      "S0044-59671973000300041\n",
      "S0044-59671973000300043\n",
      "S0044-59671973000300045\n",
      "S0044-59671973000300051\n",
      "S0044-59671973000300053\n",
      "S0044-59671973000300059\n",
      "S0044-59671978000400523\n",
      "S0044-59671978000400543\n",
      "S0044-59671978000400545\n",
      "S0044-59671978000400549\n",
      "S0044-59671978000400557\n",
      "S0044-59671978000400561\n",
      "S0044-59671978000400577\n",
      "S0044-59671978000400591\n",
      "S0044-59671978000400595\n",
      "S0044-59671978000400601\n",
      "S0044-59671978000400605\n",
      "S0044-59671978000400613\n",
      "S0044-59671978000400621\n",
      "S0044-59671978000400629\n",
      "S0044-59671978000400639\n",
      "S0044-59671978000400657\n",
      "S0044-59671978000400679\n",
      "S0044-59671978000400687\n",
      "S0044-59671978000400689\n",
      "S0044-59671978000400699\n",
      "S0044-59671978000400702\n",
      "S0044-59671978000400705\n",
      "S0044-59671978000800005\n",
      "S0044-59671978000900005\n",
      "S0044-59671978001000005\n",
      "S0080-62341987000400003\n",
      "S0080-62341987000400010\n",
      "S0080-62341987000400013\n",
      "S0080-62341987000400021\n",
      "S0080-62341987000400029\n",
      "S0080-62341987000400034\n",
      "S0080-62341987000400040\n",
      "S0080-62341987000400055\n",
      "S0080-62341987000400062\n",
      "S0080-62341987000400068\n",
      "S0080-62341987000400077\n",
      "S0080-62341987000400088\n",
      "S0080-62341987000400100\n",
      "S2238-38752011000200009\n",
      "S2238-38752011000200035\n",
      "S2238-38752011000200053\n",
      "S2238-38752011000200077\n",
      "S2238-38752011000200091\n",
      "S2238-38752011000200119\n",
      "S2238-38752011000200147\n",
      "S2238-38752011000200165\n",
      "S2238-38752011000200195\n",
      "S2238-38752011000200219\n",
      "S2238-38752011000200237\n",
      "S2237-101X2012000100202\n",
      "S2238-38752012000300011\n",
      "S2238-38752012000300021\n",
      "S2238-38752012000300043\n",
      "S2238-38752012000300069\n",
      "S2238-38752012000300103\n",
      "S2238-38752012000300135\n",
      "S2238-38752012000300161\n",
      "S2238-38752012000300183\n",
      "S2238-38752012000300203\n",
      "S2238-38752012000300237\n",
      "S2238-38752012000300269\n",
      "S2238-38752012000300279\n",
      "S2238-38752012000300309\n",
      "S1413-80502014000400001\n",
      "S1516-05722014000400008\n",
      "S1516-05722014000400009\n",
      "S1516-05722014000700020\n",
      "S1981-81222014000200004\n",
      "S1413-85572015000100105\n",
      "S1413-85572015000100173\n",
      "S1414-98932015000100248\n",
      "S1414-98932015000100249\n",
      "S0001-37652018000602991\n",
      "S0001-37652018000603105\n",
      "S0103-84782018001000300\n",
      "S0103-84782018001000400\n",
      "S0103-84782018001000601\n",
      "S0103-84782018001000650\n"
     ]
    }
   ],
   "source": [
    "# Lista dos PIDs para serem Processados\n",
    "print(\"\\n\".join(df_doi_not_processed.pid.to_list()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Analizando erros do DOI ja processados"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Foi extraido os dados de `Situação de depósito` e `XML de resultado do depósito` diretamente do banco de dados do PostgreSQL atraves do script utilizado anteriormente."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "status: error \tTotal 8\n",
      "status: failure \tTotal 1411\n",
      "status: notapplicable \tTotal 7\n",
      "status: semValor \tTotal 18\n",
      "status: success \tTotal 67\n",
      "status: waiting \tTotal 14\n"
     ]
    }
   ],
   "source": [
    "# tratando cada caso de retorno\n",
    "gb = df_doi_processed.groupby(\"feedback_status\")\n",
    "data_errors = {\n",
    "    x: gb.get_group(x) for x in gb.groups\n",
    "}\n",
    "\n",
    "for k,v in data_errors.items():\n",
    "    print(\"status:\", k, \"\\tTotal\", v.journal.count())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Caso de  statos com `failure`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total de item com failure 1411\n"
     ]
    }
   ],
   "source": [
    "status_failure = data_errors[\"failure\"]\n",
    "print(\"Total de item com failure\", status_failure.journal.count())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>size</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>msg_feedback</th>\n",
       "      <th>journal</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>ISSN \"01007203\" has already been assigned, issn (01007203) is assigned to another title (Revista Brasileira de Ginecologia e Obstetr&amp;#258;&amp;#173;cia / RBGO Gynecology and Obstetrics)</th>\n",
       "      <th>Revista Brasileira de Ginecologia e Obstetrícia</th>\n",
       "      <td>145</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"16783166\" has already been assigned, title/issn: Scientiae Studia/16783166 is owned by publisher: 10.11606</th>\n",
       "      <th>Scientiae Studia</th>\n",
       "      <td>105</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"01032070\" has already been assigned, title/issn: Tempo Social/01032070 is owned by publisher: 10.11606</th>\n",
       "      <th>Tempo Social</th>\n",
       "      <td>89</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"00217557\" has already been assigned, title/issn: Jornal de Pediatria/00217557 is owned by publisher: 10.1016</th>\n",
       "      <th>Jornal de Pediatria</th>\n",
       "      <td>77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"19805098\" has already been assigned, title/issn: Ci&amp;#258;&amp;#350;ncia Florestal/19805098 is owned by publisher: 10.5902</th>\n",
       "      <th>Ciência Florestal</th>\n",
       "      <td>75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"19825676\" has already been assigned, title/issn: Tropical Plant Pathology/19825676 is owned by publisher: 10.1007</th>\n",
       "      <th>Tropical Plant Pathology</th>\n",
       "      <td>68</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"18071775\" has already been assigned, issn (18071775) is assigned to another title (Journal of Information Systems and Technology Management)</th>\n",
       "      <th>JISTEM - Journal of Information Systems and Technology Management</th>\n",
       "      <td>66</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"18075932\" has already been assigned, title/issn: Clinics/18075932 is owned by publisher: 10.6061</th>\n",
       "      <th>Clinics</th>\n",
       "      <td>60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"0066782X\" has already been assigned, title/issn: Arquivos Brasileiros de Cardiologia/0066782X is owned by publisher: 10.5935</th>\n",
       "      <th>Arquivos Brasileiros de Cardiologia</th>\n",
       "      <td>59</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"01013289\" has already been assigned, title/issn: Revista Brasileira de Ci&amp;#258;&amp;#350;ncias do Esporte/01013289 is owned by publisher: 10.1016</th>\n",
       "      <th>Revista Brasileira de Ciências do Esporte</th>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"18078621\" has already been assigned, title/issn: Acta Scientiarum Agronomy/18078621 is owned by publisher: 10.4025</th>\n",
       "      <th>Acta Scientiarum. Agronomy</th>\n",
       "      <td>46</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"19835175\" has already been assigned, title/issn: Revista Brasileira de Cirurgia Pl&amp;#3619;&amp;#3585;stica/19835175 is owned by publisher: 10.5935</th>\n",
       "      <th>Revista Brasileira de Cirurgia Plástica</th>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Deposit contains title error: ISSN: [01008404] assigned to another title, please contact Crossref support.</th>\n",
       "      <th>Brazilian Journal of Botany</th>\n",
       "      <td>37</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"14138050\" has already been assigned, title/issn: Economia Aplicada/14138050 is owned by publisher: 10.11606</th>\n",
       "      <th>Economia Aplicada</th>\n",
       "      <td>37</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"00347140\" has already been assigned, title/issn: Revista Brasileira de Economia/00347140 is owned by publisher: 10.5935</th>\n",
       "      <th>Revista Brasileira de Economia</th>\n",
       "      <td>37</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"1519566X\" has already been assigned, title/issn: Neotropical Entomology/1519566X is owned by publisher: 10.1007</th>\n",
       "      <th>Neotropical Entomology</th>\n",
       "      <td>36</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"01027972\" has already been assigned, title/issn: Psicologia Reflex&amp;#258;&amp;#321;o e Cr&amp;#258;&amp;#173;tica/01027972 is owned by publisher: 10.1186</th>\n",
       "      <th>Psicologia: Reflexão e Crítica</th>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"16785878\" has already been assigned, title/issn: Journal of the Brazilian Society of Mechanical Sciences and Engineering/16785878 is owned by publisher: 10.1007</th>\n",
       "      <th>Journal of the Brazilian Society of Mechanical Sciences and Engineering</th>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"01004042\" has already been assigned, title/issn: Qu&amp;#258;&amp;#173;mica Nova/01004042 is owned by publisher: 10.21577</th>\n",
       "      <th>Química Nova</th>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"19806523\" has already been assigned, title/issn: Revista Odonto Ci&amp;#258;&amp;#350;ncia/19806523 is owned by publisher: 10.15448</th>\n",
       "      <th>Revista Odonto Ciência</th>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"19828837\" has already been assigned, title/issn: Pandaemonium Germanicum/19828837 is owned by publisher: 10.11606</th>\n",
       "      <th>Pandaemonium Germanicum</th>\n",
       "      <td>27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"18070302\" has already been assigned, title/issn: Computational &amp;amp; Applied Mathematics/18070302 is owned by publisher: 10.1007</th>\n",
       "      <th>Computational &amp; Applied Mathematics</th>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"16789199\" has already been assigned, title/issn: Journal of Venomous Animals and Toxins including Tropical Diseases/16789199 is owned by publisher: 10.1186</th>\n",
       "      <th>Journal of Venomous Animals and Toxins including Tropical Diseases</th>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"16785320\" has already been assigned, title/issn: ARS (S&amp;#258;&amp;#321;o Paulo)/16785320 is owned by publisher: 10.11606</th>\n",
       "      <th>ARS (São Paulo)</th>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"00802107\" has already been assigned, issn (00802107) is assigned to another title (Revista de Administra&amp;#258;&amp;#167;&amp;#258;&amp;#321;o)</th>\n",
       "      <th>Revista de Administração (São Paulo)</th>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"21798451\" has already been assigned, title/issn: TEMA (S&amp;#258;&amp;#321;o Carlos)/21798451 is owned by publisher: 10.5540</th>\n",
       "      <th>TEMA (São Carlos)</th>\n",
       "      <td>21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"01039733\" has already been assigned, title/issn: Brazilian Journal of Physics/01039733 is owned by publisher: 10.1007</th>\n",
       "      <th>Brazilian Journal of Physics</th>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"18094872\" has already been assigned, title/issn: Arquivos Internacionais de Otorrinolaringologia/18094872 is owned by publisher: 10.1055</th>\n",
       "      <th>Arquivos Internacionais de Otorrinolaringologia</th>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"01035053\" has already been assigned, title/issn: Journal of the Brazilian Chemical Society/01035053 is owned by publisher: 10.21577</th>\n",
       "      <th>Journal of the Brazilian Chemical Society</th>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"15169332\" has already been assigned, issn (15169332) is assigned to another title (Revista Brasileira de Ci?ncias Farmac?uticas)</th>\n",
       "      <th>Revista Brasileira de Ciências Farmacêuticas</th>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"0102261X\" has already been assigned, title/issn: Revista Brasileira de Geof&amp;#258;&amp;#173;sica/0102261X is owned by publisher: 10.22564</th>\n",
       "      <th>Revista Brasileira de Geofísica</th>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"15187012\" has already been assigned, title/issn: Intera&amp;#258;&amp;#167;&amp;#258;&amp;#318;es (Campo Grande)/15187012 is owned by publisher: 10.20435</th>\n",
       "      <th>Interações (Campo Grande)</th>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"18066690\" has already been assigned, title/issn: Revista Ci&amp;#258;&amp;#350;ncia Agron&amp;#258;&amp;#180;mica/18066690 is owned by publisher: 10.5935</th>\n",
       "      <th>Revista Ciência Agronômica</th>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"01023098\" has already been assigned, title/issn: Revista Brasileira de Estudos de Popula&amp;#258;&amp;#167;&amp;#258;&amp;#321;o/01023098 is owned by publisher: 10.20947</th>\n",
       "      <th>Revista Brasileira de Estudos de População</th>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"01026909\" has already been assigned, title/issn: Revista Brasileira de Ci&amp;#258;&amp;#350;ncias Sociais/01026909 is owned by publisher: 10.17666</th>\n",
       "      <th>Revista Brasileira de Ciências Sociais</th>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"19833288\" has already been assigned, title/issn: Psychology &amp;amp; Neuroscience/19833288 is owned by publisher: 10.1037</th>\n",
       "      <th>Psychology &amp; Neuroscience</th>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"14133555\" has already been assigned, title/issn: Brazilian Journal of Physical Therapy/14133555 is owned by publisher: 10.1016</th>\n",
       "      <th>Brazilian Journal of Physical Therapy</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"16784170\" has already been assigned, title/issn: Arquivos Brasileiros de Cardiologia/16784170 is owned by publisher: 10.5935</th>\n",
       "      <th>Arquivos Brasileiros de Cardiologia</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"01027638\" has already been assigned, title/issn: Brazilian Journal of Cardiovascular Surgery/01027638 is owned by publisher: 10.21470</th>\n",
       "      <th>Brazilian Journal of Cardiovascular Surgery</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"21798397\" has already been assigned, title/issn: Revista Brasileira de Cardiologia Invasiva/21798397 is owned by publisher: 10.1016</th>\n",
       "      <th>Revista Brasileira de Cardiologia Invasiva</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"23180331\" has already been assigned, title/issn: RBRH/23180331 is owned by publisher: 10.21168</th>\n",
       "      <th>RBRH</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"18078672\" has already been assigned, title/issn: Acta Scientiarum Animal Sciences/18078672 is owned by publisher: 10.4025</th>\n",
       "      <th>Acta Scientiarum. Animal Sciences</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"01007203\" has already been assigned, issn (01007203) is assigned to another title (Revista Brasileira de Ginecologia e Obstetr&amp;#195;&amp;#173;cia / RBGO Gynecology and Obstetrics)</th>\n",
       "      <th>Revista Brasileira de Ginecologia e Obstetrícia</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"19833083\" has already been assigned, title/issn: Revista da Educa&amp;#231;&amp;#227;o F&amp;#237;sica / UEM/19833083 is owned by publisher: 10.4025</th>\n",
       "      <th>Revista da Educação Física / UEM</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"14138670\" has already been assigned, title/issn: Brazilian Journal of Infectious Diseases/14138670 is owned by publisher: 10.1016</th>\n",
       "      <th>Brazilian Journal of Infectious Diseases</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"19844670\" has already been assigned, title/issn: Zoologia (Curitiba)/19844670 is owned by publisher: 10.3897</th>\n",
       "      <th>Zoologia (Curitiba)</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISSN \"14137372\" has already been assigned, title/issn: Psicologia em Estudo/14137372 is owned by publisher: 10.4025</th>\n",
       "      <th>Psicologia em Estudo</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                                       size\n",
       "msg_feedback                                       journal                                                 \n",
       "ISSN \"01007203\" has already been assigned, issn... Revista Brasileira de Ginecologia e Obstetrícia      145\n",
       "ISSN \"16783166\" has already been assigned, titl... Scientiae Studia                                     105\n",
       "ISSN \"01032070\" has already been assigned, titl... Tempo Social                                          89\n",
       "ISSN \"00217557\" has already been assigned, titl... Jornal de Pediatria                                   77\n",
       "ISSN \"19805098\" has already been assigned, titl... Ciência Florestal                                     75\n",
       "ISSN \"19825676\" has already been assigned, titl... Tropical Plant Pathology                              68\n",
       "ISSN \"18071775\" has already been assigned, issn... JISTEM - Journal of Information Systems and Tec...    66\n",
       "ISSN \"18075932\" has already been assigned, titl... Clinics                                               60\n",
       "ISSN \"0066782X\" has already been assigned, titl... Arquivos Brasileiros de Cardiologia                   59\n",
       "ISSN \"01013289\" has already been assigned, titl... Revista Brasileira de Ciências do Esporte             50\n",
       "ISSN \"18078621\" has already been assigned, titl... Acta Scientiarum. Agronomy                            46\n",
       "ISSN \"19835175\" has already been assigned, titl... Revista Brasileira de Cirurgia Plástica               40\n",
       "Deposit contains title error: ISSN: [01008404] ... Brazilian Journal of Botany                           37\n",
       "ISSN \"14138050\" has already been assigned, titl... Economia Aplicada                                     37\n",
       "ISSN \"00347140\" has already been assigned, titl... Revista Brasileira de Economia                        37\n",
       "ISSN \"1519566X\" has already been assigned, titl... Neotropical Entomology                                36\n",
       "ISSN \"01027972\" has already been assigned, titl... Psicologia: Reflexão e Crítica                        35\n",
       "ISSN \"16785878\" has already been assigned, titl... Journal of the Brazilian Society of Mechanical ...    35\n",
       "ISSN \"01004042\" has already been assigned, titl... Química Nova                                          32\n",
       "ISSN \"19806523\" has already been assigned, titl... Revista Odonto Ciência                                29\n",
       "ISSN \"19828837\" has already been assigned, titl... Pandaemonium Germanicum                               27\n",
       "ISSN \"18070302\" has already been assigned, titl... Computational & Applied Mathematics                   26\n",
       "ISSN \"16789199\" has already been assigned, titl... Journal of Venomous Animals and Toxins includin...    26\n",
       "ISSN \"16785320\" has already been assigned, titl... ARS (São Paulo)                                       22\n",
       "ISSN \"00802107\" has already been assigned, issn... Revista de Administração (São Paulo)                  22\n",
       "ISSN \"21798451\" has already been assigned, titl... TEMA (São Carlos)                                     21\n",
       "ISSN \"01039733\" has already been assigned, titl... Brazilian Journal of Physics                          12\n",
       "ISSN \"18094872\" has already been assigned, titl... Arquivos Internacionais de Otorrinolaringologia       11\n",
       "ISSN \"01035053\" has already been assigned, titl... Journal of the Brazilian Chemical Society             11\n",
       "ISSN \"15169332\" has already been assigned, issn... Revista Brasileira de Ciências Farmacêuticas           9\n",
       "ISSN \"0102261X\" has already been assigned, titl... Revista Brasileira de Geofísica                        9\n",
       "ISSN \"15187012\" has already been assigned, titl... Interações (Campo Grande)                              8\n",
       "ISSN \"18066690\" has already been assigned, titl... Revista Ciência Agronômica                             8\n",
       "ISSN \"01023098\" has already been assigned, titl... Revista Brasileira de Estudos de População             8\n",
       "ISSN \"01026909\" has already been assigned, titl... Revista Brasileira de Ciências Sociais                 6\n",
       "ISSN \"19833288\" has already been assigned, titl... Psychology & Neuroscience                              5\n",
       "ISSN \"14133555\" has already been assigned, titl... Brazilian Journal of Physical Therapy                  3\n",
       "ISSN \"16784170\" has already been assigned, titl... Arquivos Brasileiros de Cardiologia                    3\n",
       "ISSN \"01027638\" has already been assigned, titl... Brazilian Journal of Cardiovascular Surgery            3\n",
       "ISSN \"21798397\" has already been assigned, titl... Revista Brasileira de Cardiologia Invasiva             3\n",
       "ISSN \"23180331\" has already been assigned, titl... RBRH                                                   3\n",
       "ISSN \"18078672\" has already been assigned, titl... Acta Scientiarum. Animal Sciences                      2\n",
       "ISSN \"01007203\" has already been assigned, issn... Revista Brasileira de Ginecologia e Obstetrícia        1\n",
       "ISSN \"19833083\" has already been assigned, titl... Revista da Educação Física / UEM                       1\n",
       "ISSN \"14138670\" has already been assigned, titl... Brazilian Journal of Infectious Diseases               1\n",
       "ISSN \"19844670\" has already been assigned, titl... Zoologia (Curitiba)                                    1\n",
       "ISSN \"14137372\" has already been assigned, titl... Psicologia em Estudo                                   1"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Regex para tratar os retornos dos xml do CrossRef\n",
    "regex = r\"<msg>(.*)<\\/msg>\"\n",
    "result = []\n",
    "\n",
    "for item in status_failure.itertuples():\n",
    "    group = re.findall(regex, item.feedback_xml, re.MULTILINE)\n",
    "    if group:\n",
    "        msg_feedback = group[0]\n",
    "    else:\n",
    "        msg_feedback = \"\"\n",
    "    result.append(\n",
    "        [item.journal, msg_feedback]\n",
    "    )\n",
    "\n",
    "df_result = pd.DataFrame(result, columns=[\"journal\", \"msg_feedback\"])\n",
    "g_df_result = df_result.groupby([\"msg_feedback\", \"journal\"]).size().to_frame('size')\n",
    "g_df_result.sort_values(by=\"size\", ascending=False)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Caso de  statos com `success`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Esse item deve ser analidados em separado pois aparentemente eles foram processados com sucesso pelo CrossRef, mais seu DOI acusa erro no relatorio"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total de item com success 67\n"
     ]
    }
   ],
   "source": [
    "status_success = data_errors[\"success\"]\n",
    "print(\"Total de item com success\", status_success.journal.count())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>journal</th>\n",
       "      <th>pid</th>\n",
       "      <th>doi</th>\n",
       "      <th>submission_status</th>\n",
       "      <th>feedback_status</th>\n",
       "      <th>feedback_xml</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>Anais da Academia Brasileira de Ciências</td>\n",
       "      <td>S0001-37652019000400516</td>\n",
       "      <td>10.1590/0001-3765201920180614</td>\n",
       "      <td>success</td>\n",
       "      <td>success</td>\n",
       "      <td>&lt;doi_batch_diagnostic status=\"completed\" sp=\"d...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Anais da Academia Brasileira de Ciências</td>\n",
       "      <td>S0001-37652019000600402</td>\n",
       "      <td>10.1590/0001-3765201920190208</td>\n",
       "      <td>success</td>\n",
       "      <td>success</td>\n",
       "      <td>&lt;doi_batch_diagnostic status=\"completed\" sp=\"d...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>Anais da Academia Brasileira de Ciências</td>\n",
       "      <td>S0001-37652019000600611</td>\n",
       "      <td>10.1590/0001-3765201920190218</td>\n",
       "      <td>success</td>\n",
       "      <td>success</td>\n",
       "      <td>&lt;doi_batch_diagnostic status=\"completed\" sp=\"d...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>Bragantia</td>\n",
       "      <td>S0006-87052019005013101</td>\n",
       "      <td>10.1590/1678-4499.20180178</td>\n",
       "      <td>success</td>\n",
       "      <td>success</td>\n",
       "      <td>&lt;doi_batch_diagnostic status=\"completed\" sp=\"d...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>Bragantia</td>\n",
       "      <td>S0006-87052019005013102</td>\n",
       "      <td>10.1590/1678-4499.20180251</td>\n",
       "      <td>success</td>\n",
       "      <td>success</td>\n",
       "      <td>&lt;doi_batch_diagnostic status=\"completed\" sp=\"d...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0                                   journal  \\\n",
       "0           0  Anais da Academia Brasileira de Ciências   \n",
       "1           1  Anais da Academia Brasileira de Ciências   \n",
       "2           2  Anais da Academia Brasileira de Ciências   \n",
       "3           3                                 Bragantia   \n",
       "4           4                                 Bragantia   \n",
       "\n",
       "                       pid                            doi submission_status  \\\n",
       "0  S0001-37652019000400516  10.1590/0001-3765201920180614           success   \n",
       "1  S0001-37652019000600402  10.1590/0001-3765201920190208           success   \n",
       "2  S0001-37652019000600611  10.1590/0001-3765201920190218           success   \n",
       "3  S0006-87052019005013101     10.1590/1678-4499.20180178           success   \n",
       "4  S0006-87052019005013102     10.1590/1678-4499.20180251           success   \n",
       "\n",
       "  feedback_status                                       feedback_xml  \n",
       "0         success  <doi_batch_diagnostic status=\"completed\" sp=\"d...  \n",
       "1         success  <doi_batch_diagnostic status=\"completed\" sp=\"d...  \n",
       "2         success  <doi_batch_diagnostic status=\"completed\" sp=\"d...  \n",
       "3         success  <doi_batch_diagnostic status=\"completed\" sp=\"d...  \n",
       "4         success  <doi_batch_diagnostic status=\"completed\" sp=\"d...  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "status_success.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "import urllib.request\n",
    "exclude_pid = []\n",
    "# for row in status_success.itertuples():\n",
    "#     req = urllib.request.Request(\"https://www.doi.org/{0}\".format(row.doi))\n",
    "#     try: \n",
    "#         urllib.request.urlopen(req)\n",
    "#     except urllib.error.URLError as e:\n",
    "#         print(e.reason)\n",
    "#     else:\n",
    "#         exclude_pid.append(row.pid)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Esses casos os PIDs abaixo deve ser excluidos da lista de erros, pois eles existem e estao validos e registrados"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total de PIDS:  0\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Lista dos PIDs para excluir do relatorio\n",
    "print(\"Total de PIDS: \", len(exclude_pid))\n",
    "\n",
    "print(\"\\n\".join(exclude_pid))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "S0001-37652019000400516\n",
    "S0001-37652019000600402\n",
    "S0001-37652019000600611\n",
    "S0006-87052019005013101\n",
    "S0006-87052019005013102\n",
    "S0006-87052019005013103\n",
    "S0006-87052019005013104\n",
    "S0036-46652019005000219\n",
    "S0036-46652019005000220\n",
    "S0036-46652019005000504\n",
    "S0100-39842019005014101\n",
    "S0100-39842019005014102\n",
    "S0100-39842019005014103\n",
    "S0100-39842019005014104\n",
    "S0101-28002019005023101\n",
    "S0101-73302019000100703\n",
    "S0102-46982019000100901\n",
    "S0104-07072019000600310\n",
    "S0104-92242019000100212\n",
    "S1413-41522019005012101\n",
    "S1413-70542019000100220\n",
    "S1413-70542019000100221\n",
    "S1413-70542019000100222\n",
    "S1413-70542019000100401\n",
    "S1516-14392019000400214\n",
    "S1516-14392019000400215\n",
    "S1516-14392019000400216\n",
    "S1516-31802019005004101\n",
    "S1516-31802019005004102\n",
    "S1519-69842019005009101\n",
    "S1519-69842019005009102\n",
    "S1519-69842019005009103\n",
    "S1807-76922019000200304\n",
    "S1807-76922019000200305\n",
    "S1980-00372019000100317\n",
    "S1980-00372019000100318\n",
    "S1980-00372019000100319\n",
    "S1980-00372019000100320\n",
    "S1980-00372019000100321\n",
    "S1980-00372019000100322\n",
    "S1980-00372019000100323\n",
    "S1980-00372019000100324\n",
    "S1980-00372019000100325\n",
    "S1980-00372019000100326\n",
    "S1980-00372019000100327\n",
    "S1980-00372019000100328\n",
    "S1981-67232019000100442\n",
    "S1981-67232019000100443\n",
    "S1981-77462019000300200\n",
    "S1981-77462019000300508\n",
    "S1981-77462019000300701\n",
    "S2175-78602019000100232\n",
    "S2175-78602019000100233\n",
    "S2175-78602019000100234\n",
    "S2175-78602019000100235\n",
    "S2175-78602019000100236\n",
    "S2175-78602019000100237\n",
    "S2175-78602019000100238\n",
    "S2175-78602019000100239\n",
    "S2175-78602019000100240\n",
    "S2175-78602019000100241\n",
    "S2175-78602019000100601\n",
    "S2175-78602019000100602\n",
    "S2179-80872019000400117\n",
    "S2179-80872019000400401\n",
    "S2179-975X2019000100318\n",
    "S2318-03312019000100402\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Caso de  statos com `error` e `waiting`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total de item com waiting 14\n",
      "Total de item com error 8\n"
     ]
    }
   ],
   "source": [
    "status_error = data_errors[\"error\"]\n",
    "status_waiting = data_errors[\"waiting\"]\n",
    "\n",
    "print(\"Total de item com waiting\", status_waiting.journal.count())\n",
    "print(\"Total de item com error\", status_error.journal.count())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>journal</th>\n",
       "      <th>pid</th>\n",
       "      <th>doi</th>\n",
       "      <th>submission_status</th>\n",
       "      <th>feedback_status</th>\n",
       "      <th>feedback_xml</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>654</th>\n",
       "      <td>654</td>\n",
       "      <td>Physis: Revista de Saúde Coletiva</td>\n",
       "      <td>S0103-73312015000100332</td>\n",
       "      <td>10.1590/1982&amp;#8211;37030332ERRATA</td>\n",
       "      <td>success</td>\n",
       "      <td>error</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>689</th>\n",
       "      <td>689</td>\n",
       "      <td>Acta Ortopédica Brasileira</td>\n",
       "      <td>S1413-78522017000300099</td>\n",
       "      <td>10.1590/1413-785220172503153742</td>\n",
       "      <td>success</td>\n",
       "      <td>error</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>729</th>\n",
       "      <td>729</td>\n",
       "      <td>Psicologia: Ciência e Profissão</td>\n",
       "      <td>S1414-98932014000300704</td>\n",
       "      <td>10.1590/1982&amp;#8211;3703001452013</td>\n",
       "      <td>success</td>\n",
       "      <td>error</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>735</th>\n",
       "      <td>735</td>\n",
       "      <td>Revista Brasileira de Plantas Medicinais</td>\n",
       "      <td>S1516-05722015000100051</td>\n",
       "      <td>10.1590/1983-084X/ 12_191</td>\n",
       "      <td>success</td>\n",
       "      <td>error</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1330</th>\n",
       "      <td>1330</td>\n",
       "      <td>Alfa : Revista de Linguística (São José do Rio...</td>\n",
       "      <td>S1981-57942014000200252</td>\n",
       "      <td>10.1590/1981-5794-1405-0 251</td>\n",
       "      <td>success</td>\n",
       "      <td>error</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Unnamed: 0                                            journal  \\\n",
       "654          654                  Physis: Revista de Saúde Coletiva   \n",
       "689          689                         Acta Ortopédica Brasileira   \n",
       "729          729                    Psicologia: Ciência e Profissão   \n",
       "735          735           Revista Brasileira de Plantas Medicinais   \n",
       "1330        1330  Alfa : Revista de Linguística (São José do Rio...   \n",
       "\n",
       "                          pid                                doi  \\\n",
       "654   S0103-73312015000100332  10.1590/1982&#8211;37030332ERRATA   \n",
       "689   S1413-78522017000300099    10.1590/1413-785220172503153742   \n",
       "729   S1414-98932014000300704   10.1590/1982&#8211;3703001452013   \n",
       "735   S1516-05722015000100051          10.1590/1983-084X/ 12_191   \n",
       "1330  S1981-57942014000200252       10.1590/1981-5794-1405-0 251   \n",
       "\n",
       "     submission_status feedback_status feedback_xml  \n",
       "654            success           error          NaN  \n",
       "689            success           error          NaN  \n",
       "729            success           error          NaN  \n",
       "735            success           error          NaN  \n",
       "1330           success           error          NaN  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "status_error.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Esses casos nessecitam ser reprocessados pelo SciELO - DOI Manager"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>journal</th>\n",
       "      <th>pid</th>\n",
       "      <th>doi</th>\n",
       "      <th>submission_status</th>\n",
       "      <th>feedback_status</th>\n",
       "      <th>feedback_xml</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>214</th>\n",
       "      <td>214</td>\n",
       "      <td>Radiologia Brasileira</td>\n",
       "      <td>S0100-39842019000100060</td>\n",
       "      <td>10.1590/0100-3984.2017.0091</td>\n",
       "      <td>success</td>\n",
       "      <td>waiting</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>655</th>\n",
       "      <td>655</td>\n",
       "      <td>Ciência Rural</td>\n",
       "      <td>S0103-84782019000300150</td>\n",
       "      <td>10.1590/0103-8478cr20180444</td>\n",
       "      <td>success</td>\n",
       "      <td>waiting</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>673</th>\n",
       "      <td>673</td>\n",
       "      <td>Revista da Associação Médica Brasileira</td>\n",
       "      <td>S0104-42302019000300319</td>\n",
       "      <td>10.1590/1806-9282.65.3.319</td>\n",
       "      <td>success</td>\n",
       "      <td>waiting</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>678</th>\n",
       "      <td>678</td>\n",
       "      <td>Anais Brasileiros de Dermatologia</td>\n",
       "      <td>S0365-05962018000400026</td>\n",
       "      <td>10.1590/abd1806-4841.20187329</td>\n",
       "      <td>success</td>\n",
       "      <td>waiting</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>679</th>\n",
       "      <td>679</td>\n",
       "      <td>Revista Brasileira de Educação</td>\n",
       "      <td>S1413-24782018000100603</td>\n",
       "      <td>10.1590/s1413-24782018230099</td>\n",
       "      <td>success</td>\n",
       "      <td>waiting</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Unnamed: 0                                  journal  \\\n",
       "214         214                    Radiologia Brasileira   \n",
       "655         655                            Ciência Rural   \n",
       "673         673  Revista da Associação Médica Brasileira   \n",
       "678         678        Anais Brasileiros de Dermatologia   \n",
       "679         679           Revista Brasileira de Educação   \n",
       "\n",
       "                         pid                            doi submission_status  \\\n",
       "214  S0100-39842019000100060    10.1590/0100-3984.2017.0091           success   \n",
       "655  S0103-84782019000300150    10.1590/0103-8478cr20180444           success   \n",
       "673  S0104-42302019000300319     10.1590/1806-9282.65.3.319           success   \n",
       "678  S0365-05962018000400026  10.1590/abd1806-4841.20187329           success   \n",
       "679  S1413-24782018000100603   10.1590/s1413-24782018230099           success   \n",
       "\n",
       "    feedback_status feedback_xml  \n",
       "214         waiting          NaN  \n",
       "655         waiting          NaN  \n",
       "673         waiting          NaN  \n",
       "678         waiting          NaN  \n",
       "679         waiting          NaN  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "status_waiting.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Esses casos nessecitam ser reprocessados pelo SciELO - DOI Manager"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "status_waiting\n",
      "\n",
      "S0100-39842019000100060\n",
      "S0103-84782019000300150\n",
      "S0104-42302019000300319\n",
      "S0365-05962018000400026\n",
      "S1413-24782018000100603\n",
      "S1414-98932018000600003\n",
      "S1415-43662018000500355\n",
      "S1415-43662018000800564\n",
      "S1516-31802018000500449\n",
      "S1516-31802018000500464\n",
      "S1517-86922018000500366\n",
      "S1517-86922018000500382\n",
      "S1806-37132018000100069\n",
      "S1983-21172018000100203\n",
      "\n",
      "status_error\n",
      "\n",
      "S0103-73312015000100332\n",
      "S1413-78522017000300099\n",
      "S1414-98932014000300704\n",
      "S1516-05722015000100051\n",
      "S1981-57942014000200252\n",
      "S1981-81222017000200453\n",
      "S2237-101X2002000100039\n",
      "S2237-101X2012000100029\n"
     ]
    }
   ],
   "source": [
    "# Lista dos PIDs para serem Reprocessados\n",
    "print(\"\\nstatus_waiting\\n\")\n",
    "print(\"\\n\".join(status_waiting.pid.to_list()))\n",
    "print(\"\\nstatus_error\\n\")\n",
    "print(\"\\n\".join(status_error.pid.to_list()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Caso de  statos com `notapplicable` "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total de item com status_notapplicable 7\n"
     ]
    }
   ],
   "source": [
    "status_notapplicable = data_errors[\"notapplicable\"]\n",
    "\n",
    "print(\"Total de item com status_notapplicable\", status_notapplicable.journal.count())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>journal</th>\n",
       "      <th>pid</th>\n",
       "      <th>doi</th>\n",
       "      <th>submission_status</th>\n",
       "      <th>feedback_status</th>\n",
       "      <th>feedback_xml</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>251</th>\n",
       "      <td>251</td>\n",
       "      <td>Revista Brasileira de Educação Médica</td>\n",
       "      <td>S0100-55022015000100171</td>\n",
       "      <td>10.15910/1981-52712015v39n1e00232013er</td>\n",
       "      <td>notapplicable</td>\n",
       "      <td>notapplicable</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>393</th>\n",
       "      <td>393</td>\n",
       "      <td>Revista Brasileira de Ginecologia e Obstetrícia</td>\n",
       "      <td>S0100-72032014000700315</td>\n",
       "      <td>10.159/S0100-720320140004977</td>\n",
       "      <td>notapplicable</td>\n",
       "      <td>notapplicable</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>394</th>\n",
       "      <td>394</td>\n",
       "      <td>Revista Brasileira de Ginecologia e Obstetrícia</td>\n",
       "      <td>S0100-72032014000800340</td>\n",
       "      <td>10.1509/SO100-720320140005034</td>\n",
       "      <td>notapplicable</td>\n",
       "      <td>notapplicable</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>506</th>\n",
       "      <td>506</td>\n",
       "      <td>Cadernos de Saúde Pública</td>\n",
       "      <td>S0102-311X2017000702001</td>\n",
       "      <td>10.15090/0102-311x00138516</td>\n",
       "      <td>notapplicable</td>\n",
       "      <td>notapplicable</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>507</th>\n",
       "      <td>507</td>\n",
       "      <td>Cadernos de Saúde Pública</td>\n",
       "      <td>S0102-311X2017001400501</td>\n",
       "      <td>10.15090/0102-311x00058116</td>\n",
       "      <td>notapplicable</td>\n",
       "      <td>notapplicable</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Unnamed: 0                                          journal  \\\n",
       "251         251            Revista Brasileira de Educação Médica   \n",
       "393         393  Revista Brasileira de Ginecologia e Obstetrícia   \n",
       "394         394  Revista Brasileira de Ginecologia e Obstetrícia   \n",
       "506         506                        Cadernos de Saúde Pública   \n",
       "507         507                        Cadernos de Saúde Pública   \n",
       "\n",
       "                         pid                                     doi  \\\n",
       "251  S0100-55022015000100171  10.15910/1981-52712015v39n1e00232013er   \n",
       "393  S0100-72032014000700315            10.159/S0100-720320140004977   \n",
       "394  S0100-72032014000800340           10.1509/SO100-720320140005034   \n",
       "506  S0102-311X2017000702001              10.15090/0102-311x00138516   \n",
       "507  S0102-311X2017001400501              10.15090/0102-311x00058116   \n",
       "\n",
       "    submission_status feedback_status feedback_xml  \n",
       "251     notapplicable   notapplicable          NaN  \n",
       "393     notapplicable   notapplicable          NaN  \n",
       "394     notapplicable   notapplicable          NaN  \n",
       "506     notapplicable   notapplicable          NaN  \n",
       "507     notapplicable   notapplicable          NaN  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "status_notapplicable.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Esse caso deve se verificar o *Prefix* utilizado para registro dos DOI, pois aparetentemente ouve erro de digitação necesses dados. e não há como reprocesser sem a correçao do valor do prefixo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10.15910/1981-52712015v39n1e00232013er\n",
      "10.159/S0100-720320140004977\n",
      "10.1509/SO100-720320140005034\n",
      "10.15090/0102-311x00138516\n",
      "10.15090/0102-311x00058116\n",
      "10.15090/0102-311x00087416\n",
      "10.1509/0104-4060.47764\n"
     ]
    }
   ],
   "source": [
    "print(\"\\n\".join(status_notapplicable.doi.to_list()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}