Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IntegrityError: NOT NULL constraint failed: exchangedataset.input_code #187

Open
OliverLyon opened this issue Oct 30, 2024 · 14 comments
Open

Comments

@OliverLyon
Copy link

OliverLyon commented Oct 30, 2024

I tried to transform the in built scenarios into superstructure databases, and I received the following error:

IntegrityError Traceback (most recent call last)
File ~.conda\envs\activity-browser\Lib\site-packages\peewee.py:3322, in Database.execute_sql(self, sql, params, commit)
3321 cursor = self.cursor()
-> 3322 cursor.execute(sql, params or ())
3323 return cursor

IntegrityError: NOT NULL constraint failed: exchangedataset.input_code

And I also got a number of outputs such as: ('Vanadium V', ('water', 'ground-, long-term'), 'kilogram', '3.7') not found in biosphere dictionary.

Does anyone know why this is happening and how to solve this? I pasted the code I use below.
Thanks.

base_setup_with_ecoinvent(
"bw25_project_test",
"ecoinvent_3.7.1_cutoff_ecoSpold02",
r"C:\Users\Desktop\EcoInvent\ecoinvent 3.7.1_cutoff_ecoSpold02\datasets",
reparametrize_lognormals=True,
)

from premise import NewDatabase
from datapackage import Package
from premise import *

bd.projects.set_current("bw25_project_test")

scenarios = [
{"model": "REMIND", "pathway": "SSP1-NPi", "year": 2020},
{"model": "REMIND", "pathway": "SSP1-NPi", "year": 2030},
{"model": "REMIND", "pathway": "SSP1-NPi", "year": 2040},
{"model": "REMIND", "pathway": "SSP1-NPi", "year": 2050},
]

ndb = NewDatabase(
scenarios=scenarios,
#scenarios=[
# {"model": "image", "pathway": "SSP2-Base", "year": 2050}
#],
source_db="ecoinvent_3.7.1_cutoff_ecoSpold02",
source_version="3.7.1",
key='tUePmX_S5B8ieZkkM7WUU2CnO8SmShwmAeWK9x2rTFo=',
use_cached_inventories=True, # This will use any cached inventories
keep_source_db_uncertainty=False, # False by default, set to True if you want to keep ecoinvent's uncertainty data
keep_imports_uncertainty=False, # False by default, set to True if you want to keep the uncertainty data of the additional inventories
use_absolute_efficiency=False, # False by default, set to True if you want to use the IAM's absolute efficiencies
)

@romainsacchi
Copy link
Collaborator

Can you tell which premise version do you use?
Do I understand correctly that you use ecoinvent ecospold files?

These errors appear in the Activity-Browser terminal?

The project in which premise exports the database has the mention "25" in it: is it because it is created with brightway 2.5? If so, any brightway25-generated database cannot be read in Activity-Browser yet. AB only reads bw2-generated databases.

Also, when you say:

And I also got a number of outputs such as: ('Vanadium V', ('water', 'ground-, long-term'), 'kilogram', '3.7') not found in biosphere dictionary.

Is it during premise execution?

Can you print conda list from the conda environment in which you operate premise?

@OliverLyon
Copy link
Author

I am using version 2.1.9 importing a ecospold file. The error appears in when I run the code in jupyter notebook. The outputs such as ('Vanadium V', ('water', 'ground-, long-term'), 'kilogram', '3.7') not found in biosphere dictionary. are generated in the premise execution. And I run it in brightway 2 environment, it shows brightway25 because I previously run it in bw25 environment in another trail and did not change the name. The name of the environment is activity-browser because I also downloaded ab in the environment.

packages in environment at C:\Users.conda\envs\activity-brower:

Name Version Build Channel

ab-plugin-scenariolink 0.1.1 py_0 romainsacchi
activity-browser 2.10.3 py_0 bsteubing
annotated-types 0.7.0 pyhd8ed1ab_0 conda-forge
anyio 4.6.2.post1 pyhd8ed1ab_0 conda-forge
appdirs 1.4.4 pyh9f0ad1d_0 conda-forge
argon2-cffi 23.1.0 pyhd8ed1ab_0 conda-forge
argon2-cffi-bindings 21.2.0 py311he736701_5 conda-forge
arrow 1.3.0 pyhd8ed1ab_0 conda-forge
asteval 1.0.5 pyhd8ed1ab_0 conda-forge
asttokens 2.4.1 pyhd8ed1ab_0 conda-forge
astunparse 1.6.3 pyhd8ed1ab_2 conda-forge
async-lru 2.0.4 pyhd8ed1ab_0 conda-forge
attrs 24.2.0 pyh71513ae_0 conda-forge
aws-c-auth 0.7.31 h459cf4e_5 conda-forge
aws-c-cal 0.7.4 h0da4a7a_4 conda-forge
aws-c-common 0.9.31 h2466b09_0 conda-forge
aws-c-compression 0.2.19 h0da4a7a_4 conda-forge
aws-c-event-stream 0.5.0 h520d0cd_0 conda-forge
aws-c-http 0.8.10 h2b94654_5 conda-forge
aws-c-io 0.14.20 he6ac336_2 conda-forge
aws-c-mqtt 0.10.7 h5d974fa_5 conda-forge
aws-c-s3 0.7.0 h6498dec_0 conda-forge
aws-c-sdkutils 0.1.19 h0da4a7a_6 conda-forge
aws-checksums 0.1.20 h0da4a7a_3 conda-forge
aws-crt-cpp 0.29.0 hb4a7b61_0 conda-forge
aws-sdk-cpp 1.11.407 hdc23f3d_6 conda-forge
azure-core-cpp 1.14.0 haf5610f_0 conda-forge
azure-identity-cpp 1.10.0 hd6deed7_0 conda-forge
azure-storage-blobs-cpp 12.13.0 h3241184_1 conda-forge
azure-storage-common-cpp 12.8.0 hd6deed7_1 conda-forge
babel 2.14.0 pyhd8ed1ab_0 conda-forge
beautifulsoup4 4.12.3 pyha770c72_0 conda-forge
bleach 6.1.0 pyhd8ed1ab_0 conda-forge
blinker 1.8.2 pyhd8ed1ab_0 conda-forge
blosc 1.21.6 h85f69ea_0 conda-forge
boto3 1.35.51 pypi_0 pypi
botocore 1.35.51 pypi_0 pypi
bottleneck 1.4.2 pypi_0 pypi
brightway2 2.4.6 pyhd8ed1ab_0 conda-forge
brotli 1.1.0 h2466b09_2 conda-forge
brotli-bin 1.1.0 h2466b09_2 conda-forge
brotli-python 1.1.0 py311hda3d55a_2 conda-forge
brotlicffi 1.1.0.0 py311hda3d55a_2 conda-forge
bw2analyzer 0.10 pyhd8ed1ab_0 conda-forge
bw2calc 1.8.2 py311h1ea47a8_0 conda-forge
bw2data 3.6.6 pyhd8ed1ab_0 conda-forge
bw2io 0.8.12 pyhd8ed1ab_0 conda-forge
bw2parameters 1.1.0 pyhd8ed1ab_0 conda-forge
bw_migrations 0.2 pyhd8ed1ab_0 conda-forge
bzip2 1.0.8 h2466b09_7 conda-forge
c-ares 1.34.2 h2466b09_0 conda-forge
ca-certificates 2024.8.30 h56e8100_0 conda-forge
cached-property 1.5.2 hd8ed1ab_1 conda-forge
cached_property 1.5.2 pyha770c72_1 conda-forge
cairo 1.18.0 h32b962e_3 conda-forge
cchardet 2.1.7 py311h12c1d0e_5 conda-forge
certifi 2024.8.30 pyhd8ed1ab_0 conda-forge
cffi 1.17.1 py311he736701_0 conda-forge
cfitsio 4.4.1 hae32d5d_2 conda-forge
chardet 5.2.0 py311h1ea47a8_2 conda-forge
charset-normalizer 3.4.0 pyhd8ed1ab_0 conda-forge
click 8.1.7 win_pyh7428d3b_0 conda-forge
click-plugins 1.1.1 py_0 conda-forge
cligj 0.7.2 pyhd8ed1ab_1 conda-forge
colorama 0.4.6 pyhd8ed1ab_0 conda-forge
comm 0.2.2 pyhd8ed1ab_0 conda-forge
constructive_geometries 1.0 pyhd8ed1ab_1 conda-forge
contourpy 1.3.0 py311h3257749_2 conda-forge
country_converter 1.2 pyhd8ed1ab_0 conda-forge
cpython 3.11.10 py311hd8ed1ab_3 conda-forge
cryptography 43.0.3 pypi_0 pypi
cycler 0.12.1 pyhd8ed1ab_0 conda-forge
dataflows-tabulator 1.54.3 pyhd8ed1ab_0 conda-forge
datapackage 1.15.4 pypi_0 pypi
debugpy 1.8.7 py311hda3d55a_0 conda-forge
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge
dill 0.3.9 pyhd8ed1ab_0 conda-forge
docopt 0.6.2 py_1 conda-forge
ecoinvent_interface 2.4.1 pyhd8ed1ab_0 conda-forge
eight 1.0.1 pyhd8ed1ab_4 conda-forge
entrypoints 0.4 pyhd8ed1ab_0 conda-forge
et_xmlfile 2.0.0 pyhd8ed1ab_0 conda-forge
exceptiongroup 1.2.2 pyhd8ed1ab_0 conda-forge
executing 2.1.0 pyhd8ed1ab_0 conda-forge
expat 2.6.3 he0c23c2_0 conda-forge
fasteners 0.17.3 pyhd8ed1ab_0 conda-forge
fiona 1.10.1 py311h2494c99_1 conda-forge
flask 3.0.3 pyhd8ed1ab_0 conda-forge
flexcache 0.3 pyhd8ed1ab_0 conda-forge
flexparser 0.3.1 pyhd8ed1ab_0 conda-forge
fmt 11.0.2 h7f575de_0 conda-forge
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 h77eed37_3 conda-forge
fontconfig 2.14.2 hbde0cde_0 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
fonttools 4.54.1 py311h5082efb_1 conda-forge
fqdn 1.5.1 pyhd8ed1ab_0 conda-forge
freetype 2.12.1 hdaf720e_2 conda-forge
freexl 2.0.0 h8276f4a_0 conda-forge
future 0.18.3 pypi_0 pypi
gdal 3.9.3 py311h689aae6_2 conda-forge
geos 3.13.0 h5a68840_0 conda-forge
geotiff 1.7.3 h496ac4d_3 conda-forge
glib 2.82.2 h7025463_0 conda-forge
glib-tools 2.82.2 h4394cf3_0 conda-forge
greenlet 3.1.1 py311hda3d55a_0 conda-forge
gst-plugins-base 1.24.7 hb0a98b8_0 conda-forge
gstreamer 1.24.7 h5006eae_0 conda-forge
h11 0.14.0 pyhd8ed1ab_0 conda-forge
h2 4.1.0 pyhd8ed1ab_0 conda-forge
hdf4 4.2.15 h5557f11_7 conda-forge
hdf5 1.14.3 nompi_h2b43c12_105 conda-forge
hpack 4.0.0 pyh9f0ad1d_0 conda-forge
httpcore 1.0.6 pyhd8ed1ab_0 conda-forge
httpx 0.27.2 pyhd8ed1ab_0 conda-forge
hyperframe 6.0.1 pyhd8ed1ab_0 conda-forge
icu 75.1 he0c23c2_0 conda-forge
idna 3.10 pyhd8ed1ab_0 conda-forge
ijson 3.3.0 pyhd8ed1ab_0 conda-forge
importlib-metadata 8.5.0 pyha770c72_0 conda-forge
importlib_metadata 8.5.0 hd8ed1ab_0 conda-forge
importlib_resources 6.4.5 pyhd8ed1ab_0 conda-forge
inflate64 1.0.0 py311he736701_2 conda-forge
intel-openmp 2024.2.1 h57928b3_1083 conda-forge
ipykernel 6.29.5 pyh4bbf305_0 conda-forge
ipython 8.29.0 pyh7428d3b_0 conda-forge
isodate 0.7.2 pyhd8ed1ab_0 conda-forge
isoduration 20.11.0 pyhd8ed1ab_0 conda-forge
itsdangerous 2.2.0 pyhd8ed1ab_0 conda-forge
jedi 0.19.1 pyhd8ed1ab_0 conda-forge
jinja2 3.1.4 pyhd8ed1ab_0 conda-forge
jmespath 1.0.1 pyhd8ed1ab_0 conda-forge
json5 0.9.25 pyhd8ed1ab_0 conda-forge
jsonlines 4.0.0 pyhd8ed1ab_0 conda-forge
jsonpointer 3.0.0 py311h1ea47a8_1 conda-forge
jsonschema 4.23.0 pyhd8ed1ab_0 conda-forge
jsonschema-specifications 2024.10.1 pyhd8ed1ab_0 conda-forge
jsonschema-with-format-nongpl 4.23.0 hd8ed1ab_0 conda-forge
jupyter-lsp 2.2.5 pyhd8ed1ab_0 conda-forge
jupyter_client 8.6.3 pyhd8ed1ab_0 conda-forge
jupyter_core 5.7.2 pyh5737063_1 conda-forge
jupyter_events 0.10.0 pyhd8ed1ab_0 conda-forge
jupyter_server 2.14.2 pyhd8ed1ab_0 conda-forge
jupyter_server_terminals 0.5.3 pyhd8ed1ab_0 conda-forge
jupyterlab 4.2.5 pyhd8ed1ab_0 conda-forge
jupyterlab_pygments 0.3.0 pyhd8ed1ab_1 conda-forge
jupyterlab_server 2.27.3 pyhd8ed1ab_0 conda-forge
kealib 1.5.3 h6c43f9b_2 conda-forge
kiwisolver 1.4.7 py311h3257749_0 conda-forge
krb5 1.21.3 hdf4eb48_0 conda-forge
lcms2 2.16 h67d730c_0 conda-forge
lerc 4.0.0 h63175ca_0 conda-forge
libabseil 20240722.0 cxx17_he0c23c2_1 conda-forge
libaec 1.1.3 h63175ca_0 conda-forge
libarchive 3.7.4 haf234dc_0 conda-forge
libblas 3.9.0 25_win64_mkl conda-forge
libbrotlicommon 1.1.0 h2466b09_2 conda-forge
libbrotlidec 1.1.0 h2466b09_2 conda-forge
libbrotlienc 1.1.0 h2466b09_2 conda-forge
libcblas 3.9.0 25_win64_mkl conda-forge
libclang13 19.1.2 default_ha5278ca_1 conda-forge
libcrc32c 1.1.2 h0e60522_0 conda-forge
libcurl 8.10.1 h1ee3ff0_0 conda-forge
libdeflate 1.22 h2466b09_0 conda-forge
libexpat 2.6.3 he0c23c2_0 conda-forge
libffi 3.4.2 h8ffe710_5 conda-forge
libflang 19.1.2 he0c23c2_0 conda-forge
libgdal 3.9.3 h57928b3_2 conda-forge
libgdal-core 3.9.3 h042995d_2 conda-forge
libgdal-fits 3.9.3 h0a0b71e_2 conda-forge
libgdal-grib 3.9.3 hd2a089b_2 conda-forge
libgdal-hdf4 3.9.3 h430f241_2 conda-forge
libgdal-hdf5 3.9.3 had131a1_2 conda-forge
libgdal-jp2openjpeg 3.9.3 hed4c6cb_2 conda-forge
libgdal-kea 3.9.3 h95b1a77_2 conda-forge
libgdal-netcdf 3.9.3 h55e78d3_2 conda-forge
libgdal-pdf 3.9.3 ha1c78db_2 conda-forge
libgdal-pg 3.9.3 hfaa227e_2 conda-forge
libgdal-postgisraster 3.9.3 hfaa227e_2 conda-forge
libgdal-tiledb 3.9.3 hb8b5d01_2 conda-forge
libgdal-xls 3.9.3 hd0e23a6_2 conda-forge
libglib 2.82.2 h7025463_0 conda-forge
libgoogle-cloud 2.30.0 ha00044d_0 conda-forge
libgoogle-cloud-storage 2.30.0 he5eb982_0 conda-forge
libgrpc 1.65.5 ha20e22e_0 conda-forge
libhwloc 2.11.1 default_h8125262_1000 conda-forge
libiconv 1.17 hcfcfb64_2 conda-forge
libintl 0.22.5 h5728263_3 conda-forge
libintl-devel 0.22.5 h5728263_3 conda-forge
libjpeg-turbo 3.0.0 hcfcfb64_1 conda-forge
libkml 1.3.0 h538826c_1021 conda-forge
liblapack 3.9.0 25_win64_mkl conda-forge
libnetcdf 4.9.2 nompi_h92078aa_114 conda-forge
libogg 1.3.5 h2466b09_0 conda-forge
libpng 1.6.44 h3ca93ac_0 conda-forge
libpq 17.0 h7ec079e_4 conda-forge
libprotobuf 5.27.5 hcaed137_2 conda-forge
libre2-11 2024.07.02 h4eb7d71_1 conda-forge
librttopo 1.1.0 hd4c2148_17 conda-forge
libsodium 1.0.20 hc70643c_0 conda-forge
libspatialite 5.1.0 h939089a_11 conda-forge
libsqlite 3.47.0 h2466b09_1 conda-forge
libssh2 1.11.0 h7dfc565_0 conda-forge
libtiff 4.7.0 hfc51747_1 conda-forge
libvorbis 1.3.7 h0e60522_0 conda-forge
libwebp 1.4.0 h2466b09_0 conda-forge
libwebp-base 1.4.0 hcfcfb64_0 conda-forge
libxcb 1.16 h013a479_1 conda-forge
libxml2 2.12.7 h0f24e4e_4 conda-forge
libxslt 1.1.39 h3df6e99_0 conda-forge
libzip 1.11.1 h25f2845_0 conda-forge
libzlib 1.3.1 h2466b09_2 conda-forge
linear-tsv 1.1.0 pypi_0 pypi
llvmlite 0.43.0 pypi_0 pypi
lxml 5.3.0 py311hf779c20_2 conda-forge
lz4-c 1.9.4 hcfcfb64_0 conda-forge
lzo 2.10 hcfcfb64_1001 conda-forge
m2w64-gcc-libgfortran 5.3.0 6 conda-forge
m2w64-gcc-libs 5.3.0 7 conda-forge
m2w64-gcc-libs-core 5.3.0 7 conda-forge
m2w64-gmp 6.1.0 2 conda-forge
m2w64-libwinpthread-git 5.0.0.4634.697f757 2 conda-forge
markupsafe 3.0.2 py311h5082efb_0 conda-forge
matplotlib-base 3.9.2 py311h8f1b1e4_1 conda-forge
matplotlib-inline 0.1.7 pyhd8ed1ab_0 conda-forge
minizip 4.0.6 hb638d1e_0 conda-forge
mistune 3.0.2 pyhd8ed1ab_0 conda-forge
mkl 2024.2.2 h66d3029_14 conda-forge
mrio_common_metadata 0.2.1 pyhd8ed1ab_0 conda-forge
msys2-conda-epoch 20160418 1 conda-forge
multiprocess 0.70.17 py311he736701_0 conda-forge
multivolumefile 0.2.3 pyhd8ed1ab_0 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
nbclient 0.10.0 pyhd8ed1ab_0 conda-forge
nbconvert-core 7.16.4 pyhd8ed1ab_1 conda-forge
nbformat 5.10.4 pyhd8ed1ab_0 conda-forge
nest-asyncio 1.6.0 pyhd8ed1ab_0 conda-forge
networkx 3.4.2 pyhd8ed1ab_0 conda-forge
notebook-shim 0.2.4 pyhd8ed1ab_0 conda-forge
numba 0.60.0 py311h0673bce_0 conda-forge
numpy 1.26.4 py311h0b4df5a_0 conda-forge
openjpeg 2.5.2 h3d672ee_0 conda-forge
openpyxl 3.1.5 py311ha68e1ae_1 conda-forge
openssl 3.3.2 h2466b09_0 conda-forge
overrides 7.7.0 pyhd8ed1ab_0 conda-forge
packaging 24.1 pyhd8ed1ab_0 conda-forge
pandas 2.2.3 py311hcf9f919_1 conda-forge
pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge
parso 0.8.4 pyhd8ed1ab_0 conda-forge
pathos 0.3.3 pyhd8ed1ab_0 conda-forge
patsy 0.5.6 pyhd8ed1ab_0 conda-forge
pcre2 10.44 h3d7b363_2 conda-forge
peewee 3.17.7 py311h1eea93c_0 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 10.4.0 py311h5592be9_1 conda-forge
pint 0.21 pyhd8ed1ab_0 conda-forge
pip 24.3.1 pyh8b19718_0 conda-forge
pixman 0.43.4 h63175ca_0 conda-forge
pkgutil-resolve-name 1.3.10 pyhd8ed1ab_1 conda-forge
platformdirs 4.3.6 pyhd8ed1ab_0 conda-forge
poppler 24.08.0 h9415970_1 conda-forge
poppler-data 0.4.12 hd8ed1ab_0 conda-forge
postgresql 17.0 heca7946_4 conda-forge
pox 0.3.5 pyhd8ed1ab_0 conda-forge
ppft 1.7.6.9 pyhd8ed1ab_0 conda-forge
premise 2.1.9 pypi_0 pypi
premise-gwp 0.9.7 pypi_0 pypi
prettytable 3.11.0 pyhd8ed1ab_0 conda-forge
proj 9.5.0 hd9569ee_0 conda-forge
prometheus_client 0.21.0 pyhd8ed1ab_0 conda-forge
prompt-toolkit 3.0.48 pyha770c72_0 conda-forge
psutil 6.1.0 py311he736701_0 conda-forge
pthread-stubs 0.4 hcd874cb_1001 conda-forge
pthreads-win32 2.9.1 h2466b09_4 conda-forge
pure_eval 0.2.3 pyhd8ed1ab_0 conda-forge
py7zr 0.22.0 pyhd8ed1ab_0 conda-forge
pyarrow 18.0.0 pypi_0 pypi
pybcj 1.0.2 py311he736701_1 conda-forge
pycountry 24.6.1 pypi_0 pypi
pycparser 2.22 pyhd8ed1ab_0 conda-forge
pycryptodomex 3.20.0 py311he736701_1 conda-forge
pydantic 2.9.2 pyhd8ed1ab_0 conda-forge
pydantic-core 2.23.4 py311h533ab2d_0 conda-forge
pydantic-settings 2.6.0 pyh3cfb1c2_0 conda-forge
pyecospold 4.0.0 py_0 cmutel
pygments 2.18.0 pyhd8ed1ab_0 conda-forge
pypardiso 0.4.6 pyhd8ed1ab_0 conda-forge
pyparsing 3.2.0 pyhd8ed1ab_1 conda-forge
pyperclip 1.9.0 pyh7428d3b_0 conda-forge
pyppmd 1.1.0 py311hda3d55a_1 conda-forge
pyprind 2.11.2 py311h1ea47a8_1006 conda-forge
pyside2 5.15.8 py311h3d699ce_4 conda-forge
pysocks 1.7.1 pyh0701188_6 conda-forge
python 3.11.10 hce54a09_3_cpython conda-forge
python-dateutil 2.9.0 pyhd8ed1ab_0 conda-forge
python-dotenv 1.0.1 pyhd8ed1ab_0 conda-forge
python-fastjsonschema 2.20.0 pyhd8ed1ab_0 conda-forge
python-json-logger 2.0.7 pyhd8ed1ab_0 conda-forge
python-tzdata 2024.2 pyhd8ed1ab_0 conda-forge
python_abi 3.11 5_cp311 conda-forge
pytz 2024.1 pyhd8ed1ab_0 conda-forge
pywin32 307 py311hda3d55a_3 conda-forge
pywinpty 2.0.14 py311hda3d55a_0 conda-forge
pyxlsb 1.0.10 pyhd8ed1ab_0 conda-forge
pyyaml 6.0.2 py311he736701_1 conda-forge
pyzmq 26.2.0 py311h484c95c_3 conda-forge
pyzstd 0.16.2 py311h53056dc_0 conda-forge
qhull 2020.2 hc790b64_5 conda-forge
qt-main 5.15.15 h264fbc2_0 conda-forge
qt-webengine 5.15.15 h087ee03_1 conda-forge
re2 2024.07.02 hd3b24a8_1 conda-forge
referencing 0.35.1 pyhd8ed1ab_0 conda-forge
requests 2.32.3 pyhd8ed1ab_0 conda-forge
rfc3339-validator 0.1.4 pyhd8ed1ab_0 conda-forge
rfc3986 2.0.0 pyhd8ed1ab_0 conda-forge
rfc3986-validator 0.1.1 pyh9f0ad1d_0 conda-forge
rpds-py 0.20.0 py311h533ab2d_1 conda-forge
s3transfer 0.10.3 pyhd8ed1ab_0 conda-forge
salib 1.5.0 pyhd8ed1ab_0 conda-forge
schema 0.7.7 pypi_0 pypi
scipy 1.14.1 py311hf16d85f_1 conda-forge
seaborn 0.13.2 hd8ed1ab_2 conda-forge
seaborn-base 0.13.2 pyhd8ed1ab_2 conda-forge
send2trash 1.8.3 pyh5737063_0 conda-forge
setuptools 75.1.0 pyhd8ed1ab_0 conda-forge
setuptools-scm 8.1.0 pyhd8ed1ab_0 conda-forge
shapely 2.0.6 py311hd54bd37_2 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
snappy 1.2.1 h23299a8_0 conda-forge
sniffio 1.3.1 pyhd8ed1ab_0 conda-forge
soupsieve 2.5 pyhd8ed1ab_1 conda-forge
sparse 0.15.4 pyh267e887_1 conda-forge
spdlog 1.14.1 h9f2357e_1 conda-forge
sqlalchemy 2.0.36 py311he736701_0 conda-forge
sqlite 3.47.0 h2466b09_1 conda-forge
stack_data 0.6.2 pyhd8ed1ab_0 conda-forge
stats_arrays 0.7 pyhd8ed1ab_0 conda-forge
statsmodels 0.14.4 py311h0a17f05_0 conda-forge
tableschema 1.20.11 pyhd8ed1ab_0 conda-forge
tabulate 0.9.0 pyhd8ed1ab_1 conda-forge
tabulator 1.53.5 pyhd8ed1ab_0 conda-forge
tbb 2021.13.0 hc790b64_0 conda-forge
terminado 0.18.1 pyh5737063_0 conda-forge
texttable 1.7.0 pyhd8ed1ab_0 conda-forge
tiledb 2.26.2 h2e08f1e_4 conda-forge
tinycss2 1.4.0 pyhd8ed1ab_0 conda-forge
tk 8.6.13 h5226925_1 conda-forge
tomli 2.0.2 pyhd8ed1ab_0 conda-forge
toolz 1.0.0 pyhd8ed1ab_0 conda-forge
tornado 6.4.1 py311he736701_1 conda-forge
tqdm 4.66.6 pyhd8ed1ab_0 conda-forge
traitlets 5.14.3 pyhd8ed1ab_0 conda-forge
types-python-dateutil 2.9.0.20241003 pyhff2d567_0 conda-forge
typing-extensions 4.12.2 hd8ed1ab_0 conda-forge
typing_extensions 4.12.2 pyha770c72_0 conda-forge
typing_utils 0.1.0 pyhd8ed1ab_0 conda-forge
tzdata 2024b hc8b5060_0 conda-forge
ucrt 10.0.22621.0 h57928b3_1 conda-forge
unfold 2024.09.29 py_0 romainsacchi
unicodecsv 0.14.1 pyhd8ed1ab_2 conda-forge
unicodedata2 15.1.0 py311he736701_1 conda-forge
unidecode 1.3.8 pyhd8ed1ab_0 conda-forge
uri-template 1.3.0 pyhd8ed1ab_0 conda-forge
uriparser 0.9.8 h5a68840_0 conda-forge
urllib3 2.2.3 pyhd8ed1ab_0 conda-forge
vc 14.3 h8a93ad2_22 conda-forge
vc14_runtime 14.40.33810 hcc2c482_22 conda-forge
voluptuous 0.15.2 pyhd8ed1ab_1 conda-forge
vs2015_runtime 14.40.33810 h3bf8584_22 conda-forge
wcwidth 0.2.13 pyhd8ed1ab_0 conda-forge
webcolors 24.8.0 pyhd8ed1ab_0 conda-forge
webencodings 0.5.1 pyhd8ed1ab_2 conda-forge
websocket-client 1.8.0 pyhd8ed1ab_0 conda-forge
werkzeug 3.0.6 pyhd8ed1ab_0 conda-forge
wheel 0.44.0 pyhd8ed1ab_0 conda-forge
whoosh 2.7.4 py311h1ea47a8_9 conda-forge
win_inet_pton 1.1.0 pyh7428d3b_7 conda-forge
winpty 0.4.3 4 conda-forge
wrapt 1.16.0 py311he736701_1 conda-forge
wurst 0.4 pyhd8ed1ab_0 conda-forge
xarray 2024.2.0 pypi_0 pypi
xerces-c 3.2.5 he0c23c2_2 conda-forge
xlrd 2.0.1 pyhd8ed1ab_3 conda-forge
xlsxwriter 3.2.0 pyhd8ed1ab_0 conda-forge
xorg-libxau 1.0.11 hcd874cb_0 conda-forge
xorg-libxdmcp 1.1.3 hcd874cb_0 conda-forge
xz 5.2.6 h8d14728_0 conda-forge
yaml 0.2.5 h8ffe710_2 conda-forge
zeromq 4.3.5 ha9f60a1_6 conda-forge
zipfile-deflate64 0.2.0 py311ha68e1ae_4 conda-forge
zipp 3.20.2 pyhd8ed1ab_0 conda-forge
zlib 1.3.1 h2466b09_2 conda-forge
zstandard 0.23.0 py311h53056dc_1 conda-forge
zstd 1.5.6 h0ea2cb4_0 conda-forge

@OliverLyon
Copy link
Author

I tried to create a new environment and run the same code. The error and the output of " not found in biosphere dictionary": remains. Could it be the result of the unlinked changes when importing the database?

@romainsacchi
Copy link
Collaborator

OK. The message outputs given by premise regarding biosphere flows like:

('Vanadium V', ('water', 'ground-, long-term'), 'kilogram', '3.7') not found in biosphere dictionary

is normal. That is because the biosphere database of ecoinvent 3.7 does not have all the biosphere flows required by the new inventories premise imports. So these flows are just ignored.

@romainsacchi
Copy link
Collaborator

But I doubt this is related to that other issue you mention:

IntegrityError Traceback (most recent call last)
File ~.conda\envs\activity-browser\Lib\site-packages\peewee.py:3322, in Database.execute_sql(self, sql, params, commit)
3321 cursor = self.cursor()
-> 3322 cursor.execute(sql, params or ())
3323 return cursor

IntegrityError: NOT NULL constraint failed: exchangedataset.input_code

If I understand correctly, this error occurs when you use the databases through activity-browser, right?

@OliverLyon
Copy link
Author

But I doubt this is related to that other issue you mention:

IntegrityError Traceback (most recent call last)
File ~.conda\envs\activity-browser\Lib\site-packages\peewee.py:3322, in Database.execute_sql(self, sql, params, commit)
3321 cursor = self.cursor()
-> 3322 cursor.execute(sql, params or ())
3323 return cursor
IntegrityError: NOT NULL constraint failed: exchangedataset.input_code

If I understand correctly, this error occurs when you use the databases through activity-browser, right?

Yes, "activity-browser " is the name of the environment I run the code through.

@romainsacchi
Copy link
Collaborator

Hi @OliverLyon, unfortunately, I could not reproduce your error. I exported a superstructure database with the same scenarios and the same ei 3.7.1 database as you used, and I ran it with Activity Browser (same version) and obtained some results.

Screenshot 2024-11-01 at 09 16 23

Are you able to export your faulty AB project (Project -> Export project) and somehow send it to me via file transfer?

@OliverLyon
Copy link
Author

OliverLyon commented Nov 3, 2024

Hi @romainsacchi, Sorry about the confusion of activity-browser. I am not using the activity browser in this case. "acrtivity-browser" is just the name of the environment I created since I installed brightway2, activity browser and premise in this environment. The code went well when extracting the 3.7.1 database and creating new database with inbuilt scenarios.

base_setup_with_ecoinvent(
"bw_project_test",
"ecoinvent_3.7.1_cutoff_ecoSpold02",
r"C:\Users\EcoInvent\ecoinvent 3.7.1_cutoff_ecoSpold02\datasets",
reparametrize_lognormals=True,
)

from premise import NewDatabase
from datapackage import Package
from premise import *

bd.projects.set_current("bw2_project_test")

scenarios = [
{"model": "REMIND", "pathway": "SSP1-NPi", "year": 2020},
{"model": "REMIND", "pathway": "SSP1-NPi", "year": 2030},
{"model": "REMIND", "pathway": "SSP1-NPi", "year": 2040},
{"model": "REMIND", "pathway": "SSP1-NPi", "year": 2050},
]

ndb = NewDatabase(
scenarios=scenarios,
source_db="ecoinvent_3.7.1_cutoff_ecoSpold02",
source_version="3.7.1",
key='tUePmX_S5B8ieZkkM7WUU2CnO8SmShwmAeWK9x2rTFo=',
use_cached_inventories=True, # This will use any cached inventories
keep_source_db_uncertainty=False, # False by default, set to True if you want to keep ecoinvent's uncertainty data
keep_imports_uncertainty=False, # False by default, set to True if you want to keep the uncertainty data of the additional inventories
use_absolute_efficiency=False, # False by default, set to True if you want to use the IAM's absolute efficiencies
)

The only part that have and error is the superstructure output:

ndb.write_superstructure_db_to_brightway(name="bw_project_test_superstructure"),

At this stage I have not import the superstructure to activity browser yet.

@romainsacchi
Copy link
Collaborator

romainsacchi commented Nov 4, 2024

Can you maybe run:

from premise import *
clear_inventory_cache()

and try the script again?

@OliverLyon
Copy link
Author

Can you maybe run:

from premise import *
clear_inventory_cache()

and try the script again?

Thank you for the reply. I tried this:
from premise import *
clear_inventory_cache()

But the error remains:
IntegrityError: NOT NULL constraint failed: exchangedataset.input_code

@romainsacchi
Copy link
Collaborator

Is "ecoinvent_3.7.1_cutoff_ecoSpold02" a brightway database in a brightway project that you installed from ecospold2 files?

My advice would be to create a new brightway project, with a new biosphere and ecoinvent database and try again. Could eb that something went wrong during the import and is now creating some issue.

@OliverLyon
Copy link
Author

OliverLyon commented Nov 8, 2024

Is "ecoinvent_3.7.1_cutoff_ecoSpold02" a brightway database in a brightway project that you installed from ecospold2 files?

My advice would be to create a new brightway project, with a new biosphere and ecoinvent database and try again. Could eb that something went wrong during the import and is now creating some issue.

Yes, "ecoinvent_3.7.1_cutoff_ecoSpold02" is the brightway database I installed from a ecospold2 file. I tried to directly import the ecospold2 file instead of from brightway project then I'm not receiving the previous error. Now I have a new error for writing the superstructure:
UnknownObject: Exchange between ('3.7.1', 'f2d84834-d0b3-42e5-b41a-f04cc80337a4') and ('test_superstructure', '0007a913-a253-4d6e-bb9c-a9dac9b6c0c2') is invalid - one of these objects is unknown (i.e. doesn't exist as a process dataset)

Is it related to the biosphere database name it requires me to type in when running the code? I tried to type in biosphere3 alternatively, then I get the following error when writing superstructure database:
OperationalError: database is locked

@romainsacchi
Copy link
Collaborator

biosphere_name should be the name of your biosphere database name in your brightway project.

database is locked sometimes appears when a previous connection to the SQL database has not ended correctly. Restarting the notebook's kernel helps.

@OliverLyon
Copy link
Author

biosphere_name should be the name of your biosphere database name in your brightway project.

database is locked sometimes appears when a previous connection to the SQL database has not ended correctly. Restarting the notebook's kernel helps.

I restarted the kernel, now it comes back to the previous error:
IntegrityError: NOT NULL constraint failed: exchangedataset.input_code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants