is current code compatible with tf 2.13 ?

[thegodone]

I would like to move to 2.13 but have lot of errors with 3.0.1 version is the 3.0.2 or current code is fixing it ?
requirement file is not limiting the tf version indeed.

[PatReis]

Hello, yes with TF 2.13 there are a few changes that causes some problems with kgcnn 3.0.1 .
I am currently trying to repair them.
Do you have some examples? I know that tf-addons seem to cause and error and deserialization has changed a little.
For activation functions, I keept the functions but added keras layers in kgcnn.layers.activ since deserialization does not translates the functions correctly anymore and you can not change the alpha in e.g. Leaky Relu. But with layers you can now make the leak learnable. I also had to modify callbacks to allow deserialization.

[PatReis]

Hallo, can you check again with 3.0.2 ?
If there are problems, please let me know...

[thegodone]

Hello again, Try to run tf 2.13 with v.2.2.4 and got the following error :

(tf213) tgg@arm64-apple-darwin20 KGCNN_test % python keras-gcnv1.py config.cfg                     
ERROR:root:Module 'kgcnn.utils.models' is deprecated and will be removed in future versions. Please move to 'kgcnn.model.utils'.
ERROR:kgcnn.mol.convert:Can not import `OpenBabel` package for conversion.
Load config file:  config.cfg
Architecture selected: AttFP
My parameters
Loss RMSE
LSTM 16
DENSE 16
PROBA 0
LR start 0.01
using cpu
before True
INFO:kgcnn.hyper.hyper:Adding 'postfix_file' to 'info' category in hyperparameter.
{'model': {'class_name': 'make_model', 'module_name': 'kgcnn.literature.AttentiveFP', 'config': {'name': 'AttentiveFP', 'inputs': [{'shape': [None, 41], 'name': 'node_attributes', 'dtype': 'float32', 'ragged': True}, {'shape': [None, 11], 'name': 'edge_attributes', 'dtype': 'float32', 'ragged': True}, {'shape': [None, 2], 'name': 'edge_indices', 'dtype': 'int64', 'ragged': True}], 'input_embedding': {'node': {'input_dim': 95, 'output_dim': 100}, 'edge': {'input_dim': 5, 'output_dim': 100}}, 'attention_args': {'units': 200}, 'depthato': 2, 'depthmol': 2, 'dropout': 0.2, 'verbose': 10, 'output_embedding': 'graph', 'output_mlp': {'use_bias': [True, True, True], 'units': [200, 100, 1], 'activation': ['kgcnn>leaky_relu', 'selu', 'linear']}}}, 'training': {'fit': {'batch_size': 200, 'epochs': 100, 'validation_freq': 1, 'verbose': 2, 'callbacks': []}, 'compile': {'optimizer': {'class_name': 'Addons>AdamW', 'config': {'lr': 0.0031622776601683794, 'weight_decay': 1e-05}}}, 'cross_validation': {'class_name': 'KFold', 'config': {'n_splits': 5, 'random_state': None, 'shuffle': True}}, 'scaler': {'class_name': 'StandardScaler', 'config': {'with_std': True, 'with_mean': True, 'copy': True}}, 'execute_folds': 1}, 'data': {'dataset': {'class_name': 'MoleculeNetDataset', 'config': {}, 'methods': []}, 'data_unit': 'mol/L'}, 'info': {'postfix': '', 'kgcnn_version': '2.0.3', 'postfix_file': ''}}
training
is overwrite True
no 3D model
INFO:kgcnn.data.MoleculeNetDataset:Found SDF train.sdf of pre-computed structures.
INFO:kgcnn.data.MoleculeNetDataset:Read molecules from mol-file.
INFO:kgcnn.data.MoleculeNetDataset: ... process molecules 0 from 24
INFO:kgcnn.mol.encoder:OneHotEncoder Symbol found ['O', 'C', 'N']
INFO:kgcnn.mol.encoder:OneHotEncoder Hybridization found [rdkit.Chem.rdchem.HybridizationType.SP2, rdkit.Chem.rdchem.HybridizationType.SP3]
INFO:kgcnn.mol.encoder:OneHotEncoder TotalDegree found [1, 3, 2, 4]
INFO:kgcnn.mol.encoder:OneHotEncoder TotalNumHs found [0, 1, 3, 2]
INFO:kgcnn.mol.encoder:OneHotEncoder CIPCode found [None]
INFO:kgcnn.mol.encoder:OneHotEncoder ChiralityPossible found [None]
INFO:kgcnn.mol.encoder:OneHotEncoder BondType found [rdkit.Chem.rdchem.BondType.DOUBLE, rdkit.Chem.rdchem.BondType.SINGLE, rdkit.Chem.rdchem.BondType.AROMATIC]
INFO:kgcnn.mol.encoder:OneHotEncoder Stereo found [rdkit.Chem.rdchem.BondStereo.STEREONONE, rdkit.Chem.rdchem.BondStereo.STEREOANY]
INFO:kgcnn.data.MoleculeNetDataset:Read molecules from mol-file.
INFO:kgcnn.data.MoleculeNetDataset: ... process molecules 0 from 24
INFO:kgcnn.mol.encoder:OneHotEncoder Symbol found ['O', 'C', 'N']
INFO:kgcnn.mol.encoder:OneHotEncoder Hybridization found [rdkit.Chem.rdchem.HybridizationType.SP2, rdkit.Chem.rdchem.HybridizationType.SP3]
INFO:kgcnn.mol.encoder:OneHotEncoder TotalDegree found [1, 3, 2, 4]
INFO:kgcnn.mol.encoder:OneHotEncoder TotalNumHs found [0, 1, 3, 2]
INFO:kgcnn.mol.encoder:OneHotEncoder CIPCode found [None]
INFO:kgcnn.mol.encoder:OneHotEncoder ChiralityPossible found [None]
INFO:kgcnn.mol.encoder:OneHotEncoder BondType found [rdkit.Chem.rdchem.BondType.DOUBLE, rdkit.Chem.rdchem.BondType.SINGLE, rdkit.Chem.rdchem.BondType.AROMATIC]
INFO:kgcnn.mol.encoder:OneHotEncoder Stereo found [rdkit.Chem.rdchem.BondStereo.STEREONONE, rdkit.Chem.rdchem.BondStereo.STEREOANY]
INFO:kgcnn.data.MoleculeNetDataset:No invalid graphs for assigned properties found.
data length:  24
{'model': {'class_name': 'make_model', 'module_name': 'kgcnn.literature.AttentiveFP', 'config': {'name': 'AttentiveFP', 'inputs': [{'shape': [None, 41], 'name': 'node_attributes', 'dtype': 'float32', 'ragged': True}, {'shape': [None, 11], 'name': 'edge_attributes', 'dtype': 'float32', 'ragged': True}, {'shape': [None, 2], 'name': 'edge_indices', 'dtype': 'int64', 'ragged': True}], 'input_embedding': {'node': {'input_dim': 95, 'output_dim': 100}, 'edge': {'input_dim': 5, 'output_dim': 100}}, 'attention_args': {'units': 200}, 'depthato': 2, 'depthmol': 2, 'dropout': 0.2, 'verbose': 10, 'output_embedding': 'graph', 'output_mlp': {'use_bias': [True, True, True], 'units': [200, 100, 1], 'activation': ['kgcnn>leaky_relu', 'selu', 'linear']}}}, 'training': {'fit': {'batch_size': 200, 'epochs': 100, 'validation_freq': 1, 'verbose': 2, 'callbacks': []}, 'compile': {'optimizer': {'class_name': 'Addons>AdamW', 'config': {'lr': 0.0031622776601683794, 'weight_decay': 1e-05}}}, 'cross_validation': {'class_name': 'KFold', 'config': {'n_splits': 5, 'random_state': None, 'shuffle': True}}, 'scaler': {'class_name': 'StandardScaler', 'config': {'with_std': True, 'with_mean': True, 'copy': True}}, 'execute_folds': 1}, 'data': {'dataset': {'class_name': 'MoleculeNetDataset', 'config': {}, 'methods': []}, 'data_unit': 'mol/L'}, 'info': {'postfix': '', 'kgcnn_version': '2.0.3', 'postfix_file': ''}}
dataset length: 24
INFO:kgcnn.model.utils:Updated model kwargs:
INFO:kgcnn.model.utils:{'name': 'AttentiveFP', 'inputs': [{'shape': [None, 41], 'name': 'node_attributes', 'dtype': 'float32', 'ragged': True}, {'shape': [None, 11], 'name': 'edge_attributes', 'dtype': 'float32', 'ragged': True}, {'shape': [None, 2], 'name': 'edge_indices', 'dtype': 'int64', 'ragged': True}], 'input_embedding': {'node': {'input_dim': 95, 'output_dim': 100}, 'edge': {'input_dim': 5, 'output_dim': 100}}, 'attention_args': {'units': 200}, 'depthmol': 2, 'depthato': 2, 'dropout': 0.2, 'verbose': 10, 'output_embedding': 'graph', 'output_to_tensor': True, 'output_mlp': {'use_bias': [True, True, True], 'units': [200, 100, 1], 'activation': ['kgcnn>leaky_relu', 'selu', 'linear']}}
Traceback (most recent call last):
  File "/Users/tgg/Documents/KGCNN_test/keras-gcnv1.py", line 1060, in <module>
    model = make_model(**hyperparam['model']["config"])
  File "/Users/tgg/miniforge3/envs/tf215/lib/python3.10/site-packages/kgcnn/model/utils.py", line 138, in update_wrapper
    return func(*args, **updated_kwargs)
  File "/Users/tgg/miniforge3/envs/tf215/lib/python3.10/site-packages/kgcnn/literature/AttentiveFP.py", line 104, in make_model
    ck = AttentiveHeadFP(use_edge_features=True, **attention_args)([nk, ed, edi])
  File "/Users/tgg/miniforge3/envs/tf215/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py", line 70, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/__autograph_generated_fileli890uzx.py", line 45, in tf__call
    ag__.if_stmt(ag__.ld(self).use_edge_features, if_body, else_body, get_state, set_state, ('n_in', 'n_out'), 2)
  File "/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/__autograph_generated_fileli890uzx.py", line 35, in if_body
    n_in = ag__.converted_call(ag__.ld(self).lay_fc1, (ag__.ld(n_in),), dict(**ag__.ld(kwargs)), fscope)
  File "/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/__autograph_generated_file3ulrc2a4.py", line 43, in tf__call
    ag__.if_stmt(ag__.converted_call(ag__.ld(isinstance), (ag__.ld(inputs), ag__.ld(tf).RaggedTensor), None, fscope), if_body, else_body, get_state, set_state, ('do_return', 'retval_'), 2)
  File "/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/__autograph_generated_file3ulrc2a4.py", line 30, in if_body
    retval_ = ag__.converted_call(ag__.ld(tf).ragged.map_flat_values, (ag__.ld(self)._layer_dense, ag__.ld(inputs)), dict(**ag__.ld(kwargs)), fscope)
  File "/Users/tgg/miniforge3/envs/tf215/lib/python3.10/site-packages/kgcnn/ops/activ.py", line 69, in __init__
    self.alpha = float(alpha)
TypeError: Exception encountered when calling layer "attentive_head_fp" (type AttentiveHeadFP).

in user code:

    File "/Users/tgg/miniforge3/envs/tf215/lib/python3.10/site-packages/kgcnn/layers/conv/attentivefp_conv.py", line 93, in call  *
        n_in = self.lay_fc1(n_in, **kwargs)
    File "/Users/tgg/miniforge3/envs/tf215/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py", line 70, in error_handler  **
        raise e.with_traceback(filtered_tb) from None
    File "/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/__autograph_generated_file3ulrc2a4.py", line 43, in tf__call
        ag__.if_stmt(ag__.converted_call(ag__.ld(isinstance), (ag__.ld(inputs), ag__.ld(tf).RaggedTensor), None, fscope), if_body, else_body, get_state, set_state, ('do_return', 'retval_'), 2)
    File "/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/__autograph_generated_file3ulrc2a4.py", line 30, in if_body
        retval_ = ag__.converted_call(ag__.ld(tf).ragged.map_flat_values, (ag__.ld(self)._layer_dense, ag__.ld(inputs)), dict(**ag__.ld(kwargs)), fscope)
    File "/Users/tgg/miniforge3/envs/tf215/lib/python3.10/site-packages/kgcnn/ops/activ.py", line 69, in __init__
        self.alpha = float(alpha)

    TypeError: Exception encountered when calling layer 'dense_embedding_4' (type DenseEmbedding).
    
    in user code:
    
        File "/Users/tgg/miniforge3/envs/tf215/lib/python3.10/site-packages/kgcnn/layers/modules.py", line 85, in call  *
            return tf.ragged.map_flat_values(self._layer_dense, inputs, **kwargs)
        File "/Users/tgg/miniforge3/envs/tf215/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py", line 70, in error_handler
            raise e.with_traceback(filtered_tb) from None
        File "/Users/tgg/miniforge3/envs/tf215/lib/python3.10/site-packages/kgcnn/ops/activ.py", line 69, in __init__
            self.alpha = float(alpha)
    
        TypeError: Exception encountered when calling layer 'dense_4' (type Dense).
        
        float() argument must be a string or a real number, not 'Tensor'
        
        Call arguments received by layer 'dense_4' (type Dense):
          • inputs=tf.Tensor(shape=(None, 200), dtype=float32)
    
    
    Call arguments received by layer 'dense_embedding_4' (type DenseEmbedding):
      • inputs=tf.RaggedTensor(values=Tensor("attentive_head_fp/gather_nodes_ingoing/GatherV2_1:0", shape=(None, 200), dtype=float32), row_splits=Tensor("attentive_head_fp/gather_nodes_ingoing/RaggedFromRowLengths/RowPartitionFromRowLengths/concat:0", shape=(None,), dtype=int64))
      • kwargs={'training': 'None'}


Call arguments received by layer "attentive_head_fp" (type AttentiveHeadFP):
  • inputs=['tf.RaggedTensor(values=Tensor("Placeholder:0", shape=(None, 200), dtype=float32), row_splits=Tensor("Placeholder_1:0", shape=(None,), dtype=int64))', 'tf.RaggedTensor(values=Tensor("Placeholder_2:0", shape=(None, 11), dtype=float32), row_splits=Tensor("Placeholder_3:0", shape=(None,), dtype=int64))', 'tf.RaggedTensor(values=Tensor("Placeholder_4:0", shape=(None, 2), dtype=int64), row_splits=Tensor("Placeholder_5:0", shape=(None,), dtype=int64))']
  • kwargs={'training': 'None'}

using 2.12 =>

Successfully installed keras-2.12.0 tensorboard-2.12.3 tensorflow-estimator-2.12.0 tensorflow-macos-2.12.0
(tf212) tgg@arm64-apple-darwin20 KGCNN_test % python keras-gcnv1.py config.cfg                     
ERROR:kgcnn.mol.convert:Can not import `OpenBabel` package for conversion.
Load config file:  config.cfg
Architecture selected: AttFP
My parameters
Loss RMSE
LSTM 16
DENSE 16
PROBA 0
LR start 0.01
using cpu
before True
INFO:kgcnn.hyper.hyper:Adding 'postfix_file' to 'info' category in hyperparameter.
{'model': {'class_name': 'make_model', 'module_name': 'kgcnn.literature.AttentiveFP', 'config': {'name': 'AttentiveFP', 'inputs': [{'shape': [None, 41], 'name': 'node_attributes', 'dtype': 'float32', 'ragged': True}, {'shape': [None, 11], 'name': 'edge_attributes', 'dtype': 'float32', 'ragged': True}, {'shape': [None, 2], 'name': 'edge_indices', 'dtype': 'int64', 'ragged': True}], 'input_embedding': {'node': {'input_dim': 95, 'output_dim': 100}, 'edge': {'input_dim': 5, 'output_dim': 100}}, 'attention_args': {'units': 200}, 'depthato': 2, 'depthmol': 2, 'dropout': 0.2, 'verbose': 10, 'output_embedding': 'graph', 'output_mlp': {'use_bias': [True, True, True], 'units': [200, 100, 1], 'activation': ['kgcnn>leaky_relu', 'selu', 'linear']}}}, 'training': {'fit': {'batch_size': 200, 'epochs': 100, 'validation_freq': 1, 'verbose': 2, 'callbacks': []}, 'compile': {'optimizer': {'class_name': 'Addons>AdamW', 'config': {'lr': 0.0031622776601683794, 'weight_decay': 1e-05}}}, 'cross_validation': {'class_name': 'KFold', 'config': {'n_splits': 5, 'random_state': None, 'shuffle': True}}, 'scaler': {'class_name': 'StandardScaler', 'config': {'with_std': True, 'with_mean': True, 'copy': True}}, 'execute_folds': 1}, 'data': {'dataset': {'class_name': 'MoleculeNetDataset', 'config': {}, 'methods': []}, 'data_unit': 'mol/L'}, 'info': {'postfix': '', 'kgcnn_version': '2.0.3', 'postfix_file': ''}}
training
is overwrite True
no 3D model
INFO:kgcnn.data.MoleculeNetDataset:Found SDF train.sdf of pre-computed structures.
INFO:kgcnn.data.MoleculeNetDataset:Read molecules from mol-file.
INFO:kgcnn.data.MoleculeNetDataset: ... process molecules 0 from 24
INFO:kgcnn.mol.encoder:OneHotEncoder Symbol found ['O', 'C', 'N']
INFO:kgcnn.mol.encoder:OneHotEncoder Hybridization found [rdkit.Chem.rdchem.HybridizationType.SP2, rdkit.Chem.rdchem.HybridizationType.SP3]
INFO:kgcnn.mol.encoder:OneHotEncoder TotalDegree found [1, 3, 2, 4]
INFO:kgcnn.mol.encoder:OneHotEncoder TotalNumHs found [0, 1, 3, 2]
INFO:kgcnn.mol.encoder:OneHotEncoder CIPCode found [None]
INFO:kgcnn.mol.encoder:OneHotEncoder ChiralityPossible found [None]
INFO:kgcnn.mol.encoder:OneHotEncoder BondType found [rdkit.Chem.rdchem.BondType.DOUBLE, rdkit.Chem.rdchem.BondType.SINGLE, rdkit.Chem.rdchem.BondType.AROMATIC]
INFO:kgcnn.mol.encoder:OneHotEncoder Stereo found [rdkit.Chem.rdchem.BondStereo.STEREONONE, rdkit.Chem.rdchem.BondStereo.STEREOANY]
INFO:kgcnn.data.MoleculeNetDataset:Read molecules from mol-file.
INFO:kgcnn.data.MoleculeNetDataset: ... process molecules 0 from 24
INFO:kgcnn.mol.encoder:OneHotEncoder Symbol found ['O', 'C', 'N']
INFO:kgcnn.mol.encoder:OneHotEncoder Hybridization found [rdkit.Chem.rdchem.HybridizationType.SP2, rdkit.Chem.rdchem.HybridizationType.SP3]
INFO:kgcnn.mol.encoder:OneHotEncoder TotalDegree found [1, 3, 2, 4]
INFO:kgcnn.mol.encoder:OneHotEncoder TotalNumHs found [0, 1, 3, 2]
INFO:kgcnn.mol.encoder:OneHotEncoder CIPCode found [None]
INFO:kgcnn.mol.encoder:OneHotEncoder ChiralityPossible found [None]
INFO:kgcnn.mol.encoder:OneHotEncoder BondType found [rdkit.Chem.rdchem.BondType.DOUBLE, rdkit.Chem.rdchem.BondType.SINGLE, rdkit.Chem.rdchem.BondType.AROMATIC]
INFO:kgcnn.mol.encoder:OneHotEncoder Stereo found [rdkit.Chem.rdchem.BondStereo.STEREONONE, rdkit.Chem.rdchem.BondStereo.STEREOANY]
INFO:kgcnn.data.MoleculeNetDataset:No invalid graphs for assigned properties found.
data length:  24
{'model': {'class_name': 'make_model', 'module_name': 'kgcnn.literature.AttentiveFP', 'config': {'name': 'AttentiveFP', 'inputs': [{'shape': [None, 41], 'name': 'node_attributes', 'dtype': 'float32', 'ragged': True}, {'shape': [None, 11], 'name': 'edge_attributes', 'dtype': 'float32', 'ragged': True}, {'shape': [None, 2], 'name': 'edge_indices', 'dtype': 'int64', 'ragged': True}], 'input_embedding': {'node': {'input_dim': 95, 'output_dim': 100}, 'edge': {'input_dim': 5, 'output_dim': 100}}, 'attention_args': {'units': 200}, 'depthato': 2, 'depthmol': 2, 'dropout': 0.2, 'verbose': 10, 'output_embedding': 'graph', 'output_mlp': {'use_bias': [True, True, True], 'units': [200, 100, 1], 'activation': ['kgcnn>leaky_relu', 'selu', 'linear']}}}, 'training': {'fit': {'batch_size': 200, 'epochs': 100, 'validation_freq': 1, 'verbose': 2, 'callbacks': []}, 'compile': {'optimizer': {'class_name': 'Addons>AdamW', 'config': {'lr': 0.0031622776601683794, 'weight_decay': 1e-05}}}, 'cross_validation': {'class_name': 'KFold', 'config': {'n_splits': 5, 'random_state': None, 'shuffle': True}}, 'scaler': {'class_name': 'StandardScaler', 'config': {'with_std': True, 'with_mean': True, 'copy': True}}, 'execute_folds': 1}, 'data': {'dataset': {'class_name': 'MoleculeNetDataset', 'config': {}, 'methods': []}, 'data_unit': 'mol/L'}, 'info': {'postfix': '', 'kgcnn_version': '2.0.3', 'postfix_file': ''}}
dataset length: 24
INFO:kgcnn.model.utils:Updated model kwargs:
INFO:kgcnn.model.utils:{'name': 'AttentiveFP', 'inputs': [{'shape': [None, 41], 'name': 'node_attributes', 'dtype': 'float32', 'ragged': True}, {'shape': [None, 11], 'name': 'edge_attributes', 'dtype': 'float32', 'ragged': True}, {'shape': [None, 2], 'name': 'edge_indices', 'dtype': 'int64', 'ragged': True}], 'input_embedding': {'node': {'input_dim': 95, 'output_dim': 100}, 'edge': {'input_dim': 5, 'output_dim': 100}}, 'attention_args': {'units': 200}, 'depthmol': 2, 'depthato': 2, 'dropout': 0.2, 'verbose': 10, 'output_embedding': 'graph', 'output_to_tensor': True, 'output_mlp': {'use_bias': [True, True, True], 'units': [200, 100, 1], 'activation': ['kgcnn>leaky_relu', 'selu', 'linear']}}
WARNING:absl:At this time, the v2.11+ optimizer `tf.keras.optimizers.Adam` runs slowly on M1/M2 Macs, please use the legacy Keras optimizer instead, located at `tf.keras.optimizers.legacy.Adam`.
WARNING:absl:There is a known slowdown when using v2.11+ Keras optimizers on M1/M2 Macs. Falling back to the legacy Keras optimizer, i.e., `tf.keras.optimizers.legacy.Adam`.
Model: "AttentiveFP"
__________________________________________________________________________________________________
 Layer (type)                   Output Shape         Param #     Connected to                     
==================================================================================================
 node_attributes (InputLayer)   [(None, None, 41)]   0           []                               
                                                                                                  
 optional_input_embedding (Opti  (None, None, 41)    0           ['node_attributes[0][0]']        
 onalInputEmbedding)                                                                              
                                                                                                  
 edge_attributes (InputLayer)   [(None, None, 11)]   0           []                               
                                                                                                  
 dense_embedding (DenseEmbeddin  (None, None, 200)   8400        ['optional_input_embedding[0][0]'
 g)                                                              ]                                
                                                                                                  
 optional_input_embedding_1 (Op  (None, None, 11)    0           ['edge_attributes[0][0]']        
 tionalInputEmbedding)                                                                            
                                                                                                  
 edge_indices (InputLayer)      [(None, None, 2)]    0           []                               
                                                                                                  
 attentive_head_fp (AttentiveHe  (None, None, 200)   203200      ['dense_embedding[0][0]',        
 adFP)                                                            'optional_input_embedding_1[0][0
                                                                 ]',                              
                                                                  'edge_indices[0][0]']           
                                                                                                  
 gru_update (GRUUpdate)         (None, None, 200)    241200      ['dense_embedding[0][0]',        
                                                                  'attentive_head_fp[0][0]']      
                                                                                                  
 attentive_head_fp_1 (Attentive  (None, None, 200)   120600      ['gru_update[0][0]',             
 HeadFP)                                                          'optional_input_embedding_1[0][0
                                                                 ]',                              
                                                                  'edge_indices[0][0]']           
                                                                                                  
 gru_update_1 (GRUUpdate)       (None, None, 200)    241200      ['gru_update[0][0]',             
                                                                  'attentive_head_fp_1[0][0]']    
                                                                                                  
 dropout_embedding (DropoutEmbe  (None, None, 200)   0           ['gru_update_1[0][0]']           
 dding)                                                                                           
                                                                                                  
 pooling_nodes_attentive (Pooli  (None, 200)         281801      ['dropout_embedding[0][0]']      
 ngNodesAttentive)                                                                                
                                                                                                  
 mlp (MLP)                      (None, 1)            60401       ['pooling_nodes_attentive[0][0]']
                                                                                                  
==================================================================================================
Total params: 1,156,802
Trainable params: 1,156,802
Non-trainable params: 0
__________________________________________________________________________________________________
None
Epoch 1/5
2024-05-02 14:01:50.318792: W tensorflow/tsl/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
1/1 - 1s - loss: 1.0235 - root_mean_squared_error: 1.0235 - val_loss: 1.3885 - val_root_mean_squared_error: 1.3885 - lr: 0.0010 - 1s/epoch - 1s/step
Epoch 2/5
1/1 - 0s - loss: 1.2250 - root_mean_squared_error: 1.2250 - val_loss: 0.7972 - val_root_mean_squared_error: 0.7972 - lr: 0.0010 - 16ms/epoch - 16ms/step
Epoch 3/5
1/1 - 0s - loss: 0.6049 - root_mean_squared_error: 0.6049 - val_loss: 0.5340 - val_root_mean_squared_error: 0.5340 - lr: 0.0010 - 16ms/epoch - 16ms/step
Epoch 4/5
1/1 - 0s - loss: 0.4878 - root_mean_squared_error: 0.4878 - val_loss: 0.5237 - val_root_mean_squared_error: 0.5237 - lr: 0.0010 - 16ms/epoch - 16ms/step
Epoch 5/5
1/1 - 0s - loss: 0.3952 - root_mean_squared_error: 0.3952 - val_loss: 0.3796 - val_root_mean_squared_error: 0.3796 - lr: 0.0010 - 15ms/epoch - 15ms/step
Saved model to disk
Relax!

my current settings maybe not accurate :

    hyper = {
        "model": {
            "class_name": "make_model",
            "module_name": "kgcnn.literature.AttentiveFP",
            "config": {
                "name": "AttentiveFP",
                "inputs": [{"shape": [None, 41], "name": "node_attributes", "dtype": "float32","ragged": True},
                           {"shape": [None, 11], "name": "edge_attributes", "dtype": "float32","ragged": True},
                           {"shape": [None, 2], "name": "edge_indices", "dtype": "int64","ragged": True}],
                "input_embedding": {"node": {"input_dim": 95, "output_dim": 100},
                                    "edge": {"input_dim": 5, "output_dim": 100}},
                "attention_args": {"units": 200},
                "depthato": 2, 
                "depthmol": 2,
                "dropout": 0.2,
                "verbose": 10,
                "output_embedding": "graph",
                "output_mlp": {"use_bias": [True, True, True], "units": [200, 100, 1],
                               "activation": ["kgcnn>leaky_relu", "selu", "linear"]}
            }
        },
        "training": {
            "fit": {
                "batch_size": 200, "epochs": 100, "validation_freq": 1, "verbose": 2,
                "callbacks": []
            },
            "compile": {
                "optimizer": {"class_name": "Addons>AdamW", "config": {"lr": 0.0031622776601683794,
                                                                       "weight_decay": 1e-05}
                              }
            },
            "cross_validation": {"class_name": "KFold",
                                 "config": {"n_splits": 5, "random_state": None, "shuffle": True}},
            "scaler": {"class_name": "StandardScaler", "config": {"with_std": True, "with_mean": True, "copy": True}},
            "execute_folds": 1
        },
        "data": {
            "dataset": {
                "class_name": "MoleculeNetDataset",
                "config": {},
                "methods": [
                ]
            },
            "data_unit": "mol/L"
        },
        "info": {
            "postfix": "",
            "kgcnn_version": "2.0.3"
        }
    }