How to train ATM_S_reconstruction_scale_0_1000

[thebigeagle]

Congratulations on your achievement. Thank you for making the code public.
But I have some questions about the code. How do we train the ATM_S_reconstruction_scale_0_1000 model and save the parameters? I didn't see the relevant process in the readme. I only saw the process of downloading the model and it didn't guide me on how to train it myself.

[dongyangli-del]

To generate the pre-trained weights for subject 8, you'll indeed want to run the ATMS_retrieval.py script in the retrieval directory. This script is designed to train the model using the data from all subjects, which includes subject 8.

The weights you're referring to (ATM_S_reconstruction_scale_0_1000, ATM_S_eeg_features_sub-08, ATM_S_eeg_features_sub08_test) are part of the process. Here's a bit more about them:

• ATM_S_reconstruction_scale_0_1000: These are the weights used for testing the model on the time range of 0 to 1000ms.
• ATM_S_eeg_features_sub-08: This is the EEG features file for the training set of subject 8.
• ATM_S_eeg_features_sub08_test: This is the EEG features file for the test set of subject 8.

To evaluate the model performance, you can use the Generation_metrics_sub8.ipynb as mentioned in the README. For a more comprehensive evaluation, including both low-level and high-level metrics, I recommend running the Reconstruction_Metrics_ATM.ipynb as well.

I've also updated all the EEG features on Hugging Face, so you can directly use them in the Generation directory without needing to

[thebigeagle]

Thank you very much for your reply, I've been able to run the code under the Retrieval file correctly so far, but I'm still having a lot of problems running the code under Generation.

1. when I run ‘Generation_metrics_sub8.ipynb’ I still don't know how the ATM_S_reconstruction_scale_0_1000 model is trained, so I changed the model to ATMS.
2. ‘from image_generate import *’ I didn't find the image_generate file, and a lot of the code under Generation file has ‘import’ error. Some of them are importing files that aren't there, and some of them need to import files under Retrieval. 3.
3. I didn't find how to get ‘Generation/generated_imgs’.
4. I followed the README steps to run the above code, and many of the steps don't work, so I'm not sure what the problem is.
   Thank you very much for taking the time to answer my questions!

[dongyangli-del]

Thank you for reaching out and sharing the issues you've encountered while working with the codebase. I apologize for the inconvenience caused.

Regarding the model training: It appears there was some confusion regarding the 'ATM_S_reconstruction_scale_0_1000' model. Your decision to switch to the 'ATMS' model was a good workaround. We've since updated our documentation to clarify the model training process.

Missing 'image_generate' file: We don't have to use this package. We've identified and resolved the import errors that were causing issues with the code execution. Some of these errors were due to missing files, while others were due to incorrect paths. We've corrected the paths and ensured that all necessary files are present.

Generation/generated_imgs directory: This directory is created dynamically when you run the 'Generation_metrics_sub8.ipynb' notebook. If it's not appearing, it could be due to a path issue or a problem with the notebook's execution. Ensure that the notebook is running in the correct environment and that the path to the 'generated_imgs' directory is correctly specified.

README file updates: We've made updates to the README file to address the steps that were not working as expected. Please ensure that you're following the latest version of the README by forking the entire code repository. This will help you keep your local copy up-to-date with our continuous updates.

[thebigeagle]

Thank you very much for your prompt reply, I have successfully run the code and generated the image. However, I still encounter many problems when I run Reconstruction_Metrics_ATM.ipynb. This code is supposed to be for the NSD dataset and does not run the EEG dataset. And there is no result of a successful run in the ipynb file. Can you update this file?

[dongyangli-del]

I've made some adjustments to the Reconstruction_Metrics_ATM.ipynb file. Currently, it only retains the code cells necessary for evaluating the EEG reconstruction performance. Some of the less critical cells, such as visualizations, may fail due to indexing issues. You can skip those parts and proceed to run the subsequent code blocks directly.

[thebigeagle]

However, there are still many problems in the code.

1. Generated_image has 200*10 images, while test_images has only 200 images. So I'm just using the first of each category of generated images when calculating the metrics.
2. When using ‘Image’ to read the image, the image format is BHWC, so the image dimension needs to be changed to BCHW.
3. There are some other minor errors in the code.
4. When calculating the metrics, the value of SwAV is abnormal, only 0.074198.