We will open source our code as soon as possible, likely within a week, as the main contributors of FoPE are currently in their exam period.
Our code is totally based on OLMo(https://github.com/allenai/OLMo), which already provides the training and data preprocessing code. The core code of FoPE and main setup is available in the appendix of our paper. If you'd like to reproduce the results immediately, you can also integrate FoPE directly into the OLMo repository.