Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issues writing external flash chip with esptool. (ESPTOOL-962) #1034

Open
1 task done
jackgassett opened this issue Nov 17, 2024 · 7 comments
Open
1 task done

Issues writing external flash chip with esptool. (ESPTOOL-962) #1034

jackgassett opened this issue Nov 17, 2024 · 7 comments

Comments

@jackgassett
Copy link

jackgassett commented Nov 17, 2024

Operating System

Windows 10

Esptool Version

esptool.py v4.8.1

Python Version

Python 3.12.6

Chip Description

ESP32S3

Device Description

Esp32S3 Supermini attached to a Tang Primer 20K dev board with the SPI Flash pins routed to external pins, via verilog, which are connected to gpio pins on esp32s3 board.

Hardware Configuration

Only ground and the 4 spi pins MOSI, MISO, SCLK, and CS are connected from the FPGA to the Esp32S3.

How is Esptool Run

VSCODE IDE

Full Esptool Command Line that Was Run

esptool --port COM10 --baud 9600 write_flash 0x0000 write.bin --spi-connection 5,4,6,26,7 --flash_mode dout --flash_freq 12m

Esptool Output

$ esptool --port COM10 --baud 9600 write_flash 0x0000 write.bin --spi-connection 5,4,6,26,7 --flash_mode dout --flash_freq 12m
esptool.py v4.8.1
Serial port COM10
Connecting...
Detecting chip type... ESP32-S3
Chip is ESP32-S3 (QFN56) (revision v0.2)
Features: WiFi, BLE, Embedded Flash 4MB (XMC), Embedded PSRAM 2MB (AP_3v3)
Crystal is 40MHz
MAC: 64:e8:33:72:3a:d0
Uploading stub...
Running stub...
Stub running...
Configuring SPI flash mode (CLK:5, Q:4, D:6, HD:26, CS:7)...
Configuring flash size...
Flash will be erased from 0x00000000 to 0x00000fff...
Warning: Image file at 0x0 doesn't look like an image file, so not changing any flash settings.
Compressed 8 bytes to 16...
Wrote 8 bytes (16 compressed) at 0x00000000 in 0.0 seconds (effective 6.4 kbit/s)...
File  md5: 9c2cbeb2501b5d68f30adb5197a91e3a
Flash md5: dcd01c291740a2b2942d565da15c9b04
MD5 of 0xFF is c2cb56f4c5bf656faca0986e7eba0308

A fatal error occurred: MD5 of file does not match data in flash!

More Information

We are currently exploring the option of using an esp32s3 to replace the traditional FT2232 devices used with FPGAs to program a bitstream to the SPI Flash of the FPGA.

We noticed the option in esptool to change the SPI pins to external SPI pins. This seems like a very elegant way to be able to program both the ESP32 and the FPGA using the same tool.

We have routed the SPI Flash pins of the FPGA to GPIO pins which are connected to the GPIO pins of the ESP32. We are getting reliable reads and erases with this configuration, but unfortunately writes are not working. Every attempt to write gets the first couple of bytes correct but then there will be sections of incorrect data with sections of correct data...

We have tried to lower the frequency and use both dio and dout modes. I'm beginning to suspect that maybe the flash mode is not actually changing to dio or dout when gpio pins are specified with --spi-connection 5,4,6,26,7.

We have also tried multiple different pin locations with the same results.

Any ideas or help is greatly appreciated.

Other Steps to Reproduce

No response

I Have Read the Troubleshooting Guide

  • I confirm I have read the troubleshooting guide.
@github-actions github-actions bot changed the title Issues writing external flash chip with esptool. Issues writing external flash chip with esptool. (ESPTOOL-962) Nov 17, 2024
@jackgassett
Copy link
Author

Update: Have also tried it with --no-compress with the same results:

$ esptool --port COM10 --baud 9600 write_flash 0x0000 write.bin --no-compress --spi-connection 5,4,6,26,7 --flash_mode dout --flash_freq 12m
esptool.py v4.8.1
Serial port COM10
Connecting...
Detecting chip type... ESP32-S3
Chip is ESP32-S3 (QFN56) (revision v0.2)
Features: WiFi, BLE, Embedded Flash 4MB (XMC), Embedded PSRAM 2MB (AP_3v3)
Crystal is 40MHz
MAC: 64:e8:33:72:3a:d0
Uploading stub...
Running stub...
Stub running...
Configuring SPI flash mode (CLK:5, Q:4, D:6, HD:26, CS:7)...
Configuring flash size...
Flash will be erased from 0x00000000 to 0x00000fff...
Warning: Image file at 0x0 doesn't look like an image file, so not changing any flash settings.
Wrote 16384 bytes at 0x00000000 in 0.1 seconds (1053.6 kbit/s)...
File md5: 9c2cbeb2501b5d68f30adb5197a91e3a
Flash md5: dcd01c291740a2b2942d565da15c9b04
MD5 of 0xFF is c2cb56f4c5bf656faca0986e7eba0308

A fatal error occurred: MD5 of file does not match data in flash!

@jackgassett
Copy link
Author

With a little more testing it looks like maybe flash_mode and flash_freq are not taking affect. The amount of time and the kbits/s with and without the options are too close. I would expect the run with --flash_mode dout --flash_freq 12m to take much more time, but both are taking around 4.3 seconds. Here are the results:

Without flash_mode and flash_freq:
`$ esptool --port COM10 write_flash 0x0000 bitfile.bin --no-compress --spi-connection 10,11,9,26,8
esptool.py v4.8.1
Serial port COM10
Connecting...
Detecting chip type... ESP32-S3
Chip is ESP32-S3 (QFN56) (revision v0.2)
Features: WiFi, BLE, Embedded Flash 4MB (XMC), Embedded PSRAM 2MB (AP_3v3)
Crystal is 40MHz
MAC: 64:e8:33:72:3a:d0
Uploading stub...
Running stub...
Stub running...
Configuring SPI flash mode (CLK:10, Q:11, D:9, HD:26, CS:8)...
Configuring flash size...
Flash will be erased from 0x00000000 to 0x0008cfff...
Wrote 589824 bytes at 0x00000000 in 4.3 seconds (1086.3 kbit/s)...
File md5: 9492a3d37da4b9cef57bc109602104eb
Flash md5: cbfb29229cd7bb6fd4ffedab0e288490
MD5 of 0xFF is 2645bf14d83cd5d7af79e7aa8aa44225

A fatal error occurred: MD5 of file does not match data in flash!`

With flash_mode and flash_freq:
`$ esptool --port COM10 --baud 115200 write_flash 0x0000 bitfile.bin --no-compress --spi-connection 10,11,9,26,8 --flash_mode dout --flash_freq 12m
esptool.py v4.8.1
Serial port COM10
Connecting...
Detecting chip type... ESP32-S3
Chip is ESP32-S3 (QFN56) (revision v0.2)
Features: WiFi, BLE, Embedded Flash 4MB (XMC), Embedded PSRAM 2MB (AP_3v3)
Crystal is 40MHz
MAC: 64:e8:33:72:3a:d0
Uploading stub...
Running stub...
Stub running...
Configuring SPI flash mode (CLK:10, Q:11, D:9, HD:26, CS:8)...
Configuring flash size...
Flash will be erased from 0x00000000 to 0x0008cfff...
Warning: Image file at 0x0 doesn't look like an image file, so not changing any flash settings.
Wrote 589824 bytes at 0x00000000 in 4.3 seconds (1106.6 kbit/s)...
File md5: 9492a3d37da4b9cef57bc109602104eb
Flash md5: a33baa47802dff3c72e879d03f136ee6
MD5 of 0xFF is 2645bf14d83cd5d7af79e7aa8aa44225

A fatal error occurred: MD5 of file does not match data in flash!`

@Dzarda7
Copy link
Contributor

Dzarda7 commented Nov 17, 2024

Hi @jackgassett interesting usecase you have here. We will take a look at it but I am not really sure if this will be possible. I took a quick look and noticed that you are right, flash_mode and flash_freq are not taking an effect, as you can see in the following line:

Warning: Image file at 0x0 doesn't look like an image file, so not changing any flash settings.

Here is the line that checks if the image is correct. There are some other checks that image file needs to pass to be able to change these parameters. You can try to install esptool in editable mode (pip install -e . in the esptool folder), comment out the checks and flash it to see if this is the only problem.

@jackgassett
Copy link
Author

jackgassett commented Nov 17, 2024

I commented out those checks and it says it is passing the flash parameters, but I'm not seeing a slowdown, it's still taking around 4 seconds at ~1200 kbits/s.. It also looks like if I write to address 0x2000 it skips the image check. Here is with the checks commented out:

$ esptool/esptool.py --port COM10 --baud 9600 write_flash 0x0000 bitfile.bin --no-compress --spi-connection 10,11,9,26,8 --flash_mode dout --flash_freq 20m
esptool.py v4.8.1
Serial port COM10
Connecting...
Detecting chip type... ESP32-S3
Chip is ESP32-S3 (QFN56) (revision v0.2)
Features: WiFi, BLE, Embedded Flash 4MB (XMC), Embedded PSRAM 2MB (AP_3v3)
Crystal is 40MHz
MAC: 64:e8:33:72:3a:d0
Uploading stub...
Running stub...
Stub running...
Configuring SPI flash mode (CLK:10, Q:11, D:9, HD:26, CS:8)...
Configuring flash size...
Flash will be erased from 0x00000000 to 0x0008cfff...
Skipping image checks, Jack
Flash params set to 0x03f2
Wrote 589824 bytes at 0x00000000 in 4.2 seconds (1111.8 kbit/s)...
File md5: b69c247e7161e02a08b8cb2748674f20
Flash md5: 11326fce3b2ff562bb6851202cfb81c5
MD5 of 0xFF is 2645bf14d83cd5d7af79e7aa8aa44225

A fatal error occurred: MD5 of file does not match data in flash!

Here is a comparison of what the bitfile looks like compared to what is read back. It is close, seems like it just needs to be slowed down...
image

@Dzarda7
Copy link
Contributor

Dzarda7 commented Nov 18, 2024

As you can see from the documentation, the parameters are taken into an account only when address is set to bootloader address (it is 0x0 in case of ESP32S3). You can also try enable --trace option to see what is really written and compare it.

@jackgassett
Copy link
Author

jackgassett commented Nov 20, 2024

Yes, that makes sense, it looks like I need to generate a valid header with the flash_mode and flash_freq in the image header. Writing to locations other than 0x00 and 0x1000 bypass the image header and so probably never set flash_mode and flash_freq.

In the esptool.py code I see where it is updating the flash_mode and flash_freq in the header for the image. But I never see a command sent over the serial port to the bootloader to set flash_mode and flash_freq. With the SPI pins there is the spi_attach (if I remember the name right) function that sends a command and the pins to the bootloader. There is another function that sets flash parameters but it does not include flash_mode and flash_freq.

So I'm guessing that flash_mode and flash_freq get put into the image header, sent over the serial line to the bootloader on the esp32s3 with the image, and then the bootloader unpacks the headers and applies those settings. I'm trying to find the code where those parameters are set in the bootloader, I've looked through the stub code but don't see it there. So I'm guessing it must be in the bootloader code that runs out of the ROM. Is that code published anywhere or is it closed? Or am I missing it in the stub somewhere? Any push in the right direction is appreciated.

Thanks!
Jack.

@Dzarda7
Copy link
Contributor

Dzarda7 commented Nov 21, 2024

You are right. When I proposed commenting out the checks it did apply the settings to the chip probably, because of the missing header (for which the checks are in esptool).

The code that takes care of the header is not in stub, it is present in ROM code, which is not public and even I do not have access to it now unfortunately.

But esptool is great source of these information. As I said, with esp32s3, you need to flash at 0x00 address. Then I would try to add the header. It is specified here. Flash mode and flash frequency can be either put into the header or rewritten using esptool. One thing I noticed unfortunately for you is that I think that the header is also written into the flash.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants