Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pico-sdk only rp2040 implementation #472

Open
bog-dan-ro opened this issue Feb 22, 2024 · 18 comments
Open

pico-sdk only rp2040 implementation #472

bog-dan-ro opened this issue Feb 22, 2024 · 18 comments

Comments

@bog-dan-ro
Copy link

Hello there,

I'd like to add support for rp2040 but based only on pico-sdk, I'd love to upstream it, so I'd like to ask if such a contribution is something that the maintainer(s) will accept.

Why pico-sdk only? Mostly because the arduino-pico can't be used with cmake projects .

@greiman
Copy link
Owner

greiman commented Feb 22, 2024

I have no interest in supporting SdFat on new platforms. I will not being doing much much new development of SdFat.

A file system should be integrated into the operating system not a library like SdFat. When I started SdFat in 2008 for an Arduino with 1 KB of RAM, I assumed it would be replaced with a proper filesystem.

The pico-sdk should have an integrated file system with SD being one of the file devices.

It would be fairly easy to port SdFat to pico-sdk. I use SdFat on the ChibiOS/RT RTOS. You need to replace a couple of calls to millis(), digitalWrite(), and pinMode().

You need a wrapper for SPI like in this example.

I have a very fast pio SDIO driver for rp2040 that can do over 20 MB/sec. I will post it as an example eventually.

@bog-dan-ro
Copy link
Author

Thanks a lot for your quick reply!

have no interest in supporting SdFat on new platforms. I will not being doing much much new development of SdFat.

In this case I'll not waste time to try to isolate the Arduino stuff and I'll just remove it or replace it with pico-sdk.

The pico-sdk should have an integrated file system with SD being one of the file devices.

It should, but ... Right now I'm using https://github.com/carlk3/no-OS-FatFS-SD-SDIO-SPI-RPi-Pico but is buggy.

Thanks a lot for the link. Regarding PIO SDIO implementation, how big is it? I'm asking as I might not be able to use it as I'm a bit short on PIO space (on both of PIOs :) ).

@bog-dan-ro
Copy link
Author

@greiman did you manage to post the pio SDIO driver?
Thanks in advance :) !

@greiman
Copy link
Owner

greiman commented Mar 10, 2024

I looked at the SDIO driver but became busy with other things.

You probably can't use it since it uses almost all memory in one PIO. I implemented it to be fast for small transfers like the SdFat dedicated SPI mode but that takes more memory.

I also need to decide whether to do more development. There are lots of possible configuration options so I need to make it easy to choose the pins and speed options. This can't be done at run time since it must be compiled into the PIO code.

I also need to decide whether to develop a second mode. SD cards start in the very old "Standard mode" which was limited to 25 MHz. Modern cards can run much faster in that mode. I find all cards I have tested can run at 62.6 MHz with 250 MHz CPU speed.

Implementing "High Speed" mode is a big job since it has totally different bus timing but the SD start in "Standard" mode. So a reload of the PIO code is required during the initialization process. 25 years ago 50 MHz bus speed required an unusual timing.

I also don't want to increase the amount time I spend on maintenance of SdFat. I don't enjoy development on the Arduino platform. I had high hope for Arduino15 years ago when I released the first version of SdFat. I now don't use Adruino for my projects.

@bog-dan-ro
Copy link
Author

First and foremost, I'd like to say thanks a lot Bill for all your effort on this project. As a FOSS contributor myself, I know it is hard to maintain a project that is used by so many people even if we like it or not ;-).

I managed to free one of the PIOs, next I'll try to see if I can adapt ZuluSCSI's SDIO implementation. Is your implementation much different?

@greiman
Copy link
Owner

greiman commented Mar 11, 2024

Is your implementation much different?

My implementation is very different. I started from scratch, I didn't copy any existing code.

Modern SD cards require vary large transfers for maximum speed. I do this for both read and write by keeping the SD in read or write mode as long as possible. To do this I stop the SDIO clock between 512 byte transfers.

Here is a typical result of the bench example for 512 byte reads and writes. This is overclocked at 250 MHz CPU.

This is on an Adafruit Metro RP2040. I also test on a Pico.

Type is exFAT
Card size: 128.18 GB (GB = 1E9 bytes)

Manufacturer ID: 0X1B
OEM ID: SM
Product: YD4QD
Revision: 3.0
Serial number: 0XCBDD66A0
Manufacturing date: 9/2023

FILE_SIZE_MB = 50
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
24366.41,48,19,20
24366.41,37,19,20

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
24402.08,77,20,20
24402.08,169,20,20

It is slower for smaller transfers. In this case I can't transfer directly to the user buffer and must cache the 512 byte block and do an extra copy using the CPU and memcpy.

FILE_SIZE_MB = 50
BUF_SIZE = 64 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
13488.00,34,1,4
13484.36,34,1,4

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
14425.85,75,1,4
14421.69,204,1,4

@greiman
Copy link
Owner

greiman commented Mar 11, 2024

I looked at the current state of my RP2040 SDIO. The source needs to be cleaned up since I was still playing with different approaches.

I looked at the High Speed mode again. Card output valid timing has a long delay from the rising edge of clock. I might try it again at some point.

Here is High Speed mode timing:

SdioHighSpeed

I have a 350 MHz mixed signal scope and a fast logic analyzer so I can debug the PIO.

@androda
Copy link

androda commented Aug 2, 2024

You probably can't use it since it uses almost all memory in one PIO. I implemented it to be fast for small transfers like the SdFat dedicated SPI mode but that takes more memory.

Existing RP2040 SDIO systems like the aforementioned https://github.com/carlk3/no-OS-FatFS-SD-SDIO-SPI-RPi-Pico already use most (or all) of the instruction memory in one of the PIO units. It's not a downside for yours to use most of the instruction memory, rather it seems to be the status quo. :)

@greiman
Copy link
Owner

greiman commented Aug 3, 2024

@androda

I had hoped to post a version of SdFat with RP2040 SDIO some time ago but have been very busy and away from home so not had time to finish testing all the mods in the new version.

@androda
Copy link

androda commented Aug 15, 2024

@greiman
If you'd like another set of eyes on it, let me know. I have an Agilent logic analyzer frame with 16717A card which ought to be more than up to the task of checking SD waveforms.

@greiman
Copy link
Owner

greiman commented Aug 15, 2024

Here is a test version of SdFat with RP2040 SDIO. Try it and let me know the result.

SdFatRp2040.zip

It is setup with two examples, Rp2040SdioSetup and bench. I test with an Adafruit RP2040 Metro and a Pi Pico.

I use the AdaFruit PiCowBell on Pico. The blue proto board has pins for a logic analyzer.

PioDbg

I tested at 133 MHz and 250 MHz. Here is my results for the bench example:

At 133MHz:

Type is exFAT
Card size: 128.18 GB (GB = 1E9 bytes)

Manufacturer ID: 0X1B
OEM ID: SM
Product: YD4QD
Revision: 3.0
Serial number: 0XF024666C
Manufacturing date: 7/2023

FILE_SIZE_MB = 5
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
12819.69,2005,38,38
12657.42,2414,38,38

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
12593.65,88,39,39
12562.01,83,39,39

At 250 MHz:

Type is exFAT
Card size: 128.18 GB (GB = 1E9 bytes)

Manufacturer ID: 0X1B
OEM ID: SM
Product: YD4QD
Revision: 3.0
Serial number: 0XF024666C
Manufacturing date: 7/2023

FILE_SIZE_MB = 5
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
23472.68,1181,20,21
23472.68,1220,20,21

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
23808.00,45,20,21
23808.00,49,20,21

@greiman
Copy link
Owner

greiman commented Aug 16, 2024

@androda

Have you looked at boards like the Pico 2 with the new RP2350 family of chips?

I have become bored with most new MCU boards but Raspberry PI has another interesting processor.

@androda
Copy link

androda commented Aug 19, 2024

@greiman
Yes, pre-ordered some of the Pico 2. I'm also excited about the RP2350B, given the additional IO pins. We should see some really interesting projects with this new chip.

@androda
Copy link

androda commented Aug 19, 2024

Got things set up this morning for building the RP2040 library, here are the first results (no clock speed changes yet, just compiled as-is and run).

FreeStack: 253848
Type is exFAT
Card size: 31.91 GB (GB = 1E9 bytes)

Manufacturer ID: 0X3
OEM ID: SD
Product: WX32G
Revision: 8.0
Serial number: 0X830F12B8
Manufacturing date: 3/2023

FILE_SIZE_MB = 5
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
12406.15,3134,37,39
12164.67,6934,38,39

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
12593.65,84,39,39
12562.01,68,39,39

Done
Type any character to start

edit: Results at a 500MB file are nearly the same

FILE_SIZE_MB = 500
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
12348.42,47941,37,40
12280.78,52137,37,41

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
12591.28,55,39,39
12591.28,66,39,39

edit the 2nd (son of edit):
At 250MHz my speeds aren't quite as high as yours but this is a WD Purple card, not really "made for speed".

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
22192.62,46673,20,22
21952.92,51420,20,22

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
23771.03,30,21,21
23771.03,28,21,21

edit the third (son of edit returns!):
These tests were executed on a BlueSCSI V2 PCB.

@greiman
Copy link
Owner

greiman commented Aug 19, 2024

Edited: had ms where μs should have been.

Write is often slower for big writes since there will be occasional large latencies while the card programs flash.

Your card has max latencies of about 50 ms. For 133 MHz the average latency was 3-4 μs greater than the min latency.

For the card I used with a 500 MB file, the max latency is 8-10 ms and average latency is one μs greater than the min.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
12961.08,10603,37,38
12952.69,8179,37,38

For some apps occasion large write latency is very undesirable.

Here is my card for 500 MB at 250 MHz. Max of 8 ms write latency.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
24273.01,8285,20,20
24271.83,8275,20,20

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
23776.68,85,20,21
23774.42,197,20,21

@greiman
Copy link
Owner

greiman commented Sep 10, 2024

I did first tests with a Pico 2. I need to do some mods so the third PIO can be used and there are problems with some cards at 250 MHz.

Here are two tests with a Lexar Silver Plus at 250 MHz.

512 byte buffer with a 5 MB file:

FreeStack: 515984
Type is exFAT
Card size: 125.67 GB (GB = 1E9 bytes)

Manufacturer ID: 0XAD
OEM ID: LS
Product: LX128
Revision: 2.1
Serial number: 0X7009FC3A
Manufacturing date: 4/2024

FILE_SIZE_MB = 5
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
26453.33,31,19,19
26314.11,22,19,19

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
26314.11,243,19,19
26314.11,216,19,19

32,768 byte buffer with a 50 MB file.

FreeStack: 483216
Type is exFAT
Card size: 125.67 GB (GB = 1E9 bytes)

Manufacturer ID: 0XAD
OEM ID: LS
Product: LX128
Revision: 2.1
Serial number: 0X7009FC3A
Manufacturing date: 4/2024

FILE_SIZE_MB = 50
BUF_SIZE = 32768 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
27963.74,1179,1170,1171
27792.66,11351,1170,1177

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
27381.48,1208,1194,1196
27396.49,1198,1194,1196

@DatanoiseTV
Copy link

DatanoiseTV commented Sep 18, 2024

I did first tests with a Pico 2. I need to do some mods so the third PIO can be used and there are problems with some cards at 250 MHz.

Here are two tests with a Lexar Silver Plus at 250 MHz.

512 byte buffer with a 5 MB file:

FreeStack: 515984
Type is exFAT
Card size: 125.67 GB (GB = 1E9 bytes)
Manufacturer ID: 0XAD
OEM ID: LS
Product: LX128
Revision: 2.1
Serial number: 0X7009FC3A
Manufacturing date: 4/2024
FILE_SIZE_MB = 5
BUF_SIZE = 512 bytes
Starting write test, please wait.
write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
26453.33,31,19,19
26314.11,22,19,19
Starting read test, please wait.
read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
26314.11,243,19,19
26314.11,216,19,19

32,768 byte buffer with a 50 MB file.

FreeStack: 483216
Type is exFAT
Card size: 125.67 GB (GB = 1E9 bytes)
Manufacturer ID: 0XAD
OEM ID: LS
Product: LX128
Revision: 2.1
Serial number: 0X7009FC3A
Manufacturing date: 4/2024
FILE_SIZE_MB = 50
BUF_SIZE = 32768 bytes
Starting write test, please wait.
write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
27963.74,1179,1170,1171
27792.66,11351,1170,1177
Starting read test, please wait.
read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
27381.48,1208,1194,1196
27396.49,1198,1194,1196

Does it mean it would use all 3 PIOs or 3rd PIO? I am using 1 PIO for I2S Audio on the PicoADK v2, so it would be good to have 1-2 spare PIOs. In the best case, I would like to stream samples from SDXC while using I2S for audio output.

@greiman
Copy link
Owner

greiman commented Sep 19, 2024

Does it mean it would use all 3 PIOs or 3rd PIO?

It uses one PIO. It looks for a PIO with almost all memory available. Then it claims two state machines.

See static bool pioInit() at line 260 of SdFat/src/SdCard/Rp2040Sdio/PioSdioCard.cpp.

Currently 27 of 32 memory location are used in the PIO. I may need some of the free 5 locations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants