-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cannot download any file from local FS but preferences.json #10
Comments
There are several issues at play here. When removing the commenting and fixing some related issues things improves. Saving files from Flash still does not work, due to However, saving from the maintenance page works: Also the dark theme does nothing even if it is loaded, due to it beeing gzipped? The purple theme works as does the dark theme if I download it (from the controller to be sure it has not been corrupted), unzip and upload it again. |
Thanks, the header was missing - I need to make the header generation more generic... Fixed for plain .gz files.
Upload of from files panel after you have added HostPath.
Ready now - also from the Web Builder. |
Ok I tested :
Everything seems Ok now If ok on your side too I will update repository |
Downloads of .gz files from the FS File Panel saves the uncompressed data, from the maintenance page the compressed data. |
it is because you apply header gzip I guess |
I did not check maintenance page yet, will do once fs panel is ok |
Is the behaviour the same with your backend? |
I am out today, I will confirm you later |
Yes on esp3d everything is working as expected click on any .gz does not expand them but download them
I cannot check your maintenance page/embedded page source because I did not found where it is sorry- but on mine, indeed I do not use the |
What are the response headers when you download a .gz file? Here is what I return for index.html.gz:
Another odd issue is that downloading preferences.json behaves different as well, from the maintenance page it opens it in a new browser window, in the Files Panel it downloads it. The only difference I can see is that the maintenance page does not add the Could it be browsers behaving differently?
It is here, compressed. I cannot remember changing the source in any way - and it obviously handles the path to the files correctly. If not using |
if the called url is with : xxxxx.gz if the called url is without gz : xxxxx
No, I just forget to handled it, that is all ..., I focused on webui and forget the embedded, I have just added the support today (luc-github/ESP3D@247e848d2c75bb2a51f773e5a50c343fe61bb7e2)- I was struggling when testing on grblHAL today because I randomly got I though it was linked to version but even using same I got same behavior - to get proper output I first need to I suggest you to copy this https://github.com/luc-github/ESP3D/tree/3.0/embedded to your webUI plugin repository and do the links changes according grblHALs for reference as I still not not have unified version yet |
But I am little bit confused as upload was working before introducing the fix |
Is as this stated in the relevant RFC? Prehaps this: "If the media type includes an inherent encoding, such as a data format that is always compressed, then that encoding would not be restated in Content-Encoding even if it happens to be the same algorithm as one of the content codings." or is it just something that works with the ESP webserver? However - it does not explain why the Files Panel and maintenance page behaves differently with the current code. The only difference is the cachebusting parameter, the base request URI is the same and the response headers too... Confusing... |
Yes it is more like, it was suggested since I started esp8266 dev to save space on FS to serve files (https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266WebServer/examples/FSBrowser/extras/index.htm.gz) very old trick now also used on esp32 About different behavior: |
You were right - when a file is requested without the .gz extension and is only found with it it is the same as compression has been applied and the "If one or more encodings have been applied to a representation, the sender that applied the encodings MUST generate a Content-Encoding header field that lists the content codings in the order in which they were applied." I'll soon commit a fix for this.
This is a bug on my side - I set HostPath to the directory where index.html.gz is found but I did not do that when the maintenance page was loaded. The remaining difference in behaviour between the maintenance page and the Files Panel is IMO minor and can be ignored. |
I am always impressed by your ability to find answers in RFC 👍 Not related but is ssdp already working ? I have enabled it but windows 11 neither android sspd scanner found grblHAL ssdp messages |
Since I have been involved in standardization work I am keen to follow standards where possible. Not that I always understand the wording used - that can be hard sometimes...
Did you enable SSDP in Network Services? |
…preferences.json. Updated to latest maintenance page version.
Yes I do I can see my esp3d-tft but not grblHAL, I initially though |
Send
I am on Win10 and 7 it works with both. |
Sorry I was out all day :
Also checked not visible under windows 10 Home edition |
I have tested latest grbHAL and indeed all works now as expected - thank you I agree that embedded page opening known content type is a minor issue. I will try to solve it later without increasing the footprint About ssdp issue I think it is better to track in another ticket for readibility - let me know if you want me to do more test I close issue as fixed - thank you |
I'll come back to this - I have to get hold of a Win11 machine for testing. |
You have a www folder with index.html.gz in it on the SD card, this sets There is a minor bug in grblHAL where saving to flash does not work from the maintenance page when index.html.gz is not found in flash file system nor in the /www folder on the SD card and |
@terjeio I tried with WEBUI_INFLASH=1 and no www folder in the SD card and the 12kB index.html.gz from your repo goes to the maintenance page and is unable to upload the 84kB index.html.gz WebUI3.0. Next attempt with WEBUI_INFLASH=1 (and no www folder on SD card). I replaced the12kB index.html.gz in the embedded folder with 84kB index.html.gz WebUI3.0. This enabled WebUI 3.0 but no settings could be saved and it returned a preferences.json file missing error. Then finally I tried WEBUI_INFLASH=0 and added 84kb index.html.gz WebUI3.0 to the www folder on SD card. I also added a preferences.json file to that directory. Likewise it enabled WebUI 3.0 but this time without the preferences,json error, and yet again no settings could be saved. For good hygiene I run a clean and erase_flash command from ESP-IDF. It seems the ?forcefallback=yes does enable writing to Flash so the main problem is fixed. I prefer WEBUI_INFLASH=1 in any case and since I can now write to flash there is no need to use the SD card. I have a couple of other things to report...
If it is at all helpful, I tried MKS DLC32 on their WebUI (which is a custom version of Luc's with a frame button for laser gCode) Thank you. |
I have a fix pending for this bug.
The embedded filing system is read-only so saving is not possible, I could try saving to the root of the SD-card.
The Emergency Stop button sends a safety door triggered command (0x84), I'll have to check the handling of this.
I have a fix pending fo this too.
Uploading a 4.66 MB file takes 30 seconds to a no-brand SD card with my test board. Bad wiring on yours is somehow blocking update to a higher SPI speed from the one used to initialize the card?
Is this the same board you are running grblHAL on? |
@terjeio ?forcefallback=yes enables writing to the Flash filesystem. Is writing to flash this way expected? I am not running grblHAL on the DLC32. I am using a WROOM ESP32 dev board with a MicroSD wired in. All pins are pulled up to 3.3V with a 10k resistor. I suppose MISO and MOSI pins could be mismatched a little on length. |
Yes, when index.html.gz is found in the flash file system you have to add it to open the maintenance page.
Using long wires? Reducing the SPI clock frequency may help - but better to fix any underlying issues.
|
@terjeio I tried matching MISO/MISO again and got the same result. The wire length is approximately 75mm (less than 100mm). I could try higher valued pullup resistors, though I believe the driver should not have an issue with 10k. Also, the filter caps are placed near the 3.3V bus directly next to the SD card connector. Unless it is some inherent issue with the WROOM Dev board, I'm a bit at a loss. But I will try a few more things since you are getting much better results. Otherwise I'm very happy with WebUI 3.0 as SD writes are dependable (but still slow). Also I am using 2209s in UART mode and when I add
|
Yes, and you have to add your board define here as well. BTW I can clear the door state in the WebUI by clicking the soft reset button. |
Yes that works!
Yes, I'm aware of needing to add the board to trinamic_if.c and of course 2209 must be turned on in cMakeLists.txt
One odd thing about this error is that the line number is 2235, but the board_init function is not called until about 39 lines later and it is a linking error. 2273 #ifdef HAS_BOARD_INIT Using this same setup 2209 had been working in GRBL_BUILD 20221009 but I get this error now in GRBL_BUILD 20230411 |
@terjeio please ignore the HAS_BOARD_INIT error. After trying to double check the HAS_BOARD_INIT error on another PC I am no longer having errors during linking. I'm not sure why it was happening. I am still trying more things to speed up SD card. I will try wiring with coax to debug, but not sure why reads are so fast and writes are so slow. Thank you! |
I see that you have a pullup on the CS line and not the CLK line - the opposite is correct? At least that is what I have on my test board and on a separate commercial breakout I use for the RP2040. |
Hi @terjeio from all I've researched pullup on CLK should not be required, but I will definitely try it. It's really good that WebUI 3 is reading all the grblHAL settings, I sometimes have an issue where I have to power cycle the board because some of the settings are not parsed properly. The error is "parsing data error, data are incorrect" This should be Enable (not Disable) laser during hold Also please note some of the units are annotated in the comments incorrectly in the config.h file. Current is millamps and acceleration is mm/s^2 (not mm/min^2). Previously in the config file units for accel were mm/min^2 and we had to multiply accel6060 in the config file even though $12x has always been reported as mm/s. Thank you! |
Which ones? Note that some settings are dependent on others - if processed out of sequence errors may occur.
The label is correct but not the underlying functionality/default value. Fixed in the latest commit.
Fixed in the latest commit. @troth530 Do you have a Win11 computer and can test SSDP? @luc-github I have tested the ssdpscan app you linked above and it works for me, both an ESP32 and a RP2040 controller shows up correctly. Odd that the ESP32 controller does not for you. |
let me know if you want any test/debug from me |
Hi @terjeio, I do not know which settings are and are not being parsed correctly. I only get a blank screen when the settings are not parsed correctly (see below). If preferences.json is out of sync due to changes to $$ settings through serial could it be that the data cannot be parsed? Maybe full $$ grbl settings should be read first, then parse the grblHAL specific ones. There is a typo in grbl/settings.c (double D and needs comma at the end of the line) The behavior of $63 is now correct, it seems you corrected some of the logic as well. However, stopping the job while running does not spin the spindle down, but it will disable the Spindle_Enable pin. Also, after stopping the job, M5 will not spin the spindle down either, it seems a M4 is required then M5 will spin the spindle down once the job is stopped. I've confirmed this in several senders including IOSender. Pause and Resume work as expected though. I am using laser mode by the way. $14x (motor current) has a typo in the units in comments of config.h, should be milliamps (not Amps). $73=3 seems a little less useful now. Previously if it did not connect to a 2.4GHz router in station mode it would revert back to AP mode and when you connected to the AP you had the full WebUI (though 192.168.5.1). Now if it does not connect to the router in station mode, connecting to the AP will enable an AP connect page which allows you to only change $74 and $75 and no WebUI. So really $73=3 enables only station mode or station mode through AP connect page, but does not allow AP mode. Yes, I can test SSDP with a Win11 computer, but I am still debugging a few more things. First of all I have tried pullups on CLK but no improvement. Here is the MKS DLC 2.1 schematic as a reference and speed was good, no pullups on CLK, a pulldown on MISO otherwise the same schematic as I am using. Is there a specific way to format the SD cards that may improve. I can always have a prototype board, but not sure how to proceed. So far I continue to find very slow uploads and very fast downloads with the SD card. |
@troth530 you can do a |
For some reason the window is blank. Thank you! |
This could be due to the controller crashing and resetting itself? grblHAL supports both WebUI v2 and 3, either at the same time or a specific version. Local builds enable both at the same time and which version to activate is determined from the content of the [ESP800] request sent when the WebUI is loaded. If the controller was reset then a page reload should bring the settings back. Connecting to the controller with a terminal program before the WebUI is opened and monitoring the output will reveal if a crash is happening.
Which previous version? For the the SD card issue, can you look at the signalling with a scope when writing. Is the clock signal looking good and what is its frequency. Same frequency when reading? The SD card driver code is by Espressif, either check their forums for your problem or raise a new issue with them? BTW I have fixed the typos and the spindle off bug. However, please open new issues instead of adding new ones to this as you are not the OP and this issue has been closed. |
Hi @terjeio, I believe the parsing bug is related to more than 3 axes. If I treat the 4th axis as ganged it will parse correctly (sort of), but it seems to only be able to parse 3 axes and even with ganged, it will read the first 3 axes properly but will not read the A axis or M3. I will open up a new issue. #12 (GrblHAL WebUI 3.0 unable to parse $$ settings from controller) It looks like you have fixed the bug with the laser (spindle) not spinning down, but the spindle enable pin still remains active even when M5 is issued and even if one issues M5, one must still issue M4, then M5 to turn off the spindle enable pin. At least it is now safe as the laser will turn off when stop command is given or M5 is given. I will open up a new issue for this as well. Other odd behavior such as $73=3 now reverts to a fully functional AP if STA does not connect to router. So this has definitely been fixed! SSDP does not seem to work at all, the device will not boot if only that setting is enabled. I'll scope out the SD card clk and TX/RX pair and figure out what's going on with the signals. Hi @luc-github yes I had verbose mode on, but when it does not parse, there is no debugging data in the preview window or the console window. Using the same settings and 3 axes (4th ganged) looks like this Thank you! |
I am not able to replicate that. FYI the SSDP daemon will not be started unless HTTP is also enabled - since clients reads a xml file via http to get details about the service. Win10 info screen:
As a test I wired up a bare board to a commercial breakout with rather long wires, still getting nearly 300Kb/sec write speed: |
Hi @terjeio you give me hope! Can you give me more details on the SD breakout board such as a link you where you ordered the SD breakout. I may have to strip down to a bare bones setup for debug. Can you also share your cMakeLists.txt and any other non default settings. At the risk of posing a seemingly silly question- it also it looks like you have a wire connected to GPIO13, is that used for anything? Also please post your cMakeLists.txt file so I can test SSDP. The controller is crashing (or just not booting) when I simply enable SSDP setting with my current working settings. Thank you! |
The link is long gone, this is similar.
It is for measuring the spindle enable signal - without risking shorting pins.
|
Hi @terjeio, I have tried the SD card module wired directly to the ESP32 as you have done and it still reads very slow (4-8kB/sec) uploading through WebUI. I don't know whether your ESP32 WROOM board uses a Silabs or CH340g USB->UART. I wired it as short as possible with similar wire lengths. The breakout board schematic is similar to as I'm using on my board that I posted April 24. It seems you are using a different upload method other than WebUI!? If using WebUI, what version you using? Also, I am using a Class 4 brand name SD card (Sandisk) 4GB formatted at FAT32. Still scratching my head as to why this occurs, but it is definitely slow. Based on your CMakeList.txt is seems you have SoftAP turned off. As soon as I turn on SoftAP, then SSDP does not work, the device will not boot. Also you have SD streaming off, I typically leave this on. Thank you! |
I am using a Silabs CP210x driver, I guess that matches the chip - I can no longer read the markings. However, which chip is used for USB should not matter at all for WebUI performance. But to be sure, do you power the ESP32 via the USB port?
I have tested with both WebUI and FTP, both completes a 4.7Mbyte upload in ~30s. I just retested with the latest version:
I'll look into this again, SSDP should not be started before a client connection is established? Or perhaps not at all in SoftAP mode?
Does not matter, it is automatically switched on when the WebUI is enabled. |
Fix for SSDP comitted. |
Hi @terjeio, great work on SSDP and in general. However, I am still very confused about the SD card. Can you post your CMakesList.txt and a bin file. For some reason I cannot get more than ultra-slow writes to SD card. I have tested both powered from USB (which uses a LM117 5V to 3.3V LDO) and powered from a mainboard that is providing 3.3V directly the WROOM (with the LM117 removed from the WROOM). Thank you! |
Last test I did was with Web Builder generated firmware. Generally I test with a CNC BoosterPack that has on-board EEPROM, this will not start if the EEPROM is not present. You can build for this with EEPROM disabled in the Web Builder. |
Hi @terjeio, I am trying to build bare bones and just with the SD card and the WROOM device currently to match our result. I am definitely not changing the SPI pins in any testing I've done. I posted the error with Web Builder in this issue and will wait for a fix to test again. My goal is to replicate exactly your simplified setup and then systematically determine the source, but currently cannot even replicate your result after ordering the exact same SD card breakout and using what I believe to be the exact same hardware. Thank you! |
It seems on ESP32 files can be uploaded on local FS, but no files but preferences.json can be downloaded
Like the webui theme files.
I know on other platform the FS is limited and that is why we introduced the
HostPath:"/"
parameter in [ESP800], should we add a new parameterReadOnlyFS
to let know webui to disable the theme features and any FS uploadIn same idea, it seems webui (index.html.gz) can be updated but not downloaded so it is weird - I did not digged grblHAL code to be honest
Original issue was raised here : luc-github/ESP3D-WEBUI#350
The text was updated successfully, but these errors were encountered: