QGIS 3.34 takes >10 minute to load up with WMS/WFS datasources

[frida-161]

What is the bug or the crash?

With the current LTR (3.34.7) it takes more then 10 minutes to start QGIS when I have a large number of predefined WMS/WFS datasources in my profile. I tested old LTR (3.28) and current (3.36) version with the same profile and the startup takes a only a few seconds.

Apparently QGIS does some changes to the QGIS3.ini file in the profile during the startup and shutdown processes .
I tried making a copy of the QGIS3.ini while qgis was running. After the shutdown process which takes a few minutes and overwrites the ini again, I replaced it with my copy. The next startup was almost instant. But during the shutdown the ini gates rewritten again so the next startup again takes more then 10 minutes.

I could reproduce the bug on Arch Linux (qgis-ltr build from the AUR) as well as Windows 11 (Installer from the qgis.org website).

I build qgis with Debugging enabled but I dont find much about whats going on in the log except for the second line which seems to be the part that takes > 10 minutes.

I found this issue about rewriting of connections, maybe it is related #56713

.cpp:514 : (main) [0ms] Starting qgis main
messagelog.cpp:34 : (logMessage) [673134ms] 2024-06-05T08:40:49 [0] Successfully loaded globalsettingsfile path: /usr/share/qgis/resources/qgis_global_settings.ini
viders/qgsproviderregistry.cpp:367 : (init) [1245ms] Loaded 30 providers (OAPIF;WFS;arcgisfeatureserver;arcgismapserver;arcgisvectortileservice;cesiumtiles;copc;delimitedtext;ept;gdal;gpx;mbtilesvectortiles;mdal;memory;mesh_memory;mssql;ogr;pdal;postgres;postgresraster;spatialite;tiledscene;vectortile;virtual;virtualraster;vpc;vtpkvectortiles;wcs;wms;xyzvectortiles) 
messagelog.cpp:34 : (logMessage) [6086ms] 2024-06-05T08:40:56 Plugins[0] Loaded MetaSearch Catalog Client (package: MetaSearch)
messagelog.cpp:34 : (logMessage) [86ms] 2024-06-05T08:40:57 Plugins[0] Loaded DB Manager (package: db_manager)
messagelog.cpp:39 : (logMessage) [1754ms] 2024-06-05T08:40:58 Processing[2] Problem with GRASS installation: GRASS was not found or is not correctly installed
messagelog.cpp:34 : (logMessage) [7ms] 2024-06-05T08:40:58 Plugins[0] Loaded GRASS GIS provider (package: grassprovider)
messagelog.cpp:34 : (logMessage) [1256ms] 2024-06-05T08:41:00 Plugins[0] Loaded Processing (package: processing)
messagelog.cpp:34 : (logMessage) [598ms] 2024-06-05T08:41:00 [0] Application state:
QGIS_PREFIX_PATH env var:		
Prefix:		/usr
Plugin Path:		/usr/lib/qgis/plugins
Package Data Path:	/usr/share/qgis
Active Theme Name:	default
Active Theme Path:	/usr/share/qgis/resources/themes/default/icons/
Default Theme Path:	:/images/themes/default/
SVG Search Paths:	/usr/share/qgis/svg/
		/home/frida/.local/share/QGIS/QGIS3/profiles/rvr_wms_laden/svg/
User DB Path:	/usr/share/qgis/resources/qgis.db
Auth DB Path:	/home/frida/.local/share/QGIS/QGIS3/profiles/rvr_wms_laden/qgis-auth.db

messagelog.cpp:39 : (logMessage) [358230ms] 2024-06-05T08:46:58 Processing[2] Problem with GRASS installation: GRASS was not found or is not correctly installed
True

Steps to reproduce the issue

1. Create a fresh QGIS profile
2. Go to Datasource manager
3. Go to WMS and click Load and load the wms.xml file contained in the zip file below
4. Go to WFS and click Load and load the wfs.xml file contained in the zip file below
5. close QGIS
6. start QGIS again and wait..

connections.zip

Versions

QGIS version	3.34.7-Prizren	QGIS code branch	Release 3.34
Qt version	5.15.14
Python version	3.12.3
GDAL/OGR version	3.8.5
PROJ version	9.4.0
EPSG Registry database version	v11.004 (2024-02-24)
GEOS version	3.12.1-CAPI-1.18.1
SQLite version	3.46.0
PDAL version	2.7.1
PostgreSQL client version	unknown
SpatiaLite version	5.1.0
QWT version	6.2.0
QScintilla2 version	2.14.1
OS version	Arch Linux
This copy of QGIS writes debugging output.
Active Python plugins
MetaSearch	0.3.6
db_manager	0.1.20
grassprovider	2.12.99
processing	2.12.99

QGIS version 3.34.7-Prizren QGIS code branch [Release 3.34](https://github.com/qgis/QGIS/tree/release-3_34) Qt version 5.15.14 Python version 3.12.3 GDAL/OGR version 3.8.5 PROJ version 9.4.0 EPSG Registry database version v11.004 (2024-02-24) GEOS version 3.12.1-CAPI-1.18.1 SQLite version 3.46.0 PDAL version 2.7.1 PostgreSQL client version unknown SpatiaLite version 5.1.0 QWT version 6.2.0 QScintilla2 version 2.14.1 OS version Arch Linux

This copy of QGIS writes debugging output.

Active Python plugins
MetaSearch
0.3.6
db_manager
0.1.20
grassprovider
2.12.99
processing
2.12.99

Supported QGIS version

• I'm running a supported QGIS version according to the roadmap.

New profile

• I tried with a new QGIS profile

Additional context

No response

[3nids]

@nyalldawson since you have more or less followed the story, what do you think about dropping the backward compatibility and make the forward migration a one time operation (on demand, on first start, you would be asked if you want to migrate former settings, and once it's done, you don't go over it once more)?

[nyalldawson]

@3nids

dropping the backward compatibility and make the forward migration a one time operation

+1

(on demand, on first start, you would be asked if you want to migrate former settings, and once it's done, you don't go over it once more)?

Instead of on-demand , could we just change the key and then add a new setting which tracks which ones were migrated so we never do that more than once. (And store the migrated list outside of somewhere were a user action such as deleting a connection will remove it?)

[frida-161]

@3nids It seems like I'm not up to date on the story :)

Are there any Issues/CRs I can read to get in the loop?

We have a customer who wants to upgrade to the current LTR but this loading times are a dealbreaker for them.
I'm willing to do my part in getting this resolved by I'll probably need some pointers.

Frida

[3nids]

@nyalldawson what about this:

On first load QGIS imports the settings with the current forward import code. Whenever done, it saves in a settings that this was done. That import is never run again. We bring this functionnality to current release and 3.34.

We drop the backward compatibility code.
In the forward compatibility code, we obviously drop the deletion of old settings part (which were recreated in backward compatibility).

This means that the settings won't be synchronized anymore with 3.32- which I found OK now that 3.34 is LTR.

[3nids]

@frida-161 there is a new system for storage of settings which has some backward/forward compatibility code to ensure integrity across old versions of QGIS. This compatibility code is the reason of slowness and has brought some painful issues.

[frida-161]

@3nids thanks for the answer.

I understand that the compatibility code runs on 3.34 LTR and 3.36. What I don't understand is why this process takes minutes on the 3.34 and only seconds on 3.36.
It seems to be that there are some changes to the compatibility code. So maybe backporting these changes to LTR would be another option to fix this?
Can you point me to the relevant parts in the code?

Frida

[D-MM-62]

Hello everyone,
as a workaround we moved the connections to “qgis_global_setting.ini”.
Both the ones in the [connections] and the [qgis] section. It's best to copy them while QGIS is running.

@frida-161 maybe this is also a solution for you.

[stefan-overkamp]

@D-MM-62 The workaround is not working for me.
The connections are restored in QGIS3.ini when QGIS app is closed.
If you change connections or adds new ones, they are added to QGIS3.ini not qgis_global_setting.ini
3.34 LTR is not usable for us with this behaviour.

[3nids]

I implemented the suggestion here: #58250

[stefan-overkamp]

QGIS 3.34.10 LTR solved this issue.
There is now an additional setting: settings-pre3.30-migrated=true and opening QGIS is done in seconds.
Many thanks.

[weca-theo]

Also can confirm that 3.34.10 solved this issue for me, taking seconds to load now.
3.34.9 was taking 3 minutes to open any project, due to my many (~100) WMS connections stored to my profile.