sqlite-jdbc's compilation flags cause sqlite's devtest to fail

[JacksonBailey]

Describe the bug

sqlite-jdbc's compilation flags cause sqlite's devtest to fail.

To Reproduce

mkdir testing
cd testing

# blobless clone for quicker cloning, --filter can be removed if you want the full history
git clone [email protected]:sqlite/sqlite.git --filter=blob:none -b version-3.46.0
cd sqlite
git rev-parse HEAD # Should be bebe2d8be8acfd02592c4972f4ba32c3b4e4a33f
cd ..

mkdir bld
cd bld

# From here: https://github.com/xerial/sqlite-jdbc/blob/17dffe95ba2ce34d70e6fe54f69636cd3efbc45e/Makefile#L81-L104
export CFLAGS="\
-DSQLITE_ENABLE_LOAD_EXTENSION=1 \
-DSQLITE_HAVE_ISNAN \
-DHAVE_USLEEP=1 \
-DSQLITE_ENABLE_COLUMN_METADATA \
-DSQLITE_CORE \
-DSQLITE_ENABLE_FTS3 \
-DSQLITE_ENABLE_FTS3_PARENTHESIS \
-DSQLITE_ENABLE_FTS5 \
-DSQLITE_ENABLE_RTREE \
-DSQLITE_ENABLE_STAT4 \
-DSQLITE_ENABLE_DBSTAT_VTAB \
-DSQLITE_ENABLE_MATH_FUNCTIONS \
-DSQLITE_THREADSAFE=1 \
-DSQLITE_DEFAULT_MEMSTATUS=0 \
-DSQLITE_DEFAULT_FILE_PERMISSIONS=0666 \
-DSQLITE_MAX_VARIABLE_NUMBER=250000 \
-DSQLITE_MAX_MMAP_SIZE=1099511627776 \
-DSQLITE_MAX_LENGTH=2147483647 \
-DSQLITE_MAX_COLUMN=32767 \
-DSQLITE_MAX_SQL_LENGTH=1073741824 \
-DSQLITE_MAX_FUNCTION_ARG=127 \
-DSQLITE_MAX_ATTACHED=125 \
-DSQLITE_MAX_PAGE_COUNT=4294967294 \
-DSQLITE_DISABLE_PAGECACHE_OVERFLOW_STATS \
"

../sqlite/configure && make
make devtest

Expected behavior

sqlite's tests should pass when using sqlite-jdbc's compilation options.

Logs

The tests in error are here (select displayname from jobs where state = 'failed'; on testrunner.db).

• test/mutex1.test
• test/wal.test
• test/attach4.test

grep '^!' testrunner.log gives this output. I snipped a portion to keep it brief. The full output from this command visible here.

! mutex1.2.multithread.4 expected: [fast static_app1 static_app2 static_app3 static_main static_mem static_open static_pmem static_prng static_vfs1 static_vfs2 static_vfs3]
! mutex1.2.multithread.4 got:      [fast static_app1 static_app2 static_app3 static_main static_mem static_open static_prng static_vfs1 static_vfs2 static_vfs3]
! mutex1.2.serialized.4 expected: [fast recursive static_app1 static_app2 static_app3 static_main static_mem static_open static_pmem static_prng static_vfs1 static_vfs2 static_vfs3]
! mutex1.2.serialized.4 got:      [fast recursive static_app1 static_app2 static_app3 static_main static_mem static_open static_prng static_vfs1 static_vfs2 static_vfs3]
!Failures on these tests: mutex1.2.multithread.4 mutex1.2.serialized.4
!Failures on these tests: wal-26.1.82 [... snipped for brevity, many many failures here ...]
!Failures on these tests: attach4-1.3 attach4-1.4 attach4-1.5 attach4-1.6 attach4-1.7 attach4-1.8

Environment (please complete the following information):

• OS: macOS Ventura 13.6.7
• CPU architecture: x86_64
• sqlite-jdbc version: the compilation options from 3.46.0.2-SNAPSHOT

Additional context

• The options fail in the newly released 3.46.1 as well.
• I have created a post on sqlite's forum here.

[gotson]

I don't think this has anything to do with this project. You can follow up on the thread you created at sqlite forum, and we can reassess depending on their feedback.

[JacksonBailey]

Sounds good. I was a little at a loss because while it's not sqlite-jdbc's "fault", it felt like something to bring up.

I've been doing a small amount of troubleshooting on it and it seems like at least some of the failures have to do with shared cache which sqlite describes as "discouraged" and better served by WAL mode, but I'm seeing failures related to WAL mode in the tests as well so I'm not sure. (Plus it's probably not so simple as just removing shared cache mode from sqlite-jdbc.) Edit: That was not the culprit.

[JacksonBailey]

I've narrowed this down to specifically -DSQLITE_DISABLE_PAGECACHE_OVERFLOW_STATS and -DSQLITE_MAX_ATTACHED=125 (even when used by themselves) causing the failures. I'll notify sqlite.