PyPy: commits fail when cursors not closed

[tilboerner]

The underlying problem which lead to #676 is actually not related to Python 2 as @MaxLeiter already noted, but to PyPy. Unfortunately, it's a bug that causes database commits to fail, and probably results in 500 errors to HTTP requests that involve persisting data. PyPy support is broken because of this.

It's caused by a combination of implicitly creating db cursors through connection.execute..., and trusting them to be implicitly closed by garbage collection by the time we want to commit. Unlike CPython, PyPy uses a different garbage collection strategy than refcounting, and does not collect objects right when their last pointer is removed. Therefore, cursors remain open, and sqlite sees their statements as not finalized, which means it will refuse to commit, and possibly also to rollback, with the following error:

OperationalError: cannot commit transaction - SQL statements in progress

You can see this happen in the failing sqlitecache unit tests on travis. I could also reproduce it locally.

I made the tests work by explicitly closing every cursor created in sqlitecache.py. To fix the bug, we need to make sure that db cursors are closed in all code that touches the database.

[tilboerner]

Here's the commit that fixes the issue in sqlitecache.py as an example: aa4eea. (The formatting is borked, but in this legacy code that's hardly noticeable. =:0 Ay.)