Windows: after acquiring a lock (even Exclusive) process can't write to file

[jmitchell]

I adapted tests.hs and provided directions to reproduce the problem in jmitchell@4a850dd.

Is there a different way I should secure a lock, write to the locked file, and then release the lock?

I suspect what's happening is some low-level write/close operation is attempting to lock as well and causing a FILE LOCK CONFLICT. However, writing to a file without using System.FileLock results in no LockFile operations according to Process Monitor. This suggests low-level file writing/closing operations behavior varies depending on the existence of a lock, and inexplicably re-lock when there's already a lock.

It's possible System.FileLock's upstream APIs broke this use case, or perhaps there's an issue with the LockFileEx wrapper.

[jmitchell]

jmitchell@ce03f14 illustrates the problem as concisely as possible. For convenience, the important part along with explanatory comments is copied below:

lockAndWriteFile :: String -> String -> IO ()
lockAndWriteFile fname contents = do
  withFileLock fname Exclusive $ \_ -> do
    putStrLn "took exclusive lock; attempting write..."

    -- Following line fails, printing "test-filelock.exe: testLockFile.txt:
    -- hClose: permission denied (Permission denied)" and process exits
    -- prematurely with exit code 1.
    --
    -- However, there are no issues when it's commented out.
    writeFile fname contents

  -- Never prints (unless writeFile line is commented out).
putStrLn "released exclusive lock"

The observations hold even when lockAndWriteFile is given a file path which has never been used on the system before. It is not a matter of an external process already having a lock on the file (although could multiple threads in this test program cause it?)

[jmitchell]

The latest commit (jmitchell@24857fb) includes tracing in System.FileLock, System.FileLock.Internal.LockFileEx, and the test file prefixed with the labels FL, LFE, and Test, respectively.

Over several runs I'm consistently getting this same output from bin\test-filelock.exe (and exit code 1):

Test.lockAndWriteFile: start
FL.withLockFile: start
FL.lockFile: start
LFE.lock: start
LFE.open: start
LFE.open: end
LFE.lockFirstByte: start
LFE.lockFirstByte: end (success)
LFE.lock: end
FL.newLock: start
FL.newLock: end
FL.lockFile: end
  attempting write...
FL.unlockFile: start
LFE.unlock: start
LFE.unlock: end
FL.unlockFile: end
test-filelock.exe: testLockFile.txt: hClose: permission denied (Permission denied)

As I'd expect, the attempted write occurs after FL.lockFile has returned a FileLock and before FL.unlock has unlocked that FileLock.

[domenkozar]

It might be worth comparing code to https://github.com/joeyh/git-annex/blob/master/Utility/LockFile/Windows.hs

[takano-akio]

I haven't looked into the details yet, but just a quick reply: this mode of operation (writing to the lock file while holding a lock) is not currently supported by the library. In fact, the only supported thing is to use lock files purely as inter-process mutexes.

I'm aware that it is often useful to be able to write to a locked file. Perhaps there should be an interface for locking a Handle. Unfortunately I haven't had a chance to implement this.

[domenkozar]

I'd say actionable item from this issue would be to improve documentation.

[jmitchell]

Thank you for the quick response, @takano-akio.

After understanding the supported use case better I implemented a roughly equivalent behavior using an independent lock file. Granted, this approach doesn't prevent another process from writing to the target file associated with the lock file, but it does prevent the process and its threads from unintentionally clobbering the file.

I agree with @domenkozar that more documentation would have helped. Let me know if I can help.

[takano-akio]

Currently the doc on System.FileLock says:

It is not recommended to open or otherwise use lock files for other purposes, because it tends to expose differences between operating systems. For example, on Windows openFile for a lock file will fail when the lock is held, but on Unix it won't.

If you have an idea on how to improve the doc situation, feel free to open a pull request.

[domenkozar]

Maybe the lock file could be an internal implementation detail for the library. Something like passing a filepath would result into filepath.XXX lock file where XXX would be a hash of filepath.

For now, I'd mention that on windows it will result into permission denied error and mark it documentation as warning/note.