Losing offset

[jmrichardson]

Hi, I have this code to write to the offset every line. However, if I ctrl-c the script, the offset file occasionally (about 1 out of 4) is empty and it starts from beginning. Shouldn't this write be atomic or have some test to verify:

from pygtail import Pygtail
import time

sleep = 1
symbol = 'data.csv'
symbol = 'data/kibot/IBM.txt'

pt = Pygtail(symbol, every_n=1, paranoid=True)
while True:
    try:
        line = pt.next().strip()
        print(line)
    except StopIteration:
        print("sleep")
        time.sleep(sleep)
    except Exception as e:
        print(str(e))

[jmrichardson]

I ended up extending the _update_offset_file method:

from atomicwrites import atomic_write
class File(Pygtail):
    def _update_offset_file(self):
        if self.on_update:
            self.on_update()
        offset = self._filehandle().tell()
        inode = os.stat(self.filename).st_ino
        with atomic_write(self._offset_file, overwrite=True) as fh:
            fh.write("%s\n%s\n" % (inode, offset))
        self._since_update = 0

[omersiar]

I can reproduce this. read_from_end=True seems to not work as expected as well