Refactor calculation logic to support larger heartbeat / keystroke timeouts

[em411]

I've been eagerly anticipating this feature for years, and I'm deeply disappointed to see that the maximum configurable value for the keystroke timeout is only 5 minutes. This falls significantly short of expectations, especially considering that WakaTime, has a default value of 15 minutes.

While I appreciate the effort to make the timeout configurable, the current range of 30-300 seconds is far too restrictive for many users needs. WakaTime allows configuration between 5 and 240 minutes, offering much more flexibility.

This limitation severely impacts the usefulness of the feature for those of us who work on tasks that involve longer periods of thinking, searching or reference checking between keystrokes.

Could you please consider expanding the upper limit of the configurable range? Ideally, it would be great to have a range that at least matches or exceeds WakaTime's capabilities.

[taciturnaxolotl]

I might be blind and missing something here but I'm pretty sure a single change on this line should fix your issue!

wakapi/models/user.go

Line 17 in 758be88

MaxHeartbeatsTimeout = 5 * time.Minute

[em411]

I might be blind and missing something here but I'm pretty sure a single change on this line should fix your issue!

wakapi/models/user.go

Line 17 in 758be88

MaxHeartbeatsTimeout = 5 * time.Minute

I've seen this line and I'm pretty confident, there's a reason behind.

[muety]

First of all, Wakapi doesn't aim to be a drop-in replacement for WakaTime. It's its own, standalone project and may well bring its own ideas and philosophy, while being partly API-compatible with WakaTime.

Second, please note the way Wakapi calculates durations is different from WakaTime (see FAQ section in the README), so Wakapi's "hearbeat timeout" is not the exact same parameter as WakaTime's "keystroke timeout". While WakaTime has a second parameter hard-coded to 1 min for "padding" heartbeats (see here), Wakapi simply uses the timeout / offset for this as well (sort of "dual" functionality in our case).

That being said, we could, of course, refactor our durations calculation logic to match WakaTime's. But I'm hesitant to do so, (a) because it's quite a fundamental change (coding times would look vastly different suddenly) and (b) because I personally found Wakapi's approach to work quite well, i.e. well reflect my actual coding times on average.

Can you explain what is the rationale behind setting the timeout to an exceedingly large value? Why would you - in the most extreme case - set it to 240 mins? Say you open your IDE, then meet a friend and come back after 3h 59 mins - you'd be credited for 4 hours of coding.

Also, keep in mind that this would make things a little "unfair" in regard to the leaderboard. Someone setting the timeout to several hours would accumulate coding time much easier (even though it probably doesn't remotely reflect reality) compared to someone who set it to 2 mins.

I'm open to become convinced about very large timeout to make sense. But currently, I don't really see the need. In any case, of we decide to allow timeouts much larger than 5 minutes, we'd first need to refactor the entire calculation logic in a way similar to what WakaTime does.

Also, be aware that - if you're deeply unhappy with how Wakapi works - there's always the option to compile your own version and self-host it.

[em411]

I've been following Wakapi for over three years now, and I fully understand and appreciate that it's a standalone project with its own unique ideas. I'm sharing my thoughts here because I was really looking forward to this feature, and I'm a bit disappointed with how it turned out. I admit that my initial reaction came from high expectations and a misunderstanding of what the feature would cover.

Wakapi's current approach excels at reflecting pure coding time, but it struggles to capture my work time accurately. Let me explain what I mean by work time, based on how I use WakaTime:

In my job as a programmer, I use WakaTime to set daily goals of 8 hours. This helps me keep track of my performance for personal reasons. But my day isn't just coding. I also have meetings, talk with colleagues, take breaks, read documentation, and do other work-related tasks. WakaTime can't track all of these, of course. That's why I set my Keystroke Timeout to 30 minutes. This longer timeout helps cover most of my work activities.

If the timeout is too short, the report might show that I'm not working enough. This can make me feel bad about my work. A longer timeout gives a more accurate picture of my whole workday. It helps me balance my work and personal life better, and track my time at work more reliably.

I understand the concern about extremely long timeouts like 240 minutes. While I can't speak to the usefulness of such an extended period as it doesn't align with my use case, I also find very short timeouts (like 2 minutes) equally impractical for my needs.

Regarding the leaderboard fairness, WakaTime addresses this by always calculating leaderboard stats based on a standardized 15-minute keystroke timeout, regardless of individual user settings.

I'm glad you're open to talking about this. I hope this helps explain why some of us would like more flexibility in setting timeouts. It's not about trying to inflate our work hours, but about getting a more comprehensive and realistic picture of our full workday.

[muety]

Thanks a lot for elaborating on this! It helped me understand the rationale behind your request much better, now.

I came across this "conflict" a couple of times already: I agree that it would be super useful if Wakapi could reflect your whole work day as a programmer in its entirety. But I think the information gained from pure editor keystrokes is just not sufficient / high-resolution enough to accomplish this. I, personally, have settled for the fact that Wakapi can well track coding time, but not my entire work - not to speak of actually using it for billing work hours to a customer or so.

I'm willing to increase maximum heartbeat timeout to > 5 min. But, in summary, it would require two fundamental changes first:

• Refactor calculation logic to the approach followed by WakaTime (see my explanation above)
• Refactor leaderboard calculation to use a standardized (e.g. 15 min) timeout and thus not rely on users' models.Summary entities, but create its own, second set of summaries instead (higher computational- and code complexity)

I'll keep the issue open for a while and if a lot of users consider this a necessary change, I'll think about implementing it. But otherwise, I think the effort that would have to go into this and the sudden change in every user's coding times is not worth it...

[juztim]

Im totally with @em411 here, I personally count reading documentations, stackoverflow or whatever else towards coding time so, I personally think 15-30 minutes timeout is a great sweet spot.

[Grommash9]

I am personally use it for invoicing my clients, and if it will lost some time when i was googling or on meeting it will mean I am losing money

But maybe it's me using tool in the wrong way really because it was created not for that

It's really hard to use browser extensions for that because I am not sure I am able to provide url list related to project

Maybe we can add option to add meting time manually on dashboard? I have created telegram bot for that but it looks I cant send future heartbeats to book time up front only backwards

[muety]

First of all, thanks a lot for all your input (across various issues), @Grommash9, it's super valuable.

Maybe we can add option to add meting time manually on dashboard?

I think this would be too much of a stretch for Wakapi. Even though many people seem to be wanting this, Wakapi is really not meant to be used for actual billing. I think even if we supported longer timeouts, the data provided by Wakapi would still be too inaccurate to create invoices from it with a good conscience. I'd recommend to use a tool like Toggl, Kimai2 or sth. in addition.

cant send future heartbeats to book time up front only backwards

You're right. Part of the validation process of incoming heartbeats is to check that they're not sent "from the future".

[em411]

I've found a partial workaround for this issue by using Desktop Wakatime, which can count activity across all configured applications. This allows me to track everything from communication on Slack to reading docs on the browser.

[ricristian]

I discovered this project yesterday while searching for a way to “store” my stats on my own “infrastructure.” After experimenting with it a bit, I’ve grown to like it so much that I even updated the Helm chart to include the environment variables, as it hadn’t been updated in over 8 months. But that’s a topic for another day. 😊

@muety if I may, I understand that the scope of this project may not be geared toward billing purposes but rather to help us (DevOps professionals, developers, or whatever fancy corporate titles we have) monitor our coding stats—and that’s fantastic.

Some of us work as freelancers or on small, personal projects that (hopefully) generate some income. While platforms like WakaTime or similar services are available, I personally have concerns about the type of data being collected by some of these companies. Regardless of the good intentions of their owners, there’s no guarantee that in five years, only heartbeats will be sent from “my device.” Moreover, there are GDPR concerns. Many of these companies are based in the US, and their data is stored there, which introduces potential legal complications.

In my case, I’ve already deployed Wakapi, and I’ve modified the heartbeat timeout according to my needs ( update user directly from db ) . However, this feels more like a hack, which I’m not a fan of in public projects—mainly because hacks aren’t officially supported by anyone.

What I don’t understand is why this option, while “customizable,” has a limit on its value. Some users might want a timeout of two days, while others (like you) might prefer 2–5 minutes. I understand that the issue is related to the leaderboard, but for people like me who host their own Wakapi instance, this reason becomes irrelevant. Self-hosted instances typically disable the leaderboard, so this limitation doesn’t make much sense in those cases.

[muety]

Thanks for your thoughts. Had another bunch of thoughts on this, too, and also revisited WakaTime's aggregation logic again.

From my understanding, the only actual difference is that WakaTime doesn't add a "padding" to singular heartbeats, that is, if you only send one single heartbeat, WakaTime will report a duration of zero minutes, while Wakapi will still "extrapolate" to 2 minutes (or whatever your configured timeout is). So the only change required to match WakaTime's logic would essentially be to drop this "padding" concept. Actually less complicated than I first thought.

Additionally, we'd need to have two parallel sets of summaries per user of different granularity to still ensure a fair leaderboard (as described above). Implementing #716 would come in handy for this as well.

We could think about implementing this in two steps:

1. step: refactor calculation logic
   • That is to remove padding
   • Also add a global config param for maximum allowed timeout (and restrict it to 5 min for wakapi.dev for fairness for now)
2. step: implement "multi-level" summaries
   • Then, allow arbitrary large timeouts on wakapi.dev as well

After the first step, this feature of "large keystroke timeouts" would already be available to admins of self-hosted instance, who potentially don't care about the leaderboard, but remain "disabled" on wakapi.dev until after step 2.

That being said, and even though many of you guys seem to be wanting this, I'm still not entirely convinced about the reasonableness of timeouts larger than a few minutes. Say you configure your timeout to one hour. You code a bit, then go for lunch, then carry on coding. Wakapi would credit you coding time for your whole lunch break, messing up your entire coding stats (and resulting in unreasonably high invoices for your customers when actually used for billing purposes)?