Skip to content
This repository has been archived by the owner on Feb 8, 2018. It is now read-only.

repair paydays table #204

Closed
chadwhitacre opened this issue Jul 27, 2012 · 13 comments
Closed

repair paydays table #204

chadwhitacre opened this issue Jul 27, 2012 · 13 comments

Comments

@chadwhitacre
Copy link
Contributor

This isn't really used much yet but it should be (nice historical data). Two repairs needed:

@chadwhitacre
Copy link
Contributor Author

Here's how I fixed nparticipants:

update paydays set nparticipants=(select count(*) from participants where claimed_time < ts_start);

In addition to the early issue where we were counting unclaimed accounts as registered users, it turns out that rerunning the paydays script throws this number off as well (#261). We got bit by that in week 7 (#169) but it wasn't enough to notice. We got bit by it again this week (#257) and it was enough to notice. :-)

@zbynekwinkler
Copy link
Contributor

Am I missing something major here or is updating the exchange_volume (that's actually ach_volume, right?) as simple as:

UPDATE paydays
SET ach_volume = (
        SELECT count(amount) 
        FROM exchanges 
        WHERE timestamp > ts_start 
        AND timestamp < ts_end
        AND amount < 0
    );

And the same for fees:

UPDATE paydays
SET ach_fees_volume = (
        SELECT count(fee) 
        FROM exchanges 
        WHERE timestamp > ts_start 
        AND timestamp < ts_end
        AND amount < 0
    );

@ghost ghost assigned chadwhitacre Oct 12, 2013
@zbynekwinkler
Copy link
Contributor

Following the self check #1705 idea I have

select count(id)
from paydays
where charge_volume != (
    select sum(amount+fee) 
    from exchanges 
    where timestamp > ts_start 
    and timestamp < ts_end 
    and amount > 0
)

returning 0.

@zbynekwinkler
Copy link
Contributor

There are two bugs in charge_fees_volume. See

select * from (
    select id, ts_start, charge_fees_volume, (
            select sum(fee) 
            from exchanges 
            where timestamp > ts_start 
            and timestamp < ts_end 
            and amount > 0
        ) as ref
    from paydays
    order by id
) as foo
where charge_fees_volume != ref;

returning 2 rows

 id |           ts_start            | charge_fees_volume |  ref  
----+-------------------------------+--------------------+-------
 13 | 2012-07-27 20:51:21.754544+00 |              71.36 | 99.31
 14 | 2012-08-03 14:57:42.935184+00 |              45.27 | 56.83

@zbynekwinkler
Copy link
Contributor

There seems to be one bug in ach_volume. See

select * from (
    select id, ts_start, ach_volume, (
            select sum(amount) 
            from exchanges 
            where timestamp > ts_start 
            and timestamp < ts_end 
            and amount < 0
        ) as ref
    from paydays
    order by id
) as foo
where ach_volume != ref;

returning

 id |           ts_start            | ach_volume |   ref    
----+-------------------------------+------------+----------
 74 | 2013-06-20 12:12:53.292609+00 |   -1885.73 | -2024.73
(1 row)

@zbynekwinkler
Copy link
Contributor

ach_fees_volume seems to be fine:

select * from (
    select id, ts_start, ach_fees_volume, (
            select sum(fee) 
            from exchanges 
            where timestamp > ts_start 
            and timestamp < ts_end 
            and amount < 0
        ) as ref
    from paydays
    order by id
) as foo
where ach_fees_volume != ref;

Returns 0 rows.

@ghost ghost assigned zbynekwinkler Jan 7, 2014
@zbynekwinkler
Copy link
Contributor

@whit537 Is it worth to start looking for a bugs in the payday code or do we just fix it for now, add it to the self checks and wait for it to break? That would be my suggestion.

@chadwhitacre
Copy link
Contributor Author

Agreed, add it to self-checks and wait for it to break. Once we're comfortable with web app db usage we'll turn our attention to payday.

@chadwhitacre
Copy link
Contributor Author

Is it worth to start looking for a bugs in the payday code

Reviewing this again w/ the PR (#1902) ... can we trace the problem to the bugs on #202? We may not have to go debugging anew, we should have the answer right there on #202.

At first glance the dates in #204 (comment) seem like they could be Gittip <= 8 as the description on this ticket states. I'm not sure about #204 (comment). Was there another payday failure around that time that accounts for the corruption in ach_volume?

@zbynekwinkler
Copy link
Contributor

None of the problems are Gittip <= 8 (see the id that is output with each problem). I have no idea where they came from. The issue #202 is a year old and closed so I was not exploring it. BTW: that is why I asked if it is worth looking for some bugs in payday. We have agreed that it is not worth it - if there is a bug it will show up and will be caught by the self checks - we can debug right then. I don't feel like doing payday.py forensics while trying to find out what code was there when a particular payday has been run.

@chadwhitacre
Copy link
Contributor Author

BTW: that is why I asked if it is worth looking for some bugs in payday.

Right. I was suggesting that the bugs might already have been discovered, on #202 ...

None of the problems are Gittip <= 8 (see the id that is output with each problem).

... but it sounds like they weren't. So our decision holds. :-)

@zbynekwinkler
Copy link
Contributor

I am confused. Are we merging #1902 and closing this issue or do we want to do something else?

@chadwhitacre
Copy link
Contributor Author

We're merging. :-) IRC

chadwhitacre added a commit that referenced this issue Jan 17, 2014
Repair paydays and add self checks (fixes #204).
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants