-
Notifications
You must be signed in to change notification settings - Fork 115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The sum of mAP is not 100 #53
Comments
Hey, not sure if this is still relevant to you but mAP is the abbreviation of "mean Average Precision". To get mAP you take the mean over APs, not sum. |
Thanks for your reply very much! |
Ah I see. The dAP values will not sum to 100% because they are not calculated progressively (i.e. they are not applied togehter). If you would fix errors progressively (one after another until everything is fixed), then they will sum to 100%. However, this is not ideal because if you switch the order in which you fix the errors, you will get different results for their perceived importance which can be missleading (it's biased towards the error wich is fixed last). You can check section 2.3 and figure 2 in the paper which explains this in more depth. So to answer your question, the dAP values in Tide are computed individually and represent the expected gain in AP if you fix one error type (but not the others), always starting from the base AP, e.g. AP50 by default. Here is a short pseude code like example of the difference: # fix errors progressively
# when visualized, you will think bkg and miss are important, as they are fixed last
mAP = AP50
dAP = fix_errors(base=mAP, erros=[cls, loc, both, dupe, bkg, miss])
mAP + dAP = 1
# fix errors progressively
# now you will think cls is more important than bkg because their order was switched
dAP = fix_errors(base=mAP, erros=[bkg, loc, both, dupe, cls, miss])
mAP + dAP = 1
# fix errors individually (tide)
for e in [cls, loc, both, dupe, bkg, miss]:
dAP_e = fix_errors(base=mAP, erros=[e])
mAP += dAP_e
mAP != 1 Hope this helps :) |
as the picture show,all mAP' sum is not 100.
If you can give me some insight about this issue, I will appreciate.
The text was updated successfully, but these errors were encountered: