Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Phi Fei Fo Fum #548

Merged
merged 72 commits into from
Apr 23, 2023
Merged

Phi Fei Fo Fum #548

merged 72 commits into from
Apr 23, 2023

Conversation

mattansb
Copy link
Member

This PR is for writing the Phi Fei (V, W, T) paper, potentially for https://github.com/orgs/easystats/teams/core-team/discussions/5

@mattansb
Copy link
Member Author

Paper is currently a first draft, I invite any one willing to participate to lend a hand in editing / writing / formatting (don't mind switching to qmd instead or rmd, or even MSWord tbh...) @easystats/core-team

@codecov-commenter
Copy link

codecov-commenter commented Dec 30, 2022

Codecov Report

Merging #548 (180d8cf) into main (bb38e03) will not change coverage.
The diff coverage is n/a.

❗ Current head 180d8cf differs from pull request most recent head b775fc7. Consider uploading reports for the commit b775fc7 to get more accurate results

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@           Coverage Diff           @@
##             main     #548   +/-   ##
=======================================
  Coverage   89.56%   89.56%           
=======================================
  Files          56       56           
  Lines        3391     3391           
=======================================
  Hits         3037     3037           
  Misses        354      354           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@IndrajeetPatil
Copy link
Member

Thanks, @mattansb, for getting the ball rolling! Looks like a great start.

For now, I think we should work off of an .Rmd doc, which supports a lot more article formats via the {rticles} package.

I have already added some commits, and will keep editing as I find time.

Btw, currently, fei is not printed in the PDF output. I found this solution to fix this. Need to try it out.

@IndrajeetPatil
Copy link
Member

Team, please include [skip ci] in your commit messages and avoid unnecessary energy wastage through CI/CD, which is not relevant for this PR, at least as long as it remains laser-focused on the paper.

@rempsyc
Copy link
Member

rempsyc commented Jan 11, 2023

I started working on the paper a bit and made some changes. I had [skip ci] in my working memory (and memory pad) right up until I committed the message 🙈 so cancelled all corresponding GHAs after they started. For now, I've fixed the blank פ issue by using unicode. I don't know if having it as a math-mode symbol has a distinct advantage for us here though.

[skip ci]
@mattansb
Copy link
Member Author

Thanks Remi (and ChatGPT)! Please add yourself to the paper (:

Merge branch 'phi-fei-fo-fum' of https://github.com/easystats/effectsize into phi-fei-fo-fum

# Conflicts:
#	papers/Phi Fei Fo Fum (WIP)/paper.pdf
@rempsyc
Copy link
Member

rempsyc commented Jan 11, 2023

Ok! @mattansb what do you think of this new abstract/intro/conclusion?

@mattansb
Copy link
Member Author

Looks good to me! Thanks Remi!

@strengejacke
Copy link
Member

I looked at the current draft of the paper - do you think it will be realistic to submit it to the special issue? There's still some work to be done, in particular some kind of methods section, or at least some separation of the "theoretical" part and "results" part, and the discussion. Futhermore, I think we should use proper tables or in-text reporting of results, and exclude the R code (this is for supplemental).

Copy link
Contributor

@bwiernik bwiernik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few revisions and comments on sections we should add for clarity. Looking good!

papers/Phi Fei Fo Fum (WIP)/paper.Rmd Outdated Show resolved Hide resolved
papers/Phi Fei Fo Fum (WIP)/paper.Rmd Outdated Show resolved Hide resolved
papers/Phi Fei Fo Fum (WIP)/paper.Rmd Outdated Show resolved Hide resolved
papers/Phi Fei Fo Fum (WIP)/paper.Rmd Outdated Show resolved Hide resolved
Unfortunately, popular software do not always offer the necessary implementations of the specialized effect sizes necessary for a given research design. In this paper, we review the most commonly used effect sizes for analyses of categorical variables that use the $\chi^2$ (chi-square) statistic, and introduce a new effect size---`r "\U05E4"` (Fei).

Importantly, we offer researchers an applied walk-through on how to use these effect sizes in practice thanks to the `{effectsize}` package [@benshachar2020effectsize] in the R programming language [@base2023], which implements these measures and their confidence intervals. We cover in turn tests of independence (phi, Cramér’s *V*) and tests of goodness of fit (Cohen’s *w* and a new proposed effect size, Fei).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need a short section here on effect sizes in general and describing Cohen's d and Pearson r and why they are inadequate for categorical data. Also commenting on odds/risk ratios and why we might not like them (hard to interpret, not bounded 0-1). Basically say that we want effect sizes appropriate for categorical data that are in a correlation like metric.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something like this?

  • Odds ratios and risk ratios are only good for 2x2 table
    • RR is also not symmetrical (columns and rows are not exchangeable)
  • Pearson's r is appropriate for two qualitative variables (cf. biserial correlation) for which (co)variance is meaningful.
  • Cohen's d is applicable for a qualitative outcome (and is not symmetrical) for which a mean and variance are meaningful.

What else?

papers/Phi Fei Fo Fum (WIP)/paper.Rmd Outdated Show resolved Hide resolved
fei(O, p = p_E)
cohens_w(O, p = p_E)
```

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add a short section to each part above demonstrating not only the tables that produce 0 and 1 but also some "small", "medium", and "large" correlations to help readers get a sense of what different values of these effect sizes look like in data.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what you mean..?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in addition to the real-data example, we should also demonstrate, eg, what different "benchmark" small, medium, and large correlation values (e.g., .1, .3, .5, .7, .9) look like in a contingency table

papers/Phi Fei Fo Fum (WIP)/paper.bib Outdated Show resolved Hide resolved
papers/Phi Fei Fo Fum (WIP)/paper.Rmd Outdated Show resolved Hide resolved
papers/Phi Fei Fo Fum (WIP)/paper.Rmd Outdated Show resolved Hide resolved
@bwiernik
Copy link
Contributor

bwiernik commented Mar 6, 2023

@rempsyc did those "new statistics" changes make it into the Rmd?

@rempsyc
Copy link
Member

rempsyc commented Mar 6, 2023

No, as we have abandoned the Rmd to work exclusively on the word doc, in my understanding.

@mattansb mattansb merged commit 65600c4 into main Apr 23, 2023
@mattansb mattansb deleted the phi-fei-fo-fum branch April 23, 2023 05:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants