diff --git a/36_ReproduciblePaper/README.md b/36_ReproduciblePaper/README.md new file mode 100644 index 0000000..f3c062e --- /dev/null +++ b/36_ReproduciblePaper/README.md @@ -0,0 +1,6 @@ +# How to (try to) publish a reproducible paper +- November 21 +- 10:00 am - 12:00 pm +- CCN (160 5th Ave) 4th floor classroom + +- [slides](https://flatironinstitute.github.io/sciware/36_ReproduciblePaper/slides.html) ([source](main.md)) diff --git a/36_ReproduciblePaper/assets/abstrusegoose.png b/36_ReproduciblePaper/assets/abstrusegoose.png new file mode 100644 index 0000000..909814a Binary files /dev/null and b/36_ReproduciblePaper/assets/abstrusegoose.png differ diff --git a/36_ReproduciblePaper/assets/communist.svg b/36_ReproduciblePaper/assets/communist.svg new file mode 100644 index 0000000..2099295 --- /dev/null +++ b/36_ReproduciblePaper/assets/communist.svg @@ -0,0 +1,58 @@ + + diff --git a/36_ReproduciblePaper/assets/future-you.png b/36_ReproduciblePaper/assets/future-you.png new file mode 100644 index 0000000..3e54ca9 Binary files /dev/null and b/36_ReproduciblePaper/assets/future-you.png differ diff --git a/36_ReproduciblePaper/assets/overlyhonestmethods.svg b/36_ReproduciblePaper/assets/overlyhonestmethods.svg new file mode 100644 index 0000000..fc6fde4 --- /dev/null +++ b/36_ReproduciblePaper/assets/overlyhonestmethods.svg @@ -0,0 +1,66 @@ + + diff --git a/36_ReproduciblePaper/assets/reproducibility.png b/36_ReproduciblePaper/assets/reproducibility.png new file mode 100644 index 0000000..beb0d28 Binary files /dev/null and b/36_ReproduciblePaper/assets/reproducibility.png differ diff --git a/36_ReproduciblePaper/main.md b/36_ReproduciblePaper/main.md new file mode 100644 index 0000000..fbad4f2 --- /dev/null +++ b/36_ReproduciblePaper/main.md @@ -0,0 +1,127 @@ +# Sciware + +## How to (try to) publish a reproducible paper + +https://sciware.flatironinstitute.org/36_ReproduciblePaper + +https://github.com/flatironinstitute/sciware/tree/main/36_ReproduciblePaper + + +## Rules of Engagement + +### Goal: + +Activities where participants all actively work to foster an environment which encourages participation across experience levels, coding language fluency, *technology choices*\*, and scientific disciplines. + +\*though sometimes we try to expand your options + + +## Rules of Engagement + +- Avoid discussions between a few people on a narrow topic +- Provide time for people who haven't spoken to speak/ask questions +- Provide time for experts to share wisdom and discuss +- Work together to make discussions accessible to novices + + +(These will always be a work in progress and will be updated, clarified, or expanded as needed.) + + + +## Future Sessions + +- Suggest topics or contribute to content in #sciware Slack +- We are recording. Link will be posted to [https://sciware.flatironinstitute.org/](https://sciware.flatironinstitute.org/) + + +## Today's Agenda + +- What is reproducibility and why should you care? +- Big picture questions to consider +- Steps towards reproducibility + + + +## What is reproduciblity? + +![](assets/reproducibility.png) + +#note: before we get started, let's define our terms. I'm using the definitions from the Turing Way, which locates reproducible research here: can I run the same analysis on the same data and get the same output? these other quadrants are also really important for good science, but beyond the scope of this talk. + +of these quadrants, reproducible should be the easiest. but that doesn't mean it's *easy* + + +## Why be reproducible at all? + +#note: why should you do this at all? this any of these steps requires extra time, which is time that you're not spending working on your next brilliant idea. If you look around online, people make a lot of different arguments for this, but for me I think there are two main reasons, one practical and one normative. + + +## The number one person you're helping is future you + +![](assets/future-you.png) + +#note: at some point, you will return to your code. maybe you submitted the paper and now have to revise it. maybe someone reaches out to you with a question about your paper, or because they want to follow up. maybe someone in your lab wants to use your data or analysis for a new project. maybe this work will be the beginning of a new research line. assuming you think your work has some value, it is unlikely that you're done with it when you submit a paper about it; or, more accurately, that you can accurately predict when you're done with it. + +the way I've heard this described is "your most important collaborator is yourself, six months from now, or five years from now". try not to make your life harder. + + +## Science requires transparency + +![](assets/communist.svg) + +#note: secondly, more normatively, doing science means being transparent. this is a paper from some philosophers of science, addressing what's known as the "demarcation problem": how do you determine whether something counts as science? their answer is that what really distinguishes science from other forms of inquiry is that scientists must share their work widely for public scrutiny and evaluation. you cannot just say "trust me", you have to explain why. + + +## But I've already written the paper! + +![](assets/abstrusegoose.png) + +#note: but you might say, I've already published the paper, surely if anyone has any questions about my analysis, they can just read my beautifully-written and completely clear paper + + +## Papers are insufficient + +![](assets/overlyhonestmethods.svg) + +#note: papers are insufficient for reproducibility. that can be for a lot of reasons. some of those reasons are silly, but some of them are simply that it can be hard to convey all the info required to run an analysis in prose in a way that's not totally mind-numbing. not to mention, there's a lot of implicit knowledge you have, which you might not realize is relevant. and *that* problem is exacerbated by the fact that you're writing for a broad audience: the info a senior professor in your field would need to reproduce your analysis is different than that needed by a junior grad student, and different again than that required by someone outside your field (if you're writing in a journal with a broader audience). + + +## Points to consider + +