Skip to content

Commit

Permalink
Last minute updates relatively complete.
Browse files Browse the repository at this point in the history
  • Loading branch information
larsvilhuber committed Jul 1, 2024
1 parent 4fc689d commit c1269c7
Show file tree
Hide file tree
Showing 11 changed files with 568 additions and 313 deletions.
14 changes: 8 additions & 6 deletions presentation/01-run_it_again.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,26 +18,28 @@ While the code, once set to run, can do so on its own, *you* might need to spend

---

![](images/Red-Warning-PNG-Clipart.png)
![](images/Red-Warning-PNG-Clipart.png){.center width="350" height="300"}


*This should be a warning sign:* if it takes you a long time to get it to run, or to manually reproduce the results, it might take others even longer.[^warning-sign]
*This should be a warning sign:*

If it takes you a long time to get it to run, or to manually reproduce the results, it **might take others even longer**.[^warning-sign]


[^warning-sign]: Source: [Red Warning PNG Clipart](https://www.pngall.com/warning-sign-png/download/69408), CC-BY.

---

Furthermore, it may suggest that you haven't been able to re-run your own code very often, which can be correlated with fragility or even lack of reproducibility.
Furthermore, it may suggest that you **haven't been able to re-run** your own code very often, which can be indicate **fragility** or even **lack of reproducibility**.

## Takeaways
## Takeaways {.smaller}

::: {.incremental}

- your code runs without problem, after all the debugging.
- [x] your code runs without problem, after all the debugging.
- [ ] your code runs without manual intervention, and with low effort
- [ ] it actually produces all the outputs
- [ ]your code generates a log file that you can inspect, and that you could share with others.
- [ ] your code generates a log file that you can inspect, and that you could share with others.
- [ ] it will run on somebody else's computer

:::
2 changes: 1 addition & 1 deletion presentation/02-00-intermezzo.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Automation and robustness checks, as well as efficiency.

Generating a log file means that you can inspect it, and you can share it with others. Also helps in debugging, for you and others.

## Will it run on somebody else's computer?
## Will it run on somebody else's computer? {.smaller}

Running it again does not help:

Expand Down
20 changes: 14 additions & 6 deletions presentation/02-hands_off_running.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,23 @@ Did it take you a long time to run everything again?

![](https://c.tenor.com/4qs0klfg8nMAAAAC/tenor.gif)

# Let's ramp it up a bit.
## Let's ramp it up a bit.

Your code must run, beginning to end, top to bottom, without error, and without any user intervention. This should in principle (re)create all figures, tables, and numbers you include in your paper.
- Your code must run, **beginning to end**, top to bottom, without error, and **without any user intervention**.

- This should in principle (re)create all **figures**, **tables**, and **in-text numbers** you include in your paper.


::: {.notes}
We have seen users who appear to highlight code and to run it interactively, in pieces, using the program file as a kind of notepad. This is not reproducible, and should be avoided. It is fine for debugging.
:::

## Seem trivial?

> Out of **8280** replication packages in ~20 top econ journals, only **2594** (**31.33%**) had a main/controller script.[^resultsmain]
[^resultsmain]: Results computed on Nov 26, 2023 based on a scan of replication packages conducted by Sebastian Kranz. 2023. "Economic Articles with Data". https://ejd.econ.mathematik.uni-ulm.de/, searching for the words `main`, `master`, `makefile`, `dockerfile`, `apptainer`, `singularity` in any of the program files in those replication packages. Code not yet integrated into this presentation.

## TL;DR

- Create a "main" file that runs all the other files in the correct order.
Expand All @@ -21,7 +29,7 @@ We have seen users who appear to highlight code and to run it interactively, in

## Creating a main or master script

In order to be able to enable "hands-off running", the main script is key. I will show here a few simple examples for single-software replication packages. We will discuss more complex examples in one of the next chapters.
In order to be able to enable "hands-off running", the **main (controller) script is key**. I will show here a few simple examples for single-software replication packages. We will discuss more complex examples in one of the next chapters.

## Examples

Expand Down Expand Up @@ -210,11 +218,11 @@ Do not use `Rscript`, as it will not generate enough output!

For examples for **Julia, Python, MATLAB,** and **multi-software scripts**, see the [full text](https://larsvilhuber.github.io/self-checking-reproducibility/02-hands_off_running.html).

## Takeaways
## Takeaways {.smaller}


- your code runs without problem, after all the debugging.
- your code runs without manual intervention, and with low effort
- [x] your code runs without problem, after all the debugging.
- [x] your code runs without manual intervention, and with low effort
- [ ] it actually produces all the outputs
- [ ] your code generates a log file that you can inspect, and that you could share with others.
- [ ] it will run on somebody else's computer
Expand Down
12 changes: 4 additions & 8 deletions presentation/03-automatically_saving_figures.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,15 +79,11 @@ Learn how to save tables in robust, reproducible ways. Do not try to copy-paste
`xtable`, `stargazer`, others.


## Takeaways
## Takeaways {.smaller}


::: {.incremental}

- ✅ your code runs without problem, after all the debugging.
- ✅your code runs without manual intervention, and with low effort
- ✅it actually produces all the outputs
- [x] your code runs without problem, after all the debugging.
- [x] your code runs without manual intervention, and with low effort
- [x] it actually produces all the outputs
- [ ] your code generates a log file that you can inspect, and that you could share with others.
- [ ] it will run on somebody else's computer

:::
10 changes: 5 additions & 5 deletions presentation/04-creating_log_files.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,11 +197,11 @@ which will create a log file with everything that would normally appear on the c



## Takeaways
## Takeaways {.smaller}


- your code runs without problem, after all the debugging.
- your code runs without manual intervention, and with low effort
- it actually produces all the outputs
- your code generates a log file that you can inspect, and that you could share with others.
- [x] your code runs without problem, after all the debugging.
- [x] your code runs without manual intervention, and with low effort
- [x] it actually produces all the outputs
- [x] your code generates a log file that you can inspect, and that you could share with others.
- [ ] it will run on somebody else's computer
Loading

0 comments on commit c1269c7

Please sign in to comment.