Skip to content

Tracking Responses to the "Reproducibility in Computer Science" Repository (http://reproducibility.cs.arizona.edu/)

Notifications You must be signed in to change notification settings

vraychev/repro-in-cs

 
 

Repository files navigation

Welcome to repo-repe-repro: the repository to repeat an experiment in
``reproducibility''! Recently, the results from

http://reproducibility.cs.arizona.edu/

were noticed by several people. This resulted in significant
discussion on social media sites:

https://plus.google.com/+ShriramKrishnamurthi/posts/Tf4D2HKQRKi
https://www.facebook.com/shriram.krishnamurthi/posts/10153931516105581?stream_ref=10

A quick reading would suggest that a large number of systems papers
have unusable artifacts. However, several authors reported that their
work had not been represented fairly. In addition, some disinterested
people found problems with the evaluation of work by others.

As the claims multiplied, it became clear that the evaluation done
here warranted further investigation. This repository is meant to
enable and publicize the findings of that investigation.

The data from this repository is visualized here: 

http://cs.brown.edu/~sk/Memos/Examining-Reproducibility/

You can focus on the data/ subdirectory and ignore everything else.

If you really want to build the Web site:

- install DrRacket 6.0 (http://download.racket-lang.org/)
- make sure racket is in your path
- run "make"

This will generate data/index.html.

Note: metadata/summary.csv and most of the data files in data/ are
from Collberg, et al. We thank them for making their data public,
which has enabled this project.

About

Tracking Responses to the "Reproducibility in Computer Science" Repository (http://reproducibility.cs.arizona.edu/)

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • TeX 60.2%
  • Shell 33.3%
  • Racket 6.3%
  • CSS 0.2%