---
title: "Related software"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Related software}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

This is a list of other packages and apps for error detection, with no claim to completeness. They are roughly grouped by how much they have been used in practice. If you know of relevant software that isn't listed here, please contact me (email: jung-lukas\@gmx.net).

## Established methods

*Techniques that come up often in discussions of error checking.*

-   [statcheck](https://michelenuijten.shinyapps.io/statcheck-web/) by Michèle Nuijten and Sacha Epskamp is the best-known error detection software. It recomputes *p*-values and tests them for consistency with their test statistic, such as *t* or *F*. It reads PDF files automatically, so it can scan [large numbers](https://link.springer.com/article/10.3758/s13428-015-0664-2) of published articles. Steve Haroz built a [simple edition](https://steveharoz.shinyapps.io/statchecksimple) of the statcheck web app.

-   James Heathers' [SPRITE](https://web.archive.org/web/20241128150317/https://peerj.com/preprints/26968v1/) algorithm reconstructs possible distributions of raw data from summary statistics; see also his [light introduction](https://web.archive.org/web/20240716175701/https://medium.com/hackernoon/introducing-sprite-and-the-case-of-the-carthorse-child-58683c2bfeb). In R, it is implemented in [rsprite2](https://lukaswallrich.github.io/rsprite2/) by Lukas Wallrich, building on code by Nick Brown.

## Up and coming

*Recent and ongoing development in forensic meta-science methods.*

-   The [unsum](https://lhdjung.github.io/unsum/) package implements CLOSURE, a technique by Nathanael Larigaldie that generalizes SPRITE to find all possible samples, not just a few. Its core is written in Rust, which makes it fast enough for moderately large samples.

-   Ian Hussey's raft of checks for summary statistics:

    -   The [strait](https://github.com/ianhussey/strait) R package tests whether a mean and SD are possible for data with a known minimum and maximum, such as Likert scales, and assesses the magnitude of variability given the scale bounds. Its methods were formerly called TIDES, as is the [Shiny app](https://errors.shinyapps.io/TIDES/) that implements them.

    -   The [ANCHOR app](https://errors.shinyapps.io/ANCHOR/) checks the consistency between a whole sample and its subgroups.

    -   The [PORT app](https://errors.shinyapps.io/PORT/) tests correlation tables for consistency.

-   The [ellipse of insignificance app](https://drg85.shinyapps.io/EOIROAR/) by David Robert Grimes tests the robustness of dichotomous outcome trials.

-   [ScrutiPy](https://github.com/nrposner/scrutipy) by Nicolas Roman Posner brings some of scrutiny's functionality to Python. It also features CLOSURE and methods to recalculate confusion matrices. Like unsum, it is built on Rust.

## Possibly helpful

*Less well-known projects, not widely used in forensic meta-science, that may be useful for error checking. Their forensic utility has not been thoroughly examined.*

-   The R package [validate](https://cran.r-project.org/package=validate) by Mark P.J. van der Loo provides many tools for data checking.

-   The delta-F test for linearity, a.k.a. the "Förster test", was implemented in Dale J. Barr's R package [forsterUVA](https://github.com/dalejbarr/forsterUVA).

-   [tidychain](https://usrbinr.codeberg.page/tidychain/) by Alejandro Hagan is an R package for "calcChain" Excel analysis, as used in Data Colada's investigation into the work of Francesca Gino.

-   Several R packages use Benford's law, the distribution of leading digits in many naturally occurring numbers, to assess whether reported numbers are natural:

    -   [benford.analysis](https://github.com/carloscinelli/benford.analysis) by Carlos Cinelli provides tools for inspecting data with the Benford distribution.

    -   [jfa](https://koenderks.github.io/jfa/index.html) by Koen Derks offers a full statistical auditing suite (including Benford analysis).

-   R software for analyzing sequences emerged from the Pruitt investigations:

    -   The package [twopointzerothree](https://github.com/Sorbus-torminalis/twopointzerothree/) (by an anonymous developer) finds sequences of perfectly correlated numbers: duplicates, or duplicates offset by a constant.

    -   Similarly, the [sequenceSniffer](https://github.com/alrutten/sequenceSniffer) app by Anne Rutten detects repetitions in sequences.