Package {scrutiny}

scrutiny: Error Detection in Science

Description

Test published summary statistics for consistency (Brown and Heathers, 2017, doi:10.1177/1948550616673876; Allard, 2018, https://aurelienallard.netlify.app/post/anaytic-grimmer-possibility-standard-deviations/; Heathers and Brown, 2019, https://osf.io/5vb3u/). The package also provides infrastructure for implementing new error detection techniques.

Author(s)

Maintainer: Lukas Jung jung-lukas@gmx.net

Other contributors:

• Aurélien Allard [contributor]

• Nicolas Roman Posner [contributor]

See Also

Useful links:

• https://lhdjung.github.io/scrutiny/

• https://github.com/lhdjung/scrutiny/

• Report bugs at https://github.com/lhdjung/scrutiny/issues

Rename key columns according to the key arguments

Description

absorb_key_args() is superseded by function_map(), whose mappers rename key columns themselves.

Call it in a handwritten mapper. If a key argument such as x names a column of data, it renames that column to the key argument's name, so that the rest of the mapper finds it.

Usage

absorb_key_args(data, reported, key_cols_call = NULL)

Arguments

Value

data, possibly with columns renamed. Reassign the result to data.

Examples

df <- tibble::tibble(mean = c(5.19, 4.56), n = c(28, 30))
absorb_key_args(df, c("x", "n"), key_cols_call = c(x = "mean"))

Do reported counts add up to their total?

Description

add_up_to() checks whether reported subgroup sizes, or other counts, sum to the reported total.

The function is vectorized over cases, but add_up_to_map() is more convenient for testing many cases.

Usage

add_up_to(summands, total)

Arguments

Value

Logical. TRUE if the counts add up to total, FALSE if not, and NA if a value is missing, negative, or not a whole number, or if their sum or the total reaches 2^53, beyond which a sum is no longer exact.

Examples

add_up_to(c(40, 37, 43), total = 120)
add_up_to(c(40, 37, 44), total = 120)

# Several cases:
add_up_to(list(c(40, 37, 43), c(10, 12)), total = c(120, 23))

Test whether reported counts add up to their totals

Description

add_up_to_map() applies add_up_to() to every row of a data frame.

Call audit() on the output for summary statistics.

Usage

add_up_to_map(data, summands, total = NULL, ...)

Arguments

Value

A tibble with the summands columns, total, consistency, and any other columns from data, with the class scrutiny_add_up_to_map.

Summaries with audit()

audit() on add_up_to_map() output returns a tibble with one row and these columns:

1. all_cases: total number of rows.

2. incon_cases: number of inconsistent rows.

3. emp_incon: proportion of inconsistent rows.

Examples

df <- tibble::tibble(
  n_control = c(40, 25, 31),
  n_treatment = c(37, 26, 30),
  n_waitlist = c(43, 24, 30),
  total = c(120, 75, 90)
)

df |>
  add_up_to_map(summands = starts_with("n_"))

Summarize scrutiny objects

Description

audit() summarizes the output of scrutiny functions that test data frames, such as grim_map(). It is an S3 generic with methods for the functions listed below. The columns of each summary are described in the documentation of the respective function.

On the output of a sequence or total-n mapper, such as grim_map_seq(), it returns one row per case; see audit-special.

Usage

audit(data, ..., .by = NULL)

Arguments

Value

A tibble with summary statistics.

Run before audit()

Examples

pigs1 |>
  grim_map(digits_x = 2, items_averaged = 1, data_type = "integer") |>
  audit()

pigs4 |>
  duplicate_count() |>
  audit()

# Summaries by group:
pigs1 |>
  grim_map(digits_x = 2, items_averaged = 1, data_type = "integer") |>
  dplyr::mutate(study = rep(c("A", "B", "C"), each = 4)) |>
  audit(.by = study)

Classes of sequence and total-n mapper output

Description

audit() summarizes the output of functions made by function_map_seq(), such as grim_map_seq(), and of those made by function_map_total_n(), such as grim_map_total_n(), with one row per case. The columns of each summary are described in the documentation of the respective mapper.

Such output carries the generic class "scrutiny_map_seq" or "scrutiny_map_total_n", which audit() dispatches on, and a class for the particular mapper, listed below, so that you can write your own S3 methods for it. The summaries have the class "scrutiny_audit_seq" or "scrutiny_audit_total_n", which audit() summarizes further.

Sequence mappers

Total-n mappers

Examples

out <- pigs1 |>
  grim_map_seq(digits_x = 2, items_averaged = 1, data_type = "integer") |>
  audit()
out

# `audit()` summarizes the summary:
audit(out)

Compute minimal audit() summaries

Description

Call audit_cols_minimal() within an audit() method for the output of a consistency test mapper such as grim_map(). It returns a tibble with the three columns every such method must have:

1. all_cases: the number of rows, including undecidable ones.

2. incon_cases: the number of rows where consistency is FALSE. A case the test could not decide is NA there and not counted.

3. emp_incon: the empirical rate of inconsistency, incon_cases / all_cases.

Add further columns as needed. See Examples for how to name the method.

Usage

audit_cols_minimal(data)

Arguments

Value

A tibble with the three columns above.

See Also

vignette("consistency-tests-in-depth").

Examples

# For a mapper `schlim_map()` that applies a test called SCHLIM and
# returns a data frame with the class `"scrutiny_schlim_map"`:
audit.scrutiny_schlim_map <- function(data, ...) {
  audit_cols_minimal(data)
}

Deprecated summaries of sequence and total-n mapper output

Description

audit_seq() and audit_total_n() were folded into audit() in scrutiny 1.0.0. It returns the same summaries for the output of sequence and total-n mappers, and also has .by.

Usage

audit_seq(data)

audit_total_n(data)

Arguments

Check that a mapper's input has the right column names

Description

check_mapper_input_colnames() is superseded by function_map(), whose mappers check their input columns themselves.

Call it in a handwritten mapper. It errors if data lacks a key column, or if it already has a column named like the key result column ("consistency" by default).

Usage

check_mapper_input_colnames(
  data,
  reported,
  name_test,
  name_key_result = "consistency"
)

Arguments

Value

No return value; might throw an error.

See Also

vignette("consistency-tests-in-depth").

Is an object the output of a mapper function?

Description

• is_map_df() tests whether an object is the output of a mapper of any tier, i.e., of a function made by function_map(), function_map_seq(), or function_map_total_n(), such as grim_map().

• is_map_basic_df(), is_map_seq_df(), and is_map_total_n_df() test for one tier each.

Usage

is_map_df(x)

is_map_basic_df(x)

is_map_seq_df(x)

is_map_total_n_df(x)

Arguments

Details

See vignette("consistency-tests-in-depth") for the three tiers and the table at its end.

Value

Logical (length 1).

Examples

df1 <- pigs1 |>
  grim_map(digits_x = 2, items_averaged = 1, data_type = "integer")
df2 <- pigs1 |>
  grim_map_seq(digits_x = 2, items_averaged = 1, data_type = "integer")
df3 <- tibble::tribble(
  ~x1,  ~x2,  ~n,
  3.43, 5.28, 90,
  2.97, 4.42, 103
) |>
  grim_map_total_n(digits_x = 2, items_averaged = 1, data_type = "integer")

# All three are mapper output:
is_map_df(df1)
is_map_df(df2)
is_map_df(df3)

# Each is the output of one tier:
is_map_basic_df(df1)
is_map_seq_df(df2)
is_map_total_n_df(df3)

is_map_basic_df(df2)
is_map_seq_df(df3)

The DEBIT (descriptive binary) test

Description

debit() checks whether the reported mean and standard deviation of binary data are consistent with the reported sample size. With k ones among n values, the mean is k / n and the SD follows from k and n; the test asks whether any k produces both reported numbers at their decimal places.

The function is vectorized, but debit_map() is more convenient for testing many cases.

Usage

debit(
  x,
  sd,
  n,
  digits_x,
  digits_sd,
  data_type,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se"),
  formula = c("exact", "mean_n"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE
)

Arguments

Details

x, sd, n, digits_x, digits_sd, and data_type are vectorized and recycled to a common length by the tidyverse rules. All other arguments must have length 1.

x and sd are limited to about 15 significant digits, and x with n as in grim(). Within these limits, debit() decides value sets of any size exactly. With x missing, n must be below 2^52, about 4.5e15.

Value

Logical. TRUE if x, sd, and n are consistent, FALSE if not, and NA if the case cannot be decided: if n is not a positive whole number, or if a missing value could decide the case either way. See vignette("technical-notes"), section Missing values, on which cases a missing value leaves open. A sample SD divides by n - 1, so under sd_meaning = "sd_sample" (the default) and "se", n must also be at least 2.

References

Heathers, James A. J., and Brown, Nicholas J. L. 2019. DEBIT: A Simple Consistency Test For Binary Data. https://osf.io/5vb3u/.

See Also

debit_explain() to explain a result in plain words, debit_map() for data frames, and debit_values() and debit_plot_line() for the means and SDs behind a value set.

Examples

debit(
  x = 0.36,
  sd = 0.11,
  n = 20,
  digits_x = 2,
  digits_sd = 2,
  data_type = "integer"
)

Explain a DEBIT test step by step

Description

debit_explain() tests a single mean and SD of binary data like debit() does, but also lays out the calculation in plain words: which numbers of ones the mean allows, and whether the SD of any of them would be reported as the given one.

The text is built from the numbers that decide the test itself, so it always agrees with debit().

Usage

debit_explain(
  x,
  sd,
  n,
  digits_x,
  digits_sd,
  data_type,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se"),
  formula = c("exact", "mean_n"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE
)

Arguments

Value

An object of class scrutiny_explain: a list with the result as consistency (TRUE, FALSE, or NA), the explanation's paragraphs, and the test and its inputs. It prints as the explanation.

Examples

debit_explain(
  x = 0.36,
  sd = 0.11,
  n = 20,
  digits_x = 2,
  digits_sd = 2,
  data_type = "integer"
)

debit_explain(
  x = 0.35,
  sd = 0.49,
  n = 20,
  digits_x = 2,
  digits_sd = 2,
  data_type = "integer"
)

DEBIT-test many cases at once

Description

debit_map() applies debit() to every row of a data frame with means, standard deviations, and sample sizes of binary data.

Call audit() on the output for summary statistics and debit_plot() to visualize it.

Usage

debit_map(
  data,
  digits_x,
  digits_sd,
  x = NULL,
  sd = NULL,
  n = NULL,
  ...,
  data_type,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se"),
  formula = c("exact", "mean_n"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE,
  show_rec = FALSE
)

Arguments

Value

A tibble with these columns:

• x, sd, n: the inputs.

• data_type: the kind of data, as given.

• digits_x, digits_sd: the numbers of decimal places, as given.

• consistency: DEBIT consistency of x, sd, and n. NA for a case that cannot be decided, such as one with a missing value.

• Any other columns from data.

If show_rec is TRUE, further columns right after consistency hold the reconstructed values: the bounds of sd and x (sd_lower, sd_upper, x_lower, x_upper) with sd_incl_lower and sd_incl_upper saying whether the SD bounds are inclusive; and sum_lower and sum_upper, the least and the greatest number of ones among n values whose mean would be reported as x. Under formula = "exact", a value set is consistent exactly if some k from sum_lower to sum_upper has an SD within the bounds of sd.

The tibble has the class scrutiny_debit_map, which is recognized by audit() and debit_plot().

Summaries with audit()

audit() on debit_map() output returns a tibble with one row and these columns:

1. all_cases: total number of value sets.

2. incon_cases: number of DEBIT-inconsistent value sets.

3. emp_incon: proportion of DEBIT-inconsistent value sets.

4. mean_x: mean of the x values.

5. mean_sd: mean of the sd values.

6. distinct_n: number of distinct n values.

References

Heathers, James A. J., and Brown, Nicholas J. L. 2019. DEBIT: A Simple Consistency Test For Binary Data. https://osf.io/5vb3u/.

See Also

explain_map() and explain_print() to explain each row's result in plain words.

Examples

# Call `debit_map()` on binary summary data such as these:
pigs3

# The `consistency` column shows whether the
# values to its left are DEBIT-consistent:
pigs3 |>
  debit_map(digits_x = 2, digits_sd = 2, data_type = "integer")

# Get test summaries with `audit()`:
pigs3 |>
  debit_map(digits_x = 2, digits_sd = 2, data_type = "integer") |>
  audit()

# Explain the results in plain words:
pigs3 |>
  debit_map(digits_x = 2, digits_sd = 2, data_type = "integer") |>
  explain_print(1:2)

DEBIT-test values around the reported ones

Description

debit_map_seq() varies the reported means, standard deviations, and sample sizes a few steps up and down and DEBIT-tests each resulting value set. This shows whether a small error in reporting or computing could explain an inconsistency.

Call audit() on the output for summary statistics.

Usage

debit_map_seq(
  data,
  digits_x,
  digits_sd,
  x = NULL,
  sd = NULL,
  n = NULL,
  ...,
  data_type,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se"),
  formula = c("exact", "mean_n"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE,
  show_rec = FALSE,
  var = c("x", "sd", "n"),
  dispersion = 1:5,
  out_min = "auto",
  out_max = "auto",
  include_reported = FALSE,
  include_consistent = FALSE
)

Arguments

Value

A tibble with the columns of debit_map() output plus diff_var, case, and var, as in grim_map_seq(). As there, the other columns of data are dropped.

Summaries with audit()

audit() on debit_map_seq() output returns a tibble with one row per case and these columns:

• x, sd, and n: the reported values, with data_type and their consistency.

• hits_total: the number of DEBIT-consistent value sets found within the dispersion range.

• hits_x, hits_sd, hits_n: the number found by varying x, sd, and n, respectively. If both include_reported and include_consistent are TRUE, a consistent reported value set is counted as well: once per variable here, and once in hits_total.

• diff_x: the distance from x to the nearest consistent dispersed value, in dispersion steps. diff_x_up and diff_x_down are the signed steps to the nearest consistent value above and below (⁠_down⁠ is negative).

• diff_sd, diff_sd_up, diff_sd_down: the same for sd.

• diff_n, diff_n_up, diff_n_down: the same for n.

A count or distance is NA if it depends on a dispersed value set whose consistency is NA, on a variable whose reported value is missing, which is not dispersed, or on rows removed from the output, as by filter().

Call audit() again on that summary to summarize these columns further. na_count and na_rate there count the cases without a consistent value within the dispersion range, and those that a missing value leaves unknown.

Examples

# `debit_map_seq()` takes the same input as `debit_map()`:
pigs3

out <- pigs3 |>
  dplyr::slice(3:4) |>
  debit_map_seq(digits_x = 2, digits_sd = 2, data_type = "integer")
out

# Case-wise summaries:
out |>
  audit()

DEBIT-test with hypothetical group sizes

Description

Some studies report group means and standard deviations of binary data but only the total sample size, not the group sizes. For two groups, debit_map_total_n() tests hypothetical splits of the total, as grim_map_total_n() does.

Call audit() on the output for summary statistics.

Usage

debit_map_total_n(
  data,
  digits_x,
  digits_sd,
  x1 = NULL,
  x2 = NULL,
  sd1 = NULL,
  sd2 = NULL,
  n = NULL,
  ...,
  data_type,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se"),
  formula = c("exact", "mean_n"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE,
  show_rec = FALSE,
  dispersion = 0:5,
  n_min = NULL,
  n_max = NULL,
  constant = NULL
)

Arguments

Value

A tibble like that of grim_map_total_n(), with x and sd columns for the reported values, dir recording which group is paired with the larger group size (the equal split of an even total appears under "forth" only), the other columns of debit_map() output, and the other columns of data.

Summaries with audit()

audit() on debit_map_total_n() output returns a tibble with one row per case and these columns:

• x1, x2, sd1, sd2, and n: the reported values.

• hits_total: the number of scenarios in which both groups are DEBIT-consistent, summing hits_forth and hits_back.

• hits_forth: the number of such scenarios with x2 and sd2 paired with the larger group.

• hits_back: the same with x1 and sd1 paired with the larger group.

• scenarios_total: the number of scenarios tested, each a split of n with one pairing of groups and statistics, leaving out those that could not be decided.

• hit_rate: hits_total / scenarios_total, or NA if no scenario could be decided.

A count is NA if it depends on a scenario that could not be decided, or on rows removed from the output, as by filter().

Call audit() again on that summary to summarize these columns further.

References

Bauer, P. J., & Francis, G. (2021). Expression of Concern: Is It Light or Dark? Recalling Moral Behavior Changes Perception of Brightness. Psychological Science, 32(12), 2042–2043. https://journals.sagepub.com/doi/10.1177/09567976211058727

Heathers, James A. J., and Brown, Nicholas J. L. 2019. DEBIT: A Simple Consistency Test For Binary Data. https://osf.io/5vb3u/.

See Also

function_map_total_n(), which created this function from debit_map().

Examples

df <- tibble::tribble(
  ~x1,  ~x2,  ~sd1,  ~sd2,  ~n,
  0.30, 0.28, 0.17, 0.10, 70,
  0.41, 0.39, 0.09, 0.15, 65
)

debit_map_total_n(df, digits_x = 2, digits_sd = 2, data_type = "integer")

df |>
  debit_map_total_n(digits_x = 2, digits_sd = 2, data_type = "integer") |>
  audit()

Visualize DEBIT results

Description

debit_plot() plots the output of debit_map(). Each value set is a rectangle spanning the rounding bounds of its mean and SD, in blue if consistent and red if not, against the DEBIT line: the SD of binary data as a function of their mean, for the given sample size.

Usage

debit_plot(
  data,
  show_labels = TRUE,
  show_full_scale = TRUE,
  show_theme_other = TRUE
)

Arguments

Details

The line is that of the statistic debit_map() was called with: the sample SD by default, the population SD under sd_meaning = "sd_pop", both under "sd_sample_or_pop", or the SE under "se".

Under formula = "exact", the default of debit_map(), the means k / n that would be reported as each value set's x are marked as points on the line, and a value set is consistent if one of them lies inside its rectangle. If n is large, only the points that would not be drawn over by others are drawn, with a message.

If n varies in data, the lines for the smallest and largest n are drawn and the band between them is shaded; it contains the lines for all other n.

A rectangle can be too small to see, so each is drawn over a point at the reported mean and SD. Value sets with NA in a column needed for drawing are left out with a warning, and so are those of continuous data, to which DEBIT doesn't apply.

The labels come from ggrepel::geom_text_repel(), which places them at random to minimize overlap. A fixed seed makes the placement the same on every run; resizing the plot may give a better one.

Colors come from one scale for colour and fill, mapped to consistency. Add another to change them, as in the examples.

Value

A ggplot object.

References

Heathers, James A. J., and Brown, Nicholas J. L. 2019. DEBIT: A Simple Consistency Test For Binary Data. https://osf.io/5vb3u/.

Examples

pigs3 |>
  debit_map(digits_x = 2, digits_sd = 2, data_type = "integer") |>
  debit_plot()

# Other colors:
pigs3 |>
  debit_map(digits_x = 2, digits_sd = 2, data_type = "integer") |>
  debit_plot() +
  ggplot2::scale_colour_manual(
    values = c("TRUE" = "#2f7d74", "FALSE" = "#c1522e"),
    aesthetics = c("colour", "fill"),
    guide = "none"
  )

Visualize the standard deviations that DEBIT admits

Description

debit_plot_line() draws the DEBIT test for a single value set, as grimmer_plot_line() does for GRIMMER. Of all SDs that could have been reported with digits_sd decimal places, each square is one that n binary values with mean x could have; the others are left out. The reported SD is marked above the row and labeled as consistent or inconsistent.

For many value sets, debit_plot() shows the results of debit_map(). debit_values() lists the unrounded means and SDs behind a reported pair.

Usage

debit_plot_line(
  x,
  sd,
  n,
  digits_x,
  digits_sd,
  data_type,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se"),
  formula = c("exact", "mean_n"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE,
  zoom = c("close", "medium", "wide"),
  color_con = "#0047ff",
  color_incon = "#ff2a00",
  color_attainable = "grey75"
)

Arguments

Details

zoom works as in grim_plot_line(), but with zoom = "wide", the row runs from 0 to 1, the range of debit()'s inputs.

Under formula = "mean_n", the squares are the SDs of every mean that would be reported as x, whether or not n binary values can have it.

Value

A ggplot object.

Examples

# A mean of 0.35 with an SD of 0.49 is consistent with 20 binary values,
# but 0.41 is not:
debit_plot_line(
  x = 0.35,
  sd = 0.49,
  n = 20,
  digits_x = 2,
  digits_sd = 2,
  data_type = "integer"
)
debit_plot_line(
  x = 0.35,
  sd = 0.41,
  n = 20,
  digits_x = 2,
  digits_sd = 2,
  data_type = "integer"
)

Achievable means and SDs of binary data

Description

debit_values() returns every pair of a mean and an SD that n binary values (0 and 1) can have and that would be reported as x and sd. It is the counterpart of grim_values() and grimmer_values() for debit(), which is TRUE exactly if there is at least one such pair.

debit_closest() returns the single achievable pair whose SD is closest to sd, like grimmer_closest(). If x and sd are consistent, it is one of the pairs that debit_values() returns. Otherwise, it is closest among the pairs whose mean would be reported as x, or, if there are none, it is the pair whose mean grim_closest() returns.

Usage

debit_values(
  x,
  sd,
  n,
  digits_x,
  digits_sd,
  data_type,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se"),
  formula = c("exact", "mean_n"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE
)

debit_closest(
  x,
  sd,
  n,
  digits_x,
  digits_sd,
  data_type,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se"),
  formula = c("exact", "mean_n"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE
)

Arguments

Details

More than a million numbers of ones whose mean would be reported as x is an error, since they are listed before the SD narrows them down. That takes an n of more than about 10^(digits_x + 6).

In debit_closest(), pairs whose SDs are equally close go by the mean closer to x, then by the smaller number of ones.

Value

A tibble with one row per achievable pair. case is the index of the value set, as in grim_values(). x and sd are the achievable values, n the reported sample size, and sum the number of ones, k in the mean k / n. An inconsistent value set has no rows, and one that cannot be decided has one row of NA but for n, as in debit().

debit_closest() returns the same columns with exactly one row per value set: of NA but for n if a value is missing or the value set cannot be decided.

See Also

debit(), debit_map(), debit_plot_line(), and sd_binary_groups().

Examples

# A mean of 0.35 and an SD of 0.49 are consistent with 20 binary values,
# 7 of them ones:
debit_values(
  x = 0.35,
  sd = 0.49,
  n = 20,
  digits_x = 2,
  digits_sd = 2,
  data_type = "integer"
)

# None if the value set is inconsistent:
debit_values(
  x = 0.35,
  sd = 0.41,
  n = 20,
  digits_x = 2,
  digits_sd = 2,
  data_type = "integer"
)

# With few decimal places and a large sample, there can be several:
debit_values(
  x = 0.5,
  sd = 0.5,
  n = 300,
  digits_x = 1,
  digits_sd = 1,
  data_type = "integer"
)

# The closest pair to an inconsistent one:
debit_closest(
  x = 0.35,
  sd = 0.41,
  n = 20,
  digits_x = 2,
  digits_sd = 2,
  data_type = "integer"
)

Count decimal places

Description

decimal_places() counts the decimal places of numbers, or of numbers written as strings.

Usage

decimal_places(x, sep = ".")

Arguments

Details

The function counts the run of digits after the first sep, after trimming whitespace and applying any exponent: "5.30%" has two decimal places, "1e-5" has five, and "1..5" has none. Missing and non-finite values have NA decimal places, as do the string "NA", a blank string, and strings that as.numeric() reads as an infinity or NaN.

Numeric values with more than 15 significant digits cannot be counted accurately; enter such numbers as strings.

Value

Integer. The number of decimal places of each element of x.

Trailing zeros

A numeric value has lost its trailing zeros: 3.70 is stored as 3.7 and has one decimal place. If trailing zeros matter, count them while the values are still strings, such as "3.70", and give the counts to the tests as digits_x or digits_sd, as in dplyr::mutate(data, digits_x = decimal_places(x), x = as.numeric(x)).

Examples

decimal_places(x = 2.851)
decimal_places(x = "2.851")

# Trailing zeros are only counted in strings:
decimal_places(x = c(7.3900, "7.3900"))

# Whitespace at the end of a string is ignored:
decimal_places(x = "6.0     ")

Count decimal places in a data frame

Description

decimal_places_df() is deprecated because it replaces the values by their counts, while the tests need both. Use decimal_places() in dplyr::mutate() instead, as in mutate(digits_x = decimal_places(x)).

decimal_places_df() replaces every value in the selected columns by its number of decimal places. By default, it operates on all columns coercible to numeric.

Usage

decimal_places_df(
  data,
  cols = everything(),
  check_numeric_like = TRUE,
  sep = "."
)

Arguments

Value

Data frame with the selected columns replaced by their counts of decimal places.

See Also

decimal_places().

Examples

iris <- iris |>
  tibble::as_tibble() |>
  dplyr::mutate(across(everything(), as.character))

# All numeric-like columns, silently leaving out `Species`:
iris |>
  decimal_places_df()

# Selected columns only:
iris |>
  decimal_places_df(cols = starts_with("Sepal"))

Vary hypothetical group sizes

Description

If a study reports a total sample size but no group sizes, these functions generate pairs of hypothetical group sizes that add up to the total:

• disperse() takes half of an even total and goes up and down from it.

• disperse2() takes the two consecutive numbers around half of an odd total, and goes down from the lower and up from the upper.

• disperse_total() takes the total itself and calls one of the two, depending on whether it is even or odd.

They are the basis of grim_map_total_n() and the other functions made by function_map_total_n().

Usage

disperse(n, dispersion = 0:5, n_min = 1L, n_max = NULL, constant = NULL)

disperse2(n, dispersion = 0:5, n_min = 1L, n_max = NULL, constant = NULL)

disperse_total(n, dispersion = 0:5, n_min = 1L, n_max = NULL, constant = NULL)

Arguments

Details

A pair with a group size below n_min or above n_max is dropped. Columns from constant are named after a named list's elements, or "constant1", "constant2", etc. for an unnamed one; a single pair is "constant".

Value

A tibble with these columns:

• n: the group sizes. Each pair of consecutive rows adds up to the total.

• n_change: the number of steps from the input n, negative for the lower group size of each pair.

References

Bauer, P. J., & Francis, G. (2021). Expression of Concern: Is It Light or Dark? Recalling Moral Behavior Changes Perception of Brightness. Psychological Science, 32(12), 2042–2043. https://journals.sagepub.com/doi/10.1177/09567976211058727

See Also

function_map_total_n() and grim_map_total_n().

Examples

# For a total of 40, `n` is 20:
disperse(n = 20)

disperse(n = 20, dispersion = c(3, 6, 10))

# For an odd total, `disperse2()` takes two group sizes:
disperse2(n = c(25, 26))

# `disperse_total()` takes the total itself:
disperse_total(n = 40)
disperse_total(n = 51)

# Values that repeat alongside the pairs:
disperse_total(37, constant = c("5.24", "3.80"))

df_constant <- tibble::tibble(
  name = c("Paul", "Mathilda"), age = 27:28,
  registered = c(TRUE, FALSE)
)
disperse_total(37, constant = df_constant)

Count duplicate values

Description

duplicate_count() returns a frequency table of the values in a vector or data frame. For a data frame, it pools the values of all columns and records which columns each value appears in.

This is a blunt tool for initial checks. If many values have few characters each, duplicates occur by chance and mean little.

Call audit() on the output for summary statistics.

Usage

duplicate_count(x, ignore = NULL, locations_type = c("character", "list"))

Arguments

Value

A tibble with these columns, or only the first two if x is not named:

• value: the distinct values of x.

• frequency: how often each occurs, in descending order.

• locations: the names of the columns of x in which value appears.

• locations_n: the number of such columns.

The tibble has the class scrutiny_dup_count, which is recognized by audit().

Summaries with audit()

audit() on duplicate_count() output returns summary statistics for frequency and locations_n (or frequency alone if x is not named).

See Also

• duplicate_count_colpair() for duplicates between pairs of columns.

• duplicate_tally() for counts next to each value.

• janitor::get_dupes() for duplicate rows.

Examples

iris |>
  duplicate_count()

iris |>
  duplicate_count() |>
  audit()

# Ignore some values:
iris |>
  duplicate_count(ignore = c("setosa", "versicolor", "virginica"))

Count duplicate values by column pair

Description

duplicate_count_colpair() counts, for every pair of columns in a data frame, the values that occur in both. Results are ordered by that count, in descending order.

Usage

duplicate_count_colpair(data, ignore = NULL, show_rates = TRUE)

Arguments

Value

A tibble with these columns:

• x, y: a pair of columns of data, one pair per row.

• count: the number of values of x that also occur in y. Missing and ignored values are not counted.

• total_x, total_y: the number of values in x and y that are neither missing nor ignored.

• rate_x: the proportion of x values that occur in y, i.e., count / total_x, or NA if total_x is 0.

• rate_y: the same for y. It differs from count / total_y if a value repeats within a column.

Summaries with audit()

audit() on duplicate_count_colpair() output returns summary statistics for the numeric columns.

See Also

• duplicate_count() for a frequency table.

• duplicate_tally() for counts next to each value.

• janitor::get_dupes() for duplicate rows.

Examples

mtcars |>
  duplicate_count_colpair()

mtcars |>
  duplicate_count_colpair() |>
  audit()

Count duplicates at each observation

Description

For every value in a vector or data frame, duplicate_tally() counts how often it occurs in the whole input, and reports the count next to the value.

Call audit() on the output for summary statistics.

Usage

duplicate_tally(x, ignore = NULL, colname_end = "n")

Arguments

Details

Values with few characters are often duplicated by chance: in iris, 99% of values have a duplicate. Duplicates are more meaningful when there are fewer values and they have more characters.

Values are compared and returned as strings, so 1 and "1" are duplicates, but 2 and "2.0" are not: 2.0 becomes "2".

Value

A tibble with the columns of x (a value column if x is a vector), each followed by its tally column. It has the class scrutiny_dup_tally, which is recognized by audit().

Summaries with audit()

audit() on duplicate_tally() output returns summary statistics for each tally column, with a .total row across all of them. It finds these columns by the names that duplicate_tally() gave them, so a renamed one is left out.

See Also

• duplicate_count() for a frequency table.

• duplicate_count_colpair() for duplicates between pairs of columns.

• janitor::get_dupes() for duplicate rows.

Examples

duplicate_tally(x = pigs4)

duplicate_tally(x = pigs4$snout)

pigs4 |>
  duplicate_tally() |>
  audit()

pigs4 |>
  duplicate_tally(ignore = c(8.13, 7.57))

Explain mapper results row by row

Description

explain_map() takes the output of grim_map(), grimmer_map(), or debit_map() and adds an explanation column: the calculation behind each row's result in plain words, as printed by grim_explain(), grimmer_explain(), and debit_explain() for a single case. It also takes the output of the sequence and total-n mappers built on them, such as grim_map_seq().

To read the explanations, use explain_print().

Usage

explain_map(data, ...)

Arguments

Details

Each row is tested again from its own values and the settings of the mapper call, such as rounding and x_meaning. If the result differs from the row's consistency value, the row must have been changed after testing, and explain_map() throws an error.

DEBIT explanations cover formula = "exact" only.

Value

data with an explanation column, a string per row.

Examples

pigs1 |>
  grim_map(digits_x = 2, items_averaged = 1, data_type = "integer") |>
  dplyr::filter(!consistency) |>
  explain_map()

Print the explanations of mapper results

Description

explain_print() prints the explanations behind the results of a mapper such as grim_map(), one row after the other. It calls explain_map() on the printed rows only.

Usage

explain_print(data, ...)

Arguments

Value

data, invisibly.

Examples

pigs1 |>
  grim_map(digits_x = 2, items_averaged = 1, data_type = "integer") |>
  explain_print(1:2)

# Inconsistent rows only:
pigs1 |>
  grim_map(digits_x = 2, items_averaged = 1, data_type = "integer") |>
  dplyr::filter(!consistency) |>
  explain_print()

Create new ⁠*_map()⁠ functions

Description

function_map() creates a basic mapper: a function that applies a single-case test such as grim_scalar() to every row of a data frame. grim_map(), grimmer_map(), and debit_map() are made by it.

See vignette("consistency-tests-in-depth"), section Creating basic mappers with function_map().

function_map() is a developer tool: it may change in a minor release, with a NEWS entry.

Usage

function_map(
  .fn,
  .reported = NULL,
  .name_test,
  .name_key_result = "consistency",
  .name_class = NULL,
  .args_disabled = NULL,
  .args_by_row = NULL,
  .args_fixed = NULL,
  .cols_helper = NULL,
  .col_names = NULL,
  .cols_derived = NULL,
  .reported_variadic = NULL,
  ...
)

Arguments

Details

.reported_variadic is for a test whose number of key columns depends on the data, such as whether the values in any number of columns add up to the value in another one. .fn takes those values as a single vector, and the factory-made function selects the columns with tidyselect; see Examples. The selection must not be empty and must not include a column that already has a role in the test. The selected columns are returned as themselves. Such a mapper cannot be wrapped by function_map_seq() or function_map_total_n(), which derive their arguments from .reported.

The output has the class "scrutiny_{tolower(.name_test)}_map", with spaces as underscores, e.g., "scrutiny_grim_map", in addition to any .name_class. The arguments that applied to the whole call, such as rounding, are recorded on the output; see scrutiny_meta().

Value

A function with these arguments:

• data: a data frame with the key columns. Other columns are permitted.

• The .args_by_row arguments, such as digits_x.

• If .reported_variadic was given, an argument by that name.

• One argument per .reported value, for naming a column of data that holds that value under another name.

• ...: must be empty. Every argument after it must be named in full.

• The other arguments of .fn, i.e., those not named in .reported, .reported_variadic, .args_disabled, or .args_fixed, with the same defaults.

Value returned by the factory-made function

A tibble with the .reported_variadic columns, the .reported columns, the helper columns that were given, the .args_by_row columns, and the key result column (consistency by default), which is TRUE for consistent value sets, FALSE for inconsistent ones, and NA if the test can't decide. Any .cols_derived columns, any .col_names columns, and the other columns of data follow, in that order.

Examples

# A single-case function for SCHLIM, a mock test:
schlim_scalar <- function(y, n) {
  (y / 3) > n
}

# The mapper:
schlim_map <- function_map(
  .fn = schlim_scalar,
  .reported = c("y", "n"),
  .name_test = "SCHLIM"
)

df1 <- tibble::tibble(y = 16:25, n = 3:12)
schlim_map(df1)

# A test over any number of columns: do the parts add up to the total?
# The single-case function takes the parts as one vector...
sum_check_scalar <- function(parts, total, tolerance = 0) {
  abs(sum(parts) - total) <= tolerance
}

# ...which `.reported_variadic` names:
sum_check_map <- function_map(
  .fn = sum_check_scalar,
  .reported = "total",
  .reported_variadic = "parts",
  .name_test = "SUMCHECK"
)

df2 <- tibble::tibble(
  item_1 = c(10, 20, 30),
  item_2 = c(5, 5, 5),
  item_3 = c(1, 2, 3),
  total  = c(16, 27, 40)
)

# The `parts` argument selects columns with tidyselect syntax:
sum_check_map(df2, parts = starts_with("item"))

Create new ⁠*_map_seq()⁠ functions

Description

function_map_seq() creates a sequence mapper from a basic mapper such as grim_map(): a function that varies the reported values a few steps up and down and tests each resulting value set, as grim_map_seq() does. By default, only inconsistent value sets are dispersed.

See vignette("consistency-tests-in-depth"), section Sequence mapper.

function_map_seq() is a developer tool: it may change in a minor release, with a NEWS entry.

Usage

function_map_seq(
  .fn,
  .var = .reported,
  .reported,
  .name_test,
  .name_key_result = "consistency",
  .name_class = NULL,
  .args_disabled = NULL,
  .dispersion = 1:5,
  .out_min = "auto",
  .out_max = "auto",
  .var_bounds = NULL,
  .include_reported = FALSE,
  .include_consistent = FALSE,
  ...
)

Arguments

Details

The arguments of function_map_seq() set the defaults of the corresponding arguments of the factory-made function, which can be overridden in each call.

Each variable in var is dispersed around its own reported value while the others stay as reported.

To export a factory-made function from another package, follow the purrr FAQ on adverbs; see vignette("consistency-tests-in-depth"), section Context and export.

Value

A function such as these:

The settings of .fn, i.e., its arguments after ..., such as rounding, are arguments of this function as well, with the same defaults. So are its helper arguments, such as items_averaged. Its ... pass any other arguments on to .fn, which is only of use for a .fn without ... of its own.

Conventions

Name the factory-made function after .fn with ⁠_seq⁠ appended, as in grim_map_seq() from grim_map(). The output has the classes "scrutiny_map_seq" and "scrutiny_{tolower(.name_test)}_map_seq". Prefix any class you add via .name_class with your package's name, as in "mypkg_schlim_map_seq".

Examples

# Function definition of `grim_map_seq()`:
grim_map_seq <- function_map_seq(
  .fn = grim_map,
  .reported = c("x", "n"),
  .name_test = "GRIM"
)

Create new ⁠*_map_total_n()⁠ functions

Description

function_map_total_n() creates a total-n mapper from a basic mapper such as grim_map(): a function for two groups whose statistics are reported with the total sample size but without the group sizes, as grim_map_total_n() is. It uses disperse_total() to create pairs of hypothetical group sizes that add up to the total, and tests the reported statistics against each pair.

See vignette("consistency-tests-in-depth"), section Total-n mapper.

function_map_total_n() is a developer tool: it may change in a minor release, with a NEWS entry.

Usage

function_map_total_n(
  .fn,
  .reported,
  .name_test,
  .name_key_result = "consistency",
  .name_class = NULL,
  .dispersion = 0:5,
  .n_min = 1L,
  .n_max = NULL,
  .constant = NULL,
  ...
)

Arguments

Details

To export a factory-made function from another package, follow the purrr FAQ on adverbs; see vignette("consistency-tests-in-depth"), section Context and export.

Value

A function such as these:

The settings of .fn, i.e., its arguments after ..., such as rounding, are arguments of this function as well, with the same defaults. So are its helper arguments, such as items_averaged. Its ... pass any other arguments on to .fn, which is only of use for a .fn without ... of its own.

Conventions

Name the factory-made function after .fn with ⁠_total_n⁠ appended, as in grim_map_total_n() from grim_map(). The output has the classes "scrutiny_map_total_n" and "scrutiny_{tolower(.name_test)}_map_total_n". Prefix any class you add via .name_class with your package's name, as in "mypkg_schlim_map_total_n".

References

Bauer, P. J., & Francis, G. (2021). Expression of Concern: Is It Light or Dark? Recalling Moral Behavior Changes Perception of Brightness. Psychological Science, 32(12), 2042–2043. https://journals.sagepub.com/doi/10.1177/09567976211058727

See Also

function_map_seq()

Examples

# Function definition of `grim_map_total_n()`:
grim_map_total_n <- function_map_total_n(
  .fn = grim_map,
  .reported = "x",
  .name_test = "GRIM"
)

The GRIM test (granularity-related inconsistency of means)

Description

grim() checks whether a reported mean of integer data is consistent with the reported sample size and the number of items that compose the mean. The mean of n * items_averaged integers is a whole number divided by n * items_averaged, so only certain values are possible; the test asks whether any of them would have been reported as x.

The function is vectorized, but grim_map() is more convenient for testing many cases.

Usage

grim(
  x,
  n,
  digits_x,
  items_averaged,
  data_type,
  x_meaning = c("mean", "percentage"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE,
  percent = deprecated()
)

Arguments

Details

items_averaged must be specified because a mean over several items is a mean of more values, which changes the test. A default of 1 would let multi-item scales pass unnoticed. data_type must be specified because GRIM only applies to whole numbers. A default of "integer" would let the test flag means of continuous data, which can take any value.

x, n, digits_x, items_averaged, and data_type are vectorized and recycled to a common length by the tidyverse rules: an argument has length 1 or the length of the longest. All other arguments must have length 1.

Numbers beyond the precision of a double are an error rather than a guess. x with its digits_x may have about 15 significant digits: x * 10^(digits_x + 1) must be below 2^52, about 4.5e15, and (x * 10^(digits_x + 1) + 10) * n * items_averaged below 2^53, about 9e15. A threshold with decimal places raises the power of ten.

Value

Logical. TRUE if x, n, and items_averaged are consistent, FALSE if not, and NA if the case cannot be decided: if a value is infinite, if n or items_averaged is not a positive whole number, or if a missing value could decide the case either way. See vignette("technical-notes"), section Missing values, on which cases a missing value leaves open.

Defaults

The defaults are meant to flag a value set only if it is inconsistent under every usual convention:

• rounding = "up_or_down": Reports rarely say how they rounded, and the common methods only disagree on ties. SPSS and Excel round a 5 away from zero, while R's round() and Python's round() round it to an even digit. The default accepts a tie rounded either way, so a value set that fails is inconsistent under all of them. Choose another method only if you know which one was used. threshold and symmetric do not affect it.

• x_meaning = "mean": GRIM was devised for means, and most values that it is applied to are means. A percentage needs to be marked as one, since the same number, such as 71, can be either.

References

Brown, N. J. L., & Heathers, J. A. J. (2017). The GRIM Test: A Simple Technique Detects Numerous Anomalies in the Reporting of Results in Psychology. Social Psychological and Personality Science, 8(4), 363–369. https://journals.sagepub.com/doi/10.1177/1948550616673876

See Also

grim_explain() to explain a result in plain words, grim_map() for data frames, grim_values() for the means that n integers can produce, and grim_exp_incon() for how informative the test is in a given case.

Examples

# A mean of 5.19 is not consistent with a sample size of 28:
grim(x = 5.19, n = 28, digits_x = 2, items_averaged = 1, data_type = "integer")

# However, it is consistent with a sample size of 32:
grim(x = 5.19, n = 32, digits_x = 2, items_averaged = 1, data_type = "integer")

# For a scale composed of two items:
grim(x = 2.84, n = 16, digits_x = 2, items_averaged = 2, data_type = "integer")

# With percentages instead of means -- here, 71%:
grim(
  x = 71,
  n = 43,
  digits_x = 0,
  items_averaged = 1,
  data_type = "integer",
  x_meaning = "percentage"
)

# Means of continuous data, such as reaction times, can take any value:
grim(
  x = 5.19,
  n = 28,
  digits_x = 2,
  items_averaged = 1,
  data_type = "continuous"
)

Possible GRIM inconsistencies

Description

These functions quantify how informative a GRIM test is for a given sample size and number of decimal places:

• grim_exp_incon() returns the probability that a mean or percentage with digits_x random decimal digits is GRIM-inconsistent with n. It is the exp_incon column of grim_map().

• grim_total() returns the number of such inconsistent values between two whole numbers, out of 10^digits_x (10^(digits_x + 2) with x_meaning = "percentage").

See vignette("grim-in-depth"), section GRIM statistics.

Usage

grim_exp_incon(
  x,
  n,
  digits_x,
  items_averaged,
  data_type,
  x_meaning = c("mean", "percentage"),
  rounding = "up_or_down",
  threshold = 5,
  percent = deprecated()
)

grim_total(
  x,
  n,
  digits_x,
  items_averaged,
  data_type,
  x_meaning = c("mean", "percentage"),
  rounding = "up_or_down",
  threshold = 5,
  percent = deprecated()
)

Arguments

Details

x, n, digits_x, items_averaged, and data_type are recycled to a common length by the tidyverse rules: an argument has length 1 or the length of the longest.

Value

Double. For integer data, grim_exp_incon() and grim_total() return NA if x is missing or infinite, if digits_x is missing, or if n or items_averaged is missing or not a positive whole number. These include every case that grim() cannot decide, and some that it can: a mean may pass whatever n is, but how many fail depends on it. For continuous data, they return 0, or NA where grim() is NA. Under data_type = NA, they return NA unless the count is 0. A count above 2^53, which only digits_x of 14 or more with x_meaning = "percentage" can reach, is the nearest number a double can hold.

References

Brown, N. J. L., & Heathers, J. A. J. (2017). The GRIM Test: A Simple Technique Detects Numerous Anomalies in the Reporting of Results in Psychology. Social Psychological and Personality Science, 8(4), 363–369. https://journals.sagepub.com/doi/10.1177/1948550616673876

See Also

grim() and grim_map().

Examples

# Many value sets are inconsistent here:
grim_exp_incon(
  x = 83.29,
  n = 21,
  digits_x = 2,
  items_averaged = 1,
  data_type = "integer"
)
grim_total(
  x = 83.29,
  n = 21,
  digits_x = 2,
  items_averaged = 1,
  data_type = "integer"
)

# None are in this case...
grim_exp_incon(
  x = 5.14,
  n = 120,
  digits_x = 2,
  items_averaged = 1,
  data_type = "integer"
)
grim_total(
  x = 5.14,
  n = 120,
  digits_x = 2,
  items_averaged = 1,
  data_type = "integer"
)

# ...but most would be if `x` were a percentage:
grim_exp_incon(
  x = 5.14,
  n = 120,
  digits_x = 2,
  items_averaged = 1,
  data_type = "integer",
  x_meaning = "percentage"
)
grim_total(
  x = 5.14,
  n = 120,
  digits_x = 2,
  items_averaged = 1,
  data_type = "integer",
  x_meaning = "percentage"
)

Poisson binomial test for the rate of GRIM consistency

Description

Given the output of grim_map(), grim_binomial() tests whether the rate of GRIM-consistent value sets is above what random decimal digits would produce. This approach has not been tested extensively; use it with care.

Usage

grim_binomial(
  data,
  alternative = c("greater", "less", "two.sided"),
  alpha = 0.05
)

Arguments

Details

Under the null hypothesis, the decimal digits of each mean are random, so its probability of consistency is 1 - exp_incon, with exp_incon from grim_exp_incon(). Since this differs between value sets, the test uses the Poisson binomial distribution rather than averaging the probabilities. Value sets whose consistency or exp_incon is NA are left out; exp_incon can be NA even where the value set has a definite result (see grim_exp_incon()). Value sets with data_type = "continuous" have an exp_incon of 0, so they count as certain successes: they leave the p-value unchanged but raise estimate and the interval. Filter them out first to compare only the others.

The two-sided p-value is twice the smaller one-sided one, capped at 1.

Correct value sets are always consistent, so a collection of them should be more consistent than random digits even if some are wrong. There is no corresponding reason to expect fewer consistent sets than random, hence the one-sided default.

The confidence interval is the exact Clopper-Pearson interval for the consistency rate, as in stats::binom.test(), with the same alternative as the test. It treats the value sets as sharing one probability, so it is an interval for their average. Compare it to the mean of 1 - exp_incon.

Value

Tibble with the columns estimate (observed consistency rate), statistic (number of consistent cases), p.value, parameter (number of cases), conf.low, conf.high, conf.level (1 - alpha), method, and alternative.

Examples

pigs1 |>
  grim_map(digits_x = 2, items_averaged = 1, data_type = "integer") |>
  grim_binomial()

Power of the GRIM binomial test

Description

grim_binomial_power() computes the statistical power of grim_binomial() for given parameters, to judge whether the test is informative for a data set before interpreting its p-value. Like the test itself, it has not been tested extensively.

Usage

grim_binomial_power(
  p0,
  p1,
  k = NULL,
  alternative = c("greater", "less", "two.sided"),
  alpha = 0.05
)

Arguments

Value

Numeric (length 1). The probability of rejecting the null hypothesis when the alternative is true.

Examples

# 10 means with a null probability of 0.5 each, 90% truly consistent:
grim_binomial_power(p0 = 0.50, p1 = 0.9, k = 10)

# The same with 30 means:
grim_binomial_power(p0 = 0.50, p1 = 0.9, k = 30)

# One null probability per value set:
grim_binomial_power(p0 = c(0.5, 0.7, 0.8, 0.9, 0.6, 0.5, 0.4), p1 = 0.9)

Explain a GRIM test step by step

Description

grim_explain() tests a single mean like grim() does, but also lays out the calculation in plain words: which true means the reported one could stand for, which sums of the underlying integers follow from them, and whether any of these sums is possible.

The text is built from the numbers that decide the test itself, so it always agrees with grim().

Usage

grim_explain(
  x,
  n,
  digits_x,
  items_averaged,
  data_type,
  x_meaning = c("mean", "percentage"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE
)

Arguments

Value

An object of class scrutiny_explain: a list with the result as consistency (TRUE, FALSE, or NA), the explanation's paragraphs, and the test and its inputs. It prints as the explanation.

See Also

grimmer_explain() and debit_explain() for the other tests, and explain_map() for mapper output.

Examples

grim_explain(
  x = 5.19,
  n = 28,
  digits_x = 2,
  items_averaged = 1,
  data_type = "integer"
)

grim_explain(
  x = 5.19,
  n = 32,
  digits_x = 2,
  items_averaged = 1,
  data_type = "integer"
)

grim_explain(
  x = 71,
  n = 43,
  digits_x = 0,
  items_averaged = 1,
  data_type = "integer",
  x_meaning = "percentage"
)

Granularity of non-continuous scales

Description

grim_granularity() returns the smallest difference between two means that a scale with items_averaged items and n observations can produce: 1 / (n * items_averaged).

Usage

grim_granularity(n, items_averaged = 1)

Arguments

Details

The granularity of a scale follows from its design, not from the smallest observed difference between two values. Continuous scales have no granularity.

Value

Numeric. Granularity.

References

Brown, N. J. L., & Heathers, J. A. J. (2017). The GRIM Test: A Simple Technique Detects Numerous Anomalies in the Reporting of Results in Psychology. Social Psychological and Personality Science, 8(4), 363–369. https://journals.sagepub.com/doi/10.1177/1948550616673876

Examples

grim_granularity(n = 16)
grim_granularity(n = 16, items_averaged = 2)

GRIM results on a grid of sample sizes and means

Description

grim_grid() tests every combination of sample size and fractional part of a mean with GRIM, for one number of decimal places and one rounding method. It is meant for studying GRIM's properties directly. grim_plot_grid() draws the inconsistent cells as its background raster.

Usage

grim_grid(digits_x, data_type, rounding = "up_or_down", threshold = 5)

Arguments

Details

The grid covers sample sizes from 1 to 10^digits_x, where GRIM stops ruling out any mean, and fractional parts in steps of 1 / 10^digits_x, without zero. All fractional parts of means are positive, so there is no symmetric argument.

For percentages, use the number of decimal places of the proportion, which is two more: grim_grid(2) for whole percentages, as in grim_map() with x_meaning = "percentage".

Grids for one or two decimal places are precomputed for every rounding method that has no threshold. All others are computed on demand, which takes some 20 to 30 seconds for three decimal places. Four or more would mean a grid of 100 million cells, so digits_x is at most 3.

Value

A tibble with one row per cell of the grid and the columns frac (the fractional part of a mean), n (the sample size), and consistency (the GRIM result).

Examples

grim_grid(digits_x = 1, data_type = "integer")

# Share of means ruled out per sample size:
grim_grid(digits_x = 2, data_type = "integer") |>
  dplyr::summarise(share_inconsistent = mean(!consistency), .by = n)

Number of items from granularity

Description

grim_items() is deprecated and will be removed in a future version of scrutiny. Use 1 / (n * gran) instead.

Usage

grim_items(n, gran, tolerance = .Machine$double.eps^0.5)

Arguments

GRIM-test many cases at once

Description

grim_map() applies grim() to every row of a data frame with means (or percentages) and sample sizes.

Set x_meaning to "percentage" if x contains percentages, and show_rec to TRUE to see the reconstructed values behind each result.

Call these functions on the output of grim_map():

• audit() for summary statistics.

• grim_plot_grid() to visualize the results.

• explain_print() to explain each result in plain words.

Usage

grim_map(
  data,
  digits_x,
  x = NULL,
  n = NULL,
  ...,
  items_averaged,
  data_type,
  x_meaning = c("mean", "percentage"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE,
  show_rec = FALSE
)

Arguments

Value

A tibble with these columns:

• x, n: the inputs.

• items_averaged: the number of items averaged into each score.

• data_type: the kind of data, as given.

• digits_x: the number of decimal places in x, as given.

• consistency: GRIM consistency of x, n, and items_averaged. NA for a case that cannot be decided, such as one with a missing value.

• exp_incon: the probability of GRIM inconsistency; see grim_exp_incon().

• Any other columns from data.

  The tibble has the class scrutiny_grim_map, which is recognized by audit() and grim_plot_grid().

Reconstructed numbers

With show_rec = TRUE, the output has these further columns, the same ones for every rounding method:

• rec_sum: the product of x and n (times items_averaged), i.e., the sum total implied by the reported mean. It is generally not a whole number.

• sum_lower, sum_upper: the least and the greatest whole-number sum total that would have been reported as x. A value set is consistent exactly if sum_lower is not greater than sum_upper. If it is inconsistent, the two are the whole numbers on either side of rec_sum, so their gap is always 1 and does not measure how far off the reported values are.

• rec_x_upper, rec_x_lower: the two achievable means closest to x, i.e., rec_sum rounded up and down, divided by n (times items_averaged).

With x_meaning = "percentage", rec_sum is x / 100 times n (times items_averaged), and rec_x_upper and rec_x_lower are percentages like x.

Summaries with audit()

audit() on grim_map() output returns a tibble with one row and these columns:

1. all_cases: total number of value sets.

2. testable_cases: number of value sets with a positive exp_incon.

3. incon_cases: number of GRIM-inconsistent value sets.

4. emp_incon: proportion of GRIM-inconsistent value sets.

5. exp_incon: mean probability of GRIM inconsistency.

6. ratio: ratio of emp_incon to exp_incon, both computed over the value sets that could be decided.

References

Brown, N. J. L., & Heathers, J. A. J. (2017). The GRIM Test: A Simple Technique Detects Numerous Anomalies in the Reporting of Results in Psychology. Social Psychological and Personality Science, 8(4), 363–369. https://journals.sagepub.com/doi/10.1177/1948550616673876

Examples

# Use `grim_map()` on data like these:
pigs1

# The `consistency` column shows whether the
# values to its left are GRIM-consistent:
pigs1 |>
  grim_map(digits_x = 2, items_averaged = 1, data_type = "integer")

# Show the reconstructed numbers:
pigs1 |>
  grim_map(
    digits_x = 2,
    items_averaged = 1,
    data_type = "integer",
    show_rec = TRUE
  )

# Get summaries with `audit()`:
pigs1 |>
  grim_map(digits_x = 2, items_averaged = 1, data_type = "integer") |>
  audit()

# Explain the results in plain words:
pigs1 |>
  grim_map(digits_x = 2, items_averaged = 1, data_type = "integer") |>
  explain_print(1:2)

# A table of means and percentages, tested in parts:
df <- tibble::tibble(
  x = c(5.19, 71),
  n = c(32, 43),
  is_percent = c(FALSE, TRUE)
)

df |>
  dplyr::filter(!is_percent) |>
  grim_map(digits_x = 2, items_averaged = 1, data_type = "integer")

df |>
  dplyr::filter(is_percent) |>
  grim_map(
    digits_x = 0,
    items_averaged = 1,
    data_type = "integer",
    x_meaning = "percentage"
  )

GRIM-test values around the reported ones

Description

grim_map_seq() varies the reported means and sample sizes a few steps up and down and GRIM-tests each resulting value set. This shows whether a small error in reporting or computing could explain an inconsistency.

Call audit() on the output for summary statistics.

Usage

grim_map_seq(
  data,
  digits_x,
  x = NULL,
  n = NULL,
  ...,
  items_averaged,
  data_type,
  x_meaning = c("mean", "percentage"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE,
  show_rec = FALSE,
  var = c("x", "n"),
  dispersion = 1:5,
  out_min = "auto",
  out_max = "auto",
  include_reported = FALSE,
  include_consistent = FALSE
)

Arguments

Value

A tibble with the columns of grim_map() output plus diff_var, the number of steps between the tested value and the reported one; case, the row number of the reported value set in data; and var, the dispersed variable. Other columns of data, such as study labels, play no role in the test and are dropped; use case to join them back.

Summaries with audit()

audit() on grim_map_seq() output returns a tibble with one row per case and these columns:

• x and n: the reported values, with items_averaged, data_type, and their consistency.

• hits_total: the number of GRIM-consistent value sets found within the dispersion range.

• hits_x, hits_n: the number found by varying x and n, respectively. If both include_reported and include_consistent are TRUE, a consistent reported value set is counted as well: once per variable here, and once in hits_total.

• diff_x: the distance from x to the nearest consistent dispersed value, in dispersion steps. diff_x_up and diff_x_down are the signed steps to the nearest consistent value above and below (⁠_down⁠ is negative).

• diff_n, diff_n_up, diff_n_down: the same for n.

A count or distance is NA if it depends on a dispersed value set whose consistency is NA, on a variable whose reported value is missing, which is not dispersed, or on rows removed from the output, as by filter().

Call audit() again on that summary to summarize these columns further. na_count and na_rate there count the cases without a consistent value within the dispersion range, and those that a missing value leaves unknown.

Examples

# `grim_map_seq()` takes the same input as `grim_map()`:
pigs1

out <- grim_map_seq(
  pigs1,
  digits_x = 2,
  items_averaged = 1,
  data_type = "integer",
  include_consistent = TRUE
)
out

# Case-wise summaries:
out |>
  audit()

GRIM-test with hypothetical group sizes

Description

Some studies report group means but only the total sample size, not the group sizes. For two groups, grim_map_total_n() tests hypothetical splits of the total: it starts from an even split, moves one participant at a time from one group to the other, and GRIM-tests both means against both group sizes, in both pairings.

Call audit() on the output for summary statistics.

Usage

grim_map_total_n(
  data,
  digits_x,
  x1 = NULL,
  x2 = NULL,
  n = NULL,
  ...,
  items_averaged,
  data_type,
  x_meaning = c("mean", "percentage"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE,
  show_rec = FALSE,
  dispersion = 0:5,
  n_min = 1L,
  n_max = NULL,
  constant = NULL
)

Arguments

Value

A tibble with these columns:

• x: the reported means, in pairs of rows.

• n: the hypothetical group sizes, with n_change recording the steps from half the total.

• both_consistent: whether both means of a pair are consistent with their hypothetical group sizes.

• case: the row number in data.

• dir: "forth" where x2 is paired with the larger group size, "back" where x1 is. The split of an even total into two equal groups is the same scenario either way, so it appears under "forth" only.

• The other columns of grim_map() output, and the other columns of data.

Summaries with audit()

audit() on grim_map_total_n() output returns a tibble with one row per case and these columns:

• x1, x2, and n: the reported values.

• hits_total: the number of scenarios in which both groups are GRIM-consistent, summing hits_forth and hits_back.

• hits_forth: the number of such scenarios with x2 paired with the larger group.

• hits_back: the same with x1 paired with the larger group.

• scenarios_total: the number of scenarios tested, each a split of n with one pairing of groups and statistics, leaving out those that could not be decided.

• hit_rate: hits_total / scenarios_total, or NA if no scenario could be decided.

A count is NA if it depends on a scenario that could not be decided, or on rows removed from the output, as by filter().

Call audit() again on that summary to summarize these columns further.

References

Bauer, P. J., & Francis, G. (2021). Expression of Concern: Is It Light or Dark? Recalling Moral Behavior Changes Perception of Brightness. Psychological Science, 32(12), 2042–2043. https://journals.sagepub.com/doi/10.1177/09567976211058727

Brown, N. J. L., & Heathers, J. A. J. (2017). The GRIM Test: A Simple Technique Detects Numerous Anomalies in the Reporting of Results in Psychology. Social Psychological and Personality Science, 8(4), 363–369. https://journals.sagepub.com/doi/10.1177/1948550616673876

See Also

function_map_total_n(), which created this function from grim_map().

Examples

df <- tibble::tribble(
  ~x1,  ~x2,  ~n,
  3.43, 5.28, 90,
  2.97, 4.42, 103
)

grim_map_total_n(df, digits_x = 2, items_averaged = 1, data_type = "integer")

# `hits_total` counts the splits in which both means are consistent:
df |>
  grim_map_total_n(digits_x = 2, items_averaged = 1, data_type = "integer") |>
  audit()

# More steps away from the even split tend to find more hits:
df |>
  grim_map_total_n(
    digits_x = 2,
    items_averaged = 1,
    data_type = "integer",
    dispersion = 0:10
  ) |>
  audit()

Deprecated name of grim_plot_grid()

Description

grim_plot() was renamed to grim_plot_grid() in scrutiny 1.0.0. It keeps the signature it had in scrutiny 0.6.1, so that calls by position still work.

Usage

grim_plot(
  data = NULL,
  show_data = TRUE,
  show_raster = TRUE,
  show_gradient = TRUE,
  n = NULL,
  digits = NULL,
  rounding = "up_or_down",
  color_cons = deprecated(),
  color_incons = deprecated(),
  tile_alpha = deprecated(),
  tile_size = 1.5,
  raster_alpha = deprecated(),
  raster_color = NULL,
  ...
)

Arguments

Visualize GRIM test results

Description

grim_plot_grid() plots the output of grim_map() or grimmer_map(). Each value set is placed by n * items_averaged (the sample size for single items) and the fractional part of its mean, in blue if consistent and red if not, on a background raster that shows which combinations are GRIM-consistent at all.

Usage

grim_plot_grid(
  data = NULL,
  show_data = TRUE,
  show_raster = TRUE,
  show_gradient = TRUE,
  split_by_digits = FALSE,
  n = NULL,
  digits_x = NULL,
  rounding = "up_or_down",
  threshold = 5,
  tile_size = 1.5,
  color_raster = "grey75"
)

Arguments

Value

A ggplot object. With split_by_digits = TRUE, a named list of ggplot objects, one per distinct non-zero number of decimal places (e.g., list(digits_1 = ..., digits_2 = ...)), returned invisibly after each plot is printed.

Background raster

For one or two decimal places, the raster shows every combination of sample size and fractional part that is GRIM-consistent in grey and every inconsistent one in white. It depends on the number of decimal places, which is read from the digits_x column of data, and on the rounding method, which is read from the grim_map() call. The white tiles are the cells of grim_grid() whose consistency is FALSE. With three or more decimal places, they would be too small, so a gradient shows the probability of inconsistency instead.

A raster only applies to one number of decimal places, so grim_plot_grid() errors if digits_x varies across rows. Use split_by_digits = TRUE to get one plot per number of decimal places instead, or digits_x to choose one.

With x_meaning = "percentage" in the grim_map() call, percentages are plotted as the proportions they were tested as, with two more decimal places.

Set show_data = FALSE to plot the raster alone, controlled by digits_x, rounding, threshold, and n.

Under the default rounding = "up_or_down", fewer values are inconsistent at sample sizes 40 and 80 than at their neighbors with two decimal places (4 and 8 with one), and none at 80 (or 8), because many achievable means there are ties, which the default accepts in both directions.

Negative means

The y-axis is the fractional part of the absolute value of x, so a mean of -2.51 is drawn at 0.51. GRIM's granularity is symmetric around zero, so this is the value set's place on the grid. A one-directional rounding method treats the absolute value of a negative number the other way: "ceiling" like "floor", and "up" like "down" (unless symmetric = TRUE). If all means are negative, the raster is the one for this mirrored method, so that it agrees with the tiles. With positive and negative means under such a method, no single raster does; it is then the one for positive means, with a warning. The tile's color comes from the consistency column and is unaffected.

References

Brown, N. J. L., & Heathers, J. A. J. (2017). The GRIM Test: A Simple Technique Detects Numerous Anomalies in the Reporting of Results in Psychology. Social Psychological and Personality Science, 8(4), 363–369. https://journals.sagepub.com/doi/10.1177/1948550616673876

Examples

# `grim_plot_grid()` reads the number of decimal places
# from the `digits_x` column:
pigs1 |>
  grim_map(digits_x = 2, items_averaged = 1, data_type = "integer") |>
  grim_plot_grid()

# The raster follows the rounding method of the `grim_map()` call:
pigs1 |>
  grim_map(
    digits_x = 2,
    items_averaged = 1,
    data_type = "integer",
    rounding = "ceiling"
  ) |>
  grim_plot_grid()

# Percentages are plotted as proportions:
pigs2 |>
  grim_map(
    digits_x = 1,
    items_averaged = 1,
    data_type = "integer",
    x_meaning = "percentage"
  ) |>
  grim_plot_grid()

# Other colors for the value sets:
pigs1 |>
  grim_map(digits_x = 2, items_averaged = 1, data_type = "integer") |>
  grim_plot_grid() +
  ggplot2::scale_colour_manual(
    values = c("TRUE" = "#2f7d74", "FALSE" = "#c1522e"),
    aesthetics = c("colour", "fill"),
    guide = "none"
  )

Visualize the means that GRIM admits

Description

grim_plot_line() draws the GRIM test for a single value set as a row of squares. Of all means (or percentages) that could have been reported with digits_x decimal places, each square is one that a sample of size n can have; the others are left out. The reported value is marked above the row and labeled as consistent or inconsistent.

grim_plot_grid() shows many results at once; this function explains one. grim_values() lists the unrounded means behind a reported one.

Usage

grim_plot_line(
  x,
  n,
  digits_x,
  items_averaged,
  data_type,
  x_meaning = c("mean", "percentage"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE,
  zoom = c("close", "medium", "wide"),
  color_con = "#0047ff",
  color_incon = "#ff2a00",
  color_attainable = "grey75"
)

Arguments

Details

With zoom = "close", the row shows 10 values on either side of x, so that the marker and the strip behind it stand out. If x is inconsistent, it widens until the nearest attainable value is in view, but no farther than the row with zoom = "wide" is wide. It does not widen further for the nearest one on the other side: attainable percentages in particular can be dozens of values apart, so the squares and the strip would shrink to specks.

With zoom = "medium", the row widens until the nearest attainable value other than x is in view on either side, whether x is consistent or not, which shows how far apart the attainable values are. Again, it widens no farther than the row with zoom = "wide". Both settings test at most 20,000 values, and warn if they stop short of an attainable value for that reason.

Each attainable value is labeled below its square unless the squares are too close together for that, as they mostly are with zoom = "wide". Then the axis has the usual breaks, and if x is inconsistent, the nearest attainable values are labeled above the row.

With zoom = "wide", the row runs from the whole number at or below x to the next one up. With items_averaged = 1, about n of the values in this unit are attainable, or all if n exceeds 10 ^ digits_x. With x_meaning = "percentage", the unit is that of the proportion GRIM tests, so the row runs from 0% to 100%, and about n + 1 values are attainable, or all if n exceeds 10 ^ (digits_x + 2). Squares too narrow to be seen as such are drawn as ticks of the same height.

The strip behind the squares is the range of unrounded means that would have been reported as x, from roundwork::unround(). The value is consistent exactly if a square stands within the strip. Under methods such as "trunc" or "ceiling", one edge of the strip is not part of it, so a square on that edge does not count.

Value

A ggplot object.

See Also

grimmer_plot_line() for the same figure over standard deviations.

Examples

# A mean of 5.19 is not consistent with a sample size of 28:
# no square stands where the marker points.
grim_plot_line(x = 5.19, n = 28, digits_x = 2, items_averaged = 1, data_type = "integer")

# With `n = 26`, it is:
grim_plot_line(x = 5.19, n = 26, digits_x = 2, items_averaged = 1, data_type = "integer")

# The same for a percentage:
grim_plot_line(
  x = 67.4,
  n = 40,
  digits_x = 1,
  items_averaged = 1,
  data_type = "integer",
  x_meaning = "percentage"
)

Compare GRIM inconsistency rates to random means

Description

grim_plot_rates() plots one point per study: its observed rate of GRIM-inconsistent value sets against the rate that means with random decimal digits would show at the same sample sizes and numbers of decimal places. This helps spot both problematic studies and problems in GRIM usage. See Details.

Correctly reported means are always consistent, so a study off the dotted line at zero has a problem, either in the study or in the GRIM analysis. The dashed diagonal marks studies where GRIM inconsistencies are as frequent as expected if the means' digits were random.

Usage

grim_plot_rates(data, study)

Arguments

Details

The expected rate of a study is the mean of the exp_incon column, from grim_exp_incon(). Value sets whose exp_incon is zero or missing are left out because GRIM cannot test them; if none remain, that is an error. Value sets without a study are left out with a warning.

Point area is the number of GRIM-testable value sets. Color shows the ratio of the observed rate to the expected one: 0 means no inconsistencies, 1 as many as random means. Studies under the diagonal are above 1, in darker colors.

Nothing should push a study's rate above chance, so a large point clearly below the diagonal points to an error in the GRIM analysis. A small point can land there by chance.

Value

A ggplot object.

See Also

grim_binomial() for a test of the same comparison.

Examples

# Assume each unique code indicates one study
codes <- c(
  "B", "B", "C", "C", "A", "C",
  "A", "D", "A", "A", "A", "A"
)

pigs1 |>
  grim_map(digits_x = 2, items_averaged = 1, data_type = "integer") |>
  dplyr::mutate(study = codes) |>
  grim_plot_rates(study)

Deprecated name of grim_exp_incon()

Description

grim_probability() was renamed to grim_exp_incon() in scrutiny 1.0.0, which also requires digits_x and data_type, takes items_averaged instead of items and x_meaning instead of percent, and has a rounding argument.

Usage

grim_probability(...)

Arguments

Achievable means behind a reported one

Description

These functions return the means (or percentages) that a sample of n * items_averaged integers can have:

• grim_values() returns every achievable value that would be reported as x, with the whole-number sum behind it. The result has no rows if x is GRIM-inconsistent with n.

• grim_closest() returns the single achievable value closest to x among those that would be reported as x, or, if there are none, the closest one overall. Of two equally close values, it returns the smaller.

Both are vectorized like grim(); case says which value set a row belongs to. grimmer_values() and grimmer_closest() are the counterparts for GRIMMER.

Usage

grim_values(
  x,
  n,
  digits_x,
  items_averaged,
  data_type,
  x_meaning = c("mean", "percentage"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE
)

grim_closest(
  x,
  n,
  digits_x,
  items_averaged,
  data_type,
  x_meaning = c("mean", "percentage"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE
)

Arguments

Details

The achievable means are the whole numbers in the range that grim_map() shows as sum_lower and sum_upper under show_rec, each divided by n * items_averaged. Once n * items_averaged exceeds 10^digits_x, several of them can be reported as the same x, and their number grows with n * items_averaged. GRIM is then no longer informative. grim_closest() stays cheap in that case. grim_values() does not, and more than a million values for one value set is an error.

With x_meaning = "percentage", the values are percentages, like x.

Under rounding = "even", or its synonym "ties_even", both bounds of that range are inclusive (see roundwork::unround()), so the values include ties that base::round() may round away from x: grim_values(1.3, 20, 1, 1, "integer", rounding = "even") returns 1.25 and 1.35 along with 1.3.

With items_averaged > 1, the integers are the respondents' totals across the items, so sum is the sum of the totals.

Value

• For grim_values(), a tibble with one row per achievable mean. case is the index of the value set, i.e., the position in the inputs after they are recycled to a common length, so it also counts through n if x is a single value. x is the achievable value, n the reported sample size, items_averaged as given, and sum the whole number behind x, the sum of all n * items_averaged item scores. An inconsistent value set has no rows, and one that cannot be decided has one row of NA but for n and items_averaged, as in grim(). So does one with a missing digits_x that is consistent whatever it is: how many values would be reported as x still depends on it.

• For grim_closest(), a tibble with the same columns and exactly one row per value set: the closest achievable mean, or a row of NA but for n and items_averaged if the value set cannot be decided.

References

Brown, N. J. L., & Heathers, J. A. J. (2017). The GRIM Test: A Simple Technique Detects Numerous Anomalies in the Reporting of Results in Psychology. Social Psychological and Personality Science, 8(4), 363–369. https://journals.sagepub.com/doi/10.1177/1948550616673876

See Also

grim(), grim_map(), grimmer_values(), and grim_exp_incon().

Examples

# A mean of 5.19 is consistent with a sample size of 32,
# and there is exactly one way to get there:
grim_values(x = 5.19, n = 32, digits_x = 2, items_averaged = 1, data_type = "integer")

# With a larger sample, several means round to 5.19:
grim_values(x = 5.19, n = 300, digits_x = 2, items_averaged = 1, data_type = "integer")

# It is not consistent with a sample size of 28...
grim_values(x = 5.19, n = 28, digits_x = 2, items_averaged = 1, data_type = "integer")

# ...but 5.19 is close to a mean that is achievable:
grim_closest(x = 5.19, n = 28, digits_x = 2, items_averaged = 1, data_type = "integer")

# Both functions are vectorized. In `grim_values()`, `case` is the index of
# the value set, and an inconsistent one has no rows:
grim_values(
  x = c(5.19, 4.2),
  n = c(28, 30),
  digits_x = 2,
  items_averaged = 1,
  data_type = "integer"
)
grim_closest(
  x = c(5.19, 4.2),
  n = c(28, 30),
  digits_x = 2,
  items_averaged = 1,
  data_type = "integer"
)

The GRIMMER test (granularity-related inconsistency of means mapped to error repeats)

Description

grimmer() checks whether a reported mean and standard deviation of integer data are consistent with the reported sample size and the number of items averaged. It extends grim(): the mean must pass GRIM, and the SD must then be reachable from one of the integer sums that GRIM admits.

The function is vectorized, but grimmer_map() is more convenient for testing many cases.

Usage

grimmer(
  x,
  sd,
  n,
  digits_x,
  digits_sd,
  items_averaged,
  data_type,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE
)

Arguments

Details

x, sd, n, digits_x, digits_sd, items_averaged, and data_type are vectorized and recycled to a common length by the tidyverse rules. All other arguments must have length 1.

The implementation follows Allard's (2018) Analytic-GRIMMER algorithm, as does rsprite2::GRIMMER_test(), with a stricter first test: the sum of squares must be one that n whole numbers with the reconstructed sum can have. For n >= 5, this means at least the smallest such sum of squares; for smaller n, some larger ones are impossible, too. With this test, grimmer() is TRUE if and only if a sample of whole numbers has a mean and SD that would be reported as x and sd. The one exception is rounding = "even", or its synonym "ties_even", which takes a value on a rounding bound to round either way, as in roundwork::unround(), so it may also accept a value set that a sample only rounds to if ties are broken the other way. GRIMMER was first proposed by Anaya (2016).

For data on a bounded scale, such as a Likert scale, see the strait package. Its brimmest() decides exactly whether a mean and SD are possible within the bounds.

x and sd are limited to about 15 significant digits, and x with n and items_averaged as in grim(). Within these limits, grimmer() decides value sets of any size exactly, but refuses two kinds that no study reports: with n from 2 to 4, SDs above about 3e5 to 1e6 (divided by items_averaged), depending on n and sd_meaning; and value sets with more than a million candidate sums, which takes an n of more than a million.

Value

Logical. TRUE if x, sd, n, and items_averaged are consistent, FALSE if not, and NA if the case cannot be decided: if a value is infinite, if n or items_averaged is not a positive whole number, or if a missing value could decide the case either way. See vignette("technical-notes"), section Missing values, on which cases a missing value leaves open. A sample SD divides by n - 1, so under sd_meaning = "sd_sample" (the default) and "se", n must also be at least 2.

Defaults

sd_meaning = "sd_sample" because the sample SD is what nearly all software reports by default: R's sd(), SPSS, Stata, SAS, Excel's STDEV() and STDEV.S(), and pandas' .std(). The main exception is NumPy, whose np.std() returns the population SD unless ddof = 1 is given; so does Python's statistics.pstdev(). If the data may have been analyzed that way, or an inconsistent value set has a small n, test it under "sd_pop" as well. See SD meaning.

The rounding defaults are explained in grim().

SD meaning

The sample and population SDs differ by a factor of sqrt(n / (n - 1)), which matters most at small n, where GRIMMER is most powerful. A population SD tested as a sample SD will often seem inconsistent, so sd_meaning lets you check whether a value set is consistent only under the other convention.

"sd_sample_or_pop" is TRUE if the value set is consistent as a sample SD or as a population SD, and FALSE if it is neither. A single value has no sample SD, so at n = 1, "sd_sample_or_pop" is the same as "sd_pop". The reason column of grimmer_map() names the convention a value set passed under, trying the sample SD first.

"sd_sample_or_pop" accepts more SDs than either convention alone, so it is not the default. It also judges each value set on its own, although a single study usually computed all its SDs the same way. To tell which convention fits a set of values, test them under "sd_sample" and "sd_pop" separately.

Some papers report a standard error as if it were an SD. "se" tests whether a value set is consistent under that premise. The SE was rounded, not the SD, so the SD may lie anywhere in the SE's rounding range times sqrt(n). This range is wide: from about n = 30 on, nearly every value set whose mean passes GRIM is consistent under "se". Such a result says little, and it is no evidence that an SE was reported.

"se" is the SE of this group's mean, computed from its own sample SD and n. Model-based SEs, such as those of estimated marginal means, or SEs based on a pooled variance, are not. A value set that is inconsistent under "se" may still have been reported with one of them.

References

Allard, A. (2018). Analytic-GRIMMER: a new way of testing the possibility of standard deviations. https://aurelienallard.netlify.app/post/anaytic-grimmer-possibility-standard-deviations/

Anaya, J. (2016). The GRIMMER test: A method for testing the validity of reported measures of variability. PeerJ Preprints. https://peerj.com/preprints/2400v1/

See Also

grimmer_explain() to explain a result in plain words, grimmer_map() for data frames, and grimmer_values() for the means and SDs behind a consistent value set.

Examples

# A mean of 5.23 is not consistent with an SD of 2.55
# and a sample size of 35:
grimmer(
  x = 5.23,
  sd = 2.55,
  n = 35,
  digits_x = 2,
  digits_sd = 2,
  items_averaged = 1,
  data_type = "integer"
)

# However, mean and SD are consistent with a
# sample size of 31:
grimmer(
  x = 5.23,
  sd = 2.55,
  n = 31,
  digits_x = 2,
  digits_sd = 2,
  items_averaged = 1,
  data_type = "integer"
)

# For a scale composed of two items:
grimmer(
  x = 2.74,
  sd = 0.96,
  n = 63,
  digits_x = 2,
  digits_sd = 2,
  items_averaged = 2,
  data_type = "integer"
)

Explain a GRIMMER test step by step

Description

grimmer_explain() tests a single mean and SD like grimmer() does, but also lays out the calculation in plain words. It starts with the GRIM test of the mean, as in grim_explain(), and goes on to the sum of squares that each possible sum implies with the SD.

The text is built from the numbers that decide the test itself, so it always agrees with grimmer().

Usage

grimmer_explain(
  x,
  sd,
  n,
  digits_x,
  digits_sd,
  items_averaged,
  data_type,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE
)

Arguments

Value

An object of class scrutiny_explain: a list with the result as consistency (TRUE, FALSE, or NA), the explanation's paragraphs, and the test and its inputs. It prints as the explanation.

Examples

grimmer_explain(
  x = 5.23,
  sd = 2.55,
  n = 35,
  digits_x = 2,
  digits_sd = 2,
  items_averaged = 1,
  data_type = "integer"
)

grimmer_explain(
  x = 5.23,
  sd = 2.55,
  n = 31,
  digits_x = 2,
  digits_sd = 2,
  items_averaged = 1,
  data_type = "integer"
)

GRIMMER-test many cases at once

Description

grimmer_map() applies grimmer() to every row of a data frame with means, standard deviations, and sample sizes.

Call audit() on the output for summary statistics. grim_plot_grid() accepts it as well.

Usage

grimmer_map(
  data,
  digits_x,
  digits_sd,
  x = NULL,
  sd = NULL,
  n = NULL,
  ...,
  items_averaged,
  data_type,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE
)

Arguments

Value

A tibble with these columns:

• x, sd, n: the inputs.

• items_averaged: the number of items averaged into each score.

• data_type: the kind of data, as given.

• digits_x, digits_sd: the numbers of decimal places, as given.

• consistency: GRIMMER consistency of x, sd, n, and items_averaged. NA for a case that cannot be decided, such as one with a missing value.

• reason: "Passed all" if consistent, or "Continuous data" if the test does not apply; otherwise the test that failed, or why the case cannot be decided.

• Any other columns from data.

The reason column refers to GRIM and the GRIMMER tests of Allard (2018):

1. The reconstructed sum of squared observations must be a whole number, and one that n whole numbers with the reconstructed sum can reach: at least the sum of squares of these numbers as close together as possible. For n < 5, some larger sums of squares are impossible, too.

2. The reconstructed SD must match the reported one. Test 1 implies this, so it is never a reason.

3. The parity of the reconstructed sum of squared observations must match that of the reconstructed sum of the values.

A negative sd gives "Negative SD".

Under sd_meaning = "sd_sample_or_pop", a consistent value set gives "Passed all (sample SD)" or, if it passes as a population SD only, "Passed all (population SD)". If the two conventions fail for different reasons, the reason names both, and audit() counts the sample SD's, as under the default sd_meaning.

The tibble has the class scrutiny_grimmer_map, which is recognized by audit() and grim_plot_grid().

Summaries with audit()

audit() on grimmer_map() output returns a tibble with one row and these columns:

1. all_cases: total number of value sets.

2. incon_cases: number of GRIMMER-inconsistent value sets.

3. emp_incon: proportion of GRIMMER-inconsistent value sets.

4. fail_grim: number of value sets that fail GRIM.

5. fail_test1, fail_test3: number of value sets that fail GRIMMER's first or third test. Test 2 is implied by test 1, so it never fails.

References

Allard, A. (2018). Analytic-GRIMMER: a new way of testing the possibility of standard deviations. https://aurelienallard.netlify.app/post/anaytic-grimmer-possibility-standard-deviations/

Anaya, J. (2016). The GRIMMER test: A method for testing the validity of reported measures of variability. PeerJ Preprints. https://peerj.com/preprints/2400v1/

See Also

explain_map() and explain_print() to explain each row's result in plain words.

Examples

# Use `grimmer_map()` on data like these:
pigs5

# The `consistency` column shows whether the values to its left are
# GRIMMER-consistent. If they aren't, the `reason` column gives the reason:
pigs5 |>
  grimmer_map(
    digits_x = 2,
    digits_sd = 2,
    items_averaged = 1,
    data_type = "integer"
  )

# Get summaries with `audit()`:
pigs5 |>
  grimmer_map(
    digits_x = 2,
    digits_sd = 2,
    items_averaged = 1,
    data_type = "integer"
  ) |>
  audit()

# Explain the results in plain words:
pigs5 |>
  grimmer_map(
    digits_x = 2,
    digits_sd = 2,
    items_averaged = 1,
    data_type = "integer"
  ) |>
  explain_print(1:2)

GRIMMER-test values around the reported ones

Description

grimmer_map_seq() varies the reported means, standard deviations, and sample sizes a few steps up and down and GRIMMER-tests each resulting value set. This shows whether a small error in reporting or computing could explain an inconsistency.

Call audit() on the output for summary statistics.

Usage

grimmer_map_seq(
  data,
  digits_x,
  digits_sd,
  x = NULL,
  sd = NULL,
  n = NULL,
  ...,
  items_averaged,
  data_type,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE,
  var = c("x", "sd", "n"),
  dispersion = 1:5,
  out_min = "auto",
  out_max = "auto",
  include_reported = FALSE,
  include_consistent = FALSE
)

Arguments

Value

A tibble with the columns of grimmer_map() output plus diff_var, case, and var, as in grim_map_seq(). As there, the other columns of data are dropped.

Summaries with audit()

audit() on grimmer_map_seq() output returns a tibble with one row per case and these columns:

• x, sd, and n: the reported values, with items_averaged, data_type, and their consistency.

• hits_total: the number of GRIMMER-consistent value sets found within the dispersion range.

• hits_x, hits_sd, hits_n: the number found by varying x, sd, and n, respectively. If both include_reported and include_consistent are TRUE, a consistent reported value set is counted as well: once per variable here, and once in hits_total.

• diff_x: the distance from x to the nearest consistent dispersed value, in dispersion steps. diff_x_up and diff_x_down are the signed steps to the nearest consistent value above and below (⁠_down⁠ is negative).

• diff_sd, diff_sd_up, diff_sd_down: the same for sd.

• diff_n, diff_n_up, diff_n_down: the same for n.

A count or distance is NA if it depends on a dispersed value set whose consistency is NA, on a variable whose reported value is missing, which is not dispersed, or on rows removed from the output, as by filter().

Call audit() again on that summary to summarize these columns further. na_count and na_rate there count the cases without a consistent value within the dispersion range, and those that a missing value leaves unknown.

Examples

# `grimmer_map_seq()` takes the same input as `grimmer_map()`:
pigs5

out <- grimmer_map_seq(
  pigs5,
  digits_x = 2,
  digits_sd = 2,
  items_averaged = 1,
  data_type = "integer",
  include_consistent = TRUE
)
out

# Case-wise summaries:
out |>
  audit()

GRIMMER-test with hypothetical group sizes

Description

Some studies report group means and standard deviations but only the total sample size, not the group sizes. For two groups, grimmer_map_total_n() tests hypothetical splits of the total, as grim_map_total_n() does.

Call audit() on the output for summary statistics.

Usage

grimmer_map_total_n(
  data,
  digits_x,
  digits_sd,
  x1 = NULL,
  x2 = NULL,
  sd1 = NULL,
  sd2 = NULL,
  n = NULL,
  ...,
  items_averaged,
  data_type,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE,
  dispersion = 0:5,
  n_min = NULL,
  n_max = NULL,
  constant = NULL
)

Arguments

Value

A tibble like that of grim_map_total_n(), with x and sd columns for the reported values, dir recording which group is paired with the larger group size (the equal split of an even total appears under "forth" only), the other columns of grimmer_map() output, and the other columns of data.

Summaries with audit()

audit() on grimmer_map_total_n() output returns a tibble with one row per case and these columns:

• x1, x2, sd1, sd2, and n: the reported values.

• hits_total: the number of scenarios in which both groups are GRIMMER-consistent, summing hits_forth and hits_back.

• hits_forth: the number of such scenarios with x2 and sd2 paired with the larger group.

• hits_back: the same with x1 and sd1 paired with the larger group.

• scenarios_total: the number of scenarios tested, each a split of n with one pairing of groups and statistics, leaving out those that could not be decided.

• hit_rate: hits_total / scenarios_total, or NA if no scenario could be decided.

A count is NA if it depends on a scenario that could not be decided, or on rows removed from the output, as by filter().

Call audit() again on that summary to summarize these columns further.

References

Allard, A. (2018). Analytic-GRIMMER: a new way of testing the possibility of standard deviations. https://aurelienallard.netlify.app/post/anaytic-grimmer-possibility-standard-deviations/

Bauer, P. J., & Francis, G. (2021). Expression of Concern: Is It Light or Dark? Recalling Moral Behavior Changes Perception of Brightness. Psychological Science, 32(12), 2042–2043. https://journals.sagepub.com/doi/10.1177/09567976211058727

See Also

function_map_total_n(), which created this function from grimmer_map().

Examples

df <- tibble::tribble(
  ~x1,  ~x2,  ~sd1, ~sd2, ~n,
  3.43, 5.28, 1.09, 2.12, 70,
  2.97, 4.42, 0.43, 1.65, 65
)

grimmer_map_total_n(
  df,
  digits_x = 2,
  digits_sd = 2,
  items_averaged = 1,
  data_type = "integer"
)

# `hits_total` counts the splits in which both groups are consistent:
df |>
  grimmer_map_total_n(
    digits_x = 2,
    digits_sd = 2,
    items_averaged = 1,
    data_type = "integer"
  ) |>
  audit()

# More steps away from the even split tend to find more hits:
df |>
  grimmer_map_total_n(
    digits_x = 2,
    digits_sd = 2,
    items_averaged = 1,
    data_type = "integer",
    dispersion = 0:10
  ) |>
  audit()

Visualize the standard deviations that GRIMMER admits

Description

grimmer_plot_line() draws the GRIMMER test for a single value set, as grim_plot_line() does for GRIM. Of all SDs that could have been reported with digits_sd decimal places, each square is one that a sample of size n with mean x could have; the others are left out. The reported SD is marked above the row and labeled as consistent or inconsistent.

For many value sets, grim_plot_grid() accepts grimmer_map() output and shows the GRIM part of the test.

Usage

grimmer_plot_line(
  x,
  sd,
  n,
  digits_x,
  digits_sd,
  items_averaged,
  data_type,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE,
  zoom = c("close", "medium", "wide"),
  color_con = "#0047ff",
  color_incon = "#ff2a00",
  color_attainable = "grey75"
)

Arguments

Details

zoom works as in grim_plot_line(), but with zoom = "wide", the row runs from the whole number at or below sd to the next one up. It never goes below zero.

If x is GRIM-inconsistent with n, no SD is attainable and the row has no squares.

Value

A ggplot object.

Examples

# A mean of 5.23 with an SD of 2.55 is not consistent with a sample size of
# 35, but it is with 31:
grimmer_plot_line(
  x = 5.23,
  sd = 2.55,
  n = 35,
  digits_x = 2,
  digits_sd = 2,
  items_averaged = 1,
  data_type = "integer"
)
grimmer_plot_line(
  x = 5.23,
  sd = 2.55,
  n = 31,
  digits_x = 2,
  digits_sd = 2,
  items_averaged = 1,
  data_type = "integer"
)

Achievable means and SDs behind a reported pair

Description

grimmer_values() returns every pair of a mean and an SD that a sample of n integers can have and that would be reported as x and sd. It is the counterpart of grim_values() for grimmer(), which is TRUE exactly if there is at least one such pair.

grimmer_closest() returns the single achievable pair whose SD is closest to sd, like grim_closest() for means. If x and sd are consistent, it is one of the pairs that grimmer_values() returns. Otherwise, it is closest among the pairs whose mean would be reported as x, or, if there are none, whose mean is the one grim_closest() returns.

Each pair comes with the whole-number sum and sum of squares behind it. Statistics computed from several groups, such as a t-value, can be compared exactly through these integers.

Usage

grimmer_values(
  x,
  sd,
  n,
  digits_x,
  digits_sd,
  items_averaged,
  data_type,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE
)

grimmer_closest(
  x,
  sd,
  n,
  digits_x,
  digits_sd,
  items_averaged,
  data_type,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se"),
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE
)

Arguments

Details

The sums are the whole numbers that grim_values() divides by n * items_averaged. For each, the sums of squares are the whole numbers within the rounding bounds of sd that n integers with that sum can have. With items_averaged > 1, the integers are the respondents' totals across the items, so the sums are those of the totals.

The number of pairs grows with n, items_averaged, and sd, and shrinks with digits_x and digits_sd. More than a million pairs for one value set is an error, since the output has one row for each. So is a sum of squares of 2^53 or more, about 9e15, which a double can't hold exactly; grimmer() tests such value sets all the same. grimmer_closest() stops at 2^48, about 3e14, beyond which it can miss the closest pair. With n from 2 to 4, both stop at 2^40, about 1.1e12, as grimmer() does; grimmer_closest() stops there even if x fails GRIM.

Under rounding = "even", the pairs include ties that base::round() may round away from x or sd, as in grim_values().

In grimmer_closest(), pairs whose SDs are equally close go by the mean closer to x, then by the smaller sum, then by the smaller SD. A negative sd is closest to the least SD there is. The number of pairs doesn't matter there, but that of the sums does: for a consistent value set, it tries one sum per remainder modulo n, and more than 100,000 of them is an error. That takes an n of more than about 10^(digits_x + 5).

Value

A tibble with one row per achievable pair. case is the index of the value set, as in grim_values(). x and sd are the achievable values, n the reported sample size, items_averaged as given, and sum and sum_squares the whole numbers behind them. An inconsistent value set has no rows, and one that cannot be decided has one row of NA but for n and items_averaged, as in grimmer().

grimmer_closest() returns the same columns with exactly one row per value set: of NA but for n and items_averaged if a value is missing or the value set cannot be decided.

See Also

grimmer(), grimmer_map(), grim_values(), and grimmer_plot_line().

Examples

# A mean of 5.23 and an SD of 2.55 are consistent with a sample size of 31,
# and there is exactly one way to get there:
grimmer_values(
  x = 5.23,
  sd = 2.55,
  n = 31,
  digits_x = 2,
  digits_sd = 2,
  items_averaged = 1,
  data_type = "integer"
)

# With one decimal place, several pairs round to the same values:
grimmer_values(
  x = 5.3,
  sd = 0.7,
  n = 30,
  digits_x = 1,
  digits_sd = 1,
  items_averaged = 1,
  data_type = "integer"
)

# None at all if the value set is inconsistent:
grimmer_values(
  x = 5.23,
  sd = 2.55,
  n = 35,
  digits_x = 2,
  digits_sd = 2,
  items_averaged = 1,
  data_type = "integer"
)

# ...but there is a pair close to it:
grimmer_closest(
  x = 5.23,
  sd = 2.55,
  n = 35,
  digits_x = 2,
  digits_sd = 2,
  items_averaged = 1,
  data_type = "integer"
)

Test whether a vector is numeric or coercible to numeric

Description

is_numeric_like() is deprecated because scrutiny no longer takes numbers as strings. It may be removed in a future version.

is_numeric_like() tests whether an object contains numbers, by scrutiny's standards:

• Integer and double vectors are TRUE.

• Logical vectors and non-vectors are FALSE.

• Other vectors, such as strings, are TRUE if every non-NA value can be coerced to a non-NA number. The strings "NA" and "NaN" count as NA, and so does a blank string, as in a blank cell of a CSV file.

• Factors, dates, and times are tested as the strings they print as.

• Lists are tested like atomic vectors if every element has length 1, and are FALSE otherwise. A logical element other than NA makes a list FALSE.

• A vector of nothing but non-numeric, non-logical NA is NA.

Usage

is_numeric_like(x)

Arguments

Details

It asks whether a vector displays numbers, not how it is stored.

Value

Logical (length 1).

Examples

is_numeric_like(x = 1:5)
is_numeric_like(x = c(TRUE, FALSE))

is_numeric_like(x = c("42", "0.7", NA))
is_numeric_like(x = c("42", "xyz", NA))

is_numeric_like(x = as.factor(c("42", "0.7", NA)))

is_numeric_like(x = list("42", "0.7", NA))
is_numeric_like(x = list("42", "0.7", NA, c(1, 2, 3)))

# All `NA`:
is_numeric_like(x = as.character(c(NA, NA, NA)))
is_numeric_like(x = as.numeric(c(NA, NA, NA)))
is_numeric_like(x = c(NA, NA, NA))

Manage a helper column

Description

manage_helper_col() is superseded by the .cols_helper argument of function_map().

A helper column is a column of a mapper's input that is not itself tested but enters the test, such as items_averaged in grim_map(). It can be given either as a column of data or as an argument of the same name. manage_helper_col() checks that the two don't disagree if both are given and, by default, adds the argument's value to data as a column if data has none.

Mappers made by function_map() call it for every .cols_helper argument. Call it yourself in a handwritten mapper, once per helper column, and reassign the result to data.

Usage

manage_helper_col(
  data,
  arg,
  default,
  add_col = TRUE,
  var_arg = deprecated(),
  affix = deprecated()
)

Arguments

Value

data, possibly with the helper column added.

Extract substrings from before and inside parentheses

Description

before_parens() and inside_parens() extract the part of a string before or inside parentheses, brackets, braces, or custom delimiters. split_by_parens() does the same for the columns of a data frame.

Usage

before_parens(string, delim = "parens", sep = deprecated())

inside_parens(string, delim = "parens", sep = deprecated())

Arguments

Value

String vector as long as string, with surrounding whitespace removed. If a string has no opening delimiter, before_parens() returns all of it and inside_parens() returns NA. A blank part, such as the inside of "()", is NA, as is a part "NA" or "NaN".

Examples

x <- c(
  "3.72 (0.95)",
  "5.86 (2.75)",
  "3.06 (6.48)"
)

before_parens(string = x)
inside_parens(string = x)

Is a reported percentage consistent with its count?

Description

percentage_of() checks whether percentage is what absolute out of n would have been reported as, given the number of decimal places and the rounding method.

This is stricter than grim() with x_meaning = "percentage", which only asks whether some count out of n produces the percentage. Here, the count is reported too.

The function is vectorized, but percentage_of_map() is more convenient for testing many cases.

Usage

percentage_of(
  absolute,
  percentage,
  n,
  digits_percentage,
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE
)

Arguments

Details

The test runs in exact integer arithmetic. absolute, percentage, n, and digits_percentage are vectorized; all other arguments must have length 1.

An absolute above n is tested like any other, as for answers to a question that allows several, whose percentages can sum to more than 100.

Value

Logical. TRUE if the three values are consistent, FALSE if not, and NA if the case cannot be decided: if a value is infinite, if absolute is not a non-negative whole number, if n is not a positive whole number, or if a missing value could decide the case either way. A missing absolute gives FALSE if no count out of n has the percentage, and a missing n gives FALSE if no n makes absolute that percentage. A missing digits_percentage is treated as in grim().

Causes of inconsistencies

A FALSE result means that the three values cannot all be correct as reported, but not why. The cause is often trivial: the percentage may be based on another denominator than n, such as the number of valid responses, or on an unconventional rounding method. percentage_of_map_seq() with var = "n" shows which nearby denominators would make the percentage consistent.

See Also

grim() with x_meaning = "percentage", for a percentage without a count.

Examples

# 40 out of 120 is 33.33...%, so it can be
# reported as 33 or 33.3, but not as 34:
percentage_of(40, 33, 120, digits_percentage = 0)
percentage_of(40, 33.3, 120, digits_percentage = 1)
percentage_of(40, 34, 120, digits_percentage = 0)

Test many reported percentages against their counts

Description

percentage_of_map() applies percentage_of() to every row of a data frame with counts, percentages, and totals.

Call audit() on the output for summary statistics. See percentage_of() for causes of inconsistencies.

Usage

percentage_of_map(
  data,
  digits_percentage,
  absolute = NULL,
  percentage = NULL,
  n = NULL,
  ...,
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE
)

Arguments

Value

A tibble with the columns absolute, percentage, n, digits_percentage, consistency, and any other columns from data. It has the class scrutiny_percentage_of_map, which is recognized by audit().

Summaries with audit()

audit() on percentage_of_map() output returns a tibble with one row and these columns:

1. all_cases: total number of value sets.

2. incon_cases: number of inconsistent value sets.

3. emp_incon: proportion of inconsistent value sets.

Examples

df <- tibble::tibble(
  absolute = c(40, 37, 25),
  percentage = c(33, 31, 20),
  n = 120
)

df |>
  percentage_of_map(digits_percentage = 0)

Test percentages against their counts with values around the reported ones

Description

percentage_of_map_seq() varies the reported counts, percentages, and totals a few steps up and down and applies percentage_of() to each resulting value set. This shows whether a small error in one of them could explain an inconsistency.

Call audit() on the output for summary statistics.

Usage

percentage_of_map_seq(
  data,
  digits_percentage,
  absolute = NULL,
  percentage = NULL,
  n = NULL,
  ...,
  rounding = "up_or_down",
  threshold = 5,
  symmetric = FALSE,
  var = c("absolute", "percentage", "n"),
  dispersion = 1:5,
  out_min = "auto",
  out_max = "auto",
  include_reported = FALSE,
  include_consistent = FALSE
)

Arguments

Value

A tibble with the columns of percentage_of_map() output plus diff_var, case, and var, as in grim_map_seq(). As there, the other columns of data are dropped.

Examples

df <- tibble::tibble(
  absolute = c(40, 37, 25),
  percentage = c(33, 31, 20),
  n = 120
)

df |>
  percentage_of_map_seq(digits_percentage = 0) |>
  audit()

Means and sample sizes for GRIM-testing

Description

A fictional dataset with means and sample sizes of flying pigs. For demonstrating grim_map() and functions built on it.

Usage

pigs1

Format

A tibble (data frame) with 12 rows and 2 columns. The columns are:

x

Numeric. Means.

n

Numeric. Sample sizes.

See Also

pigs2 for GRIM-testing percentages instead of means, pigs3 for DEBIT-testing, pigs4 for detecting duplicates, and pigs5 for GRIMMER-testing.

Percentages and sample sizes for GRIM-testing

Description

A fictional dataset with percentages and sample sizes of flying pigs. For demonstrating grim_map() and functions built on it, with x_meaning = "percentage".

Usage

pigs2

Format

A tibble (data frame) with 6 rows and 2 columns. The columns are:

x

Numeric. Percentages.

n

Numeric. Sample sizes.

See Also

pigs1 for GRIM-testing means instead of percentages, pigs3 for DEBIT-testing, pigs4 for detecting duplicates, and pigs5 for GRIMMER-testing.

Binary means and standard deviations for DEBIT-testing

Description

A fictional dataset with means and standard deviations from a binary distribution related to flying pigs. For demonstrating debit_map() and functions built on it.

Usage

pigs3

Format

A tibble (data frame) with 7 rows and 3 columns. The columns are:

x

Numeric. Means.

sd

Numeric. Standard deviations.

n

Numeric. Sample sizes.

See Also

pigs1 for GRIM-testing means, pigs2 for GRIM-testing percentages instead of means, pigs4 for detecting duplicates, and pigs5 for GRIMMER-testing.

Data with duplications

Description

A fictional dataset with observations of flying pigs. It contains multiple duplicates. For demonstrating the ⁠duplicate_*()⁠ functions, such as duplicate_count().

Usage

pigs4

Format

A tibble (data frame) with 5 rows and 3 columns, describing various body measures of the fictional pigs. The columns are:

snout

Numeric. Snout width.

tail

Numeric. Tail length.

wings

Numeric. Wingspan.

See Also

pigs1 for GRIM-testing means, pigs2 for GRIM-testing percentages, pigs3 for DEBIT-testing, and pigs5 for GRIMMER-testing.

Means, SDs, and sample sizes for GRIMMER-testing

Description

A fictional dataset with means, standard deviations (SDs), and sample sizes of flying pigs. For demonstrating grimmer_map() and functions built on it.

Usage

pigs5

Format

A tibble (data frame) with 12 rows and 3 columns. The columns are:

x

Numeric. Means.

sd

Numeric. Standard deviations.

n

Numeric. Sample sizes.

See Also

pigs1 for GRIM-testing the same means, with other sample sizes, pigs2 for GRIM-testing percentages instead of means, pigs3 for DEBIT-testing, and pigs4 for detecting duplicates.

Restore trailing zeros

Description

These functions are deprecated because the tests no longer take strings, so there is no need to restore trailing zeros for them. Give the number of decimal places as digits_x or digits_sd instead. To format numbers for display, use formatC(x, format = "f", digits = 2).

restore_zeros() formats numbers as strings with a given number of decimal places, padding with zeros. By default, the target is the number of decimal places of the value that has the most. Do not rely on this default without checking it, since the value with the most decimal places may itself have lost trailing zeros.

restore_zeros_df() applies it to the columns of a data frame, by default all those coercible to numeric.

Usage

restore_zeros(
  x,
  width = NULL,
  sep_in = ".",
  sep_out = sep_in,
  check_width = c("capped", "never")
)

restore_zeros_df(
  data,
  cols = everything(),
  check_numeric_like = TRUE,
  check_decimals = FALSE,
  width = NULL,
  sep_in = ".",
  sep_out = sep_in,
  check_width = c("capped", "never")
)

Arguments

Details

Displayed numbers are often rounded, so a value may have more decimal places than it shows. x |> decimal_places() |> max() gives the actual maximum.

As in decimal_places(), numeric values with more than 15 significant digits cannot be padded accurately; enter such numbers as strings.

Value

• For restore_zeros(), a string vector.

• For restore_zeros_df(), a data frame.

Examples

vec <- c(212, 75.38, 4.9625)

# By default, pad to the most decimal places in `vec`:
restore_zeros(vec)

# Or to a given width:
restore_zeros(vec, width = 6)

iris <- tibble::as_tibble(iris)

# All numeric columns, but not the factor:
restore_zeros_df(iris)

# Selected columns:
restore_zeros_df(iris, starts_with("Sepal"), width = 3)

Reconstruct the input of a sequence mapper

Description

reverse_map_seq() takes the output of a function made by function_map_seq() and reconstructs the tested columns of the cases it dispersed, without ⁠digits_*⁠. audit() builds on it for sequence output.

Usage

reverse_map_seq(data)

Arguments

Value

A tibble with the tested columns, one row per case in data.

Examples

pigs1

out <- grim_map_seq(
  pigs1,
  digits_x = 2,
  items_averaged = 1,
  data_type = "integer",
  include_consistent = TRUE
)

reverse_map_seq(out)

Reconstruct the input of a total-n mapper

Description

reverse_map_total_n() takes the output of a function made by function_map_total_n() and reconstructs the reported columns of the data frame it was called on. audit() builds on it for total-n output.

Usage

reverse_map_total_n(data)

Arguments

Value

A tibble with the reported columns, such as x1, x2, and n, one row per case.

Examples

df <- tibble::tribble(
  ~x1,  ~x2,  ~n,
  3.43, 5.28, 90,
  2.97, 4.42, 103
)

out <- grim_map_total_n(
  df,
  digits_x = 2,
  items_averaged = 1,
  data_type = "integer"
)

reverse_map_total_n(out)

Turn row values into column names

Description

Tables extracted from PDFs, e.g. with tabulapdf, sometimes have their column names in the first row (or rows) and nondescript names above. row_to_colnames() replaces the column names by the values of those rows and drops the rows.

Usage

row_to_colnames(data, row = 1L, collapse = " ", drop = TRUE)

Arguments

Details

With several rows, their values are pasted together per column. Missing values, blank strings, "NA", and "NaN" are skipped, and whitespace around a value is trimmed; if all of a column's values in these rows are missing or blank, it keeps its name. The new names must be unique: a header that repeats, as in "M, SD, M, SD", is an error unless a second row tells the columns apart. A label that spans several columns, such as a group name above "M, SD", is often only in the first of them; repeat it in each one first.

Value

data with new column names, as a data frame of the same class, or a tibble if data was a matrix.

See Also

unheadr::mash_colnames(), a more general solution to the same problem.

Read the settings a mapper recorded on its output

Description

Every mapper, such as grim_map(), records the settings of the call that produced its output. scrutiny_meta() returns them as a tibble, so that a report can state the rounding method or SD convention behind a result.

Settings are the arguments that apply to the whole call, with defaults resolved, such as rounding, x_meaning, or sd_meaning. Arguments that may vary by row, such as digits_x and items_averaged, are columns of the output instead, even if given as a single value. Sequence and total-n mappers, such as grim_map_seq(), add their own settings, such as dispersion.

Printed output of grim_map(), grimmer_map(), debit_map(), or percentage_of_map(), including sequence and total-n output, has a header line that names the settings that are not at their defaults, as in ⁠# GRIM: rounding = "up"⁠, or says ⁠default settings⁠.

Some dplyr verbs, such as summarise(), drop the record, and with it the header line; filter(), arrange(), and mutate() keep it.

Usage

scrutiny_meta(data)

Arguments

Value

A tibble with one row per setting and these columns, or NULL if data has no record of its settings:

• setting: the name of the argument.

• value: its value, written as R code, such as 'up' or 1:5.

• default: TRUE if the value is the mapper's default, FALSE if not.

• added_by: the mapper whose argument it is, such as "grim_map" or "grim_map_seq".

Examples

pigs1 |>
  grim_map(
    digits_x = 2,
    items_averaged = 1,
    data_type = "integer",
    rounding = "up"
  ) |>
  scrutiny_meta()

# Only the settings that are not at their defaults:
pigs1 |>
  grim_map_seq(
    digits_x = 2,
    items_averaged = 1,
    data_type = "integer",
    dispersion = 1:10
  ) |>
  scrutiny_meta() |>
  dplyr::filter(!default)

Standard deviation of binary data

Description

Compute the SD of binary data (0 and 1 values) from different inputs:

• sd_binary_groups() from the counts of 0s and 1s.

• sd_binary_0_n() from the count of 0s and the total.

• sd_binary_1_n() from the count of 1s and the total.

• sd_binary_mean_n() from the mean and the total.

These are the SDs that debit() tests under the same sd_meaning.

sd_binary_0_n() and sd_binary_1_n() are superseded by sd_binary_groups(): sd_binary_0_n(group_0, n) is sd_binary_groups(group_0, n - group_0).

Usage

sd_binary_groups(
  group_0,
  group_1,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se")
)

sd_binary_0_n(
  group_0,
  n,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se")
)

sd_binary_1_n(
  group_1,
  n,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se")
)

sd_binary_mean_n(
  x,
  n,
  sd_meaning = c("sd_sample", "sd_pop", "sd_sample_or_pop", "se"),
  mean = deprecated()
)

Arguments

Value

Numeric. The SD or standard error; NA where n or a count is not a whole number, and where n is below 2 under "sd_sample" and "se", which divide by n - 1.

References

Heathers, James A. J., and Brown, Nicholas J. L. 2019. DEBIT: A Simple Consistency Test For Binary Data. https://osf.io/5vb3u/.

Examples

# 127 values are 0 and 153 are 1, so n = 280:
sd_binary_groups(group_0 = 127, group_1 = 153)
sd_binary_0_n(group_0 = 127, n = 280)
sd_binary_1_n(group_1 = 153, n = 280)

# From the mean, given enough decimal places:
sd_binary_mean_n(x = 0.5464286, n = 280)

# The population SD and the standard error:
sd_binary_groups(group_0 = 127, group_1 = 153, sd_meaning = "sd_pop")
sd_binary_groups(group_0 = 127, group_1 = 153, sd_meaning = "se")

Is a vector a certain kind of sequence?

Description

These functions are deprecated because scrutiny no longer uses them: the sequence mappers record how far each value was moved, in diff_var. They may be removed in a future version.

Predicates for numeric vectors, or vectors coercible to numeric:

• is_seq_linear(): do all consecutive elements differ by the same amount?

• is_seq_ascending(), is_seq_descending(): is every difference positive, or negative?

• is_seq_dispersed(): is x centered on from, with each pair of values equidistant from it?

By default (test_linear = TRUE), the last three also test for linearity.

Usage

is_seq_linear(x, tolerance = .Machine$double.eps^0.5)

is_seq_ascending(x, test_linear = TRUE, tolerance = .Machine$double.eps^0.5)

is_seq_descending(x, test_linear = TRUE, tolerance = .Machine$double.eps^0.5)

is_seq_dispersed(
  x,
  from,
  test_linear = TRUE,
  tolerance = .Machine$double.eps^0.5
)

Arguments

Value

Logical (length 1). If x contains NA:

• NA if some values substituted for the NAs would make x the kind of sequence in question and others would not.

• FALSE if none would, e.g., because x is too short for it (two values for is_seq_ascending() and is_seq_descending(), three for is_seq_dispersed()).

• TRUE if all would, as any one or two values are linear.

A missing from is handled the same way.

See Also

validate::is_linear_sequence(), which is more permissive with NA and supports date-times.

Examples

is_seq_linear(x = 3:7)
is_seq_linear(x = c(3:7, 9))

# The other predicates test for linearity by default:
is_seq_ascending(x = c(2, 7, 9))
is_seq_ascending(x = c(2, 7, 9), test_linear = FALSE)

is_seq_descending(x = c(9, 7, 2))
is_seq_descending(x = c(9, 7, 2), test_linear = FALSE)

is_seq_dispersed(x = c(2, 3, 5, 7, 8), from = 5)
is_seq_dispersed(x = c(2, 3, 5, 7, 8), from = 5, test_linear = FALSE)

# These fail their own test regardless of linearity:
is_seq_ascending(x = c(1, 7, 4), test_linear = FALSE)
is_seq_descending(x = c(9, 15, 3), test_linear = FALSE)
is_seq_dispersed(1:10, from = 5, test_linear = FALSE)

Sequences dispersed around a value

Description

seq_disperse() creates a sequence of values around from, a given number of steps up and down, where a step is one unit of the last decimal place: 7.93 is surrounded by 7.91, 7.92, 7.94, 7.95, and so on.

seq_disperse_df() returns the sequence as a data frame column, with further columns added as in tibble::tibble(). Its arguments are those of seq_disperse() with a leading dot.

Usage

seq_disperse(
  from,
  by = NULL,
  dispersion = 1:5,
  offset_from = 0L,
  out_min = "auto",
  out_max = NULL,
  include_reported = TRUE,
  track_diff_var = FALSE
)

seq_disperse_df(
  .from,
  .by = NULL,
  ...,
  .dispersion = 1:5,
  .offset_from = 0L,
  .out_min = "auto",
  .out_max = NULL,
  .include_reported = TRUE,
  .track_diff_var = FALSE
)

Arguments

Details

Unlike base::seq(), these functions need not return a regular sequence: dispersion can be any non-negative whole numbers, in any order, such as c(1, 4, 9). The values follow that order, mirrored around from. Steps of 0 and repeated steps are dropped.

Value

• seq_disperse() returns a numeric vector, which is integer if from is an integer and every value is a whole number. With track_diff_var = TRUE, it returns a list of that vector and an integer vector of steps.

• seq_disperse_df() returns a tibble with the sequence in the x column, the steps in diff_var if requested, and any columns added via the dots.

See Also

disperse() and base::seq(), whose ideas seq_disperse() combines.

Examples

seq_disperse(from = 4.02)

# A number loses trailing zeros, so state the step size:
seq_disperse(from = 5.30, by = 0.01)

# More steps:
seq_disperse(from = 4.02, dispersion = 1:10)

# Irregular steps:
seq_disperse(from = 4.02, dispersion = c(2, 3, 7))

# The data frame variant takes further columns:
seq_disperse_df(.from = 4.02, n = 45)

Split columns by parentheses, brackets, braces, or similar

Description

Summary statistics are often reported like "2.65 (0.27)". split_by_parens() splits such columns into the part before the parentheses and the part inside them. By default, it splits every column that contains the delimiters. Set transform to TRUE to pivot the result into a longer format.

Usage

split_by_parens(
  data,
  cols = everything(),
  check_delim = TRUE,
  keep = FALSE,
  transform = FALSE,
  delim = "parens",
  end1 = "x",
  end2 = "sd",
  check_sep = deprecated(),
  sep = deprecated()
)

Arguments

Value

An ungrouped data frame: a tibble unless data is a plain data frame and transform is FALSE.

See Also

• before_parens() and inside_parens() for the two parts of a string vector.

• tidyr::separate_wider_delim(), which is more general but does not recognize closing delimiters.

Examples

df1 <- tibble::tribble(
  ~drone,           ~selfpilot,
  "0.09 (0.21)",    "0.19 (0.13)",
  "0.19 (0.28)",    "0.53 (0.10)",
  "0.62 (0.16)",    "0.50 (0.11)",
  "0.15 (0.35)",    "0.57 (0.16)",
)

df1 |>
  split_by_parens()

# Only some columns:
df1 |>
  split_by_parens(cols = drone)

# Longer format:
df1 |>
  split_by_parens(transform = TRUE)

# Other suffixes:
df1 |>
  split_by_parens(end1 = "beta", end2 = "se")

# Other delimiters:
df2 <- tibble::tribble(
  ~drone,           ~selfpilot,
  "0.09 [0.21]",    "0.19 [0.13]",
  "0.19 [0.28]",    "0.53 [0.10]",
)

df2 |>
  split_by_parens(delim = "brackets")

df3 <- tibble::tribble(
  ~drone,           ~selfpilot,
  "0.09 <0.21>",    "0.19 <0.13>",
  "0.19 <0.28>",    "0.53 <0.10>",
)

df3 |>
  split_by_parens(delim = c("<", ">"))