Declare post-processing for model predictions

Tailors compose iterative adjustments to model predictions. After initializing a tailor with this function, add adjustment specifications with adjust_*() functions:

For probability distributions: adjust_probability_calibration()
For transformation of probabilities to hard class predictions: adjust_probability_threshold(), adjust_equivocal_zone()
For numeric distributions: adjust_numeric_calibration(), adjust_numeric_range()

For ad-hoc adjustments, see adjust_predictions_custom().

Tailors must be trained with fit() before being applied to new data with predict().

Usage

tailor()

Value

An object of class tailor with elements:

type: The type of task (e.g., regression)
adjustments: A list containing the sequential options specified by the user.
columns: the data set column names for the true outcome values and the predictions of various types. If these are not specified, then NULL. These are usual
ptype: a zero-row slice of the data containing the columns.

Most of these values are set when an adjustment is added to the tailor or when fit.tailor() is used.

Ordering of adjustments

When composing multiple adjustments in a tailor object, the order matters and must follow specific rules depending on the type of predictions being adjusted (classification or regression).

For classification problems (binary and multiclass), adjustments that modify probability estimates (e.g., adjust_probability_calibration()) must be applied before adjustments that change hard class predictions (including adjust_equivocal_zone()). This ensures that class predictions are based on the final calibrated probabilities. For regression problems, adjust_numeric_calibration() must be applied before other numeric adjustments. This ensures that subsequent adjustments work with calibrated predictions.

Generally, adjustments cannot be duplicated (i.e. the same adjustment type cannot be used multiple times in a tailor object), though adjust_predictions_custom() can be used multiple times. Adjustments for different prediction types cannot be mixed—numeric adjustments (for regression) and probability adjustments (for classification) cannot be used in the same tailor object.

If these ordering rules are violated, tailor() will raise an error describing the issue.

Examples

library(dplyr)
library(modeldata)

# `predicted` gives hard class predictions based on probabilities
two_class_example |> count(predicted)
#>   predicted   n
#> 1    Class1 277
#> 2    Class2 223

# change the probability threshold to allot one class vs the other
tlr <-
  tailor() |>
  adjust_probability_threshold(threshold = .1)

tlr
#> 
#> ── tailor ──────────────────────────────────────────────────────────────────────
#> A binary postprocessor with 1 adjustment:
#> 
#> • Adjust probability threshold to 0.1.

# fit by supplying column names.
tlr_fit <- fit(
  tlr,
  two_class_example,
  outcome = c(truth),
  estimate = c(predicted),
  probabilities = c(Class1, Class2)
)

tlr_fit
#> 
#> ── tailor ──────────────────────────────────────────────────────────────────────
#> A binary postprocessor with 1 adjustment:
#> 
#> • Adjust probability threshold to 0.1. [trained]

# adjust hard class predictions
predict(tlr_fit, two_class_example) |> count(predicted)
#> # A tibble: 2 × 2
#>   predicted     n
#>   <fct>     <int>
#> 1 Class1      362
#> 2 Class2      138