--- title: "From raw sample to final weights" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{From raw sample to final weights} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r setup, include = FALSE} knitr::opts_chunk$set(collapse = TRUE, comment = "#>") library(weightflow) ``` > **New here?** This is the 15-minute tour: what problem `weightflow` solves and > the shortest recipe that produces analysis weights. For the statistical > reasoning behind each adjustment, read > [*Staged survey weighting: the adjustment logic*](weightflow.html). ## The problem is not the weights Every survey organization has weighting scripts. They grow over time, absorb new adjustments, and eventually become difficult to understand, maintain and reproduce. Two analysts can implement the *same* weighting strategy in different ways, which turns audits, updates and methodological reviews into archaeology. The weights themselves are rarely the hard part. The hard part is that the *strategy* (the sequence of methodological decisions that produced the weights) is scattered across files, intermediate objects and the memory of whoever ran them last. When the responsible methodologist moves on, the knowledge often leaves with them. A typical pipeline looks like this: Weighting spread across scripts and intermediate files Sampling feeds script1.R, which writes weights1.csv, read by script2.R, which writes weights2.csv, read by script_final_v7.R, which writes final_weights_NEW.csv. The steps zigzag between code files and intermediate CSVs. sampling script1.R weights1.csv script2.R weights2.csv script_final_v7.R final_weights_NEW.csv Logic scattered across files and run order, not kept in one place. Anyone who has worked in a statistical office recognizes this immediately. Along the way you typically reach for many different packages, spread the logic across several scripts, and generate hundreds of lines of code. The methodology is real, but it is *implicit*: hidden in file order, column names, intermediate files and undocumented tweaks. ## A map of what the steps are for Before we look at any code, one picture. Weights exist because the realized sample differs from the population we want to describe, in specific, nameable ways. Each region below is corrected by a specific step. From target population to respondents Two overlapping ellipses show the target population and the sampling frame, with undercoverage, an in-scope overlap, and out-of-scope units. The in-scope area is sampled; the sample splits into unknown eligibility, ineligible and eligible, and eligible splits into nonrespondents and respondents. A legend maps each region to a weightflow step. Target population Frame Undercoverage (missing from frame) In scope Out of scope (ineligible on frame) draw sample (design weight 1/π) Sample (known inclusion probability π) Unknown eligibility Ineligible Eligible Non- respondents Respondents Unknown eligibility → step_unknown_eligibility(): redistribute their weight Ineligible → step_drop_ineligible(): remove from the sample Nonrespondents → step_nonresponse(): respondents absorb their weight Respondents carry the final analysis weight Coverage & known totals → step_calibrate(): align to the population The frame and the population *overlap* rather than nest: some of the target population is missing from the frame (undercoverage) and some frame units are out of scope (overcoverage). From the sample downward it is pure nesting (sample ⊃ eligible ⊃ respondents), and each split is a step. The weights are not conjured; they are the consequence of this sequence of decisions. ## The same strategy as one object `weightflow` expresses the whole strategy as a single, explicit recipe. Nothing is hidden in a script; nothing lives in an intermediate CSV. Every methodological decision is a `step_*()` you can read top to bottom: The whole strategy as one recipe object A single container holds weighting_spec() and a stack of step functions: unknown eligibility, drop ineligible, select within, nonresponse, calibrate and trim. An arrow leaves the container to prep(), which produces the final weights. One object: the whole strategy, defined lazily, read top to bottom. weighting_spec(data, base_weights) # design weights 1/π step_unknown_eligibility() # resolve unknown eligibility step_drop_ineligible() # remove out-of-scope units step_select_within() # within-household selection step_nonresponse() # correct for nonresponse step_calibrate() # align to population totals step_trim() # tame extreme weights prep() final weights one output of the object > **`weightflow` doesn't just compute weights: it documents how they are > constructed.** > > It does compute the weights, of course. But a weighting strategy is a > sequence of methodological decisions, and `weightflow` turns those decisions > into an explicit object that can be inspected, reproduced, modified and > reused. The final weights are only *one* output of that object, and usually > the least valuable one. The recipe is the institutional asset. If you know the `recipes`/`tidymodels` world, the analogy is deliberate: a recipe describes how a dish is produced, not the dish itself. A `weightflow` recipe describes how the weights are produced, not the weights themselves. ## The shortest real recipe Enough theory. `weightflow` ships a small bundled example (a stratified household sample, `sample_survey`, drawn from a known `population`), so every line below actually runs. ```{r glimpse} str(sample_survey[, c("pw", "region", "sex", "unknown_elig", "responded")]) ``` A minimal but complete cascade: start from the design weights, resolve unknown eligibility, correct for nonresponse, and calibrate to the population totals of `region` and `sex`. We pass those totals in the **tidy** form: a small data frame per margin with the categories and a counts column. `table()` already produces exactly that shape (its counts column is named `Freq`): ```{r totals} m_region <- as.data.frame(table(region = population$region)) m_sex <- as.data.frame(table(sex = population$sex)) m_region ``` ```{r recipe} fit <- weighting_spec(sample_survey, base_weights = pw) |> step_unknown_eligibility(unknown = unknown_elig, by = "region") |> step_nonresponse(respondent = responded, method = "weighting_class", by = "region") |> step_calibrate(method = "raking", totals = list(m_region, m_sex), count = "Freq") |> prep() ``` `prep()` is where the recipe is actually estimated. Everything before it only *records* the strategy. `collect_weights()` returns the data with the final weight attached as `.weight`: ```{r weights} w <- collect_weights(fit) summary(w$.weight) ``` ## The object is the documentation Because the strategy is an object, it explains itself. `summary()` walks the cascade step by step, reporting how many units each stage touched, how weights changed, and diagnostics such as the design effect: ```{r summary} summary(fit) ``` That printout *is* the methodological record. You did not write a separate memo describing what the scripts did; the recipe and its summary are the memo. Re-running it next year, or handing it to a colleague, reproduces the exact same weights and the exact same explanation. ## Where to go next You now have runnable weights and a mental map. To go deeper: - [*Staged survey weighting: the adjustment logic*](weightflow.html) covers the statistical reasoning behind each stage and why order matters. - [*Preparing the sample*](preparing-the-sample.html) covers eligibility, rosters and within-household selection in detail. - [*Nonresponse: weighting classes and propensities*](nonresponse-propensities.html) and [*Calibration*](calibration.html) cover the two workhorse adjustments. - [*Variance estimation*](variance-estimation.html) covers the bootstrap and jackknife that re-apply the whole recipe, so uncertainty from nonresponse and calibration reaches your standard errors.