As opposed to the decennial Census datasets available with tidycensus, datasets from the five-year American Community Survey include estimates with an associated margin of error, as the ACS is based on an annual sample of around 3 million households. As explained by the Census Bureau:
While the main function of the decennial census is to provide counts of people for the purpose of congressional apportionment and legislative redistricting, the primary purpose of the ACS is to measure the changing social and economic characteristics of the U.S. population. As a result, the ACS does not provide official counts of the population in between censuses. 1
In turn, ACS data do not represent precise counts of
population subgroups, but are rather designed to give a general sense of
how socioeconomic indicators vary across the country. In many cases, ACS
margins of error can be quite large - at times exceeding the estimate.
Let’s say we want to study aging populations in Ramsey County, Minnesota
from the 2020-2024 ACS by Census tracts. We can pull up a dataset with
get_acs:
library(tidycensus)
library(tidyverse)
vars <- paste0("B01001_0", c(20:25, 44:49))
ramsey <- get_acs(geography = "tract",
variables = vars,
state = "MN",
county = "Ramsey",
year = 2024)
head(ramsey |> select(-NAME))## # A tibble: 6 × 4
## GEOID variable estimate moe
## <chr> <chr> <dbl> <dbl>
## 1 27123030100 B01001_020 81 40
## 2 27123030100 B01001_021 72 43
## 3 27123030100 B01001_022 47 34
## 4 27123030100 B01001_023 171 81
## 5 27123030100 B01001_024 27 26
## 6 27123030100 B01001_025 51 34The margins of error for these small-cell estimates are sizable relative to the estimates themselves, and across the full 1,716-row dataset the margin of error actually exceeds the estimate in roughly 43 percent of rows. One way to address this is through data aggregation. While the specific group estimates in any given Census tract may be unreliable, the estimate for the total population over age 65 in the tract is likely better, but is not available directly from the ACS detailed tables. In turn, we can summarize our data.
The US Census Bureau publishes guidelines on how to calculate margins
of error for derived estimates (see the footnotes). These formulas are
implemented in tidycensus in the moe_sum,
moe_prop, moe_ratio, and
moe_product functions. The example below illustrates the
use of moe_sum to calculate the margin of error around a
derived estimate for Census tract population over age 65.
ramsey65 <- ramsey |>
group_by(GEOID) |>
summarize(sumest = sum(estimate),
summoe = moe_sum(moe, estimate))
head(ramsey65)## # A tibble: 6 × 3
## GEOID sumest summoe
## <chr> <dbl> <dbl>
## 1 27123030100 972 175.
## 2 27123030201 1167 231.
## 3 27123030202 76 39.0
## 4 27123030300 971 199.
## 5 27123030400 479 160.
## 6 27123030500 413 140.The margins of error for this aggregate population are more reasonable. However, the Census Bureau does issue this warning:
These methods do not consider the correlation or covariance between the basic estimates. They may be overestimates or underestimates of the derived estimate’s standard error, depending on whether the two basic estimates are highly correlated in either the positive or negative direction. As a result, the approximated standard error may not match direct calculations of standard errors or calculations obtained through other methods. 2
Dealing with margins of error in the ACS is a complex, yet important topic. This is especially true when dealing with spatial data, and considering ways to aggregate areal units to improve the reliability of estimates. I recommend the following papers for further reading:
Spielman, S.E., and Folch, D.C. (2015). Reducing Uncertainty in the American Community Survey through Data-Driven Regionalization. PLOS ONE. http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0115626; Python implementation at https://github.com/geoss/ACS_Regionalization
Spielman, S.E., and Singleton, A. (2015) Studying Neighborhoods Using Uncertain Data from the American Community Survey: A Contextual Approach. Annals of the Association of American Geographers. http://www.tandfonline.com/doi/full/10.1080/00045608.2015.1052335; R implementation at https://github.com/geoss/acs_demographic_clusters.
Wong, D.W., and Sun, M. (2013). Handling Data Quality Information of Survey Data in GIS: A Case of Using the American Community Survey Data. Spatial Demography. https://link.springer.com/article/10.1007/BF03354884.
United States Census Bureau (2008). A Compass for Understanding and Using American Community Survey Data. https://www.census.gov/content/dam/Census/library/publications/2008/acs/ACSGeneralHandbook.pdf.↩︎
United States Census Bureau (2016). Instructions for Applying Statistical Testing to the 2011-2015 ACS 5-Year Data. https://www2.census.gov/programs-surveys/acs/tech_docs/statistical_testing/2015StatisticalTesting5year.pdf.↩︎