As opposed to the decennial Census datasets available with tidycensus, datasets from the five-year American Community Survey include estimates with an associated margin of error, as the ACS is based on an annual sample of around 3 million households. As explained by the Census Bureau:
While the main function of the decennial census is to provide counts of people for the purpose of congressional apportionment and legislative redistricting, the primary purpose of the ACS is to measure the changing social and economic characteristics of the U.S. population. As a result, the ACS does not provide official counts of the population in between censuses. 1
In turn, ACS data do not represent precise counts of
population subgroups, but are rather designed to give a general sense of
how socioeconomic indicators vary across the country. In many cases, ACS
margins of error can be quite large - at times exceeding the estimate.
Let’s say we want to study aging populations in Ramsey County, Minnesota
from the 2012-2016 ACS by Census tracts. We can pull up a dataset with
get_acs:
library(tidycensus)
library(tidyverse)
vars <- paste0("B01001_0", c(20:25, 44:49))
ramsey <- get_acs(geography = "tract",
variables = vars,
state = "MN",
county = "Ramsey",
year = 2016)
head(ramsey %>% select(-NAME))## # A tibble: 6 × 4
## GEOID variable estimate moe
## <chr> <chr> <dbl> <dbl>
## 1 27123030100 B01001_020 51 27
## 2 27123030100 B01001_021 92 40
## 3 27123030100 B01001_022 48 28
## 4 27123030100 B01001_023 8 13
## 5 27123030100 B01001_024 51 52
## 6 27123030100 B01001_025 23 18We can see that in two instances the margin of error exceeds the estimate. One way to address this is through data aggregation. While the specific group estimates in this Census tract may be unreliable, the estimate for the total population over age 65 in the tract is likely better, but is not available directly from the ACS detailed tables. In turn, we can summarize our data.
The US Census Bureau publishes guidelines on how to calculate margins
of error for derived estimates (see the footnotes). These formulas are
implemented in tidycensus in the moe_sum,
moe_prop, moe_ratio, and
moe_product functions. The example below illustrates the
use of moe_sum to calculate the margin of error around a
derived estimate for Census tract population over age 65.
ramsey65 <- ramsey %>%
group_by(GEOID) %>%
summarize(sumest = sum(estimate),
summoe = moe_sum(moe, estimate))
head(ramsey65)## # A tibble: 6 × 3
## GEOID sumest summoe
## <chr> <dbl> <dbl>
## 1 27123030100 677 124.
## 2 27123030201 899 201.
## 3 27123030202 149 53.8
## 4 27123030300 783 154.
## 5 27123030400 423 131.
## 6 27123030500 396 111.The margins of error for this aggregate population are more reasonable. However, the Census Bureau does issue this warning:
These methods do not consider the correlation or covariance between the basic estimates. They may be overestimates or underestimates of the derived estimate’s standard error, depending on whether the two basic estimates are highly correlated in either the positive or negative direction. As a result, the approximated standard error may not match direct calculations of standard errors or calculations obtained through other methods. 2
Dealing with margins of error in the ACS is a complex, yet important topic. This is especially true when dealing with spatial data, and considering ways to aggregate areal units to improve the reliability of estimates. I recommend the following papers for further reading:
Spielman, S.E., and Folch, D.C. (2015). Reducing Uncertainty in the American Community Survey through Data-Driven Regionalization. PLOS ONE. http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0115626; Python implementation at https://github.com/geoss/ACS_Regionalization
Spielman, S.E., and Singleton, A. (2015) Studying Neighborhoods Using Uncertain Data from the American Community Survey: A Contextual Approach. Annals of the Association of American Geographers. http://www.tandfonline.com/doi/full/10.1080/00045608.2015.1052335; R implementation at https://github.com/geoss/acs_demographic_clusters.
Wong, D.W., and Sun, M. (2013). Handling Data Quality Information of Survey Data in GIS: A Case of Using the American Community Survey Data. Spatial Demography. https://link.springer.com/article/10.1007/BF03354884.
United States Census Bureau (2008). A Compass for Understanding and Using American Community Survey Data. https://www.census.gov/content/dam/Census/library/publications/2008/acs/ACSGeneralHandbook.pdf.↩︎
United States Census Bureau (2016). Instructions for Applying Statistical Testing to the 2011-2015 ACS 5-Year Data. https://www2.census.gov/programs-surveys/acs/tech_docs/statistical_testing/2015StatisticalTesting5year.pdf.↩︎