Skip to content

Methodology

The Behavioral Risk Factor Surveillance System (BRFSS) is conducted by the Alaska Division of Public Health in cooperation with the National Centers for Disease Control and Prevention (CDC). It is a monthly telephone survey that utilizes a standard protocol and interviewing methods developed by the CDC.

Sample Design

Although the main purpose of the BRFSS surveys is to estimate the prevalence of behavioral risk factors in the general adult population, interviewing each person is not economically feasible. Thus, a probability or random sample is drawn in which all persons have a known chance of selection. The BRFSS in Alaska uses a stratified random sampling design; the strata or regions are combinations of geographically clustered census areas and boroughs. An equal number of interviews are conducted from each region, which purposely over-samples the non-urban areas of Alaska. Over-sampling results in an adequate sample size collected from rural areas and allows analysis of the BRFSS data by region.

Sample Size

Each month over 200 Alaska residents age 18 and older are interviewed over the telephone regarding their health practices and day to day living habits, to reach an annual sample size of at least 2,500. The data are collected from January through December, for each year.

For some indicators BRFSS data have been combined with data from a second survey, the Alaska Supplemental BRFSS. This survey was state developed and is funded by the Alaska Tobacco Prevention and Control Program. The survey focuses largely on tobacco use and attitudes. It has been collected in Alaska since 2004. The Supplemental BRFSS uses the same sample design, data collection methodology, and has the same sample size goal as the BRFSS. Combining the Supplemental BRFSS survey with the BRFSS where possible allows for a larger sample size for analysis and more stable estimates. Also note that when the combined BRFSS or Supplemental BRFSS data are used, estimates will not match those provided by the CDC for those indicators.

Figure 1: Sample Size – Landline survey

Alaska BRFSS Sample Size by Year and Survey, 1991 - 2010Alaska BRFSS Sample Size by Year and Survey, 1991 - 2010

Table 1: BRFSS Sample Size by Year and Survey

Year

Total

Standard

Supplemental*

1991

1534

1534

 

1992

1536

1536

 

1993

1534

1534

 

1994

1535

1535

 

1995

1535

1535

 

1996

1536

1536

 

1997

1545

1545

 

1998

1989

1989

 

1999

2051

2051

 

2000

2083

2083

 

2001

2875

2875

 

2002

2694

2694

 

2003

2665

2665

 

2004

5128

2656

2472

2005

5755

2813

2942

2006

4247

2113

2134

2007

5116

2552

2564

2008

4946

2664

2282

2009

4765

2432

2333

2010

3237

1964

1273


Sampling Process

Landline

The GENESYS sampling system through Marketing Systems Group provides a random telephone number sample each month. They use a Disproportionate Stratified Sample (DSS) process that is designed to improve the probability that all households in Alaska with telephones have a chance of inclusion in the study. For DSS, 100-number blocks of telephone numbers are placed into two strata based on the presumed density (high or low) of residential telephone numbers. One-plus block strata have at least one residential telephone number while zero blocks have none. The BRFSS sample is drawn from one-plus blocks; zero blocks are not sampled. The one-plus blocks are further divided based on whether the numbers are listed in a directory (listed one-plus block) or not listed (not listed one-plus block). Numbers in the listed one-plus blocks are sampled at a higher rate than those in the not listed one-plus blocks. Phone number samples are drawn quarterly and use a “with replacement” design; the result is that a phone number may be sampled in more than one quarter. The quarterly samples are divided and provided monthly.

In addition, GENESYS electronically identifies business, non-working, and cell phone numbers through its identification services and has modified its identification services to detect non-working numbers in rural Alaska. This technological adjustment has improved the process and the survey efficiency for Alaska. This pre-screening process happens on a monthly basis to provide the most up to date sample to each state. Because Alaska has such a low number of active residential lines, the study requires a large phone sample each month to operate successfully.

Participation in the BRFSS is random, anonymous and confidential. Respondents are randomly selected from household residents 18 years of age or older. Only those living in households are surveyed, omitting residents of institutions, nursing homes, dormitories and group homes.

Cell phones

During 2003 to 2009, the proportion of U.S. adults who lived in cell phone-only households increased by more than 700%, and the trend is continuing; this increase is illustrated in Figures 2 and 3 below.

Figure 2: Percentage of Adults and Children Living in Households with Only Wireless Telephone Service or No Telephone Service: United States, 2003-2011

Percentage of adults and children living in households with only wireless telephone service or no telephone service: United States, 2003 -2011
From the CDC/NCHS National Health Interview Survey
Adults living in households with only wireless service has increased to 30% in 2011.  the percentage of adults and children living in households with no telephone service has remained steady at less than 2%.Percentage of adults and children living in households with only wireless telephone service or no telephone service: United Stat
NOTE: Adults are aged 18 and over, children are under age 18.
DATA SOURCE: CDC/NCHS, National Health Interview Survey.


Figure 3: Polynomial regression equations fitted to a plot of the Percentage of Adults Living in Cell Phone Only Households by Age and Year

Percentage of Adults Living in Cell Phone Only Households by Age and Year: United States, 2003-2009 From the CDC/NCHS National HPercentage of Adults Living in Cell Phone Only Households by Age and Year: United States, 2003-2009 From the CDC/NCHS National H
Source: Blumberg SJ, Luke JV. Wireless substitution: Early release of estimates from the National Health Interview Survey, July-December 2008. National Center for Health Statistics. May 2009. (http://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless200905.htm)


Figure 3 shows that, in particular, younger adults are more likely to live in a cell phone only household. Because of differences in the characteristics of people living in households with and without landlines, all telephone surveys in the United States have had to adapt their methods to this relentless increase in cell phone-only households.1,2

Figure 4: Characteristics of Adults Living in Cell Phone Only Households
Characteristics of Adults Living in Cell Phone Only Households From “Weighting BRFSS Dual Frame Data”, Presentation by Machell TCharacteristics of Adults Living in Cell Phone Only Households From “Weighting BRFSS Dual Frame Data”, Presentation by Machell T
Source: “Weighting BRFSS Dual Frame Data”, Presentation by Machell Town and William Bartoli, 2011 BRFSS Conference, March 20, 2011


Figure 4 illustrates characteristics of people who are switching to cell phone only.

Figure 5: Age Distribution of Alaska BRFSS Respondents Compared to Adult Population by Age Group, 2000 and 2010

Age Distribution of Alaska BRFSS Respondents Compared to Adult Population by Age Group, 2000 and 2010 In 2010, the BRFSS captureAge Distribution of Alaska BRFSS Respondents Compared to Adult Population by Age Group, 2000 and 2010 In 2010, the BRFSS capture

Figure 5 shows the change in the Alaska BRFSS landline survey population, compared to the Alaska adult population, for 2000 and 2010. In 2010, the BRFSS captured more older adults and fewer younger adults than the Alaska adult population; this disparity is much larger in 2010 than it was in 2000. As noted in Figure 5, young adults are easily missed by landline surveys as they are more likely to live in a cell phone only household. Through 2008 these cell phone only households were not included in the BRFSS as only households with landline telephones were eligible. During 2009, all states conducted a small cell phone-based BRFSS using a sample of cell phone exchanges. Due to the small number of surveys, through 2010 responses from the cell phone survey have not been combined with those from the landline survey. In 2011, a large enough cell phone sample was collected to allow reporting of the cell phone and landline data combined.

The methodology for the cell phone survey is very similar to that of the landline survey, with some changes to the sampling process. Cell phone numbers are not stratified by region and instead a statewide sample is provided each month. Cell phone numbers are also not pre-screened as with the landline sample. Further as a cell phone generally considered an individual rather than a household device, the goal is to reach the cell phone of an adult 18 years and older living in a private residence or college housing in Alaska. Respondents to the cell phone survey may also have a landline but must receive 90% or more of their phone calls on their cell phone. Owners of business only cell phones are not interviewed. Content of the landline and cell phone surveys are the same.

Survey Instrument

The CDC BRFSS questionnaire has three parts:

  • Core
  • Optional standard modules
  • State-added questions

The core is a standard set of questions asked by all states within a year. It includes questions about current health related perceptions, conditions, and behaviors (e.g., health status, health insurance, diabetes, tobacco use, selected cancer screening procedures, and HIV/AIDS risks) and questions on demographic characteristics.

Optional modules are CDC-supported sets of questions on specific topics that states can choose to add to their survey. Individual states may develop their own questions or identify questions of local interest that come from other national surveys or health evaluations. These state-added questions are not edited or evaluated by CDC. States are selective with choices of modules and state-specific questions to keep the questionnaires at a reasonable length.

Each year the states and CDC agree on the content of the core component and possible optional modules. BRFSS protocol specifies that all states ask the core component questions without modification and may elect to add modules and state-added questions. Any new core or optional module questions proposed as additions to the BRFSS must go through cognitive and field-testing prior to their inclusion in the survey. The practice of utilizing questions from other surveys such as the National Health Interview Survey or the National Health and Nutrition Examination Survey allows the BRFSS to take advantage of cross-comparison between studies.

The Supplemental BRFSS Survey content has some overlap with the Standard BRFSS, including demographics, CVD status and diabetes status. The big difference is the focus of the Supplemental BRFSS Survey on tobacco use, beliefs and attitudes.

Data Collection

The Alaska BRFSS survey data collection is conducted by an experienced third-party contractor. Interviewers, who are extensively trained using a standardized CDC protocol, conduct the interviews seven days a week. Interviews are monitored for training purposes and quality control. Data are collected via computer using WinCATI (Windows-based Computer Assisted Telephone Interviewing) software. While conducting the telephone interview, the interviewer has the script and questionnaire on a computer screen, which is read verbatim. The designated answer of the respondent is selected on the screen. Incorporating edits and skip patterns into the WinCATI instrument reduces interviewer errors, data entry errors, and skip errors, while reducing respondent burden.

Data Processing and Analysis

Data processing is an integral part of the survey process, with collected data from the Standard BRFSS sent to CDC during each month of the year. Data conversion tables are developed to read the survey data and associated call history information from the WinCATI software, and to combine the information into the final format specified for the data year. CDC also creates and distributes a Windows-based editing program that can perform data validations on properly formatted survey results. This program is used to output lists of errors or warning conditions encountered in the data. These edited reports are produced monthly and corrections are made as needed, after which data files are sent to the CDC electronically.

At the end of each survey year, Standard BRFSS data are compiled and weighted by CDC, and cross tabulations and prevalence reports are prepared using SAS and SUDAAN software. To create the specific at-risk variables, such as binge drinking, several variables from the data file are combined with varying complexity. With the binge drinking example, the results from several questions in the alcohol section are combined to determine if a respondent is considered a binge drinker. The creation of some at-risk variables requires only combining codes, while others require sorting and combining selected categories from multiple variables.

Data from the Supplemental BRFSS survey are all processed, weighted and analyzed in-house following CDC protocols.

Weighting

BRFSS data are weighted to account for differences between the respondents and the population that the data should represent—that is, adults age 18 and older who are not living in institutional settings. The BRFSS weighting methodology can be divided into two parts: a) the design weight, which adjusts for the probability that the respondent would be selected, based on factors related to the sampling process, and b) adjustment based on demographic factors.

From 1991 to 2010, this second adjustment method, called post-stratification, adjusted for gender and age groups by region. However, during this time, developments and innovations in the world of survey analysis and statistics and computer processing made it possible to adjust for more factors and provide a better way to adjust for the over- or under-representation of various subpopulations in survey data. Between 2006 and 2010 CDC reviewed and applied a new adjustment method, called iterative proportional fitting, or “raking,” which improves the accuracy of BRFSS by allowing the use of more demographic variables. This new raking methodology permits the inclusion of education level, marital status, and renter/owner status, along with gender, age, and race/ethnicity, in the final weights. This method also allows adjustment by phone type—landline or cell phone, so that these groups are represented in the data, in proportion to how they occur in the population.

With the reporting of 2011 BRFSS data, the CDC has officially introduced the new methods of sampling (by cell phone as well as landline phone numbers) and raked weighting. Along with the inclusion of respondents who only have cell phones, raking improves the overall representativeness of the BRFSS data. This change in methods will mean that the way we use the data will change also. For example, we may not be able to review whether behavior has changed over time, across all the years of data collected. New trend reviews will eventually focus on years of data that include both landline and cell phone respondents, and which are adjusted using raked weighting methodology. These new methods are necessary to provide a more accurate reflection of the health behaviors and conditions measured in the BRFSS survey.

Limitations

The BRFSS uses telephone interviewing for several reasons. Telephone interviews are faster and less expensive than face-to-face interviews. Calls are monitored for quality control.

The main limitation of any landline telephone survey is that people without a landline phone cannot be reached and are not represented. This is somewhat rectified by including cell phones in the BRFSS sample. The percentage of households with a telephone varies by region in Alaska. In general, persons of lower socioeconomic status are less likely than persons of higher socioeconomic status to have phones and may be under-sampled. With surveys based on self-reported information, the potential for bias must be kept in mind when interpreting results. Survey response rates may also affect the potential for bias in the data. The literature shows that most questions on the core CDC BRFSS instrument are at least moderately reliable and valid and many were reported to be highly reliable and valid.3

Response Rates

One way to look at how well the BRFSS is reaching respondents is to look at the response rate of the survey (how many people answer the survey in comparison to how many are called). The CASRO rate is a modification of a simple response rate and is defined as the proportion of all eligible respondents in the sample for whom an interview is completed. So, phone numbers that are not working or are a business only phone are not included in this calculation. The CDC has set 40% as the lower limit for the CASRO response rate for states. If a state falls below a 40% CASRO response rate, data collection and training practices should be thoroughly examined.4 As shown in Figure 6, Alaska has had CASRO response rates well above 40% for all years data have been collected.

Figure 6: CASRO Response Rate by Year, Alaska BRFSS

CASRO response rates by survey and year, Alaska BRFSS, 1991-2010 Since 2004, when the Supplemental BRFSS started, CASRO rates fo
 

References:

1 Blumberg SJ, Luke JV, Ganesh N, et al. Wireless substitution: State-level estimates from the National Health Interview Survey, January 2007–June 2010. National health statistics reports; no 39. Hyattsville, MD: National Center for Health Statistics. 2011.

2 Link, MW, Battaglia, MP, Frankel, MR, Osborn, L, Mokdad AH. Reaching the US cell phone generation: comparison of cell phone survey results with an ongoing landline telephone survey. Public Opinion Quarterly. 2007, Vol. 71: No 5, 814-839.

3 Nelson, DE, Holtzman D, Bolen J, et al. Reliability and validity of BRFSS measures. Soz Praventivmed. 2001; Vol. 46:suppl.1

4 Behavioral Risk Factor Surveillance System 2010 Summary Data Quality Report. Version #1 – Revised: 05/02/2011. Available at: http://www.cdc.gov/brfss/technical_infodata/quality.htm