Frequently Asked Questions
***UPDATE*** This updated FAQs document incorporates changes to the CMS Nationwide CAHPS Survey of Adult Medicaid Enrollees (Nationwide CAHPS Survey) sampling specifications released in July 2014. After further review of those sampling specifications and MSIS data availability, CMS and NORC have adjusted the specifications to shift the eligibility time period used to develop the sampling frame to an earlier time periodso that more states are able to choose Sampling Option 1. Of the three sampling options, Option 1 requires the least amount of effort by an individual state, with NORC doing most of the required work to create the sample.
The FAQs are divided into five sections as shown below:
- General Sampling Related Questions
- Option 1Specifications
- Option 2 Specifications
- Option 3 Specifications
- Secure Data Transfer
What if my state has questions that aren’t answered here or in the sampling specifications?
NORC’s team of state data experts and statisticians are available by email and phone to answer additional questions you may have for implementing the sampling specifications. NORC’s email for this project is .
General Sampling Related Question
What are the changes to the updated sampling specifications?
The new sampling specifications containfour major changes:
-The eligibility time period for constructing the sampling frame is now October 1, 2013 through December 31, 2013 (originally it was January 1, 2014 through June 30, 2014).
-As a result of the updated eligibility time period, we have deleted the newly eligible indicator and added a new flag to indicate whether individuals enrolled during this period were eligible on August 31, 2014. Please see the “Sample Specifications Final v2” for more information. This information is located at:
-The long term care claims file should cover January 1, 2013 through December 31, 2013 (originally it covered July 1, 2013 through June 30, 2014).
-The sample size per stratum has changed,whichonly affects states choosing Sampling Option 3 (originally the sample size was the same across all four strata).
My state conducted a CAHPS survey this year. Will CMS and NORC take measures to ensure enrollees surveyed by my state are not surveyed again?
Deduplication of survey respondents is operationally challenging due to prohibitions against sharing personally identifiable information.However,in special circumstances, NORC may be able to remove the respondent population from the Nationwide CAHPS Survey sampling frame, provided the state is the intermediary for this informationand supplies NORC with either SSNs or MSIS identification numbers for those who participated in the earlier state CAHPS survey.This information must be transmitted from the state to NORC via the CMS secure data transfer connection. NORC will check the list of sampled enrollees and if an enrollee’s identifier is found both in the sample and in the state-provided list, a substitute enrollee will be selected for the sample. The procedure that NORC will use is described in Sampling Options and Specifications for the Nationwide CAHPS Survey of Adult Medicaid Enrollees, Appendix C: Option 3 Specification, Section III. Sample Selection, 5. Substitution Rule.
At least one Medicaid health plan has conducted a CAHPS survey in my state this year. Will CMS and NORC take measures to ensure enrollees surveyed by Medicaid health plans in my state are not surveyed again?
CMS and NORC have consulted with the National Committee for Quality Assurance (NCQA), the Association for Community Affiliated Plans (ACAP), and a CAHPS vendor regarding this issue. Based on the feedback received during these discussions, CMS has concluded it is not feasible to remove enrollees who were previously surveyed by Medicaid health plans at this time. This would require transferring personally identifiable information (PII) from vendors to states and then to NORC, which would place data at risk of inappropriate disclosure. In most states, the sample will be sufficient to ensure that most enrollees who were sampled by the health plans would not also be sampled for the Nationwide CAHPS Survey.
My state intends to conduct a CAHPS survey within the next year. Will CMS provide us with a list of enrollees that participated in the Nationwide CAHPS Survey of Adult Medicaid Enrollees?
NORC will provide CMS with a list of MSIS identification numbers and SSNs of all enrollees in your state that responded to the CAHPS survey. CMS and your state can work together on an agreement for CMS to share this information with your state.
Medicaid health plans inmy state intend to conduct CAHPS surveys of state enrollees within the next year. Will CMS provide the Medicaid health plans a list of enrollees that participated in the Nationwide CAHPS Survey of Adult Medicaid Enrollees?
NORC will provide CMS with a list of MSIS IDs and SSNs of all enrollees in your state that responded to the Nationwide CAHPS Survey. CMS and your state can work together on an agreement for CMS to share this information with your state, and determine if the data can be shared with the Medicaid health plans.
Option 1: NORC selects a sample from MSIS records.
What does Option 1 entail?
Under this option, NORC will draw a sample of enrollees from the up-to-date MSIS data files that states have previously submitted to CMS. After the sample is selected, NORC will send a file to each state through CMS’secure data transfer connection. The file will contain either the MSIS-ID and/or SSN for each sample record. Because MSIS data files do not contain contact information for the enrollees, the state will need to match on the given ID number, and deliver a file to CMS and NORC that contains the enrollee ID number that was initially provided and the appended enrollee contact information(e.g., enrollee address, telephone number),along with any additional flags that are required (e.g., prior state CAHPS survey flag). The file should be sent as a pipe-delimited (i.e., “|”) text file.
What if my state’s MSIS files are not current forFFY Quarter 1 (October – December 2013)? Can I still choose this option?
If your state has submitted its MSIS eligibility and long-term care files for FFY 2014, Quarter 1 butthe files are still under review by CMS, your state may be eligible for Option 1. In such instances, the state should consult with NORC to verify feasibility of using this information for the Nationwide CAHPS Survey sample.
How much effort will this optiontake?
We expect Option 1willrequire the least amount of time to implement. We anticipate that this option will take approximately 9 hours, on average, to complete for each state, based on results from a pilot study conducted in early 2014.1
1Note that the level of effort is an estimate based on a pilot study, and the actual amount of time spent may differ by state.
Option 2: State provides CMS with an MMIS (or other state Medicaid system) data extract.
What does Option 2 entail?
Under this option, states will send CMS and NORC two data files: one with an MMIS (or other state system) data extract of eligible enrollees and a second data file with long-term care claims (CLAIMSLT). Using these two files, NORC will select a sample of enrollees for the survey.The file of eligible enrollees provided to CMS and NORC will contain the enrollee ID number and appended enrollee contact information (e.g., enrollee address, telephone number) along with any additional flags that may be required (e.g., prior state CAHPS survey flag). The file should be sent as a pipe-delimited (i.e., “|”) text file.
Does my state need to include enrollee contact information(e.g., enrollee address, telephone number) on all eligible enrollees?
Contact information does not need to be sent with the initial data files, but will eventually need to be transmitted to CMS and NORC. If a state chooses, the enrollee contact information can be withheld for the initial file delivery, but will require an additional effort by the state to append the enrollee contact information after NORC has selected the sample of enrollees.
How much effort will this optiontake?
We expectOption 2 will require approximately 57 hours, on average, to complete, based on results from a pilot study conducted in early 2014.2 Ifenrollee contact information is excluded from the initial file transmission, then additional hours are needed for appending the enrollee contact information onto the file.
2 Note that the level of effort is an estimate based on a pilot study, and the actual amount of time spent may differ.
Why does my state need to send the CLAIMSLT file?
The long-term care file is needed so NORC can appropriately identify enrolleeswho are in institutional care settings. The Nationwide CAHPS Survey addresses respondents that are non-institutionalized, and as such, NORC will exclude these cases prior to sample selection.
My state is in the process of converting to the T-MSIS IDs. Which IDs should we send?
Either the MSIS or T-MSIS IDs are sufficient so long as both the enrollee and long-term care claims file use the same IDs. The IDs your state providesaresolely dependent on where your state is in the process of converting to the T-MSIS IDs. It is important that both the MMIS and CLAIMLT files have the same ID numbers so that NORC can identify those enrollees that are in both files.
I’m not sure how to define a variable (e.g., Basis-of-Eligibility) given our transition to T-MSIS?
In all cases, please defer to the MSIS definitions for defining plans, enrollees, etc. These are included in Appendix D of the sampling specifications.
Option 3: State constructs sample frame and selects sample of enrollees.
What does Option 3 entail?
Under this option, states will be responsible for selecting the sample of enrollees and sending the selected sample with enrollee contact information to CMS and NORC. It will require the state to spend time constructing the sampling frame and noting record counts and frequencies during the process for review by NORC. Once the state constructs the sampling frame per the sampling specifications provided by NORC, the state will need to apply a detailed sampling methodology defined by NORC to select enrollees for the survey, and again, record counts and simple frequencies will be required during the process. The file of selected enrollees provided to CMS and NORC will contain the enrollee ID number and appended enrollee contact information (e.g., enrollee address, telephone number) along with any additional flags that may be required (e.g., prior CAHPS survey flag). The file should be sent as a pipe-delimited (i.e., “|”) text file. Additionally,detailed tables that need to be filled in with record counts and/or frequencies should be sent with the final selected sample file. The tables to be filled in will be provided by NORC.
How much effort willthis option take?
We expect this option will require the most amount of time for states to complete, based on the results from a pilot study conducted in early 2014. We anticipate this option will likely take 66hours, on average, to complete across 3-5 weeks.3
3Note that the level of effort is an estimate based on a pilot study, and the actual amount of time spent may differ.
Does my state need to follow the sampling specifications exactly?We already have a program that can select random samples.
Yes, the sampling specifications need to be followed exactly. CMS and NORC are conducting this survey across the 50 states and the District of Columbia. It is crucial that the same sampling methodology is implemented in each state so that national estimates are based on the same target population. The specifications for creating the sampling frame and sample selection are extremely detailed to ensure that the same procedures are implemented across all states.
Why does mystate need to providethe detailed record counts at each step?
In working with multiple states, NORC has parameters for the number of cases that are likely to be removed at each processing step during sample frame creating, mostly due to enrollees being out-of-scope for this survey. By reviewing the sample counts for each step, we can quickly determine if the steps defined for setting up the sampling frame and selecting the sample were followed appropriately. It is crucial that the methodology for constructing the sampling frame and selecting the sample of enrollees follows the same process, whether the state draws the sample or NORC draws the sample, as in sampling Options 1 and 2.
Why does my state need to provide frequencies by the many different demographic variables?
As with the record counts needed during the sampling frame construction, the demographic distributions from the sampling frame are needed for comparison against the selected sample to ensure the procedures were implemented appropriately.NORC will provide the tables to be filled in for the states that choose this option.
How long should my statekeep the sampling frame that is constructed?
While it is hoped that the sample selection goes smoothly, there is always a chance that a new sample will need to be drawn. Therefore, we recommend the sampling frame be kept for a minimum of one year from the date of construction.
What if my statedoes not have all four strata categories? Should we increase our sample size in the other categories?
The target sample size requiredin each stratum are as follows:Full Duals = 5,556; Disabled = 5,556; Managed Care =11,110; and FFS/PCCM (Fee for Service/Primary Care Case Management) = 7,404. If there is a stratum category that does not apply in your state (e.g., managed care or fee-for-service), simply create the sample in the same way for the remaining categories with the requiredtarget per stratum. For example, if your state does not have a fee-for-service program for adult Medicaid enrollees, your state would select a sample of5,556 cases in the Full-Duals stratum, 5,556 cases in the Disabled stratum, and 11,110 in the Managed Care stratum, for a total of 22,222 selected enrollees.
Secure Data Transfer.
Who should work on creating and transferring the data files?
The people who typically send the MSIS update files to CMS are the ideal group to work on this part of the project.
Does our state need to set up a new MSIS secure data transfer connection for this project?
No. It is expected that states will use their existing MSIS secure data transfer connection for this project. The data file naming convention specified in the sampling specifications will route the data files automatically to NORC from CMS.
Do we follow the same naming conventions as with an MSIS transfer?
No. In order for the data files to be routed to NORC different naming conventions are needed. See Appendix E: Secure Data Transfer Step-by-Step Instructions of the Sampling Options and Specifications for the Nationwide CAHPS Survey of Adult Medicaid Enrollees document.
What should I do once the file is created and ready to transfer?
Send the file through the MSIS secure data transfer connection and email the details to .
What information should my state include when emailing about delivering the file?
Please include the names of each file that were delivered, a brief description of each file that was sent (i.e., this is the enrollee file with enrollee contact information), the number of records in each file and on what platform the file was created (e.g., Mainframe, UNIX). Additionally, if Option 3 is chosen, the tables provided by NORC should be filled in completely and sent as well. This can be as an attachment to the email.
What if I have issues transferring my data through the secure data transfer process?
First, confirm that the correct file naming convention is being used. If a state is still having issues transferring data files to CMS and NORC, the state should contact the CMS IT Service Desk at 1-800-562-1963 or nd open a trouble ticket for secure data transmission issues. Please include “PR 823 CAHPS EFT” in the subject line, so that it is routed to the right people.