1

Idaho STARS Child Care Research and Evaluation

Overarching Question:
What is the value of the IdahoSTARS program for children, families, providers, and community members?
Study Set I: Impact on Child
I. 1a Child Outcomes: To what extent do Idaho children in quality care and education settings (including family home and center-based care) succeed emotionally, cognitively, socially, and physically, and inlanguage and communication skills?
A preamble will be used to explain research-based evidenceregarding the criteria for quality settings, to prepare research instruments, and to compare and contrast national and international quality child care research results with Idaho-specific findings.
Study Design: Correlational;Child Outcomes and IdahoSTARS
Study will examine relationships between child outcomes, family and community demographics, types of care, availability and accessibility of programs, participation in ICCP, and involvement in IdahoSTARS Professional Development and the Quality Rating Improvement System.
Research Question(s) / Steps / Timeline
  1. What is the relationship between global child outcomes (e.g., Idaho Reading Indicator Scores—IRI; health indicators), socio-economic indicators of the community and families (e.g., free and reduced lunch, ICCP recipients,) and characteristics of child care settings (e.g., Quality Rating and Improvement sites, providers engaged in professional development, type of professional development, ICCP providers, and health and safety practices)?
/
  1. Complete literature review and preamble.
  2. Develop pilot case studies of children in two child care programs (including joint programs such as early Head Start) and considering children on IFSPs and IEPs. The results will be used to inform the design of child outcome studies for subsequent statewide studies each of the subsequent four years.
  3. Receive Human Assurances Committee approval.
  4. Identify funding source.
  5. Based on results of pilot case study(s) revise protocol for statewide implementation.
  6. Receive Human Assurances Committee approval.
  7. Implement.

I.1b Quality Care and Education for Every Child
A preamble will be developed to explain national and international evidence-basedand research-based practices that will provide a base from which to prepare research instruments and ultimately to compare and contrast Idaho-specific findings.
Study Design: Descriptive; Frequency and Survey Design; Care and Education of Every Child
Research Questions / Steps / Timeline
  1. Where are children across types of disabilities (mild to severe) and age ranges (0-3, 3-5, 5-12, 12-18) receiving care and education?
  2. To what extent are these settings inclusive and local?
  3. What type of support, training and education do Idaho providers need to (a) feel comfortable and (b) demonstrate competence to provide care for children with emotional, behavioral, cognitive, physical, and medical challenges?
  4. What are the critical variables across types of providers and communities that support quality inclusive care?
  5. What are the variables that will increase the number of children in local child care settings with special needs?
  6. What are existing interagency collaborations, agreements, and articulations necessary to promote quality early care and education for every child? What collaborations, formal agreements, and articulations are needed?
  7. What are the characteristic demographics, practices, and dispositions of those who report providing inclusive care and those who choose training in inclusive care?
/
  1. Complete literature review of national and international studies on child care and inclusion. Complete preamble.
  2. Develop a compendium of existing formal agreements between child care programs and public programs such as the Idaho Infant and Toddler Program, Head Start, Early Head Start, Medicaid, and local public school Developmental Preschool programs. Review and analyze for trends annually for each of five years.
  3. For each survey listed in this plan (parents, providers, directors, and community members) embed targeted questions regarding care and education for every child, including those being served with Individual Family Service Plans or Individualized Education Plans.
  4. Cooperate with Idaho Infant toddler Program, Early Head Start and Head Start, and Developmental Preschool program data sources regarding child care program awareness, availability, and use. (Specifically target IFSP and IEP goals and objectives for review.)
  5. Develop tools for a pre-post self rating of knowledge, dispositions, and skills of participants who take trainings about children with special needs. Incorporate the tool into each training/course that participants take. Follow a randomly selected sample of participants with interviews to gather information about application of the knowledge and skills taught in the training/course.

  1. Receive Human Assurances Committee approval for all instruments.
  2. Analyze STARS data on QRIS inclusion track for participation demographics, and cross tabulate participation with report of practices.
  3. Implement study annually in each of the next four years.
  4. Analyze and report findings annually.

Study Set II: Impact on Parents
II.1 Parent choices for child care
A preamble will be inserted here to explain national and international evidence-based and research-based practices to informdevelopment of instruments, and to ultimately compare and contrast Idaho-specific findings.
Study Design: Survey on Parent Knowledge, Dispositions, and Choices
Collect information on Parent/child demographics. Identify parental criteria for choosing child care type, and available choices. Identify parental knowledge and dispositions toward quality indicators. Identify parental perceptions of availability of quality settings that match their knowledge and dispositions toward quality indicators. Cross tabulate for quality indicator knowledge and disposition of parents with catchment area demographics for ICCP provider participation, IdahoSTARS participation, and Quality Rating Improvement System availability.
Research Questions / Steps / Timeline
  1. What do Idaho parents identify as quality and how do their dispositions, knowledge, and choices compare and contrast with national indicators?
  2. To what extent do state child care (IdahoSTARS) activities, such as QRIS ratings, impact parent choice of child care settings?
  3. What are the salient variables in parent choice (eg. health and safety, learning experiences, convenience, child satisfaction, parent support, flexibility, and cost)?
  4. To what extent do socioeconomic, cultural, ethnic/racial, and region or type of community impact parent choice? Do ICCP families have the same choice as non-ICCP families?
  5. What is the average length of a child’s enrollment in a particular program? What sort of continuity of care are children experiencing?
  6. How do parents find care? How do parents of children with special needs find child care?
  7. Are there enough child care programs to meet demand, particularly in remote rural areas?
  8. Can parent choice for quality be impacted by education? What information is most influential to promote quality care choices? What impact would an educational campaign have on family knowledge of quality indicators and practices in Idaho?
/
  1. Participate in National Survey of Early Care and Education.
  2. Completeliterature review and preamble.
  3. Design survey instruments, including protocols for interviews and focus groups and first draft of questionnaire.
  4. Receive Human Assurance Committee approval.
  5. Carry out focus groups and interviews to inform questionnaire development.
  6. Revise questionnaire based on interviews and focus groups.
  7. Disseminate questionnaire.
  8. Analyze results.
  9. Summarize and report results.
  10. Crosswalk Idaho findings with National Survey of Early Care and Education.
  11. Design a study examining the results of an educational campaign about quality in child care.

Study Set III: Impact on Providers
Improving and Maintaining Quality Care and Education
Through the Professional Development System
Overall Questions
  1. To what extent does the professional development system (incentives, training, mentors, access to formal education, apprenticeships, QRIS) impact provider and program quality?
  2. What components of the PDS most impact quality?

Staff Education and Training
III 1a Education and Training: Impacting practices
Study Design:Descriptive Study; Baseline Data from IdahoSTARS Records for Frequency Reports; and Cross Tabulation Study
Key Variables: education of provider, years of experience, type of setting, qualifications of trainers and mentors, training type and hours in formal and informal experiences.
Research Questions / Steps / Timeline
  1. What are the components of quality learning experiences (both training and formal education) for professional development across demographic groups?
  2. To what extent do training and education experiences impact child care practices?
  3. What combination of activities (i.e., incentives, mentors, training and education) best facilitates quality outcomes and provider satisfaction?
  4. Why, when, and where do providers attend training? What are common barriers to attending training?
/
  1. Complete literature review to support development ofinstruments to be used to compare and contrast Idaho results.
  2. Develop list of desired data points needed to respond to questions.
  3. Develop codes for data identification.
  4. Gather descriptive data from the IdahoSTARS database from training logs/scholarships targeting the following:
  5. How many providers have their own children in their care?
  6. What percentage of children isin ICCP programs?
  7. What number of providers is accessing training in each component?
  8. What are the characteristics of providers accessing the trainings in each of the 10 components?
  9. What are the characteristics of providers choosing to access college level course work as compared to training in each of the 10 components?

Research Questions / Steps / Timeline
  1. What regional trainings for each component are available and accessed?
  2. What are the average satisfaction ratings of each trainer across components?
  1. Carryout cross tabulation on training/education activities and provider characteristics.
  2. Report descriptive data, using frequency results.

III1b Education and Training: Quality Learning Experiences
Study Design 1:Observational Study
Research Questions / Steps / Timeline
What is a quality learning experience for Idaho providers? /
  1. Identify key variables in a quality training experience from literature.
  2. Observe trainers across each component to determine optimal/minimal level of expected training quality.
  3. Use outcomes of observations to identify expected training quality measures.

Study Design 2:Qualitative Case Studies
Research Questions / Steps / Timeline
What is a quality learning experience for Idaho providers? /
  1. Randomly select providers from family child care programs and child care centers who will do focused journaling for three years; have them articulate their experiences with training/education in the IdahoSTARS PDS.
  2. Interview mature child care providers concerning their current and historical involvement in quality initiatives.

III 1c Staff Incentives and Retention
Study Design:
Research Questions / Steps / Timeline
  1. What influence do incentives have on enrollment and retention of Idaho providers in the PDS?
  2. What are the minimal to optimal incentives needed to maintain Idaho provider interest in professional development opportunities?
/
  1. Harvest existing data from IdahoSTARS database.
  2. Survey providers.

Research Questions / Steps / Timeline
  1. What incentives influence Idaho providers to enter into the system, to seek on-going training/formal education, and to improve the quality of their practice?
  2. What type of work setting is the Idaho provider in?
  3. How is the Idaho provider influenced by leadership in the work setting?
  4. What are the tangible impacts of incentives on providers?

III1d) Quality Rating Improvement System
Study Design: Survey Study, Focus GroupInterviews, and Descriptive Data from IdahoSTARS QRIS, and Retrospective Interviews
Research Questions / Steps / Timeline
  1. What are the salient barriers and supports that impact participation and accomplishment in the QRIS?
  2. What are the benefits of a QRIS rating for child care programs in terms of attracting and keeping clients?
  3. What are the characteristics and demographics of participation in the QRIS, including: time to reach minimum qualifications for each tier; financial support and barriers; staff educational success barriers; staff turnover; program administration characteristics; retention to completion; and participant socioeconomic and educational demographics?
  4. What are the characteristics of centers that have chosen to participate in the QRIS?
/
  1. Access National Survey of Early Care and Education Study.
  2. Using descriptive data for IdahoSTARS QRIS and survey and focus group design, develop a statewide profile of directors and providers who participate in Idaho QRIS.
  3. Determine frequencies and choices of QRIS activities and progress of programs toward tier levels.
  4. Administer cross tabulations of demographics with QRIS activities and progress toward tier levels. Compare and contrast results of those programs that participate and thoseprograms that do not. Compare and contrast completers with those who do not complete. Target analyses of characteristics of those that move up or down tiers.
  5. Complete literature review.
  6. Develop survey design and quantitative study of program characteristics from IdahoSTARS records.
  7. Develop protocols for focus group, interviews, and questionnaires.
  8. Receive Human Assurances Committee approval.
  9. Carry out focus groups and interviews to inform questionnaire development.
  10. Develop and validate questionnaire.
  11. Survey directors/staff who are participating in QRIS and non-participating directors/staff.
  12. Interview mature child care providers concerning their current and historical involvement in quality initiatives (Retrospective Interviews).

Study Set IV: Community Impact
IV 1 Community knowledge about Idaho STARS and quality child care
Preamble will be inserted here to describe national and international evidence and research-based studies that show the impact of quality care on the economy and workforce.
Study Design: Survey design to explore community knowledge and attitudes about Idaho child care and IdahoSTARS role in quality care in Idaho
Research Question(s) / Steps / Timeline
  1. What do communities (rural, remote, urban, semi rural, suburban, across economic, ethnic and racial groups, family type) expect in terms of local child care provisions (e.g. health and safety, number of providers, quality and convenience of providers, education, training levels, and experience of providers)?
  2. Is there a difference between what communities expect across categories and what the early childhood professional community expects?
  3. In the case of disparity between expectations and reality, what process should the state/nation take to forge the divide?
  4. What are similarities and differences of expectations and perceptions of reality among child care providers, parents, employers, and community members?
  5. What are the demands and impacts of IdahoSTARS activities on the workforce and on local economies?
  6. What are employer expectations and awareness levels for IdahoSTARS activities?
  7. What are employer and community member’s responsibilities to child care programs?
  8. What are the community settings available to parents who have children with special needs?
/
  1. Complete literature review and preamble.
  2. Access existing data relevant to community perceptions of child care.
  3. Consider data needs and formats embedded in all aspects of the five year plan including: (a) demographics across each component of IdahoSTARS (e.g., numbers, type of providers enrolled, education level, type of provider, years of service, amount of training, type of training; (b) cost analysis questions of individual components; (c) customer satisfaction across all components; (d) awareness of program across community groups.
  4. Review Logic Model for targeted outcomes in relation to community and workforce.
  5. Develop questionnaires to address targeted outcomes.
  6. Receive Human Assurances Committee approval.
  7. Distribute questionnaires via internet survey tools and at meetings of EC3 and Early Childhood Consortium. Reduce and analyze results.
  8. Compile frequency data from IdahoSTARS database to reflect needs listed above.
  9. Report findings and conclusions.