Section B

Table N2. Long-term care quantitative studies: Columns 11-18 of 18 (pages N-7 to N-24)

Author
Year / 11. Results: KQ1 (Health Care Outcomes) / 12. Results: KQ2 (Harms) / 13. Results: KQ3 (Provider Outcomes-QI & other Behaviors) / 14. Results KQ4:(Selection by Patients & Payers) / 15. Results: KQ5 (Impact of Public Report Characteristics) / 16. Results: KQ6 (Impact of Contextual Factors) / 17. Summary/ Conclusion / 18. Funder of Research/ and Conflicts of Interest
Cai
201078 / None / None / Vaccination rates (mean) for States and DC (n=51)
(2005-6 pre-Report, 2006-7, 2007-8, change 2005-6 to 2007-8)
Short-term residents
74.64, 76.99, 80.10, 5.46%
Long-term residents
87.15, 87.88, 88.82, 1.67%
Community dwelling
65.64, 68.80, 72.05, 6.41%
38 states experienced improvement in vaccination rates for short term residents and 29 states for long term residents. / None / None / NYS NH Vaccination rates by facility and baseline score, percent
(2005-06 and 2006-07 )
Low baseline group
Short term residents: 58.53; 70.22
Long term residents: 83.43; 86.81
High baseline group
Short term residents 86.89; 85.33
Long-term residents: 93.62; 91.79 / Immunization rates at NHs increased after public reporting in NH Compare, but rates also increased in community dwelling elderly suggested the increase may not be due to public reporting.
Facilities that had low baseline scores were more likely to increase their vaccination rate. / NIA
Conflicts: Not Reported
Castle
200779 / Adjusted Odds Ratio (AOR) of highest quartile to lowest quartile (95% CI) for influence on quality measures
Competition (AOR>1 greater competition- improvement), Occupancy (AOR>1 higher occupancy-improvement), Interaction (AOR>1 lower competition and high occupancy - improvement).
Need for help with daily activities has increased 1.18 (1.03 to 1.27*) 0.85 (0.64 to0.96**) 0.92 (0.76 to 1.05)
Moderate to severe pain 1.10 (0.98 to 1.32) 0.97 (0.61 to 1.04) 0.99 (0.67 to 1.10)
Low-risk residents who have pressure sores 1.14 (1.01 to 1.26*) 0.86 (0.70 to 0.97*) 0.88 (0.71 to 0.97*)
Physically restrained 0.81 (0.76 to 1.03) 1.11 (0.90 to 1.32) 0.91 (0.86 to 1.12)
More depressed or anxious 0.95 (0.80 to 1.02) 0.96 (0.80 to 1.12) 0.97 (0.82 to 1.15)
Lost control over their bowels or bladder 0.92 (0.81 to 1.08) 0.98 (0.81 to 1.22) 0.93 (0.78 to 1.19)
Catheter inserted and left in 1.07 (0.89 to 1.10) 0.89 (0.77 to 1.00*) 0.90 (0.79 to 0.98*)
Spend most of their time in bed or in a chair 0.93 (0.81 to 1.05) 0.90 (0.72 to 1.15) 0.95 (0.77 to 1.12)
Ability to move about has become worse 0.96 (0.86 to 1.11) 0.93 (0.88 to 0.99*) 0.95 (0.82 to 0.99*)
Urinary tract infection 0.95 (0.86 to 1.05) 1.03 (0.93 to 1.12) 0.93 (0.86 to 1.10)
Lost too much weight 0.87 (0.79 to 1.08) 0.91 (0.88 to 1.13) 0.97 (0.89 to 1.10) / 5 Quality Measures (QM) have significant AOR for competition, indicating more improvement. 7 have lower AOR for occupancy also indicating more improvement.
Improvements were most likely in highly competitive markets and in markets with low occupancy rates. This supports the idea that report card encourage improvement through market-driven mechanisms.
3 of the QM that show more improvement are for short-stay residents, NH may be more open to influence by market forces for short stay (Medicare rates are higher and turn over may allow faster gains in improvement). / Funding: not reported
Competing interests: none declared
Castle
200779
Cont. / Short-stay residents with delirium 1.25 (1.04 to 1.29*) 0.85 (0.69 to 0.99*) 0.88 (0.70 to 0.97*)
Short-stay residents with moderate to severe pain 1.21 (1.07 to 1.33*) 0.73 (0.61 to 0.95*) 0.75 (0.68 to 0.98*)
Short-stay residents with pressure sores 1.15 (1.03 to 1.27*) 0.84 (0.78 to 0.97*) 0.99 (0.70 to 1.17) Overall quality measures’ difference 1.12 (1.03 to 1.16*) 0.89 (0.76 to 0.98*) 0.93 (0.79 to 0.97*)
Castle
200880 / Mean 2004, Mean 2006, Relative Change
(negative is improvement in quality):
Long-stay residents
Increased Help with Daily Activities: 15.39, 15.68, 2%*
Pain: 6.32, 5.03, -20%*
High-risk with Pressure Sores: 13.43, 12.80, -5%*
Low-risk with Pressure Sores: 2.59, 2.42, -7%*
Physically Restrained: 7.26, 6.13, -16%*
More Depressed: 14.66, 14.45, -1%*
Lose Control of Bowel or Bladder: 47.68, 48.66, 2%*
Catheter: 5.91, 5.79, -2%*
Most Time in Bed or Chair: 4.21, 4.21, 0%
Worse Ability to Move Around:12.18, 12.56, 3%*
Urinary Tract Infection: 8.64, 8.74, 1%*
Lose Too Much Weight:8.63, 8.73, 1%
Short-Stay Residents
Delirium: 2.97, 2.31, -22%*
Pain: 23.11, 21.47, -7%*
Pressure Sores: 19.16, 18.39, -4%*
*significant at .05 using a paired t-test / none / None / none / None / Influence of competition AOR (95% CI) and
Occupancy AOR (95% CI) on Quality Measures
AOR<1 = high competition associated with improvement
AOR>1 low occupancy associated with improvement
Long-stay residents
Increased Help with Daily Activities: 0.69 (.55-.85)**; 0.79 (.67-.94)**
Pain: 1.05 (.84-1.12); 1.10 (.87-1.39)
High-risk with Pressure Sores: 0.45 (.19-.77)**; .90 (.68-1.19)
Low-risk with Pressure Sores:0.89 (.69-1.44); 0 .61 (.45-.82)***
Physically Restrained: 1.41 (.86-2.32); 0.9 (.71-.96)**
More Depressed: 0.77 (.63-.97)**; 0.81 (.68-.96)*
Lose Control of Bowel or Bladder: 0.95 (0.59-1.52); 0.84 (.67-.99)*
Catheter: 1.02 (.90-1.15); 0.99 (.82-1.19)
Most Time in Bed or Chair: 0.94 (.87-.99)*; 0.93 (.75-1.16)
Worse Ability to Move Around: 0.96 (.79-1.17); 0.72 (.58-.89)**
Urinary Tract Infection:0.85 (.61-.97)*; .82 (.72-.95)**
Lose Too Much Weight: 0.43 (0.29-0.85)*; 0.89 (.59-.99)*
Short-Stay Residents
Delirium: 0.97 (.77-.99)*; 0.81 (.69-.95)*
Pain: 0.81(.67-.98)**; 1.10 (.91-1.32)
Pressure Sores: 0.93 (0.59-1.46); 0.81 (.63-.99)* / From 2004 to 2006, improvement in 9 quality measures, decline in 5 and 1 stayed the same. All but 2 (the no change and a 1% increase in % of residents who lose too much weight) were statistically significant (p<.05). Improvements ranged from a 20% reduction in residents with pain to a 1% reduction in % of residents more depressed or anxious.. The largest decline was a 3% increase in t% of residents whose ability to move around in their room got worse.
Stratifying the changes by the lowest 10% and highest 10% at baseline indicated that there may be some regression to the mean and for variables where this may be the case, an adjusted change score was calculated which reduced the magnitude but did not eliminate the improvement.
Improvements were most likely in highly competitive markets for 8 quality measures and in markets with low occupancy rates for 10 quality measures. This supports the idea that report card encourage improvement through market-driven mechanisms. / Not Reported
Castle
201081 / none / None / none / None / Impact on quality measure of having a SFF NHs in the county
Coefficient (SE) for model with all facilities.
High-risk residents with Pressure Sores -.201 (.039) **
Low-risk residents with Pressure Sores -.073 (.042)*
Residents with UTI -.261 (.101)*
Short-stay residents withe Pressure Sores -.044 (.031)*
Any deficiency .152 (.038) **
Quality deficiency citations .137 (.079)*
*p≤.01; **p≤.001
Remainder of quality indicators were not significantly different. When only the subset of NHs below the median on quality rating in the county are compared, 8 out of 22 quality indicators are higher in counties with SSF. Additional measures with significant differences are pain, depressed; lose too much weight, and flu vaccine. / The analyses provide partial and relatively weak evidence of spill over of improved quality in counties with a SFF receiving attention for poor quality for the NHs in the county that had poorer quality when the SFF was designated. The increase in deficiencies is counter to the spillover hypothesis.
In both cases, however the number of deficiencies and quality of care deficiencies cited during inspection surveys were higher for facilities in counties with a SFF. / NIA
Gaudet 201182 / Change Associated with NH Compare
Restraints:
Significant improvement at introduction (intercept) and post NH Compare slope increased.
Pressure Ulcers:
Decline at introduction and significant but small gain post NH Compare
ADL
Significant decrease at introduction and decline post NH Compare
Pain
Significant improvement after introduction and positive change rate of change post NH Compare / None / None / None / None / % of Medicare residents
minimal impact on performance change
(-0.01 change in intercept for restraints and PUs; .004 per quarter gain in ADL performance after NH compare)
Non Profit Ownership
No significant impact on performance.
Market competition
No significant impact on performance. County was used for market
% of Black Residents
Gap in baseline quality was not always in the direction expected. Impact of NH Compare is smaller on facilities with the highest portion of black residents for 3 measure of 4 measures (ADL, Restraints, Pain) when change in intercept is considered and in 2 of 4 (ADL, PU) when considering change in slope. / Improvement in quality associated with NH Compare varied by quality measure.Market and Facility Characteristics were not found to influence the impact of NH Compare. / AHRQ
Grabowski 201183 / The coefficients on the dummy variable for the introduction of NH Compare were not statistically significant in any of the models with a different QM as the dependent variable.
Coefficient/SE/N (facility quarters)
UTI :0.013 /0.017/ 369,907
ADL loss: 0.002/ 0.018/ 367,998
Physical Restraints:0.015/0.021/369,913
PU, high risk:-0.012/0.020/366,338
PU, low risk: -0.028/0.033/364,597
The estimated effects were also insignificant. / None / None / No Impact of NH Compare on Selection (Market Share)
NH quality below bottom quartile vs. /top quartile
Coefficient (NS unless noted)
UTI :-0.0001/0.007
ADL loss: 0.0009 (sig. at 10% level) /0.002
Physical Restraints:-0.0001/0.008
PU, high risk:0.006 /0.009
PU, low risk: -0.003/0.010 (sign. At 5% level) / None / The coefficients on the interaction term of the introduction of NH Compare with an measure of market competitiveness were statistically significant for two of the five QMs
Coefficient/SE/N (facility quarters)
UTI :-0.040 /0.029/ 369,907
ADL loss: 0.036/ 0.024/ 367,998
Physical Restraints:-0.022/0.051/369,913
PU, high risk:0.062/0.034/366,338significant at 10% level
PU, low risk: 0.217/0.048/364,597significant at 1% level / NH Compare has no overall effect on quality of care.NHs in more competitive markets do seem to respond to public reporting by increasing quality and the magnitude of the impact estimated by the models is meaningfully large (going from 2 to 5 facilities in a market would result in a 15% or 89% of standard deviation increase in quality for pressure ulcers in high and low risk residents respectively. / NIA
Jung
201085 / Scores improved for all five of the measures related to the management of daily activities, but the degree varied by measure from 7.1% increase (3.4 percentage points) for transferring to bed to 18.9% (5.7 percentage points for ability to walk around). Urgent care did not change and hospitalizations increased (interpreted as a decline in quality). [Data not shown in tables].
The percentage of agencies that:
Improved, No change, Worsened
1. Bathing 61.9, 10.8, 27.4
2. Transferring to bed 54.9, 10.8, 34.3
3. Taking oral meds 59.8, 11.9, 28.3
4. Have less pain 57.2, 11.5, 31.3
5. Walking or moving around 62.1, 11.1, 26.8
% of patients who
6. Need urgent care 41.5, 13.4, 45.2
7. are admitted to the hospital 47.2, 12.0, 40.8 / None / None / None / None /
Quality scores generally improved for all types of agencies. For profits were higher on some measures at baseline but by 2007 nonprofits had improved more and had better performance for all measure. Agencies with lower baseline scores improved more. Agency types associated with higher quality at baseline often had larger improvements. [Data presented graphically, unable to extract values). / Quality measures for patient’s ability to manage activities improved while urgent care and hospitalization did not. Baseline quality scores for 2003 varied by agency characteristics but the differences were small (3.6% to 11.3% of the mean depending on the measure). Not for profits did best on 4 of 7 measures, and for profits on 3 of 7. Hospital-based and larger agencies also had higher scores at baseline. There were no patterns in Medicare certification or region.
Agencies with lower baselines, nonprofits, hospital-based, and agencies with longer Medicare Tenure improved more. / Social Science Research Institute at Pennsylvania State University
Conflicts: none declared
Konetzka
201184 / None / 1.2 % point increase in discretionary hospitalizations by day 14 (sample average of 18.8%) after public reporting.Coefficient on public reporting indicator 0.0121; robust standard error 0.00007,p<.01
Controlling for secular trends (comparing the pilot and non pilot states) the increase is still significant, but smaller at 0.5 percentage points.
Non discretionary hospitalizations decreased after public reporting.The increase in discretionary hospitalizations was greater in patients at higher risk of scoring poorly on NH Compare indicators at day 14, even controlling for risk at admission. / None / None / None / None / Authors document that increase in re hospitalization is an unintended, negative consequence of public reporting. This suggests that gaming in order to improve publicly reported scores is not limited to selection of patients/residents at admission, but can also occur at other points in the care process. / Not reported
Mukamel
201088 / None / None / Ratio of clinical to hotel expenditures increased significantly (p<0.0001) by 5% after publication of NH Compare.
Average ratio: 1.78
Pre: 1.71 in 2001; 1.72 in 2002
Post: 1.76 in 2003; 1.84 in 2004; 1.85 in 2005; 1.80 in 2006
To get the same increase in expenditure ratio would require a 17% increase in case mix or a 27% increase in Medicare residents.
Controlling for differential growth in the costs of clinical verses hotel services using the CPI reduced the effect by 75%, it remained significant. / None / None / The stratified results support the author’s assumptions:
NH with lower quality scores, lower occupancy, for-profit, chain owned and in more competitive markets increased their clinical to hotel expenditures after reporting. / NHs do appear to have increased their expenditures on clinical services after the public release of NH compare. This is supported by the fact that subgroups expected to be more sensitive to public reporting (e.g. those in competitive markets) shifted more resources to clinical services. / NIA
Conflicts: Not Reported
Mukamel
200886 / Impact of Public Reporting on Quality Measures
(Time Trend Change-all NHs, Change in Level: Demo States, Change in Level. Non Demo States)
Physical Restraints 0.09, -0.92**, -0.74*
Short-term Pain 0.12, -2.78***, -2.54***
Pressure Ulcers 0.05, 0.47. 0.56*
ADLs 0.07, 0.48, 0.62
Infections -.18, -0.14, 0.23
***p<=.0001
**.001<p≤.01
* .01<p<.05 / None / Change in Level by Number of Actions Taken (1,2, 3, 4,5,)
Physical Restraints -.62, -.89**, -1.09***, -1.22***, -1.29***
Short-term Pain: -2.38**, -2.48***, -2.58***, -2.68***, -2.77***
Pressure Ulcers .52*, .52*, .52*, .52*, .52*
ADLs .64, .40, .22, .12, .08
Infections .16, .06, -.01, -.06, -.08
***p<=.0001
**.001<p≤.01
* .01<p<.05 / KQ4: none / None / None / Improvement was found in some but not all of the QM studied, all of which were publicly reported.Changes were linked to actions taken in response to reports.Based on prior improvement trends, NHC publication generated the equivalent of 3 years of improvement prior to the public report. / NIA
Stated: no disclosure or disclaimers
Mukamel
200987 / None / Significant decline (0.5 one-tailed tests) in post publication admissions
ADL limitations: none
Diabetes: none
Incontinence: none
PU stage 2 or higher: none
Pain: 2.5 percentage points; 13% fewer admissions around time of first publication
Memory loss: 0.4 percentage points; 0.7% fewer admissions around the time NH Compare changed in 1st Q 2004. / None / None / None / Significant decline (0.5 one-tailed test) in post publication admission when stratified by ownership, full occupancy status, having a low QM reported in first publication, chain affiliation and geographic region.
ADL limitations: none
Diabetes: none
Incontinence: none
PU stage 2 or higher: none
Pain: NH in bottom 20th percentile for state has a stronger and sustained decline in admissions.
Tendency to cream skim about for-profit and non-profit, but not government NH
Memory loss: Tendency to cream skim among for-profit and chain affiliated NHs. / Empirical analyses found cream skimming in 2 of 6 admission cohort characteristics but the effect sizes were not large. Four of the six characteristics did not decline in people admitted post NHC, suggesting there was no cream skimming. For the four admission characteristics in which there was no decline, a no decline was found in stratified analyses by NH types, suggesting the overall analyses were not hiding cream skimming.
For pain the evidence of some cream skimming is seen across the subgroups with no differences by chain affiliation or region while for-profits and non profits were more likely to cream skim than government-owned NH abut the strongest effect is that NH with poorer quality scores at initial publication were more likely to cream skim. For memory loss the subgroups with more cream skimming were for-profits, chain affiliation and, for only one follow-up Q, low quality NHs. / NIA
Conflicts: Not Reported
Park
2011 a89 / NHs with high quality scores or those that improve on publicly reported scores: increased market share and more Medicare admissions which lead to higher revenues and increased profit margins.The advantage comes primarily through increased Medicare admissions, which are financially advantageous. The difference was statistically significant when facilities were stratified by improvement (improved, no change, worse) and not significant when stratified by level of quality score (high, middle, low).Cost savings did not differ across the groups. / NHs that have higher quality scores or improve in quality had better financial performance (increase in revenue) after public reporting. / AHRQ
Park
2011 b90 / None / None / Interaction terms (profit margin and indicator of public reporting) were significant for three of the studied quality measures:
total staff hours per resident day (0.118 [7.08 minutes] p<0.01) ,
incidence rates of pressure sores (-0.201, p<0.01), and
Total number of deficiency citations (-0.034, p<0.05) ).
For restraint use the change pre and post public reporting was not significant (0.17 p>.1)
For a 25% increase in profit margin (mean total profit margin is 1.1%), the mean relative change in each quality measure is less than 1%.For example: Pressure Ulcer incidence predicted to decline 0.0005 percentage points (0.14% of the overall mean of 3.7%) or a reduction of 40 pressure ulcers in 786,297 residents. / None / None / For profit facilities: significantly higher quality with higher profit margin on the three of the four measures studied after public reporting
Non profit facilities: the change in the relationship between profit and quality was only significant for deficiencies.
The positive association between profit and quality increased more after public reporting in competitive markets (5.3%) then in less competitive markets (1.9%). / Public reporting changes the relationship between profit and quality in the way models of economic incentives predict, but these changes are small and may not be clinically important. / AHRQ and VA
Stevenson
200691 / None / None / None / Mean NH occupancy rate for the entire period was 86%.
Regression with DV=NH occupancy rate of next period. IV = deficiencies and staffing levels in prior period.
Regression coefficients
Prior deficiencies -0.038
Prior serious deficiencies -0.372
Prior nurse staff 0.021
Prior aide staff -0.008
all significant p<0.05
r-squared: 0.75 / None / None / While finding support for the idea that public reporting has an impact on selection of NH, the effect sizes are small. Occupancy rate may not be the most appropriate outcome measure as it is constrained in its potential to change. Regression analyses including alternative models, all find an effect of the quality or staff reporting on occupancy, but the effects are small: an increase in 10 deficiencies would result in 0.4 percent decrease in occupancy. / NIA
Werner
2009 a92 / Within NH changes associated with NH Compare
No pain 2.0 percentage points improvement (base 76%)
No delirium 0.5 percentage points improvement (base 96%)
Improved walking 0.2 percentage points improvement (base 7%)
Preventable re hospitalizations declined slightly (.075 to .05--estimated from graph)
Repeated model with small, non reporting l NHs as a control for secular trend
No pain: improvement but decreased magnitude
No delirium: no difference from pre-post model above
Improved walking: improvement and increased magnitude
Preventable hospitalizations: Slightly worsening then stable
All changes pre and post NH compare p<.01 / None / None / None / None / None / All three reported quality measures and potentially preventable re hospitalizations improved over time. (Same numbers reported as other Werner article)