CalHospitalCompare.org

About the Data and Ratings

Some hospitals provide better care than others. This site rates quality of care, including clinical quality, patient experience, and patient safety, for participating California hospitals.

As of 2011, more than 232 hospitals, representing over 82% of acute care hospital admissions in the state, are participating in this important voluntary effort. The conditions and procedures rated are among the most common reasons for being admitted to a hospital (heart attack, heart failure, heart bypass surgery, pneumonia, and maternity).

Hospitals on this Web site are rated on several aspects of health care quality including the effectiveness and timeliness of care; patient experience with overall, medical, surgical, and maternity care; and how often hospitals follow recommended patient safety practices. In addition, readmissions and hospital length of stay, indicators of hospital efficiency, are reported.

As data become available, other conditions and measures will be included in future updates to this Web site.

To produce the information available on this Web site, a rigorous process is followed:

• Data are collected by government agencies, private reporting agencies, or the hospitals themselves.
• Data are reported to a central institution, the University of California at San Francisco Philip R. Lee Institute for Health Policy Studies.
• If needed, data are "risk-adjusted" so that apples-to-apples comparisons are possible. Risk adjustment is a statistical method that allows fair comparison of hospitals with different types of patients.
• Data are then compared to a benchmark value, either a national, state, or CHART standard.
• Ratings are assigned to measures when it is technically feasible (for example, a large enough sample or an accepted risk-adjustment approach) and there is an accepted standard.

Guidelines established by an oversight committee and expert statisticians determine how the data are processed during each of these steps.

Data Sources and Collection

Data collected by California hospitals are reported to various federal and state agencies and the CHART project directly, as noted in the table below. In all cases, we use the most recent data available. The time periods vary for different measures.

Note: Data on Potentially Preventable Readmissions were produced using proprietary computer software created, owned, and licensed by the 3M Company. All copyrights in and to the 3M [APR™] Software, and to the 3M [APR™ DRG] classification system(s) (including the selection, coordination, and arrangement of all codes) are owned by 3M. All rights reserved.

Note (Nov. 2011): Due to an error by the vendor that handles its patient experience surveys, only 55% of Kaiser hospitals' patient experience surveys were submitted. Because of this error, some Kaiser hospitals are labeled "too few cases (vendor error)," despite the fact that Kaiser California has enough patients and all eligible patients are surveyed. This issue will be resolved by February 2012.

Data Validation

Data used on this Web site are reported to the following entities and validated in the following ways:

• Both CHART and CMS: subject to review and validation by CMS
• OSHPD and then supplied to CHART: periodically validated through chart review studies and other auditing procedures
• CCORP: audited through cross-referencing two databases that report the same outcomes and auditing of patient records
• CalNOC: screened for inconsistencies and internal validation checks are employed
• CHART directly: periodically audited on site at CHART hospitals by independent registered nurses certified in CHART data collection methods

Hospitals are allowed to review and resubmit data before it is publicly reported.

Risk-Adjustment Methods

Some patients respond better to treatment than others. For example, having diabetes can reduce your chances of surviving heart surgery. So it would not be fair for a hospital with more diabetic patients to have a worse rating just because they treat more sick patients. "Risk adjustment" is the process that levels the playing field among hospitals to fairly compare hospital performance.

However, some measures do not need to be risk adjusted. For example, having diabetes does not prevent the nurse from checking your wristband before giving a medication, so there should be no adjustment for diabetes in that measure.

The following organizations provided the rules for collecting data and adjusting for variations in patient mix when necessary:

• California Hospital Assessment and Reporting Taskforce (CHART)
• The Joint Commission
• The federal Medicare program
• California Office of Statewide Health Planning and Development (OSHPD)
• California Department of Public Health, Center for Family Health, Genetic Disease Screening Program, Newborn Screening Data
• California Children's Services
• California Nursing Outcomes Coalition
• National CAHPS (the national center for collecting patient surveys)
• The federal Agency for Healthcare Research and Quality (data collection and validation methods)

In general, the effect of the risk-adjustment process on hospital scores is small.

Applying a Margin of Error to Data

Another way to make sure that fair comparisons can be made between different hospitals is to apply a "margin of error." This helps compare hospitals that perform very few procedures with those that perform many. For example, a hospital that does only one heart surgery will have a death rate of either 0% (if the patient lives) or 100% (if the patient dies). Based on the outcome for one patient, it is difficult to determine how the hospital would perform for many surgeries on many patients. So we apply a wide margin of error that says the death rate is not very reliable because it is based on just one case.

For hospitals that have only a few patients with a particular condition, their margin for error in that condition is broad, while for hospitals with hundreds of patients the margin for error is narrow. (For this reason, for any single measure on this site, two hospitals with the same rating may have very different percentages shown.)

In rating the hospitals, the margin of error is taken into account by calculating an estimated range of hospital performance for each condition, with the range wider for hospitals with fewer patients and narrower for hospitals with many patients.

Performance Ratings

The performance rating icons indicate how well a hospital performed compared with other hospitals:

Depending on the type of measure, the scale will use all five (superior, above average, average, below average, and poor) or only three of these ratings (superior, average, and poor).

Case Volume Ratings

Research has shown that the more cases a hospital handles for a particular procedure, then the surgical team has more practice, which may lead to better results for the patients.

This Web site provides case volume ratings for some surgical procedures. Hospitals are rated as either having "enough cases" or "not enough cases." To determine the rating, the hospital reports the number of cases performed and it is compared to a specific "cut point" as determined by medical research.

For hospitals that meet the cut point in volume, then the Web site also shows the risk-adjusted death rate for the procedure.