Change proposal

Change proposal

Order Description

As an Advanced nursing student, you are developing new competencies in leadership, and in order to achieve mastery, you must apply those competencies to live-practice experiences and situations. You are to identify a recently completed organizational change or innovation or one currently in process and use your leadership problem-solving skills to investigate, document, and evaluate the implementation process. For this paper, think about how you, as a nurse leader, function as a detective, scientist, and manager of a healing environment.

In this hypothetical or actual assessment scenario, you will work with a Change leader in a healthcare organization in order to select an organizational change (Pain Management) that has occurred within the last six months. If you choose a change currently in progress, it must be far enough into the implementation process in order for you to gather sufficient data for the assessment. The organizational change should have been implemented in order to initiate or support a patient advocacy project, system change, or educational program to increase quality care outcomes. The Change leader you choose to work with for the identified change (Pain Management) needs to have been directly involved with the change process in the organization.

After identifying change proposal (Pain Management) complete the attached “Change Investigation Proposal Form” you should identify how you will investigate the implemented organizational change in the form.

Investigate the selected organizational change (Pain Management) using the attached NCQA “QIA Form. You may utilize the attached “QIA Form Instructions” to help guide you in how to fill out the QIA Form. Your investigation will be used to document how the change was implemented in the organization.

Requirements:
A. Submit “Change Investigation Proposal Form” that includes the following information:
• A brief description of the organizational change you plan to investigate (Pain Management)
• A process for conducting the investigation (e.g., data collection, key stakeholders to talk to), including how you will obtain the information on the “QIA Form”

B. Summarize (suggested length ½ to 1 page) the identified organizational change and the patient population that it affects.

Note: Use the attached “QIA Form” and gather data from the organization that is necessary to populate sections I–V of the form. The form should serve as a guide to complete the assessment and conduct your investigation. Refer to the attached “QIA Instructions Form” for information on how to complete the QIA Form.
C. Submit a completed copy of the attached “QIA Form” in which you record data from your investigation in sections I–IV.
1. Summarize what data you collected for each section (I–IV) of the “QIA Form.”
a. Discuss what data collection measure(s) were used by the organization.
b. Analyze the appropriateness of the data collection measures, including whether the data supported the need for change.
2. Discuss how the data collection measure(s) could have been improved, using Advanced-level nursing and interprofessional standards.

D. Analyze the effectiveness of the change project in the organizational setting by doing the following:
1. Discuss how the change was evaluated for success after implementation.
a. Discuss the effects the implementation has had on the organization and quality care outcomes.
2. Evaluate whether stakeholders involved with implementation were successful in their roles.
3. Discuss how the change project could have been improved to increase quality care outcomes.

E. Summarize your involvement with the organization and/or stakeholders as you conducted your investigation.

The Following Files Attached:

1. QIA Form
2. QIA Form Instructions

The Following Criteria must be meet for this paper
A. n/a
B. Summary: Provide a logical summary, with sufficient detail, of the identified organizational change and the patient population that it affects.
C. QIA Form: Provide a completed copy of the attached “QIA Form” in which you record data from the investigation in sections I–IV.
C1. QIA Form Summary: Provide a logical summary, with sufficient detail, of what data you collected for each section (I–IV) of the “QIA Form.”
C1a. Collection Measures Used: Provide a logical discussion, with substantial detail, of what data collection measure(s) was used by the organization.
C1b. Appropriateness: Provide a plausible analysis, with substantial support, of the appropriateness of the data collection measure(s), including whether the data supported the need for change.
C2. Collection Measures Improvement: Provide a logical discussion, with substantial support, of how the data collection measure(s) could have been improved, using advance-level nursing and interprofessional standards
D1. Evaluation: Provide a logical discussion, with substantial detail, of how the change was evaluated for success after implementation.
D1a. Implementation Effects: Provide a logical discussion, with substantial detail, of the effects the implementation has had on the organization and quality care outcomes.
D2. Stakeholder Roles: Provide a logical evaluation, with substantial detail, of whether stakeholders involved with implementation were successful in their roles.
D3. Improvement: Provides a logical discussion, with substantial detail, of how the change project could have been improved to increase quality care outcomes.
E. Involvement Summary: Provide a logical summary, with sufficient detail, of your involvement with the organization and/or stakeholders as you conducted the investigation.

Helpful references: https://www.ihi.org/resources/pages/Howtoimprove/default.aspx

QUALITY IMPROVEMENT FORM
NCQA Quality Improvement Activity Form (an electronic version is available on NCQA’s Web site)
Activity Name:
Section I: Activity Selection and Methodology
A. Rationale. Use objective information (data) to explain your rationale for why this activity is important to members or practitioners and why there is an opportunity for improvement.

B. Quantifiable Measures. List and define all quantifiable measures used in this activity. Include a goal or benchmark for each measure. If a goal was established,
list it. If you list a benchmark, state the source. Add sections for additional quantifiable measures as needed.
Quantifiable Measure #1:
Numerator:
Denominator:
First measurement period dates:
Baseline Benchmark:
Source of benchmark:
Baseline goal:

Quantifiable Measure #2:
Numerator:
Denominator:
First measurement period dates:
Benchmark:
Source of benchmark:
Baseline goal:
Quantifiable Measure #3:
Numerator:
Denominator:
First measurement period dates:
Benchmark:
Source of benchmark:
Baseline goal:
C. Baseline Methodology.

C.1 Data Sources.
[ ] Medical/treatment records
[ ] Administrative data:
[ ] Claims/encounter data [ ] Complaints [ ] Appeals [ ] Telephone service data [ ] Appointment/access data
[ ] Hybrid (medical/treatment records and administrative)
[ ] Pharmacy data
[ ] Survey data (attach the survey tool and the complete survey protocol)
[ ] Other (list and describe):
_________________________________________________________________________________________________________________________________

_________________________________________________________________________________________________________________________________

C.2 Data Collection Methodology. Check all that apply and enter the measure number from Section B next to the appropriate methodology.
If medical/treatment records, check below:
[ ] Medical/treatment record abstraction
If survey, check all that apply:
[ ] Personal interview
[ ] Mail
[ ] Phone with CATI script
[ ] Phone with IVR
[ ] Internet
[ ] Incentive provided
[ ] Other (list and describe):
_______________________________________________
_______________________________________________ If administrative, check all that apply:
[ ] Programmed pull from claims/encounter files of all eligible members
[ ] Programmed pull from claims/encounter files of a sample of members
[ ] Complaint/appeal data by reason codes
[ ] Pharmacy data
[ ] Delegated entity data
[ ] Vendor file
[ ] Automated response time file from call center
[ ] Appointment/access data
[ ] Other (list and describe):
_________________________________________________________________
_________________________________________________________________

C.3 Sampling. If sampling was used, provide the following information.
Measure Sample Size Population Method for Determining Size (describe) Sampling Method (describe)

C.4 Data Collection Cycle.
Data Analysis Cycle.
[ ] Once a year
[ ] Twice a year
[ ] Once a season
[ ] Once a quarter
[ ] Once a month
[ ] Once a week
[ ] Once a day
[ ] Continuous
[ ] Other (list and describe):
_________________________________________________________
_________________________________________________________ [ ] Once a year
[ ] Once a season
[ ] Once a quarter
[ ] Once a month
[ ] Continuous
[ ] Other (list and describe):
_________________________________________________________
_________________________________________________________
C.5 Other Pertinent Methodological Features. Complete only if needed.

D. Changes to Baseline Methodology. Describe any changes in methodology from measurement to measurement.
Include, as appropriate:
• Measure and time period covered
• Type of change
• Rationale for change
• Changes in sampling methodology, including changes in sample size, method for determining size and sampling method
• Any introduction of bias that could affect the results
___________________________________________________________________________________________________________________________________
___________________________________________________________________________________________________________________________________
___________________________________________________________________________________________________________________________________
___________________________________________________________________________________________________________________________________

Section II: Data / Results Table
Complete for each quantifiable measure; add additional sections as needed.
#1 Quantifiable Measure:
Time Period
Measurement Covers
Measurement
Numerator
Denominator Rate or Results Comparison Benchmark Comparison
Goal Statistical Test and Significance*
Baseline:
Remeasurement 1:
Remeasurement 2:
Remeasurement 3:
Remeasurement 4:
Remeasurement 5:
#2 Quantifiable Measure:
Time Period
Measurement Covers
Measurement
Numerator
Denominator Rate or Results Comparison Benchmark Comparison
Goal Statistical Test and Significance*
Baseline:
Remeasurement 1:
Remeasurement 2:
Remeasurement 3:
Remeasurement 4:
Remeasurement 5:
#3 Quantifiable Measure:
Time Period
Measurement Covers
Measurement
Numerator
Denominator Rate or Results Comparison Benchmark Comparison
Goal Statistical Test and Significance*
Baseline:
Remeasurement 1:
Remeasurement 2:
Remeasurement 3:
Remeasurement 4:
Remeasurement 5:
* If used, specify the test, p value, and specific measurements (e.g., baseline to remeasurement #1, remeasurement #1 to remeasurement #2, etc., or baseline to final remeasurement) included in the calculations. NCQA does not require statistical testing.
Section III: Analysis Cycle
Complete this section for EACH analysis cycle presented.
A. Time Period and Measures That Analysis Covers.

B. Analysis and Identification of Opportunities for Improvement. Describe the analysis and include the points listed below.
B.1 For the quantitative analysis, include the analysis of the following:
• Comparison with the goal/benchmark
• Reasons for changes to goals
• If benchmarks changed since baseline, list source and date of changes
• Comparison with previous measurements
• Trends, increases or decreases in performance or changes in statistical significance (if used)
• Impact of any methodological changes that could impact the results
• For a survey, include the overall response rate and the implications of the survey response rate

B.2 For the qualitative analysis, describe any analysis that identifies causes for less than desired performance (barrier/causal analysis) and include the following:
• Techniques and data (if used) in the analysis
• Expertise (e.g., titles; knowledge of subject matter) of the work group or committees conducting the analysis
• Citations from literature identifying barriers (if any)
• Barriers/opportunities identified through the analysis
• Impact of interventions

Section IV: Interventions Table
Interventions Taken for Improvement as a Result of Analysis. List chronologically the interventions that have had the most impact on improving the measure. Describe only the interventions and provide quantitative details whenever possible (e.g., “hired 4 UM nurses” as opposed to “hired UM nurses”). Do not include intervention planning activities.
Date Implemented (MM / YY) Check if
Ongoing

Interventions

Barriers That Interventions Address

Section V: Chart or Graph (Optional)

Attach a chart or graph for any activity having more than two measurement periods that shows the relationship between the timing of the intervention (cause) and the result of the remeasurements (effect). Present one graph for each measure unless the measures are closely correlated, such as average speed of answer and call abandonment rate. Control charts are not required, but are helpful in demonstrating the stability of the measure over time or after the implementation.

Quality Improvement Activity (QIA) Form Instructions
Overview
When to Use the QIA Form
This document is a guide for completing NCQA’s Quality Improvement Activity (QIA) form. This form can be used for the QIA required NCQA accreditation and certification programs, as applicable. It must be used to meet the Quality Improvement Projects required for Medicare Advantage Deeming.
For NCQA purposes, you are not required to use the QIA form; however, you must provide the data it requests in order for NCQA to review your QIAs completely and accurately. Submit a QIA for each activity you present by attaching it to the applicable element in the Survey Tool using the Attach Document feature in the Survey Tool.
Detailed instructions on attaching documents to the Survey Tool are found in the Survey Tool Instructions under Help on the Main Menu bar.
The purpose of the QIA form is to summarize the clinical and service quality activities that you are using to demonstrate meaningful improvement in the applicable element.
You should not complete the QIA forms for service or clinical activities that you use to demonstrate compliance with other standards that require data collection and analysis such as member/enrollee satisfaction, availability and access and satisfaction with UM. Document compliance with these standards as you would document any other standard.
All data points must be final when your organization submits the Survey Tool.
NCQA does not recommend using this form to report on activities that have only one data point (e.g., baseline only).
Consult the appropriate Explanation for the meaningful improvement standard for the accreditation or certification program for which you apply.
Remember that you cannot achieve a score of 100% with only one data point.
The activity will not be considered.
Achieving Meaningful Improvement
Submit enough data To receive “credit” for meaningful improvement, you must submit enough data to allow an evaluation of any seasonal variations that could affect the results. On the service side, open-enrollment seasons can affect such activities as ensuring access to primary care and reduction in referral time frames. In most cases you must present:
• annual measurement occurring during the same season (e.g., comparing the first quarter of one year to the first quarter of the following years) for areas that show seasonal differences, such as provision of enrollment cards
• five quarters of data
• fifteen months of data.
Note: If you do not have adequate data to satisfy the above conditions or if you believe that the results are not biased by seasonal issues, provide an explanation as it relates to QI 12 and QI 13 under Other Pertinent Methodology Features, in Section I.
The improvement must meet the time period covered in the survey To receive “credit” for meaningful improvement, the improvement must have occurred in the three-year period covered in the survey. For example, if you have annual data on member satisfaction since 1996, but the date of the survey for which this QIA is being prepared is January 2008, only data beginning in 2005 should be shown.
In other words, the improvement must have started at some point during the three years immediately prior to the survey and have been subsequently sustained.
For Renewal Surveys, you may need to present measurements for the year prior to the current survey period if these data were not available for your previous survey.
The QIA Form
The form’s five sections The QIA form is divided into five sections:
• Section I Activity Selection and Methodology
• Section II Data/Results Table
• Section III Analysis Cycle
• Section IV Interventions Table
• Section V Chart or Graph
Activity name and activity examples The form first asks you to supply an activity name. The activity name should succinctly encompass the purpose of the activity and begin with an action word that accurately states what the activity is designed to do (e.g., “improving,” “increasing,” “decreasing,” “monitoring”). Examples are listed below.
• decreasing the risk of congestive heart failure
• improving claims turn-around time to practitioners
• increasing the rate of diabetic foot exams
• improving access to behavioral health services
• decreasing practitioner complaints with the referral process.

Section I:
Activity Selection and Methodology
This section asks you to provide the rationale for choosing this QI activity for your organization. Explain why the clinical or service activity affects your members or practitioners.
NCQA requires you to choose service improvements based on their impact on members. NCQA also accepts improvements in practitioner satisfaction that relate to utilization management (UM) processes or effects (e.g., issues identified in UM 11) for one service QIA.
Examples are listed below:
• improvements in turnaround time for prior-authorization requests decrease the time that members wait to receive care requiring authorization and/or increase productivity for practitioners
• improvements in UM decision making turn-around-time ensure more satisfied members and/or practitioners
• improvements in referral to specialist turnaround time reduce the number of complaints and appeals regarding referrals.
Rationale
Define the rationale for selecting the activity This section asks you to define your rationale for selecting this activity for improvement.
• Why was it chosen over others?
• Why is it important to your members or practitioners?
• Why is it worth the resources your organization is spending on it?
Using objective information provide as much information that is specific to your organization as possible.
You do not have to provide generic defenses for most clinical or service issues. For example, do not include explanatory phrases such as “member services departments serve many important functions”, or “neuropathy of the foot is a serious condition that affects thousands of diabetics nationwide.”
Nor is it necessary to provide literature source cites on the importance of a clinical or service issue to members unless it is an unusual topic. Focus on the importance of the activity to your organization.
Importance of activity Include pertinent organization data or community demographic data that reflect the importance of the activity to your organization’s membership. Describe the magnitude of the issue related to the activity in quantifiable terms.

Activity examples Examples are listed below.
• Between 2004 and 2005, hospitalization due to diabetic foot neuropathy rose 9 percent. This was the largest increase in any diabetes related hospitalization. Research has shown that periodic foot screening of diabetics and self screening by diabetics can decrease rates of foot neuropathy.
• Practitioner dissatisfaction turnaround time with UM decisions increased from 5 to 15 percent between 2004 and 2005. This was the largest increase in practitioner dissatisfaction the organization has received for four years. In addition, this 15 percent dissatisfaction rate was the highest dissatisfaction rate on the practitioner survey.
Quantifiable Measures
Quantifiable measures clearly and accurately measure the activity This section asks you to list all quantifiable measures you use in this activity, including those added over time. Quantifiable measures should clearly and accurately measure the activity being evaluated. List your baseline benchmarks and goals and if you modify them over time, list the updated benchmark or goal in the table in Section II.
Multiple measures You may use one or more measures for each activity. For some activities, multiple measures are useful. For example, practitioner complaints and actual turn-around-time for UM decisions would be two measures that are closely linked to the timeliness of UM decisions.
In other cases, multiple measures may not be useful. For example, you may display multiple measures associated with a CHF disease management (DM) program, only one of which shows improvement. Unless the intervention is clearly focused to address that measure, NCQA may not consider the improvement meaningful.
Denominator Describe here the event being assessed or the members who are eligible for the service or care. Indicate whether all events or eligible members are included, or whether the denominator is a sample. Examples of responses are listed below:
• all physician complaints
• members 35 years of age and older during the measurement year who were hospitalized and discharged alive from January 1–December 24 of the measurement year with a diagnosis of congestive heart failure
• all survey respondents

Numerator Describe here the criteria being assessed for the service or care:
• all physician complaints concerning UM decision turn-around-time
• members meeting the criteria for inclusion in the denominator who received an ambulatory prescription for ace inhibitors within 90 days of discharge
• survey respondents who do or do not like the event in the denominator
First measurement period State here the time period covered by the initial assessment.
For clinical issues, this is typically an entire calendar year (e.g., January 1, 2008–December 31, 2008).
For service issues, the measurement period is often monthly or quarterly (e.g., January 2008 or 1Q 2008). Measurement periods may vary by measure. For example, the first measurement period for UM decision timeliness may be the first quarter of 2008, but the measure addressing timeliness may not have started until the third quarter of 2008.
Baseline benchmark Include here information on how the benchmark was derived as well as the benchmark rate. NCQA defines “benchmark” as the industry measure of best performance against which the organization’s performance is compared. It should be directly comparable to your QI measure.
You may describe the benchmark in numerical terms (e.g., the 90th percentile), or in terms of the comparison group (e.g., the best published rate in our state, 85 percent).
The benchmark may be a best practice in an industry based on published data or the best performance within a corporation with multiple organizations. NCQA requires a benchmark or a goal, but not both. Many service activities do not have benchmarks. If you are not using a benchmark, insert “NA” in response to this query.
Remember: Benchmarks are not averages; they are the best in class.
The average for a national organization or corporation with multiple organizations is not a benchmark.
The organization’s best rate would be considered a benchmark.

Benchmark source If you give a benchmark, list the organization or publication from which it was obtained and the time period to which it pertains.
Baseline goal The performance goal is the desired level of achievement for the measure within a reasonable time. It does not have to be based on actual best practices, but it should reflect the level of achievement your organization has targeted.
The goal should be quantitative and stated in numerical terms (e.g., 90 percent, 0.3 appeals per thousand, 3 days).
Most organizations do not set performance goals until after they have collected baseline results. If that is the case, enter NA here.
Words such as “improve,” “decrease” or “increase” are not acceptable in stating goals unless they are accompanied by a numerical quantifier (e.g., “improve one standard deviation from baseline” or “decrease by 5 percentage points from the last remeasure”).
Remember to use the words “percent” and “percentage” precisely.
An increase in practitioner satisfaction with the UM referral system from 35 percent to
40 percent is a 5 percentage point increase, not a 5 percent increase.

State the first goal you set (which, generally, is set after baseline results have been analyzed). NCQA expects that as you achieve your goals, you set new ones. Section II has a space to list updated goals. Examples are listed below.
Goal example Measure: Pre-service UM decisions.
Numerator: Number of preservice decisions less than 4 days.
Denominator: Number of preservice decisions.
Benchmark: NA
Baseline Goal: 80 percent of preservice decisions are made within 3 days of the request.
Note: NCQA does not consider achievement of a prespecified goal or benchmark alone as a demonstration of meaningful improvement.
Baseline Methodology
This section uses tables, check boxes and narrative to enable you to describe your methodology. The more precisely you describe the data you used and how they were obtained; the sampling procedures, if any, that were applied; and any special factors that could have influenced the results, the more easily NCQA can assess the validity and reliability of the findings.

C.1 Data sources
Check all the data sources used. If you used other sources that are not listed, check “Other” and describe the sources completely. Indicate the number of the measure from Section B next to the data source used.
C.2 Data collection methodology This section is divided into:
• medical/treatment record
• survey
• administrative.
Because you may use different data collection methodologies for different measures, check all that apply. Indicate the number of the measure from Section B next to the data source used. If you collected survey data using more than one of these techniques, check all that apply. If you used different techniques, or if you used other methods to collect administrative data, mark “Other” and describe your data sources completely. You are not limited to the options provided.
Most of these methodologies are self-explanatory. The definitions for the survey data collection methodology are listed below.

Definitions
Personal interview A face-to-face interview.
Mail A survey mailed to and returned from the respondent and involving no personal contact.
Phone with CATI script A telephone interview using a computer-assisted script containing prompts beyond the actual questions that can be used according to a set protocol.
Phone with IVR A telephone interview involving an interactive voice recognition system rather than a live person.
Internet A survey conducted using the Internet and involving no personal interaction.
Incentive provided A survey in which the respondent was given an incentive (e.g., gift certificate, cash) for participating.
Note: Regardless of the survey methodology, mark this box if the respondent is given any incentive to complete the survey.
Other Any other survey methodology different from those listed above.
C.3 Sampling For each measure that involved sampling, state the sample size, the method used to determine the size and the sampling methodology. If the size is the same for all measures, state “All Measures” and give the information only once. Also provide the size of the full population from which you drew the sample.
Remember that the sampling methodology here relates to your baseline measurement only.
Any change to this sampling methodology is reported in Section I.D of this form.

Table elements Measure. You may use the measure number from the measures listed in Section I.B and abbreviate the name.
Sample size. State the number of the full sample selected, including any oversampling. The denominator listed in Section II provides the number included in the measure.
Determining the sample size. To determine the size, explain the parameters used to determine the sample size, which typically include:
• the assumptions or requirements of the statistical test to be used to verify the significance of observed differences
• the desired degree of confidence in the statistical test (alpha level)
• statistical power (the sensitivity of the statistical test to detect differences; bigger samples yield greater power)
• the margin of error to be allowed when assessing the hypothesis
• the oversample rate
– the oversample is the extra cases included in the sample to replace cases rejected because of contraindications, ineligibility, etc. (In survey measurement, the oversample should be large enough to replace expected nonresponses.) Examples of oversampling are shown below.
Oversampling example You plan to improve the time required for members to obtain a referral. You conduct telephone surveys of different groups of members who obtained referrals at two points in time, asking them how many days it took for them to get the referral. You have these expectations about the survey:
• the distribution of responses about the “number of days to referral” is normally distributed for both the pre- and post-survey groups
• the t-test is used to test the significance of the pre- and post-differences at alpha = 0.05 and 80 percent power
• a pilot survey showed that the standard deviation of “number of days to referral” responses is 5.25
• the program reduces the average number of days from 8.5 days to 7 days
• the response rate is 85 percent.
Sample size calculations based on the above parameters indicate that you require a sample of 193 completed surveys. You expect that 15 percent of the sampled members will not respond, so you sample 227 members to account for the nonresponse (X *0.85 = 193; X = 193/85; X = 227). This calculation includes 193 members in the original sample plus an oversample of 34 patients to replace those who do not respond.
Sampling method State the sampling methodology (simple random sample, stratified random sample, convenience sample). State the reasons for exclusions from the sample, if there were any (e.g., “Simple random sampling was used. During the claims pull, three claims were excluded because they were miscoded.”).
Remember that if your sampling methodology involves a survey, it is not necessary to complete
this table because you have included the Survey Tool and the survey protocol
(requested in Section I.C.2).

C.4 Data collection cycle and data analysis cycle Check the box that applies or describe the frequency of data collection and analysis. Indicate the number of the measure from Section B next to the data source used. For many service activities, the data collection cycle is more frequent than the analysis cycle.
For example, hospitalization data may be collected weekly, but analyzed monthly or quarterly. Survey data may be collected quarterly and analyzed at six-month intervals.
C.5 Other pertinent methodology features Describe any other methodological decisions or issues that could affect the analysis of the data or influence the results, such as:
• coding definitions
• claims-processing specifications unique to your organization
• claims-processing delays
• unique survey response coding or benefit design (e.g., pharmacy benefits).
If your QIA does not include sufficient data as specified by NCQA policy, or if you believe the results are not biased by seasonal issues because of the definition of the measure, provide your rationale for considering this for
QI 12 and QI 13.
Mark this section “NA” if there are no other methodological features that need to be brought to NCQA’s attention. You are not required to complete this section past this point.
Changes to Baseline Methodology
This section asks you to describe any methodology changes that were made after the baseline measurement was taken. To compare results accurately, it is best to use the same methodology over time. However, you may need to change methodology in order to strengthen the validity and reliability of the outcome, correct inadequacies in the initial process, or accommodate for lack of resources. Specifying changes that were made is important because those changes influence analysis of the results.
For each affected measure, you must describe:
• the dates during which the changed methodology was used
• how the methodology was changed
• the rationale for the change
• the anticipated impact of the change on the analysis.
If you changed the sampling methodology in the same way for several measures you need to provide the information only once. If the sampling methodology is the same, but the sample size has changed, show only those changes.

Section II:
Data/Results Table
This section consists of a table of the results of the baseline measurement and all of the remeasurements that you are presenting for consideration for the QIA. You may substitute a table of your choice as long as it includes all of the required elements. If there are more than five remeasurement periods, add a row for each additional measure. If you measured a service issue more frequently than quarterly, combine the data by recalculating the numerator and denominator and enter the quarterly result in the table.
Table Description
Quantifiable measure You may use the measure number from the list of measures completed in Section I and abbreviate the name.
Time period covered State the time period the measurement covers. It could be quarterly (e.g., 1Q 2008), twice a year (e.g., January–June and July–December 2008), yearly (e.g., 2008), or every other year (e.g., January–December 2006 and January–December 2008).
Numerator/
denominator List the numerator and denominator for each remeasurement period.
If the measure uses survey methodology, state the number of people who met the numerator criteria (numerator) and the number of people who responded to the question (denominator).
Rate or results Convert the fraction (numerator/denominator) to a percentage.
Comparison benchmark/ comparison goal List the goal and/or benchmark period in effect during the remeasurement cycle. The comparison goal is blank for the baseline measurement unless you have established a goal prior to pulling the baseline data. A goal based on baseline data that is in effect for the first remeasurement cycle should appear in the comparison box on remeasurement line 1. If you met your goal but there is still opportunity for improvement, NCQA suggests you increase your goal.
If you changed your goal for any other reason, explain the basis for doing so in Section III: Analysis Cycle. You may also add benchmarks that you did not have at the baseline period.
Statistical test and significance NCQA does not require you to test for statistical significance. Consult the appropriate Standards and Guidelines for the accreditation or certification program for which you are applying for additional information on the requirements for achieving meaningful improvement.
If you have performed such tests and choose to report them, however, state the time periods that you compared and the type of statistical test used for each measure. The table has been left open-ended to allow you to compare any time period you choose. Most organizations compare the latest remeasurement to the previous one and the latest remeasurement to the baseline measurement.

Statistical testing is generally not necessary when measures are based on the entire eligible population, and may not be appropriate if the denominator is not based on a random or probability sample or if the measure specifications substantially changed since the last remeasurement period.
For the most common test (comparing two independent rates), the chi-square test of proportions or the z-test of proportions can be used (e.g., a z-test to compare the baseline to remeasure #1, p value = 0.2992; and baseline to remeasure #5, p value = 0.001).
These tests are not appropriate when the same members are being measured at different time periods, in which case the McNemar test for correlated proportions might be appropriate.
If you measure nonrate data, such as average wait times, the t-test or z-test for comparing means would be appropriate, depending on the size of the sample. If you have several independent remeasurements, based on samples, you may want to do an ANOVA test of linear trend to show that the rate is increasing over time.
Section III:
Analysis Cycle
In this section, you are asked to present the results of the quantitative and qualitative analyses you used to interpret the meaning of the results and to identify the opportunities for improvement that you wish to pursue. These analyses involve interpreting the data, which may include collecting additional data; identifying barriers or causes for less-than-desired performance; and designing strategies to overcome the barriers. Implementation of interventions is covered in Section IV.
Time Period and Measures Covered by the Analysis
Focus of the analysis The analysis may occur after every remeasurement or after grouping several remeasurement periods. Your analysis may focus on one measure, on all measures or on a combination of measures.
For example, an activity designed to improve Preserivce UM decision turn-around time may include three measures:
• time from request to decision
• time from request is notification
• perceived turn-around-time by member
You may collect these data quarterly but analyze the data only twice a year. The first analysis period might include only the first and second measure and the second might include all three measures.
On the clinical side, an example for improving asthma management could include:
• measures of ER visits
• inpatient admissions per thousand
• quality-of-life measures from a member survey.
For example, if you measured ER visits and inpatient admissions monthly and conducted the quality-of-life survey annually, you could analyze the first two measures quarterly and the quality-of-life measure annually.
If you have multiple analysis periods, it is helpful to label them clearly. For example:
• Analysis I: Calendar year
• Analysis II: Calendar year
• Analysis III: January–December 2005.

Identifying and Analyzing Opportunities for Improvement
In this section, you are asked to address the points specified, as appropriate, for the activity for each analysis cycle.
B.1 Quantitative analysis Compare to the goal/benchmark. Have you met your goals and or achieved the benchmark?
Why did the goals change? If you changed your goal, explain why. If you met your goal but there is still opportunity for improvement, NCQA expects you to increase your goal. If you change your goal for any other reason, explain the basis for doing so. Avoid adjusting goals without a sound rationale for doing so.
Has the benchmark changed? If you changed your benchmark, indicate the source of the new benchmark and the date it was adopted.
Compare to previous measurements. Have the results increased or decreased since the previous remeasurement? If so, does this change represent an improvement, or deterioration?
Trends and statistical significance. Describe any trends you identified and their significance. What weight do you place on the presence or absence of statistical significance?
Impact of any methodological changes. Discuss the impact of the methodological changes on the actual results. Could the results be biased, positively or negatively, by the changes in methodology? Explain why or
why not.
Overall survey response rate and implications. If any measures in the analysis are based on survey data, give the survey response rate for the entire survey.
Describe the impact that this response rate could have on the reliability of the findings. Variability in response rates in remeasurement periods should also be addressed (e.g., a 20 percent or less response rate is generally considered too low to draw reliable population-based conclusions).
B.2 Qualitative analysis Techniques and data used. Many techniques exist for determining the barriers or root causes for the results. You may have to collect additional data, stratify the data, or analyze subgroup data in order to drill down sufficiently to understand the reasons for the results. Include both how you performed the barrier analysis and any additional data collected used for barrier analysis.
Brainstorming, multivoting, pareto analysis and fishbone diagramming are common continuous quality improvement techniques used to identify barriers to improvement. In addition to stratifying the data you already have collected to calculate the measure, you may have to analyze the results of other data, such as targeted survey results, complementary data (e.g., complaints in relation to satisfaction survey rates), and results of focus groups.

Expertise of group performing analysis. List the group or committee that was involved in the analysis and state why it was qualified to perform this analysis by describing the composition of the group and its expertise in evaluating this activity. If statistical or survey research analysis is required, describe the qualifications of those involved.
For service issues, such as UM turn-around on decision, the analysis may be performed by departmental managers and staff. Clinical issues may require expertise in the clinical subject matter as well as an understanding of the delivery system, benefit structure and other distinctive aspects of the organization.
NCQA recognizes that many service issues are addressed during the normal course of business and that there may not be a formal a committee structure to address these issues as there is with clinical issues.
Citations from literature. For many clinical and service quality improvement activities, there are sources that contain information about barriers to performance that have already been identified and are generally accepted. You may use these sources to supplement, or substitute for, your own barrier analysis. Give the complete citation (i.e., name of article and journal and date of publication) for each source you have used.
Barriers/opportunities identified. List the barriers to or causes for the less than acceptable performance that you identified, if any. Although NCQA recognizes that inadequate data collection may contribute to low performance, it does not accept improvements in data collection alone as an opportunity to improve.
Barriers and opportunities for improvement must focus on variables (e.g., improving processes, changing benefits, and educating members, practitioners or both) that can result in improved performance.
The following are examples of categories that may create barriers:
• member knowledge
• practitioner knowledge
• benefit coverage
• co-pay restrictions
• organization staffing
• problems with PCP or specialist access
• referral access
• systems issues in the organization.
List opportunities for improvement that you identified from the barriers. For example, you may identify the lack of family involvement in therapy as a barrier to improving depression management for children and adolescents. Next, you may identify as opportunities for improvement the lack of knowledge by the practitioner of the importance of family involvement, the family’s unwillingness to participate in therapy, and the child’s resistance to parental involvement. You must then choose which of these opportunities to focus on and develop one or more interventions.

Although you list the interventions in relation to the barriers you identified in Section IV, you should justify here the causal link between your interventions and the results you observed. Explain how your interventions influenced the outcome; identify the interventions that were most influential and explain why; and describe any intervening or confounding factors that may have contributed to the changes.
Some barriers do not lead to opportunities because of benefit restrictions, state law or other problems outside the control of the organization.
Remember that opportunities are not the same as barriers or interventions.
Barrier example 1 Barrier: Inadequate coverage of phones during lunch and breaks
Opportunity: Improve lunchtime and break coverage
Intervention: Revised staff scheduling to provide better coverage using existing staff
Barrier example 2 Barrier: Insufficient psychiatrist availability in a region
Opportunity: Increase psychiatrist access by contracting with more psychiatrists
Intervention: Recruited six new psychiatrists to meet availability needs

Section IV:
Interventions Table
In this section, you are asked to list the interventions taken to overcome barriers you identified in the previous section.
Note: You are not required to pursue interventions for all identified barriers.
Table Description
Date implemented List the month and year during which the intervention was implemented.
Check if ongoing Some interventions occur on a regular, ongoing basis. Often the effectiveness of the intervention rests on its repetitive nature.
Check the column if the intervention occurs at some periodic interval, then state its frequency (e.g., monthly, quarterly, annually). Examples are:
• quarterly training for UM staff,
• annual mailings on the importance of colon cancer screening, and
• monthly review of quality reports of timeliness of approving referrals are examples of ongoing interventions.
Intervention List the interventions chronologically. Generally, you implement interventions after the data are analyzed. If you began interventions prior to analyzing the baseline measure or prior to this survey period and you believe they have an impact on the performance measures during this survey period, list them first. Interventions may be listed under categories, such as member, practitioner, collaborative, and systems, if doing so is useful to you.
Provide a detailed, quantitative definition of the intervention whenever possible. For example, “hired four UM nurses” is more specific than “increased UM staffing.” “Mailed lists of 455 noncompliant members to 54 pediatricians and 31 family practitioners” better describes the magnitude of the intervention than “mailed lists of noncompliant members to practitioners.” You may abbreviate the full name of the intervention after using it for the first time.
Do not include activities that have been planned but not yet implemented (e.g., developing policies, conducting committee meetings or organizing activities).
Remember that you may include interventions taken after the last remeasurement period shown on this form, but they are not used by NCQA to determine meaningful improvement.
This list also summarizes your interventions. NCQA surveyors review additional back-up material
to document the extent of the intervention and its implementation.

Barriers that interventions address List all the barriers that each intervention is designed to address, which you should have previously described in Section III. You may abbreviate the name of the barrier. It may be helpful to number the barriers and use the numbers in subsequent references to them.
Do not include barriers related to data collection. An example of a completed Section IV interventions table appears below:

Activity Name: Improving Preserivce UM decision turn-around time

Section IV: Interventions Table
Interventions Taken for Improvement as a Result of Analysis. List chronologically the interventions that have had the most impact on improving the measure. Describe only the interventions and provide quantitative details whenever possible (e.g., “hired 4 customer service reps” as opposed to “hired customer service reps”). Do not include the intervention planning activities.
Date Implemented
(MM / YY)
Check if Ongoing

Interventions Barriers That Interventions Address
03/05 Hired 3 UM nurses Inadequate UM staffing
09/05 X Instituted weekly lunchtime training sessions conducted by staff of claims, marketing, etc., departments to update UM staff about policies and discuss more efficient decision making processes UM staff not following timeliness protocols consistently
12/06 Distributed to all practitioners an updated practitioner handbook that included a description of how the UM decision making process and the time frames Inadequate practitioner knowledge about role of customer service department
4/06 X Revised session on UM procedures and processes and delivered as part of all new practitioner orientations Inadequate practitioner knowledge of UM process

Section V:
Chart or Graph (Optional)
This section supplements the information you have provided up to now by more fully clarifying the relationship between the results of the remeasurements and the timing of the interventions.
A chart or a graph that plots both the results and the dates you implemented changes designed to improve your results often provides a visual presentation that is helpful in addition to the narrative or tables.
NCQA recommends attaching this “picture” if the activity has more than two measurement periods in order to show the relationship between the timing of the interventions (the cause) and the result of the remeasurements (the effect). Present one chart or graph for each measure unless the measures are closely correlated, which may be displayed in one graphic.
Use whatever type of chart (line, bar, mixed) that clearly presents both your interventions and your performance measures. A simple line graph might be appropriate for service activities with multiple data points, while a bar chart might be more appropriate to show changes in measures with annual measurement points. Interventions are placed on the graph or bar and show the dates of implementation. You may number the interventions and provide a key to the numbering, or you may number the interventions in Section III and use those numbers on the graph or bar. NCQA encourages you to limit the interventions you use to those you have identified as being the strongest.

NCQA does not require control charts that display upper and lower confidence limits, but you may include them if you believe they are helpful in demonstrating the stability of the measure over time.

Back-Up Information
NCQA wants to review documentation that supports the information you have summarized on your QIA. In addition to the completed QIA form, NCQA may need additional documentation. Your designated ASC will let you know if this applies.
• Such information often encompasses:
– all material related to methodology, including data collection tools (e.g., medical record abstraction sheets, codes for administrative data, inter-rater reliability testing, computer algorithms)
– copies of literature cited, as appropriate
– excerpts of minutes or other documentation that show how and when analysis was performed
– tools and supplemental data used in barrier analysis
– evidence and dates of actions taken:
• Copies of mailings
• Newsletters
• Responses from practitioners or members
• Revised policies and procedures
• Excerpts from updated member or practitioner handbooks
• Revised contracts