Understanding Quality KPIs & Objectives

Before discussing “KPIs and Objectives” (Goals/Targets), we must first recognize and understand the hierarchical relationship between the Quality Policy, KPIs, and Quality Objectives.

  1. A “Quality Policy” establishes the intentions and direction of an organization (as formally expressed by its top management) related to quality.
  2. KPIs are “measures associated with goals or targets showing how well an organization is achieving its objectives”.
  3. A “Quality Objective” is a “result (goal/target) to be achieved” related to quality.

“Quality Objectives” are the specific “Goals” / “Targets” established within a KPI. For example, if “% of Rework” were a KPI, the objective might be “≤0.2% of items produced”.

The Quality Policy

The second requirement in ISO 9001 / AS9100, sec. 5.2.1 is to “establish, implement, and maintain a quality policy that provides a framework for setting quality objectives”.

5.2.1 Establishing the Quality Policy
Top management shall establish, implement, and maintain a quality policy that:
a. is appropriate to the purpose and context of the organization and supports its strategic direction;
b. provides a framework for setting quality objectives;
c. includes a commitment to satisfy applicable requirements;
d. includes a commitment to continual improvement of the quality management system.

This firmly establishes the “Quality Policy” at the top of the hierarchy pyramid.

KPIs (Key Performance Indicators)

While neither ISO 9001 nor AS9100 specifically mention KPIs (Key Performance Indicators), these must be innately identified in order to establish objectives. Clearly, you must know what you're going to measure before you can identify a goal/target.

However, AS9101 does define a KPI.

AS9101 - 3.2 Key Performance Indicator (KPI)
Measures associated with goals or targets showing how well an organization is achieving its objectives or critical success factors for a particular project. KPIs are used to objectively define a quantifiable and measurable indication of the organization's progress towards achieving its goals.

AS9101 goes further than ISO 9001 by requiring AS9100 Auditors to complete a “Process Effectiveness Assessment Report” (PEAR) during each assessment.

AS9101 - 4.2.2.5.1 Process Results
The audit team shall record measures, targets, and values of KPIs related to each audited operational process (see 9100-series standards clause 8) on the PEAR (see Form 3 Section 2), taking into account the confidentiality of information (see ISO/IEC 17021-1 clause 8.4 requirements).

A PEAR form is required to be completed for each “operational process” (e.g., Contracts, Programs, Engineering, Procurement, Production).

While the PEAR form includes fields for the Auditor to record data relating to procedures reviewed, records examined, people interviewed, etc., it also has a specific section for the auditor to record the KPIs for the process, the “Objectives” (e.g., “Target”) associated with each KPI, and the actual data (i.e., value) for the period audited (usually the previous 12 months).

The “Comments” field is for the auditor to either note that the “objective” was met OR what actions are being taken to meet it.

This is because AS9100 requires AS9100 Auditors to issue an NCR when “Quality Objectives” are not being met, and no action is being taken.

AS9101 - 4.2.2.5.1 Process Results (Cont.)
The audit team shall issue an NCR against the relevant 9100-series standard clause, when the process is not delivering the planned results and appropriate action is not being taken.

The “action” is not specifically required to be a “corrective action”. An acceptable “action” could be an “improvement” activity.

Quality Objectives (Goals/Targets)

The first requirement in sec. 6.2.1a is that “The quality objectives shall be consistent with the quality policy”.

6.2 Quality Objectives and Planning to Achieve Them
6.2.1 The organization shall establish quality objectives at relevant functions, levels, and processes needed for the quality management system.
The quality objectives shall:
a. be consistent with the quality policy;
b. be measurable;
c. take into account applicable requirements;
d. be relevant to conformity of products and services and to enhancement of customer satisfaction;
e. be monitored;
f. be communicated;
g. be updated, as appropriate. The organization shall maintain documented information on the quality objectives.

ISO 9000:2015 defines a “Quality Objective” as:

3.7.2 Quality Objective
objective (result to be achieved) related to quality.

A “Quality Objective” may be expressed as a “Goal” or “Target” (as these are considered synonyms). While management may establish many important goals/targets, ISO 9001 & AS9100 are ONLY concerned with “quality” objectives. All other objectives should be considered (by the auditors) to be “out of scope”.

To summarize, the organization (business) is required to identify measurable quality objectives in each process that are relevant to the conformity of products and services, and relate to the fulfillment of applicable requirements.

Establish “realistic” objectives. Once a KPI is identified, it should be measured and analyzed to assess the current process performance before establishing an objective to strive toward. These analyses often reveal inherent process limitations (e.g., capability, capacity) that must be escalated to management in order to be overcome.

Common KPIs and Quality Objectives

The below examples have been accepted by various AS9100 auditors. And the “Suggested Goals / Targets” are only provided as a starting point. As data is gathered, these goals/targets should be adjusted based upon (1) the observed capabilities of each process (Cpk) and (2) the value each metric provides. A key consideration in analyzing these KPIs should be “data collection ability” and “data integrity”. Beware the sirens call to mediocrity.

To put the suggested goals/targets in perspective: 99.80% is approx. 4.38 Sigma Short Term (ZST) – or 2,000 DPMO (Defects Per Million Opportunities). That is the average % products delivered “defect-free” from a “typical” machining “Job Shop” (producing high variation, low volume lots at inconsistent rates).

98.0% On-Time Delivery (OTD) is approx. 3.55 Sigma Short Term (ZST) – or 20,000 DPMO. This % is typical of many manufacturing facilities that rely upon raw material suppliers and freight carriers who function outside of their control.

Process KPI Suggested
Goal / Target
Contracts Proposals submitted prior to deadline (typically established by the customer) 100%
Contract / Customer Purchase Order revisions requested after acceptance (due to review errors) ≤0.25% of opportunities
Project Project Milestones achieved on-time 100%
Project Deliverables (e.g., CDRLs) delivered on-time 100%
Engineering Engineering Milestones achieved on-time 100%
% of ECOs (Engineering Change Order) due to “design” errors (e.g., poor manufacturability) ≤2% of ECOs submitted
% of ECOs (Engineering Change Order) submitted citing “drawing” errors ≤2% of ECOs submitted
First Article Inspection (FAI) errors (a reflection of design for manufacturability) ≤0.25% of opportunities
Software – Defects (bugs) released (usually ranked) Varies by Rank
Procurement Purchase Order Accuracy ≥99.8%
% of nonconforming products from suppliers (e.g., Dispositioned as Return To Vendor (RTV), scrap) ≤0.25% of line items
% of errors from suppliers (e.g., incorrect or missing documentation, shipped to the wrong address) ≤0.25% of line items
% On-Time Delivery from Suppliers see Note 1 ≥98%
Production / Manufacturing Production / Manufacturing Schedule met ≥99%
% First Pass Yield (FPY) ≥99.8%
Rolled Throughput Yield (RTY) ≥98%
% of Rework ≤0.2%
% of Repair ≤0.2%
% of Scrap ≤0.25%
% of Customer Returns (Warranty) ≤0.2% of opportunities
% of Customer Returns (non-warranty) ≤0.2% of opportunities
% Product Recalls (Notifications sent to customers) ≤0.1% of opportunities
% On-Time Delivery see Note 2 ≥98%

Note 1:
AS 9100, sec. 8.4.1.1
The organization shall:
c. periodically review external provider performance including process, product and service conformity, and on- time delivery performance;

[The above would include applicable SDRL (Subcontract Data Requirements List) items]

Note 2:
AS 9100, sec. 5.1.2 Customer Focus
Top management shall demonstrate leadership and commitment with respect to customer focus by ensuring that:
d. product and service conformity and on-time delivery performance are measured and appropriate action is taken if planned results are not, or will not be, achieved.

[These metrics are also required to be monitored and reviewed in AS 9100, sections: 9.1.2 & 9.3.2]