AS9100D Objective Evidence Requirements: Part 2 Records

Go to Part 1 - Required Documentation

As mentioned in Part 1, AS 9100D is an incredibly poorly written QMS standard littered with ambiguous/vague requirements that have resulted in subjective interpretations and inconsistent implementation. This article discusses the minimum “Objective Evidence” actually required by the standard. Any interpretations beyond these minimum requirements (e.g., by an internal or external auditor) are based purely on subjective opinion.

The second confusing point to address is which “records” are actually required by AS 9100D. First, let's look at how ISO 9001:2015 defines a “record”.

AS 9100:2016 (Rev. D) Annex A states:

A.6 Documented Information

Where ISO 9001:2008 used the term “records” to denote documents needed to provide evidence of conformity to requirements, this is now expressed as a requirement to retain documented information.

A row number below in Bold indicates either “Tips” or “Interpretations” are associated with the requirement(s).

# “Retained” as Documented Information (Records) Sec.
1 To the extent necessary, the organization shall:
b. retain documented information to have confidence that the processes are being carried out as planned.
[While this requirement is SUBJECTIVE, it is clarified in:
7.5.1 General
The organization’s quality management system shall include:
• documented information required by this International Standard;
• documented information DETERMINED BY THE ORGANIZATION as being necessary for the effectiveness of the quality management system.

AND Annex A.6
“THE ORGANIZATION is responsible for determining what documented information needs to be retained, the period of time for which it is to be retained, and the media to be used for its retention.“

NOTE: Some ACCREDITATION BODY AUDITORS from ANAB have incorrectly interpreted 4.4.2b to mean that EVERYWHERE the standard uses the word “PLAN”, there must be an associated record. This would significantly expand the number of records required. However, the AS9100 IDR has stated that this interpretation was incorrect.]
4.4.2b
2 The organization shall retain appropriate documented information as evidence of fitness for purpose of the monitoring and measurement resources.
[SUBJECTIVE requirement - because AS 9100D fails to adequately define what is considered to be appropriate .] 		7.1.5.1
3 When measurement traceability is a requirement, or is considered by the organization to be an essential part of providing confidence in the validity of measurement results, measuring equipment shall be:
a. calibrated or verified, or both, at specified intervals, or prior to use, against measurement standards traceable to international or national measurement standards; when no such standards exist, the basis used for calibration or verification shall be retained as documented information; 		7.1.5.2a
4 The organization shall:
d. retain appropriate documented information as evidence of competence. 		7.2d
5 The organization shall plan, implement, and control the processes (see 4.4) needed to meet the requirements for the provision of products and services, and to implement the actions determined in clause 6, by:
e. determining, maintaining, and retaining documented information to the extent necessary:
1. to have confidence that the processes have been carried out as planned;
2. to demonstrate the conformity of products and services to their requirements;
[SUBJECTIVE requirement - to be determined by the ORGANIZATION (as per 7.5.1b)… NOT the auditor.] 		8.1.e
6 8.2.3 Review of the Requirements for Products and Services
8.2.3.2 The organization shall retain documented information, as applicable:
a. on the results of the review;
b. on any new requirements for the products and services. 		8.2.3.2
7 The organization shall retain documented information on design and development inputs. 8.3.3
8 8.3.4 Design and Development Controls
The organization shall apply controls to the design and development process to ensure that:
a. the results to be achieved are defined;
b. reviews are conducted to evaluate the ability of the results of design and development to meet requirements;
c. verification activities are conducted to ensure that the design and development outputs meet the input requirements;
d. validation activities are conducted to ensure that the resulting products and services meet the requirements for the specified application or intended use;
e. any necessary actions are taken on problems determined during the reviews, or verification and validation activities;
f. documented information of these activities is retained;
g. progression to the next stage is authorized. 		8.3.4
9 8.3.4.1 When tests are necessary for verification and validation, these tests shall be planned, controlled, reviewed, and documented to ensure and prove the following:
a. test plans or specifications identify the test item being tested and the resources being used, define test objectives and conditions, parameters to be recorded and relevant acceptance criteria;
b. test procedures describe the test methods to be used, how to perform the test, and how to record the results;
c. the correct configuration of the test item is submitted for the test;
d. the requirements of the test plan and the test procedures are observed;
e. the acceptance criteria are met.
[While this requirement does not include the words “maintained” or “retained”, it is generally interpreted as applying in part to “test plans” (or specifications), “test procedures”, and to the “test results” (i.e., record)]		 8.3.4.1
10 The organization shall retain documented information on design and development outputs. 8.3.5
11 8.3.6 Design and Development Changes
The organization shall retain documented information on:
a. design and development changes;
b. the results of reviews;
c. the authorization of the changes;
d. the actions taken to prevent adverse impacts. 		8.3.6
12 The organization shall determine and apply criteria for the evaluation, selection, monitoring of performance, and re- evaluation of external providers, based on their ability to provide processes or products and services in accordance with requirements. The organization shall retain documented information of these activities and any necessary actions arising from the evaluations. 8.4.1
13 8.4.1.1 The organization shall:
b. maintain a register of its external providers that includes approval status (e.g., approved, conditional, disapproved) and the scope of the approval (e.g., product type, process family); 		8.4.1.1
14 When externally provided product is released for production use pending completion of all required verification activities, it shall be identified and recorded to allow recall and replacement if it is subsequently found that the product does not meet requirements. 8.4.2
15 Controlled conditions shall include, as applicable:
n. the availability of evidence that all production and inspection/verification operations have been completed as planned, or as otherwise documented and authorized; 		8.5.1n
16 Controlled conditions shall include, as applicable:
q. the identification and recording of products released for subsequent production use pending completion of all required measuring and monitoring activities, to allow recall and replacement if it is later found that the product does not meet requirements. 		8.5.1q
17 8.5.1.2 Validation and Control of Special Processes
For processes where the resulting output cannot be verified by subsequent monitoring or measurement, the organization shall establish arrangements for these processes including, as applicable:
f. requirements for documented information to be retained.
[SUBJECTIVE requirement - to be determined by the ORGANIZATION (as per 7.5.1b)… NOT the auditor.] 		8.5.1.2f
18 8.5.1.3 Production Process Verification
The organization shall retain documented information on the results of production process verification. 		8.5.1.3
19 8.5.2 Identification and Traceability
The organization shall control the unique identification of the outputs when traceability is a requirement, and shall retain the documented information necessary to enable traceability. 		8.5.2
20 8.5.3 Property Belonging to Customers or External Providers
When the property of a customer or external provider is lost, damaged, or otherwise found to be unsuitable for use, the organization shall report this to the customer or external provider and retain documented information on what has occurred. 		8.5.3
21 8.5.6 Control of Changes
The organization shall retain documented information describing the results of the review of changes, the person(s) authorizing the change, and any necessary actions arising from the review. 		8.5.6
22 8.6 Release of Products and Services
The organization shall retain documented information on the release of products and services. The documented information shall include:
a. evidence of conformity with the acceptance criteria;
b. traceability to the person(s) authorizing the release.

When required to demonstrate product qualification, the organization shall ensure that retained documented information provides evidence that the products and services meet the defined requirements. 		8.6
23 8.7 Control of Nonconforming Outputs
8.7.2 The organization shall retain documented information that:
a. describes the nonconformity;
b. describes the actions taken;
c. describes any concessions obtained;
d. identifies the authority deciding the action in respect of the nonconformity. 		8.7.2
24 9.1 Monitoring, Measurement, Analysis, and Evaluation
The organization shall evaluate the performance and the effectiveness of the quality management system.
The organization shall retain appropriate documented information as evidence of the results. 		9.1.1
25 9.2.2 The organization shall:
f. retain documented information as evidence of the implementation of the audit program and the audit results.
NOTE: See ISO 19011 for guidance. 		9.2.2f
26 The organization shall retain documented information as evidence of the results of management reviews. 9.3.3
27 10.2.2 The organization shall retain documented information as evidence of:
a. the nature of the nonconformities and any subsequent actions taken;
b. the results of any corrective action. 		10.2.2

# 1
When reading AS9100, sec. 4.4.2b, the interpretation that any additional records are to be determined by the ORGANIZATION (as per 7.5.1b)… NOT the auditor, is supported by interpretations relating to ISO 9001:2008 in “ISO/TC 176/SC 2 Listing of Approved Interpretations against ISO 9001:2008” (Version V1, 2011-02):
- RFI (Request for Interpretation)# 111 asked “Does Clause 6.3 require records of the maintenance of infrastructures?”. The official interpretation was a simple “No”.
- RFI# 115 asked “Does Clause 7.4.3 require records of the verification of purchased product?”. The official interpretation was a simple “No”.

Because neither of these records are specifically required in ISO 9001:2015, the same logic remains in effect.

# 2
Because AS 9100D fails to adequately address ”fitness for purpose“, knowledgable Aerospace companies such as Collins Aerospace (previously UTAS) & Pratt & Whitney identify a minimum ”accuracy ratio“ for M&TE (as 4:1) in their United Technologies ASQR-01 (Rev. 11), sec. 5.4.1.

# 3
1. To begin clause 7.1.5.2 with the words ”When measurement traceability is a requirement…“ seems odd because ALL M&TE should have metrological traceability. While it has been speculated that this requirement is intentionally vague so as to also encompass subjective “measuring tools” (e.g., Visual Representations (photographs), Representative Samples, Customer Satisfaction Surveys), due to this ambiguity, most AS 9100 auditors simply interpret this as a requirement for “calibration” (a.k.a. “Metrological Confirmation”) records. The bottom line is, if ISO / IAQG had wanted something different, they should have been more specific.
2. For clause 7.1.5.2a to require metrological traceability to “to international or national measurement standards” indicates either ignorance on the part of ISO / IAQG concerning metrological traceability OR the influence of government members mandating the use of government-funded NMIs (National Metrology Institutes). In either case, this is an antiquated way in which to address metrological traceability. To gain a better understanding of metrological traceability, read ”NIST Traceability Numbers - The Sasquatch of Metrology.
BTW, The clarification concerning metrological traceability includes the statement “The organization may have regulatory requirements to have standards traceable to NAA.” (which is possible where regulators do not understand the definition of “metrological traceability” contained in BIPM JCGM 200). The acronym “NAA” is intended to be understood to mean "National Aviation Authority".

# 4
For guidance on addressing AS 9100, clause 7.2d, see the (non-binding) IAF ISO 9001 Auditing Practices Group Guidance on: Competence.

# 15
The IAQG official "AS 9100:2016 Series Clarifications" states: Clause 8.5.1.n requires evidence that all production and inspection/verification operation steps have been completed as planned or otherwise documented and authorized. Examples of evidence can include stamps, electronic signatures, initials, or names.

# 18
In response to the question: “Does 9100 require production process verification of all assemblies?”, the IAQG official "AS 9100:2016 Series Clarifications" states: It depends. The organization defines its production process verification process to cover parts and assemblies. Assembly can include subassemblies, component assemblies, and even final product.
In response to the question: “Does 9100 mandate that a Production Process Verification be performed and the fixture verified to the first article if the tooling fixtures in the factory have been disassembled and moved to another location within the same facility?”, the IAQG official "AS 9100:2016 Series Clarifications" states: “Yes. It is expected that the organization would have some tool verification activity, commensurate with the amount of tool disassembly, to ensure the fixture is still capable of building conforming hardware. It is thought that disassembly and reassembly of a fixture would be specified as one of the requirements that would invalidate the previous PPV.”

The IAQG official "AS 9100:2016 Series Clarifications" also provides clarification of this clause stating: “The first clause 8.5.1.3 requirement was introduced so all organizations, including those with small production quantities (such as in Space industry), could apply the Production Process Validation (PPV) instead of identifying it as not applicable (exclusion). The Team wanted to open the door for other “process” methods to perform PPV that may be implemented to provide an alternative methodology to the previously written PPV requirement. The team decided to keep the second requirement for all the organizations as a FAI can be done according to internal rules (or according to the 9102 when required by contract).
The first paragraph was added since only performing a FAI does not provide the warranty that the whole “production” process will be able to product parts that meet requirements. Actually, it only provides the warranty that the “manufacturing” process is able to “manufacture” a product compliant with the requirements relating to the “product.” The other requirements regarding the “production” process (in terms of quantities to produce, lead-time, cost constraints, …) cannot be verified with only a FAI. It was not the team's intent to require PPAP or process capability for each production process. Regarding the “records” we require the organization to retain documented information on how they ensure production process verification is implemented.”

# 26
AS9100/ISO 9001, sec. “9.3.2 Management Review Inputs” states:
The management review shall be planned and carried out taking into consideration:
c. information on the performance and effectiveness of the quality management system, including trends in:
2. the extent to which quality objectives have been met;

When asked “Is every process established by an organization required to have at least one quality objective?” (ref. AS9100/ISO 9001sec. 6.2.1), the official response provided in the “US TC 176 - TG22 - Interpretations” (US# 2018-04) stated: No. (6.2.1 …shall establish objectives at “relevant” processes…) It may NOT be relevant to set an objective for some processes.

However, when asked “Is it the intent of the standard that an organization can have just a top-level requirement(s) that is used to evaluate the effectiveness of the QMS and several individual processes without those processes having specific metrics? For example, OTD of product to the customer of 98% is the top-level metric and the metric used to evaluate the effectiveness of the purchasing process, contract review process, and the manufacturing process with no additional metrics. So if they have met the OTD of 98%, then all processes are deemed as effective.” (relating to sec. 4.4.1c & g), the IAQG official "AS 9100:2016 Series Clarifications" responded: “No. 9100-series standards require the organization to determine if the identified processes are effective and achieving planned results (see clause 4.4.1c). Each process measure should evaluate the effectiveness of that process and be value-added. This is the measure that would be included in Process Effectiveness Assessment Report (PEAR) as the key performance indicator for that process.
The 9100-series standards does not mandate a certain number of process measures. Small organizations typically have fewer measures than larger organizations. These small organizations have increased visibility regarding process health due to their size. Regardless, this does not alleviate the need for determining if processes are effective and achieving planned results. The organization can have additional working level measures that may not flow up to top management or management review.”

# 27
When asked “If action is taken under section 10.2.1, c) that addresses a cause, but not the “root” cause (e.g., 1 why vs. 5 whys), is a review of the effectiveness of corrective action (part d) required?”, the the official response provided in the “US TC 176 - TG22 - Interpretations” (US# 2019-01) stated: Yes. Experts agree that the intent of 10.2.1d (review the effectiveness of any corrective action taken) is to ensure that the organization manages nonconformities, and implements corrective action, appropriately.
ISO/TS 9002:2016 states in Clause 10.2.1, “The organization should review the effectiveness of any corrective actions by confirming (through evidence) that the actions have been implemented or correction taken and as a result the nonconformities have not recurred.”

However, when asked “Is it a requirement of ISO 9001:2015, Subclause 10.2, to document the root cause analysis?”, the official response provided in the “ISO/TC 176/SC 2 Listing of Approved Interpretations against ISO 9001:2015” (RFI# 134) stated: No. This is because ISO 9001 does not address “root cause analysis”.

Source: https://www.pngrepo.com/svg/85966/road-with-broken-line If you’re ready for a path forward now…
Just click here to schedule your FREE Certification Strategy Meeting (via Zoom) with me. I’ll answer any questions you might have. No sales pitch. Just information.

Or, for my cell phone & e-mail address, visit the contact us page.

100% of our clients achieve certification on their first attempt.

This means that no CB has ever required a “follow-up” or “special” audit for any of our clients prior to being issued their certification.

We provide you with “peace of mind” that we'll take care of QMS certification!

Contact Us