Differences

This shows you the differences between two versions of the page.

--- articles:types_of_audits [2020/06/24 10:58] – [Conclusion] rrandall
+++ articles:types_of_audits [2023/02/09 11:10] (current) – [Quality Management System Audits] rrandall
@@ Line 1: / Line 1: @@
 ====== Types of Audits ======
-Despite the fact that there are many different types of audits, ISO 9001 & AS9100 registrars and consultants have been singing praises of "Process-based” audits for years. In fact, they've been promoted to the point that //some// ISO 9001 & AS9100 auditors have actually issued (unjustified) nonconformities to companies for having performed “clause/element-based" internal audits rather than the //preferred// "process-based" audits!
+Before we discuss the different "types" of audits, we must first define the "scope" of the audit.
-These nonconformities are based upon a highly subjective interpretation of ISO 9001 & AS9100.
+^  Internal Audits  ^  External Audits  ^^
+^  1st Party Audits  ^  2nd Party Audits  ^  3rd Party Audits  ^
+|  Audits conducted by an organization itself, or on its behalf (e.g., through contracted auditors - for independence), of its internal functions and processes  |  Audits of Suppliers (or audits of your company performed by your customers - or their representatives)  |  Audits performed by Certification Bodies - CBs (e.g., ISO 9001, AS 9100), or Accreditation Bodies - ABs (e.g., ISO 17020, ISO 17025)  |
-<blockquote>ISO 9001:2015 & AS9100:2016: \\
+The audit "scope" also includes defining criteria such as:
-.2.1 The organization shall conduct internal audits at planned intervals to provide information on whether the quality management system; \\
+  * The specific site(s) covered by the audit (e.g., specific addresses, Building Numbers)
-b. is EFFECTIVELY implemented and maintained.</blockquote>
+  * Any specific function(s) or process(es) to be covered (e.g., Sales, Purchasing, Engineering, Production)
+  * The Audit Criteria for: \\
+    * a QMS Internal audit (covering specific internal documents; procedures, Work Instructions, etc.), \\
+    * a QMS Standard Audit (specifying a QMS Standard such as ISO 9001, AS 9100, ISO 17020, ISO 17025, etc.), \\
+    * a Product Audit (i.e., for a specific product), \\
+    * a Configuration Audit (i.e., for a specific product configuration)
-Largely due to "indoctrination" (from the registrars) in how to think about ISO 9001 & AS9100, many auditors consider a "process-based" audit to be the ONLY way possible to determine "//whether the quality management system is effectively implemented and maintained//”. This was further supported by an "opinion" provided in the [[https://asq.org/quality-resources/iso-9001/us-tc176|US TC 176 - TG22 - Interpretations]] (Read: [[articles:re-writing_iso_9001_through_interpretation|Re-writing ISO 9001:2015... through Interpretation]]).
+<WRAP center round info 80%>
+For purposes of this article, all audits are assumed to be "quality-related" in nature (e.g., NOT Environmental or Safety).
-However, is this true?
+</WRAP>
 ===== What are the types of audits? =====
+While there are many types of audits, and audit strategies, the most common types are:
-ISO 19011:2018 Annex A states:
+  * Quality Management System Audits
+  * Product Audits
+  * Configuration Audits
-<blockquote>A.12 Audit of supply chain \\
+ALL of the above audits can be performed either internally (on the company itself) or externally (on suppliers/subcontractors).
-//The audit of the supply chain to specific requirements can be required. The supplier audit programme should be developed with applicable audit criteria for the type of suppliers and external providers. The scope of the supply chain audit can differ, e.g. complete __management system audit__, single __process audit__, __product audit__, configuration audit.//</blockquote>
-While this is NOT a complete list of audit types, neither ISO 19011:2018 nor ISO 9000:2015 defines nor describes the difference between these audit types.
+While ISO 19011:2018 Annex A mentions several different "types" of audits (as "scopes"), neither ISO 19011:2018 nor ISO 9000:2015 defines nor describes the difference between these audit types.
+<blockquote>ISO 19011:2018 A.12 Audit of supply chain \\
+//The audit of the supply chain to specific requirements can be required. The supplier audit programme should be developed with applicable audit criteria for the type of suppliers and external providers. The scope of the supply chain audit can differ, e.g. complete __management system audit__, single __process audit__, __product audit__, __configuration audit__.//</blockquote>
-You may be thinking, "//Why not just take the ISO 9000:2015 definition for the word "audit" and expand it to include the other word (product, process, or system)//".
+__Value-Added Audits (VAA)__ \\
+While all of the above-listed audit types have an objective of verifying “compliance”. a “Value-Added Audit” (VAA) focuses on __improvement__ through the elimination or reduction of non-value-added (NVA) activities/steps in a sequence. A common approach is to utilize a detailed flow chart of the process - identifying those "activities/steps" that either "add no value" or "add no value but are necessary" (e.g., an inspection activity to mitigate the possibility of nonconforming product(s) being delivered).
-Let's see how that works...
+A VAA is often used to identify opportunities for improvement. more information on VAAs is at the bottom of this article.
+==== Quality Management System Audits ====
-<blockquote>**Process Audit** \\
+The most common audits are Internal Audits of the Quality Management System (required by most QMS standards, e.g., ISO 9001, AS9100). However, some companies also perform Quality Management System Audits of their suppliers/subcontractors.
-//A systematic, independent, and documented process for obtaining objective evidence and evaluating it objectively to determine the extent to which the audit criteria are fulfilled in a set of interrelated or interacting activities that use inputs to deliver an intended result.//  (Source: ISO 9000:2015, sec. 3.13.1 & 3.4.1)</blockquote>
-<blockquote>**Product Audit** \\
+<blockquote>**System Audit** \\
-//A systematic, independent, and documented process for obtaining objective evidence and evaluating it objectively to determine the extent to which the audit criteria are fulfilled in the output of an organization that can be produced without any transaction taking place between the organization and the customer.//  (Source: ISO 9000:2015, sec. 3.13.1 & 3.7.6)</blockquote>
+An audit is conducted on a management system. It can be described as a documented activity performed to verify, by examination and evaluation of objective evidence, that applicable elements of the system are appropriate and effective and have been developed, documented, and implemented in accordance and in conjunction with specified requirements.  (Source: [[https://asq.org/quality-resources/auditing|https://asq.org/quality-resources/auditing]])</blockquote>
-<blockquote>**Management System Audit** \\
+Quality Management System (QMS) Audits can be performed at one time (e.g., as Certification Bodies do during their Re-Assessments) or of portions of the QMS (e.g., as Certification Bodies do during their Surveillance Audits).
-//A systematic, independent, and documented process for obtaining objective evidence and evaluating it objectively to determine the extent to which the audit criteria are fulfilled throughout the set of interrelated or interacting elements of an organization (that) establish policies and objectives, and processes to achieve those objectives.//  (Source: ISO 9000:2015, sec. 3.13.1 & 3.5.1)</blockquote>
-While the above //may// be technically correct (I'm not actually sure of that statement), at the very least, it is "techno-babble" that is unsuited for use in the real world.
+=== Audit Sub-Types ===
-===== ASQ describes these types of audits =====
+Most common in QMS audits, audit sub-types can include:
+  * Process Audits
+  * Clause-based Audits
+  * Horizontal Audits
+  * Vertical Audits
-Alternatively, ASQ has an article titled "[[https://asq.org/quality-resources/auditing|What is Auditing]]", which includes a section titled: "//The Three Different Types of Audits//". While not "official" definitions of the terms, this section describes these three types of audits in a way that is far more understandable.
+__Process Audits__ \\
+ASQ has an article titled "[[https://asq.org/quality-resources/auditing|What is Auditing]]", which includes a section titled: "//The Three Different Types of Audits//". While not "official" definitions of the terms, the article describes a "Process Audit" (actually, an audit sub-type) in a way that is very understandable.
 <blockquote>**Process audit** \\
@@ Line 47: / Line 63: @@
   * Check the adequacy and effectiveness of the process controls established by procedures, work instructions, flowcharts, and training and process specifications.
 (Source: [[https://asq.org/quality-resources/auditing|https://asq.org/quality-resources/auditing]])</blockquote>
+Despite the fact that there are many different types of audits, ISO 9001 & AS9100 registrars and consultants have been promoting "Process-based” audits for years. And, largely due to "indoctrination" (from the registrars) in how to think about ISO 9001 & AS9100, many auditors consider a "process-based" audit to be the ONLY way possible to determine "//whether the quality management system is effectively implemented and maintained//”. This is further supported by a "non-binding" opinion provided in the [[https://asq.org/quality-resources/iso-9001/us-tc176|US TC 176 - TG22 - Interpretations]] (Read: [[articles:re-writing_iso_9001_through_interpretation|Re-writing ISO 9001:2015... through Interpretation]]).
+In fact, they've been promoted to the point that //some// ISO 9001 & AS9100 auditors have actually issued (unjustified) nonconformities to companies for having performed “clause/element-based" internal audits rather than the //preferred// "process-based" audits! These nonconformities are based upon a highly subjective interpretation of ISO 9001 & AS9100.
+Despite the subjective interpretations/opinions of some auditors, companies should decide the type of internal audit that best provides "//information on whether the quality management system is effectively implemented and maintained//".
+If faced with an external (e.g., 3rd Party) auditor who insists that you must perform “process-based” internal audits in order to "//provide information on whether the quality management system is effectively implemented and maintained//", then there are several ways to argue this.
+  - Neither ISO 9000:2015 nor AS 9100:2016 requires that a "process audit" or "process-based audit" be performed in order to "//provide information on whether the quality management system is effectively implemented and maintained//".
+  - No ISO document (including ISO 19011:2018 nor ISO 9000:2015) defines the term "process audit" or "process-based audit".
+  - The "[[https://committee.iso.org/files/live/sites/tc176/files/documents/ISO%209001%20Auditing%20Practices%20Group%20docs/Auditing%20General/APG-DemonstrateConformity2015.pdf|ISO 9001 Auditing Practices Group Guidance on: Demonstrate conformity to the standard]]" does not require “process-based” internal audits to demonstrate conformity with ISO 9001:2015.
+  - While the "[[https://committee.iso.org/files/live/sites/tc176/files/documents/ISO%209001%20Auditing%20Practices%20Group%20docs/Auditing%20General/APG-Audit_Planning.pdf|ISO 9001 Auditing Practices Group Guidance on: Audit Planning]]" states "//Develop the plan around the processes – not the clauses of the standard//", this is contained in a section titled "USEFUL TIPS TO AUDITORS"... because this is NOT a requirement.
+  - The official ISO document "[[https://www.iso.org/files/live/sites/isoorg/files/archive/pdf/en/iso9001-2015-process-appr.pdf|The Process Approach in ISO 9001:2015]]" describes the process approach toward the QMS WITHOUT requiring (or even mentioning) a “process-based” internal audit.
+The above should be sufficient information to "[[articles:how_to_appeal_nonconformance|appeal]]" this type of invalid nonconformity.
+The point is that auditors should be there to verify compliance with "actual" requirements. Not imagine/invent requirements where none exist.
+__Clause-based Audits__ \\
+A "clause-based" (aka "element-based") is where an audit verifies compliance with individual clauses/elements contained in a QMS Standard. For example, only auditing ISO 9001, section 7.2 "Competence".
+__Horizontal Audits__ \\
+A horizontal audit is an audit of "one" process across multiple departments in the business. Common examples of this approach include:
+  * Competence (ISO 9001/AS9100, sec. 7.2)
+  * Document Control (ISO 9001/AS9100, sec. 7.5)
+  * Corrective Action (ISO 9001/AS9100, sec. 10.2)
+Using "Competence" as an example, the auditor would examine a sampling of records used to provide evidence of the competence of the personnel performing quality-related work in every area (e.g., Sales, Procurement, Engineering, Planning, Production, Inspection, Shipping).
+Using "Document Control" as another example, the auditor would verify control of documents in every area where documents are distributed (or otherwise required to be controlled - e.g., documents of external origin).
+<WRAP center round info 80%>
+AS9100 auditors use the "Horizontal Auditing" approach for all of the AS9100 requirements outside of section 8. This is recorded in AS9101 Form 2: "QMS PROCESS MATRIX REPORT".
+</WRAP>
+__Vertical Audits__ \\
+A vertical audit is an audit of "all" the processes used by a department. Common examples of this approach include:
+  * Sales (ISO 9001/AS9100, sec. 8.2)
+  * Engineering (ISO 9001/AS9100, sec. 8.3)
+  * Procurement (ISO 9001/AS9100, sec. 8.4)
+  * Production (ISO 9001/AS9100, sec. 8.5, 8.6 & 8.7)
+Using "Sales" as an example, the auditor would:
+  - interview personnel concerning the process inputs, activities, and outputs;
+  - examine a sampling of records (required by sec. 8.2.3.2) used to provide evidence of a contract/order review having been performed (with its results) and any records created to describe any new requirements for the products and services (e.g., Change Orders, Amended/Revised Orders).
+Using "Procurement" as another example, the auditor would interview personnel concerning the process inputs, activities, and outputs. And examine a sampling of the records required by ISO 9001/AS9100, sec. 8.4 to be retained.
+During a full system audit, the auditors "should" verify that the linkages between processes provide effective communication of requirements.
+<WRAP center round info 80%>
+AS9100 auditors use the "Vertical Auditing" approach for all of the AS9100 requirements contained in section 8. This is recorded in a separate 9101 Form 3: "PROCESS EFFECTIVENESS ASSESSMENT REPORT" (aka PEAR) for each department.
+</WRAP>
+==== Product Audits ====
+A “Product Audit” is performed with the objective of verifying whether a particular product (e.g., parts or assemblies, processed material, software) conforms to requirements (i.e., drawings, BOMs, technical specifications, product/material standards).
+<WRAP center round info 80%>
+A similar approach could be taken when performing a service-related audit (e.g., laboratory analysis, destructive or non-destructive testing, calibration services, repair services). However, the focus would generally be on the specification(s) defining the required service.
+</WRAP>
+In fact, the ASQ definition of a “Product Audit” includes "services".
 <blockquote>**Product Audit** \\
@@ Line 52: / Line 132: @@
-<blockquote>**System Audit** \\
+Product audits can be either performed internally or at supplier sites.
-An audit conducted on a management system. It can be described as a documented activity performed to verify, by examination and evaluation of objective evidence, that applicable elements of the system are appropriate and effective and have been developed, documented, and implemented in accordance and in conjunction with specified requirements.  (Source: [[https://asq.org/quality-resources/auditing|https://asq.org/quality-resources/auditing]])</blockquote>
+<WRAP center round important 80%>
+A “[[articles:source_inspections|Source Inspection]]" is NOT a "Product Audit" (or a “Supplier Audit”) because it is an “__inspection__” (not an audit). Unlike a Supplier/Product Audit (which follows a product through its entire production process – verifying that requirements have been met), a Source Inspection takes place at the end of the production line. A Source Inspection "can" replace the Receiving Inspection performed by the customer.
+</WRAP>
+These audits should begin by identifying the particular product(s) that the customer is purchasing from the supplier. If the company has any specific product/service-related concerns, then these should be considered when planning the audit.
+A product audit typically begins by verifying that all of the customer requirements have been communicated to production personnel e.g., through travelers, drawings, Work Instructions, Specifications). If the requirements were not adequately communicated to production, then there is typically a communication breakdown between the Sales function and Production.
+The auditor should then follow an example of the product model (e.g., Part Number) being purchased by the customer through the entire production process… verifying that all requirements were met, including confirming that:
+  * the product was manufactured to the correct product configuration – through verifying that the revision levels of all component/detailed parts match the revision levels required by the customer (typically through confirming that the detailed parts & subassemblies used match the product BOM (Bill of Material)
+  * the correct raw material was used (e.g., supported by a Material Test Report, Certificate of Analysis, Chemical Analysis Report).
+  * the work was performed in the proper environment (e.g., a Cleanroom environment)
+  * proper equipment is provided (e.g., to prevent ESD damage),
+  * any special processing was performed by a qualified process (e.g., a Nadcap certified process),
+  * measuring instruments had the appropriate accuracy (“Accuracy Ratio” between the instrument and the tolerance measured), range & resolution.
+The product audit would also examine completed records (from previous product runs) providing evidence that the practices observed are consistent (e.g., completed Job Travelers, nonconformity reports of this specific product model (part number), and, if available, a Pareto chart identifying key issues with this product. If there are issues identified (either externally, through customer complaints or internally, through employees identifying nonconformities), the auditor should evaluate the effectiveness of any analysis (e.g., Fishbone, charts, 5-Whys) and actions are being taken to mitigate (through risk management) or eliminate (through corrective action) these issues. This is an area where both ISO 9001 & AS 9100 are weak.
+All of the objective evidence collected should provide confidence to the customer that the specific product model (e.g., Part Number) is being produced in accordance with requirements.
+Is this how “Quality Auditing performs Supplier Audits?
+If not, then this may be a new area of opportunity… perhaps through “branding” these as “Product Audits” through:
+  * A series of articles educating customers as to how product audits can benefit them… even of suppliers who have a registered QMS.
+  * Creation of a generic “Product Audit” checklist to be used by “Quality Auditing” to provide consistency in both performance and reporting.
+Supplier/Product Audits are typically of suppliers identified by customers, as having significant quality issues. And a major part of supplier audits is the follow-up. Either verifying that the actions taken by the supplier are effective, or working with the supplier to determine and implement an effective plan to improve product quality. These two activities should be performed jointly in order to justify the cost of Supplier/Product Audits.
+Source Inspections
+I was talking with Oshan about these types of audits (on 12/9/21). A source inspection is where a Purchaser/Buyer (i.e., customer), OR their representative (e.g., “Quality Auditing”), verifies the conformity of a product (e.g., whether based on a sampling of a batch/lot, or 100%) AFTER production, and prior to the product(s) being shipped to the customer.
+=== Audit Strategies ===
+__Downstream Audits__ \\
+A "Downstream Audit" starts at the beginning of the value stream and follows an order from receipt (e.g., "Sales") through each functional area and process, in sequence until the end (e.g., the Shipping area). The auditor should verify that all of the requirements are communicated through each stage of the value stream such that the final product/service satisfies all of its requirements.
-If you'd like to include a definition for these terms in your own QMS (or "Supplier Quality Requirements Manual"), simply reference the above ith the link to the ASQ article.
+__Upstream Audits__ \\
+An "Upstream" audit begins with a final product/service (e.g., awaiting delivery in the Shipping area) and follows the value stream in reverse order (i.e., "upstream") to where the contract/order was received (e.g., the Sales area). The auditor should verify that all of the requirements were accurately communicated from each preceding stage of the value stream, ensuring that the final product/service satisfies all of its requirements.
-===== Configuration Audits =====
+==== Configuration Audits ====
 While "//configuration audit//" is also not defined by ISO (even in ISO 10007:2003, "//Quality management systems — Guidelines for configuration management//"), the U.S. Federal Aviation Administration (FAA) has defined the term in [[https://www.faa.gov/documentLibrary/media/Order/1800.66.pdf|Order1800.66 (Incl Chg 3), "Configuration Management Policy"]].
@@ Line 72: / Line 188: @@
 //Physical examination of the actual configuration of the item being produced. It verifies that the related design documentation matches the item as specified in the contract. The system product baseline if finalized and validated at the PCA.// (Source: [[https://www.dau.edu/glossary/Pages/Glossary.aspx|U.S. Defense Acquisition University - Glossary]])</blockquote>
-===== Conclusion =====
+==== Value-Added Audits (VAA) ====
-Ultimately, despite the subjective interpretations/opinions of some external auditors, organizations should decide the type of internal audit that best provides "//information on whether the quality management system is effectively implemented and maintained//".
+While all of the above-listed audit types have an objective of verifying "compliance". a "Value-Added Audit" (VAA) focuses on improvement through the elimination or reduction of non-value-added (NVA) activities/steps in a sequence.
-If faced with an external auditor who insists that you must perform “process-based” audits in order to "//provide information on whether the quality management system is effectively implemented and maintained//", then there are several ways to argue this.
+<WRAP center round info 80%>
+An excellent book describing these audits is "[[https://www.amazon.com/Understanding-Applying-Value-Added-Assessment-Eliminating/dp/0873893697|Understanding and Applying Value-Added Assessment - Eliminating Business Process Waste]]" by William E. Trischiler.
+</WRAP>
-- Neither ISO 9000:2015 nor AS 9100:2016 requires that a "process audit" or "process-based audit" be performed in order to "//provide information on whether the quality management system is effectively implemented and maintained//".
+"Value-Added Audits" are performed "internally" and tend to be used by companies who have adopted the use of Lean 6 Sigma tools.
+In a "Value-Added Audit", the auditor examines each activity/step in a process asking whether the activity/step adds value to the product or service. If the customer isn't paying for the activity/step being performed, then it doesn't add value. However, there may be some activities/steps that a necessary to properly complete an activity/step that __does__ add value.
-- No ISO document (including ISO 19011:2018 nor ISO 9000:2015) defines the term "process audit" or "process-based audit".
+As a part of the VAA, the auditor seeks to determine whether there is any justification for each NVA activity/step in the sequence/process. NVA results in some form of waste.
-- The "[[https://committee.iso.org/files/live/sites/tc176/files/documents/ISO%209001%20Auditing%20Practices%20Group%20docs/Auditing%20General/APG-DemonstrateConformity2015.pdf|ISO 9001 Auditing Practices Group Guidance on: Demonstrate conformity to the standard]]" does not require “process-based” internal audits to demonstrate conformity with ISO 9001:2015.
+The following eight lean manufacturing wastes, mostly derived from the TPS (Toyota Production System), have a universal application to businesses today. The acronym for the eight wastes is DOWNTIME. Downtime stands for:
+  * Defects
+  * Overproduction
+  * Waiting
+  * Not utilizing talent
+  * Transportation
+  * Inventory excess
+  * Motion waste
+  * Excess processing
-- The "[[https://committee.iso.org/files/live/sites/tc176/files/documents/ISO%209001%20Auditing%20Practices%20Group%20docs/Auditing%20General/APG-Audit_Planning.pdf|ISO 9001 Auditing Practices Group Guidance on: Audit Planning]]" states "//Develop the plan around the processes – not the clauses of the standard//" in a section titled "USEFUL TIPS TO AUDITORS" because this is NOT a requirement.
+Performing a VAA is a great way to identify significant areas for improvement. However, at a minimum, whoever is assigned to perform a VAA should be a Lean Six Sigma certified Green Belt. Otherwise, the VAA will likely fail to prove beneficial.
-- The official ISO document "[[https://www.iso.org/files/live/sites/isoorg/files/archive/pdf/en/iso9001-2015-process-appr.pdf|The Process Approach in ISO 9001:2015]]" describes the process approach toward the QMS WITHOUT requiring (or even mentioning) a “process-based” internal audit.
-The above should be sufficient information to "appeal" this type of invalid nonconformity.
-The point is that auditors should be there to verify compliance "actual" requirements. Not imagine requirements where none exists.