6.1 Qualification

Facility Qualification

Facility is the highest level of qualification and is usually the result of some of the other qualification types. Facility qualification also indicates that the facility (or the overall company) has the proper business, manufacturing, training, and other quality processes integrated throughout the operations. This qualification often involves the coordination of different organizations such as the following:

• Facility Quality and Management – Organization that will be using the machine to make parts
• Part Customer – Organization that will be receiving parts made on the machine
• Industry Group or 3rd Party Organization – Organization that represents the part customer, as part of a larger industry organization. Examples of these are NADCAP, Lloyd’s Registry, etc.
• Regulatory Agency – Governmental body or industry group

A qualified AM facility must be able to demonstrate a few key systemic, organizational, and process-oriented capacities, including having an overall quality management system, approved AM-related processes and equipment, and relevant non-AM-related processes and equipment. These three areas are described in more detail below.

Overall Quality Management System: This often requires that the facility operates in accordance with an industry-approved specification, such as ISO-9000, AS-9100 (aerospace), or ISO-13485 (medical).

Typical parts of the overall quality management system are:

• Understanding customer and contract requirements
• Organized and documented leadership roles and responsibilities
• Involvement of employees, which includes personnel training and qualification
• Control of business, manufacturing, and other processes. This includes control of materials, designs (build files) software, machines, work instructions (build programs), operators, inspectors, etc.
• Closed-loop improvement processes – Monitoring production and making improvements in a controlled fashion, or taking timely corrective action when things go wrong
• Data-based decision making
• Management of relationships with suppliers and subcontractors

Approved AM-Related Processes and Equipment: Prior to receiving a facility qualification, it is usually necessary for the AM-related processes and equipment to be qualified. Examples of these qualifications include feedstock handling and tracking, qualification of at least one AM machine, and qualifications of operators for the machines.

Non-AM-Related Processes and Equipment: This can apply to processes outside of the actual print operations. In some cases, these processes are performed within the facility, or they may be performed in sister facilities or subcontractors, which have been approved.

Examples of these processes and equipment that may require approval are:

• Heat treatment
• Machining and other post-processes
• Nondestructive testing
• Mechanical and material testing
• Dimensional inspection

A qualified AM facility consists of qualified personnel, equipment, and processes working together in an integrated fashion to reliably produce AM hardware that meets requirements and is consistently being improved in a controlled fashion.

Machine Qualification

A qualified AM machine is one that can reliably produce AM parts that meet requirements. As the AM industry matures, machine qualification is coalescing on a staged approach that closely aligns with medical equipment qualification. The stages differ in the propose of the testing, feedstock used during the testing, types of testing being done, the pass/fail criteria, and who is involved in the qualification testing, and determining a machine to be qualified.

The different types of organizations involved may include the following:

• Machine Builder – Organization that designs and builds the machine
• User – Organization (Facility) that will be using the machine to make parts
• Part Customer – Organization that will be receiving parts made on the machine
• Industry Group or 3rd Party Organization – Organization that represents the part customer, as part of a larger industry accreditation organization; examples of these are Nadcap and Lloyd’s Registry
• Regulatory Agency – Governmental body or industry group

Factory Acceptance Testing (FAT): Factory acceptance testing is usually conducted at the site at which the machine is built and performed by employees of the machine builder, to determine whether a hardware and software of the product satisfies the requirements. Since FAT occurs before the machine is accepted by the user, it is sometimes considered outside the realm of qualification, but either way, it is considered a prerequisite to the other qualification stages. FAT is also good practice as it provides data to a default or baseline known point. FAT essentially consists of the machine builder verifying that the machine operates as intended. The involved parties are the machine builder and the user. In the case of smaller machines (e.g. 250mm class Laser Powder Bed Fusion machines), this testing takes place at the factory where the machine was built. In the case of large Direct Energy Deposition or Material Extrusion machines where the machine is assembled for the first time in the user’s facility, this would take place at the user’s facility.

FAT consists of a number of operational tests to verify compliance with the machine specification. Some of these tests will check the performance of individual sub-systems, including the items below.

• Testing the optical train on a L-PBF machine by scanning the laser over the build area that has a series of detectors installed to measure beam profile, placement accuracy, consistency, and power.
• Translating the deposition head on a DED or ME machine over the full build volume to verify accuracy and repeatability over that volume.
• Measuring the protective gas flow on a machine to verify the lack of dead spots in coverage
• Measuring the heating tip temperature on a ME machine.
• Testing the powder recoating system over a range of powder sizes and layer thicknesses to verify range of operation.
• Verifying operation of the feedstock handling system by checking material feed rates or dosing rates.

While the feedstock used for the actual tests may be a standard material specified by the machine builder, it is more commonly that material the user intends to put in the machine in production. This is not only to avoid potential contamination issues, but to have a continuity of material type from FAT through Installation Qualification, Operational Qualification, and Part Qualification. After these tests have been successfully completed, FAT will often culminate with making a series of test parts that validate the performance of the machine over a range of geometries and location within the build volume. These geometries are generally determined by the machine builder as ones that provide the best information on the performance of the machine in the shortest amount of time and cost. Post-build dimensional inspection is the most common, with radiographic, CT scanning, and surface roughness measurements often employed. Additionally, the log files that were generated during the builds are analyzed for any anomalies. After successful completion of FAT, the machine is then disassembled to the extent necessary and shipped to the user’s facility.

Installation Qualification (IQ): IQ Is a process that verifies that all aspects of facility, utilities and equipment that affect product quality adhere to the specification, and that the equipment has been properly delivered and correctly installed. IQ, sometimes known as Site Acceptance Testing (SAT), almost always takes place at the user’s facility, after the machine has been received, installed, and re-assembled. In the case of machines that are only assembled at the user’s facility, IQ and FAT may be performed simultaneously to reduce time and cost. In many cases, the first stage of IQ is repeating all of the FAT testing to ensure that the newly re-assembled machine performs the same as it did at the factory. A SAT may include the FAT initially but then increase in scope to the user’s definition if they possess or add capability to the standard machine and thus becomes the default condition for the user which is then different from the original factory.

IQ will often consist of additional tests required by the machine user. In the case where final payment on the machine is contingent on passing IQ, the machine builder will also be involved in designing the tests and determining pass/fail criteria. Depending on the industry, the part customer or a 3rd party may also be involved. While some of these acceptance tests are subsystem diagnostic type tests, they will usually include building test parts that are representative of the geometries that the user is intending to make with the machine. The intended production feedstock is almost always used for this testing, for the aforementioned reasons. Figure F07_02_iq shows an example IQ component. In addition to the tests performed for FAT, additional tests sometimes include chemistry, microstructural, dissection, and mechanical property tests. Depending on the testing performed, post-processing may be necessary. Mechanical testing at this stage would generally require exceeding specification minimums.

Figure 6.2 Installation Qualification test part example from the US National Institute of Standards and Technology. (credit: Modification of Figure 3 in “Gary Mac, Hammond Pearce, Ramesh Karri, Nikhil Gupta. “Uncertainty quantification in dimensions dataset of additive manufactured NIST standard test artifact.” Data in Brief, Volume 38, 2021, 107286. CC BY 4.0)

Operational Qualification (OQ) – OQ is series of tests which ensure that equipment and its sub-systems will operate within their specified limits consistently. This 3rd stage of machine qualification almost always involves the user and the part customer or 3rd party in defining the qualification tests and the pass/fail criteria. They may even collaborate in performing the actual testing, such as the part customer performing the actual chemical, microstructural and mechanical tests. This is generally the earliest instance for a regulatory body to get involved, although usually in a review capacity. Another prerequisite for OQ is that the basic processes for building and post-processing parts is locked down and documented in what is often referred to as a process control document (PCD).

The requirements for OQ are generally found in the specification used by the part customer to procure production hardware. While in most cases, the specification used for OQ are proprietary to the part customer, the general push is to move to industry-standard specifications. Examples of this are AWS D20.1 (Specification for Fabrication of Metal Components using Additive Manufacturing) and AMS 7032 (Additive Manufacturing Machine Qualification). The specification will either include or reference a specification that contains the full range of qualification tests, along with the relevant pass/fail criteria.

OQ will use the feedstock intended for production, as the end result is the approval to use the machine to make production hardware to a specification that includes the feedstock and final part material. Feedstock from a qualified source is also generally required. Full post-processing may need to be performed to properly perform certain tests like chemistry, microstructure, and mechanical properties. These will also need to be performed by post-processors who have the necessary approvals from the part customer or 3rd party.

Because the objective of OQ is to demonstrate that the machine and feedstock combination are capable of repeatedly making hardware that conforms to specification, the following are often part of an OQ:

Test material comes from the full volume of the machine that will be used for production: While this is potentially the full build volume of the machine, in many cases, certain parts of the build volume, will not be used to make production hardware. In this case, test material would not come from these regions, and the qualification documents would state which areas of the build volume are excluded. An example of this is shown in Figure 6.3.Qualification Volume is the portion of the build volume qualified to make the specified hardware.

Figure 6.3 Relationship Between Build Volume and Qualification Volume

Multiple lots of material: While some specifications allow all of the test material to come from a single build, others will require 2 or more builds. These builds may allow the same feedstock lots, or they may require different feedstock lots. Producers consider the same options for post-processing operations. The objective is to show that the machine can produce conforming material multiple times. The specification may even require that the builds be non-consecutive or that the feedstock contain a range of chemistry within the range allowed by the feedstock specification. In the case of this example, chemistry and microstructure tests will be done on material in the grip areas of the specimens, but additional coupons or excess material could also be used. (A coupon is a small piece of material that is used as a representative of the entire build for the purposes of testing. It is common in AM and other areas of fabrication.) It should be noted that designing the test builds for qualification to meet all of the coupon size, orientation, and location requirements can be quite challenging. Making spare coupons is recommended because it is possible to have a “bad” test that would then invalidate the qualification by having insufficient tests.

Machine performance during builds meets specification requirements: This means that the machine being qualified performs as expected during the qualification builds, without any anomalies that would require the build to be stopped or the parts to be rejected, such as a loss of shielding gas or input energy.

Hardware meets specification requirements: This means that the parts and coupons built during the qualification builds meet all of the specification requirements for chemistry, microstructure, dimensions, NDT, minimum mechanical properties, and so on.

Consistent mechanical properties: While the specifications for many engineering materials require that their mechanical properties meet the minimums in the specification, newer ones are beginning to require that the population of the material used for qualification match or be more consistent that that used in generating the design values. The reason is to show that not only does the material meet minimum properties, but that the process/machine is sufficiently in control to prevent drift over the production run that could result in the part having low properties that are not detected in lot acceptance testing. While only a few AM specifications currently require this, it may become more common as AM moves to more critical applications.

An example of this is shown in Table 6.1. Note the bottom three rows of the table, which contain the mean, standard deviation, and mean – 3 standard deviations. The high scatter in the X-Direction testing (41 MPa standard deviation, versus 20 MPa for the design value population) means that the potential minimum values for the process (899 UTS and 849 YS) falls below that of the design value population, even though the actual values are all above the specification minimum. There are many methods to compare data sets to see if they are the same population, some of which are described in MMPDS. In this example, the high scatter in UTS strongly suggests that the process is different from that used to develop the design values, and the machine would fail OQ if consistency were one of the criteria.

Part Qualification

While prototyping, risk reduction, and pre-production development for a part may be performed prior to machine qualification, part qualification may only take place on a machine that has been qualified, except in instances where the engineering authority allows simultaneous machine and part qualification. In part qualification, the user is demonstrating that the combination of the feedstock, machine, general build process (as documented in the PCD), and the part-unique processes produces a part that not only conforms to the general part specification, but also to the Engineering requirements for the part. This part qualification can consist of 3 to 5 activities:

Process Qualification / Validation

The previous paragraphs described a qualification structure where machines are qualified, and then each individual part is qualified. While this approach is appropriate in many cases, it can be quite costly and time-consuming, especially when a machine or cell of identical machines is making a large number of similar parts. An alternate approach, commonly known in the medical industry as Process Qualification (PQ) or Process Validation (PV), and referred to in other industries as a Part Family approach.

Consider the manufacture of implants for hip or shoulder replacements. The shape of the typical implant stem is basically the same: A ball-like structure is connected to a curved stem, which attaches to the femur (in a hip replacement) or the humorous (in a shoulder replacement). From there, however, there is a good degree of variance: Surgeons need stems of various lengths, thicknesses, and with different surface details (ribbing, texture) for the purposes of attachment. As a medical replacement part, it can be imagined that a manufacturer would need multiple machines to meet the production rate on most of the parts, which will be discussed later. The desirability of performing PQ on this family of parts is apparent. Instead of qualifying each part individually, the manufacturer could develop a common process (location and orientation in the build chamber, build parameters, post-build thermal processing, finishing), and qualify by testing the extremes and the middle of the process windows. As new variants are added, a lower level of qualification (potentially just NDT, dimensional, and a single destruct article) may be needed.

Figure 6.4 A shoulder implant, featuring the familiar shape of the ball at one end, which is inserted into the shoulder’s ball-and-socket joint, and the stem that is implanted in the upper arm bone. Hip implants have a somewhat similar shape. Both types of implants must be manufactured in a range of sizes and lengths to accommodate the wide array of sizes of people’s bodies and bone structures. But each size must maintain the same level of quality, strength, and material makeup to ensure long-term effectiveness and no adverse reactions. (credit: Modification of “humerous endoprothessis-36” by Raimond Spekking/Wikimedia Commons, CC BY-SA 2.0)

Personnel Qualification

Intrinsic to having a facility qualified, or to have an expectation of success on a machine or part qualification, is having qualified personnel. Like many other industries, such as welding, personnel qualification consists of the operator or operators undergoing a series of classroom (increasingly virtual or web-based) and practical training on the equipment they will be qualified on. A certain number of on-the-job hours running the equipment under the supervision may also be required. After completing training, they will then have a series of oral, written, and practical (on-machine) examinations. After successfully passing these examinations, the operator certified to operate the equipment. In some instances, especially for critical parts or ones that require skilled manual operations, an additional level of qualification for that part may also be necessary. Like many of the skilled trades, AM levels of qualification could look like:

• Apprentice – Able to operate equipment under supervision of a Journeyman or Master. Generally, not qualified for critical parts.
• Journey-level – Able to operate equipment without supervision. May supervise apprentices. Could be qualified for critical parts.
• Master level – Able to operate equipment without supervision and supervise apprentices. Qualified for critical parts. Responsible for other aspects of an AM facility, such as approving build files, signing off on maintenance, administering qualification of Apprentices and Journey-level.

Some industries, professional groups, and higher education institutions have been reconsidering the use of “master” to indicate authority over other people, due to the term’s deep association with slavery. Nearly all states, certifying bodies, and college systems have decided to continue using the term, but those involved in the field should consider the implications of its use and be conscious of the context and connotations of the word.

Some skilled workers, such as plumbers and electricians, are certified in reaching these levels through state-specific evaluation and experience standards. (In other words, to progress from journey level to master level requires specific evidence of experience.) Others, such as machinists, often do not have such formalized levels, but workers can still showcase their experience and qualifications through evidence and resumes. AM may fall into the latter, less formalized category, but the field is growing significantly and there will be plenty of opportunities for career progression.

Feedstock Qualification

Without feedstock that reliably meets specification, any manufacturing process will be hard-pressed to repeatably produce hardware that meets requirements, hence the need to qualify feedstock. The use of a qualified feedstock is almost always a requirement for OQ, Part Qualification, or PQ. The specification the covers printing and processing the AM parts will generally require the use of feedstock qualified to a specification. ASTM and SAE (AMS Series) are issuing a series of feedstock specifications for many of the commonly used powders, wires, and filaments. Where a public specification does not exist, or where different controls are needed, organization will issue their own proprietary specifications.

Feedstock specifications contain requirements for items such as the following:

• Classifications – Sub-categories
• Chemistry – Intentional Constituents
• Chemistry – Unintentional Constituents or Contaminants
• Method of Manufacture
• Product Form and Size – Powder size distribution, wire/filament diameter, foil thickness
• Physical Properties – Density, Flow Rate, etc
• Types and Frequency of Testing
• Packaging, Safety, Handling, Labeling, etc.

Many of these specifications also contain qualification requirements or call out another specification that covers qualification. Qualifying a feedstock to a specification generally requires sampling a certain number of feedstock lots (often 3 or more) and testing them against the requirements in the specification. If all of the lots pass, then a feedstock supplier will be considered qualified. Additionally, each lot of material that is produced will be individually qualified to meet the specification. A powder lot that passes the qualification tests, can then be certified to meet the specification, as mentioned above.

Each company and industry has its own approach to who can be qualified against a feedstock specification. Industries that produce more critical hardware (aerospace, medical) will often only qualify the factory/plant where the feedstock is made from a precursor (monomer for polymers, bar stock for wire or powder, etc.). For critical parts, the facility that produces the precursor and its precursors (e.g. ingot or sponge for titanium powder) may need to be qualified. In other industries and applications, a distributor (facility that procures feedstock and re-sells to users) may be qualified.

Qualification of Additional Facilities, Machines, Personnel, and Feedstock Sources

The simplest and lowest-cost method of qualifying an additional feedstock source is to perform the same qualification tests (3 lots, etc) that were done for the original source. In some cases, however, the engineering authority may want to expand upon that qualification, which would include the building and testing of hardware. This may be driven by the criticality of the part. Another reason, especially in an emerging industry like AM, is to ensure that what is nominally the same feedstock from a different source also performs the same in processing. The extent of this additional qualification can range from building and testing and IQ test part, to one or more OQ builds and testing, to repeating the qualification testing for each part.

Like additional feedstock sources, there is a broad range for qualification of additional machines. This range is also highly influenced by the similarity of the additional machines to the initial machines. The range of similarity can be expressed by the categorizations listed below.

• Identical – Same model number with exact same hardware, firmware, and software
• Very Similar – Same model number but slight differences in hardware, firmware, or software that do not influence the operation of the machine. Examples of this would be a larger spool for filament, or different software for uploading the build log file to memory.
• Similar – Same model number but differences in hardware, firmware, or software that could influence the operation of the machine. Examples of this would be a larger heater tip for improved thermal stability, or a redesigned galvanometer that has higher reliability.
• Not Similar – Same model number but differences in hardware, firmware, or software that will influence the operation of the machine. Examples of this would be a different laser, process control software, etc.
• Different Machine – Different model number and design.

While all of the machines would receive a full IQ, OQ or PQ could range from a full repeat of the OQ for the first machine (definite for a different machine, with a demonstration of equivalent material property values and scatter), to something abbreviated, such as a single build and a check of the material properties against the specification (Identical or Very Similar). Likewise, part qualification could range from full component and/or destructive testing (different machine) to first article testing without a destruct (similar or very similar) to part acceptance testing only (identical). Additionally, the criticality of the part will play a role, with more critical parts requiring more intensive qualification.