Fort Belvoir, Va –
For over four decades, Alison Vuille has helped direct the quality assurance process for the Army’s sensors, goggles, night vision devices and numerous other platforms and programs. Vuille has spent the last two decades improving the PEO Soldier portfolio. She currently leads the Quality Assurance section of the Project Manager Soldier Warrior (PM SWAR) Quality Assurance-Test Directorate. In recent years, she helped refine the Enhanced Night Vision Goggle-Binocular (ENVG-B), Joint Effects Targeting System (JETS) and the Family of Weapons Sights (FWS) as these products moved through development, production, testing, and fielding.
During the 1980s and 1990s, Vuille performed quality assurance on sensors, night vision goggles, and night sights mounted on tanks during her tenure at then-Night Vision Lab and Project Manager Night Vision Reconnaissance, Surveillance, and Target Acquisition and served as a liaison between Army Communications-Electronics Command Research and Development Center (CECOM RDEC), the Natick Soldier Systems Center in Massachusetts and Project Manager Soldier.
After PEO Soldier stood up in 2002 and absorbed many of the Army’s Soldier sensor programs, Vuille transitioned into the command shortly after, providing her expertise on the development of products and equipment that were in high demand, due to the Army’s increased operational tempo overseas in support of operations in Iraq, Afghanistan, and around the world.
Vuille, who retired as a full-time government civilian in 2022, was brought back to PM SWAR to helped ensure continuity after Project Manager Soldier Maneuver and Precision Targeting (PM SMPT) merged into the then-Project Manager Integrated Visual Augmentation System (PM IVAS, now PM SWAR) during PEO Soldier’s reorganization in the summer of 2022.
PM SWAR’s leadership recognizes Vuille’s significant contributions to the development of improved products and equipment for the force.
“Alison is the cornerstone of our quality and test programs here at Project Manager Soldier Warrior,” said SWAR Deputy Project Manager Mark Stephens. “Our products delivered to Soldiers are directly tied to her 40 years of experience. She built our testing and quality standards based on MIL STD [Military Standard] 810 and is our zealot for accepting a quality product. Her Quality Assurance mentoring both to government and industry partners made our defense industry stronger.”
Currently Vuille is focused on updating SWAR’s quality/test orientation training and the Quality Assurance Standard Operating Procedure, as well as internal training programs.
Vuille shared her thoughts about her experiences as a quality assurance professional and why quality assurance is essential to developing better equipment for the force by answering a few questions.
—-
Q. What is Quality Assurance?
A. Quality can be defined as the degree to which the products we buy for Soldiers satisfies the user and specification requirements, as well as customer expectations. My definition of Quality Assurance is the process by which we ensure a high-quality product.
Q. What is the origin of Quality Assurance?
A. While the origins of Quality could be traced back to the formation of craftsmen guilds in medieval times, it became more important during the industrial revolution, and continuing through World War II. The factory environment created a much greater need for quality concepts, principles, and tools in order for manufacturers to build high quality products and be competitive. Throughout the 20th century, pioneers such as Deming and Juran promoted quality concepts, analysis, and implementation. Employment of quality practices are the reason Japanese cars became dominant in the automotive marketplace, and eventually US manufacturers followed suit. You might remember the term Total Quality Management, which was fully embraced as an overarching principle within the federal government, and after that Lean Six Sigma came in focus. An international standard for Quality, ISO 9001, was established in the latter part of the 20th century, and many manufacturing companies are certified to this standard.
Q. How did you learn about Quality, Reliability, and Testing?
A. The Army used to have an intern training center at Red River Army Depot for Quality and Reliability Engineering. I was recruited from the university I attended and spent one year covering a wide variety of associated subjects, followed by on-the-job training at the Aviation School. After transferring to the Night Vision Lab, I also took a number of technical training courses in reliability testing, environmental testing, environmental stress screening, ISO 9001 quality management, parts screening, sensor technical training, and many others, as well as courses at Defense Acquisition University.
Q. Why do you implement Quality Assurance in Army Acquisition?
A. The Army has a regulation regarding Quality, AR 702-11, and it is actually a requirement to implement Quality in Army Acquisition. Quality applies across all aspects of acquisition. The Army Regulation is principle oriented rather than process, and requires practices such as customer requirements assessment, fact-based decision making, effective root cause corrective action for failures, independence of quality personnel, use of metrics and continuous improvement, and risk-based management. From a personal standpoint, I implement Quality because I am passionate about ensuring only high-quality systems are fielded to the Army. Soldiers deserve high quality, reliable equipment in their fight for our country and freedom across the world.
Q. Why is it important to conduct environmental tests?
A. Environmental tests are a subset of Developmental Tests. Our Soldiers have very little ability to control most of their environment, so their equipment must operate in whatever environment they need to fight in. I call the environments the silent Key Performance Parameter (KPP), as they can impact whether the system meets a critical function during operational use. We must at least ensure Army systems are tested in critical environments before we field, at least to the extent that we can tell Soldiers the system limitations. Examples of limitations might be the system can’t be immersed in water, operate in winds greater than 20 mph, or won’t function below a certain temperature. Limitations must be accepted by the user representative. Military and commercial standards have been created to emulate and characterize environments, threats, and other situations that equipment may encounter. Use of standards and common test methodologies are what allow the development of facts to evaluate equipment performance.
Q. Why did we come up with CDRLs (Contract Data Requirement Lists) and DIDs (Data Item Descriptions) for testing and Quality Assurance?
A. CDRLs and DIDs for Quality are generally required so that there is sufficient information in test plans and procedures to properly evaluate the specified performance, ensure applicable standards are followed, as well as document Government concurrence on the test plans and procedures. Also, that the test report has sufficient evidence that proves a given system meets its requirements. By defining areas that should be addressed, CDRLs and DIDs for failure reports ensure contractors have done a thorough job addressing the failures and allow the government to better understand whether a root cause was determined and evaluate the effectiveness of the corrective action accomplished by the contractor.
Q. What training did you have to research and send folks to in order to implement Quality Assurance?
A. At the time I started my career in 1980, the Army had a one-year certificate program at Red River Army Depot in Quality and Reliability Engineering for basic training of new interns in acquisition, with recruitments at universities. There was also a shorter Quality Assurance certification program at Rock Island. Until recently, there was a DAU certification career program in Production and Quality Management (PQM), which was essentially the career path for a Quality professional in the government. DAU discontinued the PQM certification, but they still have basic training in quality in several courses that are mandatory across the entire functional area workforce. They also offer online courses as part of their College of Contract Management offerings. DAU is currently working on additional courses with a manufacturing and quality focus. For my team, in addition to DAU courses, I have periodically brought courses to Belvoir applicable to their work in various topics such as Quality Engineering, Auditing to ISO 9001, Environmental Test Methods, Electromagnetic Environmental Effects and Testing, and short courses in Reliability, among others.
Q. What was your most difficult challenge in Testing and Quality Assurance, and what obstacles do you have to overcome to have a successful Quality and Test team?
A. The most difficult challenge is balancing fielding equipment fast with the desire to field a high-quality product to our Soldiers. We want to field new capabilities as fast as possible so Soldiers have the latest and greatest equipment to help achieve mission success, but it would be doing Soldiers a disservice if that equipment is unreliable or does not meet required performance. There is a balance in quality versus fielding. We would never field any equipment if we were to wait until the system is perfect and without issues. The key is to gather facts from operational, developmental, and production test data, and then identify and manage risks. This problem is particularly prevalent in development of complex sensors. One of the primary obstacles is the perception that requirements for Quality will always cost more and negatively impact schedule. This may sometimes be the case, as Quality professionals are focused on performance of the system rather than cost or schedule. But as many may recall, the majority of a system’s life cycle cost is in sustainment, so efforts up front to ensure a high-quality and reliable product can reap many benefits in reduced cost over the life cycle.
Q. How do you ensure Quality and reliability are designed into the product?
A. To deliver a high-quality product, the most efficient means is to start with a good design and build process. Development should include all the appropriate system engineering tools and analysis to ensure the design meets the specified requirements, and design reviews should address the results of analyses such as thermal analysis, stress analysis, finite element analysis, tolerancing, etc., and identify how they influenced the design. All that said, no design is perfect, and there will likely be trades and faults identified, which is where a manufacturer’s failure reporting analysis and corrective action system (FRACAS) becomes critical.
Q. Why is a FRACAS system so important?
A. Effective FRACAS is a basic requirement of a Quality Management System. Since no design is perfect, and failures frequently occur during manufacturing and qualification as well as in operational use, addressing failures in a robust manner is critical to eliminating those failures and improving product performance and reliability. Effective FRACAS is critical to the Army in order to meet requirements and reduce costs over the product’s life cycle, as well as to the manufacturer to control production costs. About 20 years ago, the Army did a study which illustrated most systems that were subjected to a formal operation test failed to meet the system operational reliability requirement. That stimulated the Army to place a greater emphasis on FRACAS, design for reliability and reliability growth testing, as well as other disciplines such as Physics of Failure, Highly Accelerated Life Testing (HALT) and Highly Accelerated Stress Screening (HASS).
Q. Why do reliability growth testing?
A. Not all equipment needs to have reliability growth testing, but some of our more complex sensors have had a hundred or more failures in their first phase of development and operational testing and did not initially meet the required reliability. Because corrective actions are typically not completely effective and sometimes unearth new problems, it may require several phases of testing followed by a corrective action period to identify and resolve failures in the design and manufacturing process. Realizing this allows us to better plan a program schedule that will allow sufficient time to resolve issues found and allows the design to incrementally improve and meet the reliability requirement. Our contracts require a failed item analysis report (FIAR) to document resolution of each failure and ensure a closed loop process; the FIAR process can be painful and sometimes take a great deal of time to resolve, significantly impacting schedule.
Q. Why do you continue to conduct conformance inspection testing throughout production?
A. Manufacturers are continually improving their design and process, developing new suppliers, training new assemblers, buying new parts and materials, and there are a multitude of other things which could result in issues that are not immediately apparent. In my experience, issues continue to arise, some catastrophic, even after many years of production. No matter how much the prime believes they have supplier control, suppliers sometimes make changes they believe would not impact form, fit, and function, and fail to notify the prime contractor. To maintain a high level of quality requires some level of production testing against critical performance tests, reliability, and environments that are subject to variation. For high volume production, we frequently require lot testing on a monthly basis. As production continues, the level of testing can be reduced or adjusted, or even eliminated, based on results, evidence of process control and good field reliability performance.
Q. Tell me more about the FIAR process.
A. Resolving FIARs in both development and production phases is the most critical aspect of our job in Quality Assurance in fielding high-quality and reliable equipment. The first thing we ask ourselves when confronted with a failure is, “how will this failure impact the Soldier?”. We are particularly relentless at resolving any failures that will cause the Soldier to abort his mission or turn in his equipment. Resolution of the failure requires our contractors to dig into why the failure occurred and find the root cause, determine a corrective action, and prove the corrective action is effective and does not cause any other issues. This process can hold up production, affect a contractor’s money stream, sometimes for long periods, and therefore can be very contentious. The government conducts FIAR Risk Assessments as needed to make decisions on acceptance and fielding of product. In the long run, there are positive effects for both Government and contractor. Fewer defects means improved field reliability for Soldiers and less rework and scrap costs, and higher production rates, for the contractor.
Q. Should Quality be implemented in COTS (Commercial off-the-Shelf) products?
A. In the past, I have had some people suggest that Quality does not apply to COTS products. The fact is that Quality principles apply across the board on all products the Army purchases, and across all disciplines, whether the product is developed by the Army, COTS, GOTS (Government off-the-shelf) or modified COTS. Any COTS product must be first evaluated to ensure it is effective, suitable, and survivable, and meets Soldiers’ needs, to include required performance in the operational environmental in which it will be used. Any limitations should be identified, and agreement reached with user representatives on whether those limitations are acceptable in a fielded product. Let me just give you a broad example. Let’s say the Army needs a widget to perform a certain function, and in the commercial world there are some that would suffice, but the COTS products were only designed to operate in a limited band around room temperature. If the Army needs to use this product in Alaska or various other places during the winter, as well as in spring or fall, this product will not meet the Army’s needs. This may result in the user community requiring a modification, such as a heater, or an entirely new Army developed product to meet the requirements.
Q. Why should Quality Assurance be independent of the Product Manager?
A. Quality and test personnel must be free to identify areas of concern regarding performance without the pressure applied from cost and schedule. This enables a thorough understanding of the issues and risks to the Product Manager and Program Manager. While Program and Product Managers maintain the ultimate decision authority to procure and field equipment, the Quality Assurance team provides independent feedback on the product quality, reliability, and ability to meet test requirements that is used to understand risks and make decisions.
Q. Overall, from all your service over the last 40 years, what do you consider the most important lessons learned from a Quality Assurance perspective?
A. My team and I have a long list of detailed lessons learned over the years that we review periodically and keep adding to. But the biggest lesson is more strategic. Recently we have tried to field systems faster, without having resolved many of the failures experienced in testing prior to entering production. The result has been that too many defects perpetuate into production, production is constantly being interrupted to address failures and rework product, and in some cases field reliability is negatively impacted, which can result in Soldiers losing confidence in your equipment. The best way to a healthy production line is to fix problems early during development, eliminate failure modes with a robust FIAR process, test sufficiently to validate that your problems have been resolved, and don’t go into full rate production until you have reduced the level of defects to a very low level.
By Frederick Shear Army