Applied Test and Evaluation

Course length:

3 Days



Course dates

Interested in attending? Have a suggestion about running this course near you?
Register your interest now

Want to run this event on-site? Enquire about running this event in-house


Test and evaluation (T&E) provide the means to ensure that a product performs in the intended way and with the intended results. Testing starts very early in a product system development, however, because test planning is dependent on good requirements. This course teaches the details of test and evaluation from product concept through operations, with a view toward application in large system development. The course is highly practical, helping students to understand not only what must be done, but why it is necessary. Students learn about a stakeholder view of test and evaluation, the use of requirements as the primary staypoint for verification, planning a test program in terms of strategy, plans, and procedures, the statistical analysis of a test prior to performing the test, the differences between integration testing and final testing, and how to conduct tests at different phases of product/system development, production and support. Practice the skills on a realistic product system. Design a test program for a typical product system. Build the robotic system. Implement the test program. Analyze the results. The Robotic Test Challenge Exercise gives students the opportunity to practice the skills taught in the course from end to end during development and support of actual operating robots. The exercise happens in four segments throughout the course, encompassing a total of six hours in highly practical learning.

Follow this link for a short course overview video.

Who Should Attend:

  • Test Engineers
  • Design Engineers (any engineering discipline)
  • Systems Engineers, Project Engineers; Technical Team Leaders
  • System Support Leaders
  • Technical and Management Staff
  • Project Managers

What You Will Learn:

  • The roles of T&E in development and support
  • Nine basic principles of testing
  • A useful life-cycle model for T&E
  • How to analyze product/system requirements into test requirements that satisfy stakeholders
  • T&E development: strategy, plan, procedures
  • Test design and analysis; statistical methods
  • System integration testing and how it differs from product/system final testing
  • Test methods, test control, test results

Course Outline:

  1. Test & Evaluation Overview (1:30) – An overview of test and evaluation (T&E) principles and methods for simple products and complex systems, including T&E tasks from beginning to end of a project. Basic definitions and concepts for the course, including the differences in test, evaluation, verification, validation, developmental testing (DT&E or “alpha”), and operational testing (OT&E or “beta”). Test and evaluation relationship to other engineering and management disciplines. Cost-effective T&E, and the cost of quality. Nine basic principles of testing.
  2. Test and Evaluation In the Life Cycle (1:30) – Roles of test and evaluation throughout product development and support. A useful model of typical test activity phases (requirements, planning, conduct, analysis & reporting) and their correlation with program phases. How test concepts fit into special life cycles: commercial product development, US Department of Defense acquisitions under DoD-5000, evolutionary development, iterative development, and agile development.
  3. Developing Test Requirements (2:30) – Requirements as the primary method for measurement and control of product development. Where requirements come from in the systems engineering paradigm. Types of requirements, including traditional, model-based and agile requirements. Stakeholder values in the form of Measures of Effectiveness (MOE) and Measures of Performance (MOP). How to develop a requirements verification matrix (RVM); verification methods (Inspection, Analysis, Demonstration, Test). Test requirements differences for prototypes, first articles, production and support.
  4. Designing a Test and Evaluation Program (2:30)– Understanding stakeholder needs for T&E information Creating a T&E strategy by evaluating the product/system development concept in context of the stakeholder needs. An effective outline of T&E strategy topics, including planning for developmental and operational testing phases. An example: the US DoD “Test and Evaluation Master Plan (TEMP)” Trade-offs involved in test decisions; maturity at different phases; level of detail. What is a “verification event”? Converting the strategy to a T&E plan defining specific verification events in terms of requirements tested, time frame, equipment/skills needed, duration, and goals. Identification of test enabling products early enough to affect the development program. Modeling and simulation for test planning. Risk management as a test planning tool.
  5. Designing Tests and Evaluations (3:30) – The test procedure as a control for each verification event. Effective topics to include in a test procedure, as an outline for developing the test. Identifying the issues and goals in each verification event. Determining the requirements to include, and what not to include. “Black box” input/output analysis, choosing what to measure, and identification of observability issues. Affecting the product/system design to improve testability. Logical sequencing of the test procedure based on product/system states, input controls, and observable measurements. Review of probability and statistics principles including probability distributions. Analyzing expected variation in test data, statistical design of tests, sampling principles, selecting useful statistical methods, design of experiments, common statistical errors.
  6. Integration Testing (1:00) – How to successfully manage the intricate aspects of system integration testing; level of integration planning; managing complex system integration; work-arounds. Development test concepts; five types of integration test planning; preferred order of events; component testing; conducting integration tests for complex systems; work-arounds for anomalies and failures.
  7. Test Conduct (2:00) – How to perform testing; differences in testing for prototypes, first article qualification, recurring production acceptance, support; rules for test conduct. Testing for different purposes, verification vs. validation; test records; prerequisites management; test readiness certification, test constraints, test article configuration; troubleshooting and anomaly handling; measures of success and indicators of difficulty; test tools. Test failure analysis.
  8. Robotic Test Challenge (6:00) – A hands-on class exercise in small groups, handled in four major segments. Part A analyzes a system concept and requirements, developing an RVM and specific test requirements,. Part B creates an effective test program and test procedures for the product system, including identifying any necessary test equipment to be developed. Part C builds the robotic systems per assembly instructions, as well as building any identified test equipment. Part D implements the test program to evaluate the final robots. 


Dr. Eric Honour, CSEP, international consultant and lecturer, has a 40-year career of complex systems development & operation. Former President of INCOSE, selected as Fellow and as Founder. He has led the development of 18 major systems, including the Air Combat Maneuvering Instrumentation systems and the Battle Group Passive Horizon Extension System. BSSE (Systems Engineering), US Naval Academy; MSEE, Naval Postgraduate School; and PhD, University of South Australia.


Dr. Scott Workinger has led innovative technology development efforts in complex, risk-laden environments for 30 years in the fields of manufacturing (automotive, glass, optical fiber), engineering and construction (nuclear, pulp & paper), and information technology (expert systems, operations analysis, CAD, collaboration technology). He currently teaches courses on program management and engineering and consults on strategic management and technology issues. Scott has a B.S in Engineering Physics from Lehigh University, an M.S. in Systems Engineering from the University of Arizona, and a Ph.D. in Civil and Environment Engineering from Stanford University.


REGISTRATION: There is no obligation or payment required to enter the Registration for an actively scheduled course. We understand that you may need approvals but please register as early as possible or contact us so we know of your interest in this course offering.

SCHEDULING: If this course is not on the current schedule of open enrollment courses and you are interested in attending this or another course as an open enrollment, please contact us at (410)956-8805 or Please indicate the course name, number of students who wish to participate. and a preferred time frame. ATI typically schedules open enrollment courses with a 3-5 month lead-time. To express your interest in an open enrollment course not on our current schedule, please email us at

For on-site pricing, you can use the request an on-site quote form, call us at (410)956-8805, or email us at