Engineers evaluate repeatability and reproducibility through structured measurement system analysis, most commonly a gauge R&R study. This process quantifies how much variation in test results comes from the measurement system itself rather than real differences in the component being tested. The sections below address the specific questions that arise when applying these methods to aerothermal component testing.
What are the accepted metrics for quantifying test repeatability?
Repeatability is quantified by measuring the same component multiple times under identical conditions and calculating the variation across those measurements. The standard metrics are the standard deviation of repeated measurements, the percentage of total variation attributed to the measurement system, and the signal-to-noise ratio between part variation and measurement variation.
In practice, engineers express repeatability as a percentage of tolerance or as a percentage of total observed variation. A measurement system is generally considered acceptable when its contribution to total variation falls below 10%, marginal between 10% and 30%, and inadequate above 30%. These thresholds come from the AIAG Measurement System Analysis guidelines, which are widely adopted across industries that depend on precise experimental data.
For aerospace component testing, engineers also track the coefficient of variation across repeated test runs, since gas turbine components operate under tightly controlled thermodynamic conditions where even small measurement drift has engineering significance. Absolute uncertainty values expressed in physical units, such as pressure in Pa or temperature in K, are reported alongside percentage-based metrics to give OEM customers a complete picture of data quality.
How does reproducibility differ from repeatability in engineering tests?
Repeatability measures variation when the same operator uses the same instrument to test the same part under the same conditions. Reproducibility measures variation when different operators, different instruments, or different test rigs produce results for the same component. Repeatability is a property of the measurement system; reproducibility is a property of the broader testing process.
The distinction matters because the two sources of variation require different corrective actions. Poor repeatability usually points to instrument instability, signal noise, or inconsistent test conditions within a single run. Poor reproducibility often points to differences in operator technique, calibration offsets between instruments, or systematic differences between test facilities.
In aerothermal component testing, reproducibility is especially relevant when a component is tested at one facility for aerodynamic performance and at a second facility for acoustic or thermal behaviour. If the two data sets need to be compared or combined, the reproducibility between the facilities must be characterised and documented before any engineering conclusions are drawn from the combined dataset.
What causes poor repeatability in aerothermal component testing?
Poor repeatability in aerothermal testing typically originates from four sources: unstable boundary conditions during the test, sensor drift or degraded instrumentation, mechanical variation in how the test article is installed, and data acquisition noise introduced by the measurement chain.
Boundary condition instability is the most common cause in compressor and turbine testing. Small fluctuations in inlet total pressure, inlet temperature, or rotational speed between repeat runs translate directly into variation in measured performance parameters. Test rigs that cannot hold these conditions within tight tolerances will show poor repeatability even when the instrumentation is functioning correctly.
Sensor degradation is a subtler problem. Pressure taps can partially block, thermocouples can drift, and strain gauges can lose adhesion under thermal cycling. When sensors degrade between repeat measurements, the variation is attributed to the test rather than the instrumentation, which can mask the true source of the problem. Regular calibration checks before and after each test run are the standard way to detect this.
Installation variability also contributes. In compressor testing, small differences in how a rotor or stator stage is assembled into the rig, such as tip clearance variation or seal contact pressure, can produce measurable differences in performance between nominally identical test runs. Documented assembly procedures and post-installation inspection records are essential for ruling out this source of variation.
How is a gauge R&R study structured for gas turbine test rigs?
A gauge R&R study for a gas turbine test rig follows the same statistical structure as in manufacturing: multiple operators measure multiple parts multiple times, and the resulting data is decomposed into variation from repeatability, variation from reproducibility, and variation from actual part-to-part differences. The specific implementation is adapted to the realities of large-scale aerothermal testing.
In a typical test rig study, the “operators” may be different test engineers or different shifts, the “parts” are defined operating points on the compressor map rather than discrete physical parts, and the “measurements” are the aerodynamic performance parameters recorded at each point. The study is designed so that each operating point is visited multiple times across different test sessions, with the rig fully shut down and restarted between sessions to capture the full range of setup variation.
The statistical analysis uses analysis of variance to separate the variance components. The output is a breakdown showing how much of the total observed variation comes from the measurement system and how much reflects real differences in the component or operating condition. For gas turbine testing, this analysis is often extended to include uncertainty propagation, so that the R&R results feed directly into the measurement uncertainty budget reported with the test data.
When should R&R evaluation be repeated during a test campaign?
R&R evaluation should be repeated at the start of a test campaign, after any significant change to the measurement system or test rig configuration, and at the end of the campaign as a data quality check. For long campaigns, an interim R&R check at a defined midpoint is good practice.
The triggers for an unplanned R&R re-evaluation are equally important. If data from a test run looks inconsistent with earlier results and the cause is not immediately obvious, repeating the R&R assessment before continuing is more productive than running additional test points on a potentially compromised measurement system. Similarly, if a sensor is replaced, a data acquisition module is swapped, or the rig undergoes maintenance that touches instrumented areas, the measurement system should be re-validated before the campaign continues.
In extended test programmes where the same rig is used across multiple phases, R&R data from one phase should not be assumed to carry over to the next. Changes in ambient conditions, rig wear, and hardware upgrades all have the potential to shift the measurement system’s behaviour in ways that a baseline R&R study would not capture.
How do test facilities demonstrate R&R compliance to OEM customers?
Test facilities demonstrate R&R compliance by providing documented measurement uncertainty analyses alongside test data, with the uncertainty budget explicitly accounting for both repeatability and reproducibility contributions. OEM customers typically specify acceptance criteria for measurement uncertainty in the test contract, and compliance is demonstrated by showing that the facility’s R&R results fall within those criteria.
The documentation package generally includes calibration records for all instruments used in the test, the results of the gauge R&R study performed at the start of the campaign, and a traceability chain linking each measurement back to national or international measurement standards. For high-stakes development tests, facilities may also provide raw repeated-run data so the customer’s own engineers can independently verify the repeatability claims.
Accreditation to ISO 17025 is the most widely recognised formal demonstration of a facility’s measurement competence, as it requires independent auditing of the calibration and uncertainty procedures. Beyond accreditation, OEM customers often conduct their own facility audits before a test campaign begins, reviewing the R&R procedures, data acquisition systems, and staff qualifications directly. A facility’s ability to present clear, traceable R&R documentation is one of the practical ways it builds and maintains confidence with customers over successive test programmes.
How AneCom supports measurement system analysis in component testing
AneCom AeroTest provides end-to-end testing services for aerothermal components, with measurement system analysis built into its standard test workflows rather than treated as an add-on. For OEM customers who need validated performance data for compressors, combustors, or turbine components, this means R&R evaluation is planned and executed as part of the test programme from the outset.
- Structured gauge R&R studies conducted at the start of each test campaign, with results documented in the measurement uncertainty budget
- Calibrated instrumentation with full traceability, covering pressure, temperature, flow, and acoustic measurements across the test rig
- A modular data acquisition system across three test benches, allowing consistent measurement conditions and direct comparison of results between test configurations
- Europe’s largest anechoic chamber for fan system acoustic testing, where controlled test conditions support high repeatability in acoustic performance measurements
- Engineering support for interpreting R&R results and identifying corrective actions when measurement system variation exceeds agreed thresholds
If you are planning a test campaign and need a facility that can demonstrate measurement system compliance to your quality requirements, contact AneCom AeroTest to discuss your programme.
Related Articles
- What are the most common failures in component validation testing?
- What is the difference between prototype engineering and production engineering?
- Why do mechanical designs change during detailed engineering?
- What makes rotating machinery difficult to design?
- What is detailed design in mechanical engineering?