Engineering teams should weight concept selection criteria based on how directly each criterion affects the project’s primary objectives, customer requirements, and technical constraints. Criteria tied to non-negotiable performance thresholds or safety requirements typically carry the highest weights, while secondary preferences carry less. The sections below work through the mechanics of weighting, who should set it, and when to revisit it.
What factors determine the weight of each selection criterion?
The weight assigned to a selection criterion should reflect how much that criterion influences whether the final design succeeds or fails. Criteria connected to mandatory performance targets, regulatory compliance, or customer-defined requirements generally receive the highest weights. Criteria that represent preferences or optimizations, rather than hard requirements, receive proportionally lower weights.
Several factors feed into this judgment. First, consider the consequences of failing to meet the criterion: if a compressor stage must hit a specific pressure ratio to make the overall engine cycle work, that criterion is non-negotiable and deserves a high weight. Second, consider how distinguishable the candidate concepts are on that criterion. A criterion on which every concept performs identically contributes nothing to the decision, regardless of its theoretical importance, and can often be set aside or given a reduced weight for the purposes of comparison.
Third, consider the cost of measurement and verification. In aerospace engineering and other high-precision domains, some criteria are expensive to validate experimentally. Teams sometimes underweight criteria that are hard to measure, which introduces bias. Explicit discussion of measurement feasibility during weighting helps prevent this.
Finally, the stage of the project matters. Early in development, weights should reflect strategic priorities. Later, when test data is available, weights can be grounded in measured performance rather than estimates.
How does a weighting matrix work in concept selection?
A weighting matrix assigns a numerical importance score to each evaluation criterion, then multiplies those scores by the performance ratings given to each concept. The result is a weighted total for each concept that allows direct comparison across options that perform differently on different criteria.
The process works as follows. The team first lists all evaluation criteria in rows. Each criterion receives a weight, typically expressed as a percentage of a total of 100, or as a value on a fixed scale such as 1 to 5. Each concept is then rated against each criterion, again on a consistent scale. The weight and the rating are multiplied to produce a weighted score for that criterion-concept pair, and all weighted scores for a concept are summed to produce its total.
The Pugh matrix is the most widely used form of this approach in engineering design. In its original form, a Pugh matrix uses a reference concept rather than numerical ratings, marking each candidate as better, worse, or equivalent to the reference on each criterion. Weighted variants extend this by applying numerical scores, which gives the method more resolution when concepts are closely matched.
The matrix does not make the decision automatically. It structures the judgment so that trade-offs are visible and the reasoning behind a selection can be reviewed, challenged, and documented. This is particularly valuable in complex development programs where multiple stakeholders need to understand why one concept was chosen over another.
Who should be involved in setting criterion weights?
Criterion weights should be set by a cross-functional group that includes the customer or end-user representative, the lead systems engineer, and subject matter experts for the domains most relevant to the design problem. No single person should set weights unilaterally, because individual perspectives systematically overweight the criteria closest to their own specialism.
In practice, this means bringing together people who understand what the customer actually needs, people who understand what is physically achievable, and people who understand the cost and schedule implications of different design choices. In gas turbine development, for example, the aerodynamicist, the structural analyst, the manufacturing engineer, and the program manager will each have legitimate but different views on which criteria matter most.
A facilitator who is not directly invested in any particular concept can help the group reach agreement without letting the most senior voice dominate. Techniques such as pairwise comparison, where each criterion is compared directly against every other criterion to establish relative importance, reduce the tendency for participants to simply assign high scores to everything they care about.
What’s the difference between absolute and relative criterion weighting?
Absolute weighting assigns each criterion a score independently, based on its inherent importance to the project, without reference to the other criteria. Relative weighting ranks or scores criteria in comparison to each other, so that the total weight across all criteria is constrained to a fixed sum, typically 100%.
Absolute weighting is simpler to apply but prone to inflation. When team members score each criterion independently, there is a natural tendency to rate most criteria as highly important, which compresses the differences between them and reduces the method’s ability to distinguish between concepts. It also makes it harder to compare weighting decisions across different projects or teams.
Relative weighting forces trade-offs. If one criterion receives a higher weight, others must receive less. This constraint makes the team’s priorities explicit and produces a more honest representation of what actually matters most. Pairwise comparison is one structured way to derive relative weights: each criterion is compared one-to-one against every other criterion, and the results are tallied to produce a ranked, proportional weighting.
For most engineering concept evaluation tasks, relative weighting produces more useful results because it reflects the reality that resources, performance, and attention are finite. A design cannot simultaneously optimize for everything, and the weighting scheme should reflect that.
How do you handle conflicting weights from different stakeholders?
Conflicting weights from different stakeholders should be treated as information rather than a problem to be smoothed over. Disagreement about how to weight criteria usually signals a genuine difference in understanding of project priorities, customer requirements, or technical constraints, and that disagreement needs to be resolved explicitly before the evaluation proceeds.
The first step is to surface the reasoning behind each stakeholder’s proposed weights. Often, conflicting weights reflect different assumptions about what the customer actually requires, or different assessments of technical risk. Making those assumptions explicit allows the group to address the underlying disagreement rather than simply averaging the numbers.
Where genuine value differences exist, the customer or program owner should have the final say on criteria that relate to product requirements. Technical experts should have more influence over criteria that relate to feasibility and risk. Documenting who made the final weighting decisions, and why, protects the team if the rationale is questioned later in the program.
If consensus is genuinely unreachable, running the evaluation with two or three alternative weighting schemes can be informative. If the same concept comes out ahead under all reasonable weighting scenarios, the team can proceed with confidence. If the result is sensitive to weighting choices, that sensitivity itself is a signal that the decision warrants more investigation before committing to a concept.
When should criterion weights be revised during a project?
Criterion weights should be revised when the project’s requirements change, when new technical information materially affects what is achievable, or when the competitive or regulatory context shifts in ways that change what the customer values. Weights set at the start of a project are based on the best available information at that time and should not be treated as permanent.
In practice, the most common triggers for revision are customer feedback that reveals a misunderstanding of requirements, test results that show a previously assumed performance level is not achievable, and program-level decisions that change cost or schedule constraints. In testing and validation programs, measured data frequently challenges assumptions that were made during early-stage concept selection, and the weighting scheme should be updated to reflect what has been learned.
Revisions should be made deliberately and documented carefully. Changing weights after concepts have been evaluated, without clear justification, can introduce bias toward a preferred outcome. The team should record what changed, why the change was warranted, and what effect it has on the evaluation results. This discipline keeps the process credible and auditable.
A practical approach is to schedule a formal weighting review at each major project gate, rather than waiting for a crisis to prompt revision. This makes weight revision a normal part of the design decision-making process rather than an exception that invites suspicion.
How AneCom supports engineering concept evaluation
AneCom AeroTest works directly alongside engineering teams at the point where concept selection decisions need to be grounded in real performance data. For gas turbine and aero-engine development programs, that means providing the experimental evidence that makes criterion weighting concrete rather than speculative. Specifically, AneCom offers:
- Aerothermal component testing for compressors, combustors, and turbine parts, producing validated performance data that directly informs how criteria such as efficiency, pressure ratio, and thermal loading should be weighted
- Fan system acoustic testing in Europe’s largest anechoic chamber, where noise performance can be measured under controlled free-field conditions, removing uncertainty from noise-related criteria in the evaluation matrix
- Instrumentation and data acquisition across all test vehicles, with a modular measurement system across three test benches that supports high-resolution data collection for complex, multi-criteria evaluations
- Engineering services from design and analysis through to assembly and testing, available from a single source and at customer sites worldwide, covering the full range of gas turbine development needs
If your team is working through a concept selection process and needs experimental data to anchor your weighting decisions, contact AneCom AeroTest to discuss how testing can be integrated into your development program.
