Weighted Scoring Analysis

This post explains weighted scoring analysis (WSA) – a structured approach to evaluating alternatives using both quantitative and qualitative criteria, complete with exam questions, a worked example, and key terminology.

In a Nutshell

Weighted scoring analysis is a multi-criteria method for ranking alternatives when you need to weigh both quantitative and qualitative factors. The result is a transparent ranking based on weighted point scores.

Core Concept

WSA ranks alternatives using:

• A criteria catalog (including must-have and knock-out criteria)
• Weightings (that sum to 1.0)
• Normalized scores (typically 0–10)

The overall score emerges as the sum of weight × score for each criterion. Sensitivity analysis then checks whether the ranking remains stable if weights or scores shift.

Crucially: screen for knock-out criteria first, so unsuitable alternatives don’t inflate the final scores.

Key Exam Points

• Clearly separate must-have/knock-out criteria from nice-to-have criteria
• Weights must sum to 1.0
• Scale consistency: “higher = better” across all criteria (invert cost criteria if needed)
• IHK exam standard: document and justify both weightings and data sources
• Manipulation risk: use peer review or a four-eyes principle
• Can be paired with Total Cost of Ownership (TCO) or ROI analysis
• Full documentation required: matrix, assumptions, version history, sign-off

Key Steps

1. Define objectives
2. Build criteria catalog with knock-out conditions
3. Choose scales and utility functions
4. Select weighting method (pairwise comparison or percentage)
5. Create scoring matrix
6. Normalize and aggregate scores
7. Run sensitivity analysis
8. Visualize results (bar chart, spider diagram, or heatmap)
9. Assign roles and sign-offs
10. Sanity-check the outcome

Simple Worked Example

Alternatives: Framework A, B, C
Criteria: Performance, Maintainability, License Cost, Community
Weights: 0.35, 0.30, 0.20, 0.15
Scores (0–10 scale); cost criterion inverted
Winner: highest weighted total score

Strengths and Weaknesses

Strengths

• Combines hard and soft criteria
• Transparent decision process
• Sensitivity analysis is straightforward
• Integrates well with TCO or ROI frameworks

Weaknesses

• Weights are inherently subjective
• The scale you choose can skew results
• Compensation problem (one very strong criterion can mask a very weak one)

Typical Exam Questions (with Brief Answers)

1. What is weighted scoring analysis for? A structured way to pick between alternatives based on weighted criteria.
2. What’s the difference between must-have and nice-to-have criteria? Must-have criteria eliminate candidates; nice-to-have criteria feed into the score.
3. How do you calculate the overall score? Sum of (weight × score) for each criterion; weights sum to 1.0.
4. Why run a sensitivity analysis? To verify the ranking holds up if weights or scores change.

Key Takeaway

Weighted scoring analysis works well for software, tool, and architecture decisions. Success depends on clear criteria, well-reasoned weightings, consistent scoring scales, and thorough documentation.

How to Study It

1. Pick a real-world choice problem and list all criteria.
2. Sketch the WSA workflow as a flowchart.
3. Under time pressure, work through a complete scoring matrix.
4. Double-check scale consistency and that weights sum to 1.0.

Further Reading

1. https://en.wikipedia.org/wiki/Weighted_sum_model
2. https://en.wikipedia.org/wiki/Multi-criteria_decision_analysis