Quality Assurance Measures
This article explains quality assurance measures – including exam questions and key concepts.
In a Nutshell
Quality emerges through coordinated measures: audits, code reviews, testing methods, static code analysis, pair programming, bug tracking, a robust development process, and consistent CI/CD pipelines, made measurable through quality targets and quality gates.
Overview
Preventive vs. Detective
- Preventive: Guidelines, training, pair programming, Definition of Done
- Detective: Static code analysis, tests, code reviews, audits
Central to this is an automated pipeline with Continuous Integration that builds, analyzes, and tests each change while reporting results, and delivers code through Continuous Delivery or Continuous Deployment. Bug tracking manages the defect lifecycle (status, severity, links to commits and releases). Documentation (architecture, ADRs, operations) is maintained as part of quality and stored under version control.
Exam-Relevant Topics
Audits (internal/external)
- Audit checklists: Predefined questionnaires covering security, compliance, and quality aspects
- Evidence collection: Documentation of measures, test results, and reviews
- Action plans: Concrete steps to address identified gaps with timelines
- Sampling: Statistical review of artifacts and processes
- Audit trail: Complete record of all changes and decisions
Code Review Process
- Four-eyes principle: At least two people review and approve code changes
- Review checklist: Standardized criteria for architecture, security, performance, and tests
- Automated pre-checks: Linting, formatting, and unit tests before manual review
- Review types: Informal review, technical review, and inspection with defined roles
- Metrics: Review coverage, defect detection rate, review duration
Test Methods and Strategies
- Test pyramid: Unit tests (70%), integration tests (20%), E2E tests (10%)
- Test types: Functional tests, non-functional tests, acceptance tests, regression tests
- Test automation: CI integration, parallel execution, test data management
- Exploratory testing: Experience-based testing without predefined scripts
- Mutation testing: Quality measure of test effectiveness through code mutations
Static Code Analysis
- Quality metrics: Cyclomatic complexity, code smells, technical debt
- Security scanning: OWASP Top 10, dependency checking, SAST (Static Application Security Testing)
- Code quality gates: Defined thresholds for metrics and rule violations
- Tools: SonarQube, ESLint, Checkstyle, PMD, Fortify
- Continuous monitoring: Trend analysis and quality evolution over time
Pair/Mob Programming
- Role distribution: Driver (writes code), Navigator (observes, thinks ahead)
- Role rotation: Regular exchanges (e.g., every 25 minutes)
- Knowledge transfer: Direct experience sharing and mentoring
- Quality benefits: Real-time error detection, better architectural decisions
- Remote setup: Screen sharing, live coding tools, virtual pairing platforms
Bug Tracking and Defect Management
- Defect lifecycle: New → In Progress → Fixed → Tested → Closed
- Prioritization: Severity (impact) vs. priority (urgency)
- MTTR (Mean Time To Repair): Metric for repair speed
- Defect density: Number of defects per line of code or function point
- Root cause analysis: Systematic investigation of critical defect causes
Development Process and Standards
- Definition of Ready (DoR): Criteria for accepting work into a sprint or backlog
- Definition of Done (DoD): Completion criteria for user stories and tasks
- Branching strategies: Git Flow, GitHub Flow, trunk-based development
- Release management: Versioning, release notes, deployment strategies
- Compliance requirements: GDPR, ISO standards, industry standards
CI/CD Pipeline Architecture
- Pipeline stages: Build → Test → Analyze → Package → Deploy → Monitor
- Quality gates: Automated decision points with defined criteria
- Rollback strategies: Automated reversal on deployment failures
- Feature flags: Control feature activation without new releases
- Canary/Blue-Green: Risk-minimized deployment strategies
Quality Metrics and KPIs
- ISO 25010: Functionality, reliability, usability, efficiency, maintainability, portability
- DORA metrics: Deployment frequency, lead time, MTTR, change failure rate
- Code quality: Test coverage, code duplication, technical debt ratio
- Performance: Response time, throughput, resource utilization
- Security: Vulnerability count, security score, compliance rate
Documentation and Knowledge Management
- Architecture documentation: C4 model, ADRs (Architecture Decision Records)
- API documentation: OpenAPI/Swagger, examples, versioning
- Operations documentation: Deployment guides, monitoring manuals
- Knowledge base: FAQ, best practices, lessons learned
- Versioning: Documents as part of the repository with changelogs
Core Components
- Quality targets & metrics (ISO 25010 characteristics)
- Review practices (code, architecture, security)
- Test strategy (test pyramid, mutation testing, flaky test management)
- Static code analysis & security scans
- Pair/mob programming
- Bug tracking process (triage, prioritization)
- Development process (DoR, DoD, release flows)
- CI pipeline (build, lint, test, analysis, artifacts)
- Continuous Delivery (manual release, staging, canary, blue-green)
- Continuous Deployment (automatic when gates pass)
Practical Example (Lean QA Chain for a Web Service)
DoD: Unit tests present, coverage increasing, analysis passing, review approved, ticket linked, changelog updated, docs current
Pipeline:
1) Lint + Format
2) Unit Tests + Mutation Testing
3) Static Code Analysis (Quality Gate)
4) Build + Sign Artifact
5) Integration Tests in Container
6) Contract Tests Against Dependencies
7) Deploy to Staging (CD)
8) E2E Smoke Tests
9) Release / Automatic Go-Live (Deployment)
10) Monitoring Active
Reviews: PR Checklist (architecture, security, tests, documentation)
Bug Tracking: Ticket (high) → Reproduction → Test Case → Fix (commit) → Regression Test → Verified → Closed
Advantages and Disadvantages
Advantages
- Early defect detection
- Lower rework costs
- Reproducible quality
- Better compliance documentation
- Increased team knowledge
- Faster and safer releases
Disadvantages
- Initial setup effort
- Learning curve
- Potential slowdown without discipline
- Metric-driven approaches can distort behavior
Common Exam Questions (with Quick Answers)
-
Continuous Delivery vs. Continuous Deployment? Delivery: technically ready at any time, but requires manual release. Deployment: automatic when gates pass.
-
What should be in a review checklist? Architectural compliance, security checks, error handling, tests, naming conventions, complexity, logging, documentation.
-
How do you prepare for an audit? Define scope, gather evidence, provide policies and ADRs, select samples, prepare action plans.
-
What role does static code analysis play? It automatically detects stylistic, structural, and security violations and sets quality gates.
-
How do you measure test effectiveness? Mutation score, flaky test rate, defect detection before release, and coverage as a trend metric.
Key Resources
- https://martinfowler.com/articles/continuousIntegration.html
- https://testing.googleblog.com
- https://owasp.org
Common Interview Questions on Quality Assurance
1. What’s the difference between Continuous Delivery and Continuous Deployment?
Answer: Continuous Delivery means your software is technically ready for production at any time, but requires a manual release decision. Continuous Deployment takes this further by automating the entire process through to production once all Quality Gates pass. Think of it this way: Delivery is “ready to go”, Deployment is “already live”.
2. What should a Code Review checklist include?
Answer: A thorough review checklist covers: architectural conformance (design decisions, patterns), security (input validation, authentication), error handling (exception handling, logging), test coverage (unit tests, integration tests), naming conventions, code complexity (cyclomatic complexity), performance considerations, and documentation (comments, API docs).
3. How do you prepare for a quality assurance audit?
Answer: Audit preparation involves: defining scope (processes, systems, timeframes), gathering evidence (documentation, test results, logs), preparing policies and ADRs, creating a sampling plan, training staff, developing an action plan for identified gaps, and establishing an audit trail.
4. What role does static code analysis play in quality assurance?
Answer: Static code analysis provides automated detection of style violations, structural issues, and security problems without running the code. It enforces Quality Gates, tracks Technical Debt, validates compliance rules, and generates metrics like cyclomatic complexity. Tools such as SonarQube integrate into CI/CD pipelines and prevent low-quality code from being merged.
5. How do you measure test effectiveness?
Answer: Test effectiveness is measured through multiple metrics: Mutation Score (how well tests catch code mutations), Flaky Rate (test reliability), defect detection before release, Test Coverage as a trend (not as an absolute figure), test execution time, and regression test pass rate. Strong effectiveness is shown by catching bugs early and experiencing few production defects.
6. What are Quality Gates and how are they implemented?
Answer: Quality Gates are automated decision points in CI/CD pipelines that allow or block progress based on defined quality criteria. Implementation uses metric thresholds (Coverage < 80% = block), security scans, performance tests, and code review status. Gates are configured in pipeline tools like Jenkins, GitLab CI, or GitHub Actions.
7. Explain the test pyramid and why it matters.
Answer: The test pyramid describes the ideal distribution of test types: Unit Tests (70%) – fast, isolated, many; Integration Tests (20%) – medium speed, component interactions; E2E Tests (10%) – slow, full system coverage. This matters because it delivers fast feedback through abundant unit tests, achieves cost efficiency by minimizing expensive E2E tests, enables better fault isolation, and creates stable test suites.
8. What is Mutation Testing and when is it used?
Answer: Mutation Testing evaluates test quality by introducing small changes (mutations) into the code and checking whether tests catch them. It’s used for critical systems, test optimization, and Quality Gates. A high Mutation Score (>80%) indicates good test coverage. Tools like PIT (Java) or Stryker (JavaScript) automate this process.
9. What benefits does Pair Programming bring to quality assurance?
Answer: Pair Programming delivers real-time code review (bugs caught immediately), knowledge sharing (experience transfer), lower defect rates (two sets of eyes catch more), better architectural decisions (team discussion), and continuous learning. It’s especially valuable for complex problems, onboarding new team members, and critical code sections.
10. How does an effective bug tracking process work?
Answer: An effective bug tracking process includes: structured entry (repro steps, environment, logs), prioritization by severity and priority, assignment to responsible developers, status tracking (New → In Progress → Fixed → Tested → Closed), linking to commits and releases, root cause analysis, and metrics like MTTR and defect density.
11. What are Definition of Ready (DoR) and Definition of Done (DoD)?
Answer: Definition of Ready (DoR) sets criteria a User Story must meet before sprint entry (clear requirements, acceptance criteria, technically feasible). Definition of Done (DoD) specifies completion criteria for a task (code review passed, tests green, documentation updated, deployable). Both act as quality checkpoints in agile workflows.
12. Which branching strategies support quality assurance?
Answer: Git Flow (feature branches, develop, release, master) suits structured releases with extensive testing. GitHub Flow (feature branch → master → deploy) enables fast deployment cycles. Trunk-Based Development supports extreme Continuous Integration with minimal branches. Your choice depends on release frequency, team size, compliance requirements, and risk tolerance.
13. How are security aspects integrated into CI/CD pipelines?
Answer: Security integration uses: SAST (Static Application Security Testing) during build, dependency scanning for known vulnerabilities, DAST (Dynamic Application Security Testing) in staging, container scanning, secret management, compliance checks, and security gates. Tools like OWASP ZAP, SonarQube Security, and Trivy are wired into pipeline stages.
14. What are Feature Flags and how do they support quality assurance?
Answer: Feature Flags are runtime toggles that switch functionality on or off. They enable quality assurance through gradual rollout (canary releases), A/B testing, quick disable on problems, parallel development, and risk-reduced deployments. Implementation uses config servers, feature toggle platforms, or simple configuration files.
15. Explain Canary and Blue-Green deployment strategies.
Answer: Canary Deployment rolls out a new version gradually to small user groups while monitoring metrics. Blue-Green Deployment runs two identical production environments and switches traffic completely to the new one (Green). Both minimize deployment risk, enable fast rollback, and provide early feedback. Your choice depends on infrastructure and risk strategy.
16. What metrics does DORA (DevOps Research and Assessment) measure?
Answer: DORA metrics include: Deployment Frequency (how often you deploy), Lead Time for Changes (commit to deployment), Mean Time To Recovery (MTTR) (recovery from outages), and Change Failure Rate (percentage of failed deployments). High performers show frequent deployments, short lead times, fast recovery, and low failure rates.
17. What is Technical Debt and how do you manage it?
Answer: Technical Debt represents future costs from suboptimal technical choices. Manage it through identification (code analysis, team feedback), prioritization (impact vs. effort), tracking (Technical Debt Ratio, SonarQube), repayment strategy (regular refactoring sprints), and prevention (code reviews, architecture decisions). The goal is conscious choice rather than unchecked degradation.
18. How are non-functional requirements tested?
Answer: Non-functional testing covers: performance (load, stress, spike tests), security (penetration, vulnerability tests), usability (user experience, accessibility), availability (high availability, failover), scalability (horizontal and vertical), and compliance (GDPR, ISO standards). These tests run in specialized environments using dedicated tools.
19. What are Architecture Decision Records (ADRs) and why matter?
Answer: ADRs document key architectural decisions with context, decision, rationale, and consequences. They matter for traceability, knowledge transfer, consistency, and onboarding. ADRs are versioned, stored as Markdown, and treated as project documentation. They support architecture governance and decision-making.
20. How are test environments isolated from production?
Answer: Environment isolation uses physical separation (different servers/clusters), logical separation (database schemas, configs), network separation (firewalls, VPNs), data separation (anonymized test data), and access control (RBAC). Goals include production data security, stable tests, and parallel development. Containers (Docker, Kubernetes) simplify this separation.
21. What is Contract Testing and when is it used?
Answer: Contract Testing validates microservice compatibility through defined “contracts” (API specs). It’s used in microservice architectures to accelerate integration testing, catch breaking changes early, and enable parallel development. Tools like Pact and Spring Cloud Contract automate this and integrate into CI/CD pipelines.
22. How is documentation quality ensured?
Answer: Documentation quality is ensured through versioning (in repositories), automation (doc generation from code), review processes (technical writer sign-off), templates (standardized formats), code linkage (API docs), regular updates (part of DoD), and user feedback. Tools like Swagger/OpenAPI, Javadoc, and MkDocs support this.
23. What are Smoke Tests and when are they run?
Answer: Smoke Tests validate basic application functionality after deployment. They run after every deployment to catch critical issues early. Typical smoke tests check: login, database connectivity, API endpoints, external services. They’re fast (minutes) and deliver a go/no-go signal for further testing.
24. How is test data management handled in CI/CD?
Answer: Test data management uses Test Data Factories (programmatic generation), containerization (Docker with test data), database migrations (Flyway, Liquibase), mocking for external dependencies, data cleanup between tests, and versioning. Goals are reproducible tests, performance, and environment isolation.
25. Prepare for a typical QA interview question.
Answer: Question: “Walk me through a complete quality assurance process for a new web project.” Structure your answer: 1. Preventive measures (coding standards, architecture guidelines), 2. CI/CD pipeline (build → test → analyze → deploy), 3. Code reviews (checklist, peer review), 4. Test strategy (unit, integration, E2E), 5. Quality Gates (metrics, security), 6. Monitoring (production metrics), 7. Continuous improvement (retrospectives, analytics). This structure demonstrates systematic thinking and QA expertise.