ISTQB Advanced (CTAL-TM v3.0) Mock Exam #1 — Questions & Answers

Every question in this mock exam, with the correct answer marked and a written rationale behind each option below — for reading and review, not a timed run.

Chapter 1 · Managing the Test Activities — 26 questions

Question 1

CTAL-TM v3.0 restructures the test process around three management activities aligned with ISO/IEC/IEEE 29119-2, rather than the seven-activity view used at Foundation Level. Which set of three activities does the syllabus use?

Test planning; test monitoring and control; test completion

Correct answer

Correct. These are the three management activities named in the syllabus and mapped onto ISO/IEC/IEEE 29119-2's dynamic test process.

Test planning; test analysis and design; test execution and reporting

Analysis, design and execution are activities the test manager plans and monitors, but the syllabus classifies them as test process work rather than as management activities.

Test strategy definition; test implementation; test closure reporting

Strategy definition sits inside planning and closure reporting inside completion, so this list splits and renames the real grouping rather than reproducing it.

Risk identification; risk analysis; risk mitigation planning

These are the steps of risk management, which the syllabus treats as an input to planning and control, not as the top-level structure of the test process.

Why

The syllabus groups the test manager's work into test planning, test monitoring and control, and test completion. The remaining Foundation activities (analysis, design, implementation, execution) are the day-to-day work being managed, not management activities in their own right.

Question 2

A release has passed its exit criteria and the test manager is closing out the test effort. Which TWO of the following are described in the syllabus as test completion activities?

Archiving the testware and handing it over to the group that will maintain the product

Correct answer

Correct. Archiving and handover of testware is one of the named completion activities, so the assets remain usable for maintenance testing.

Restoring the test environment to an agreed state so it can serve the next effort

Correct answer

Correct. Restoring or decommissioning the test environment is explicitly listed among the test completion activities.

Comparing actual test progress against the planned progress and issuing control directives

This is test monitoring and control: it happens while testing is running so that deviations can still be corrected, not after the effort is closed.

Identifying product risks with stakeholders and agreeing which risk treatment applies to each

Product risk identification and treatment selection is a planning activity, performed before the risks in question have been tested.

Assessing test readiness and confirming that entry criteria for the next test level are met

Test readiness assessment is part of monitoring and control, gating the start of a level rather than closing the effort down.

Why

Test completion covers producing the test completion report, archiving and handing over testware, restoring the test environment, and capturing lessons learned. Measuring progress against the plan and issuing control directives belong to monitoring and control, which runs during execution.

Question 3

A test manager maps stakeholders onto the power-interest matrix. The chief…

A test manager maps stakeholders onto the power-interest matrix. The chief financial officer controls the test budget and can cancel the automation programme, but has shown no interest in the test approach and does not attend test reviews. Which stakeholder category does the syllabus assign to this position?

Latent — high influence over the test effort combined with currently low interest in it

Correct answer

Correct. Latents hold enough power to affect the effort but are not engaged, so the manager's task is to keep them satisfied and watch for their interest rising.

Promoter — high influence over the test effort combined with high interest in it

Promoters are actively engaged as well as powerful; this stakeholder attends nothing and has expressed no view on the approach.

Defender — low influence over the test effort combined with high interest in it

Defenders care about testing but cannot direct it. Here the situation is reversed: the stakeholder can cancel the programme outright.

Apathetic — low influence over the test effort combined with low interest in it

The low-interest half fits, but budget authority and the ability to cancel the programme are exactly what distinguishes high power from low.

Why

The power-interest matrix in the syllabus names four categories: Promoters (high power, high interest), Latents (high power, low interest), Defenders (low power, high interest) and Apathetics (low power, low interest). High influence combined with low interest is the Latent quadrant.

Question 4

A payments platform runs a quality gate between system testing and user…

A payments platform runs a quality gate between system testing and user acceptance testing. The agreed gate criteria are: no open defects of severity 1 or 2, requirements coverage at or above 95%, and all high product risks covered by at least one executed test. Two days before the gate the test manager has: zero severity 1 defects, one open severity 2 defect in the refund flow with a fix merged but not yet retested, requirements coverage at 97%, and two of the eleven high product risks covered only by tests that are currently blocked by an unavailable third-party sandbox. The programme manager asks to pass the gate now and "catch up during UAT". Analysing the situation against the purpose of a quality gate, what is the most defensible response from the test manager?

Report that two criteria are unmet — an unretested severity 2 fix and two high risks with no executed coverage — quantify the residual risk each represents for UAT, and put the exception decision to the accountable stakeholders in writing

Correct answer

Correct. The gate's evidence is compared against the agreed criteria, the shortfall is expressed as residual risk, and the waiver becomes an explicit decision owned by the stakeholders who carry that risk.

Pass the gate on the strength of the coverage figure of 97% and the absence of severity 1 defects, and record the sandbox problem in the test completion report at the end of UAT

Meeting one criterion does not compensate for another, and deferring the record of two uncovered high risks to a report written after UAT removes the stakeholders' chance to act on it.

Refuse to pass the gate until every criterion is met, since the criteria were agreed with the stakeholders and the test manager is accountable for enforcing them

The test manager evaluates and reports against the criteria, but the decision to accept residual risk sits with the accountable stakeholders; taking it away turns a business trade-off into a unilateral one.

Relax the coverage criterion to 90% and lower the two affected risks from high to medium, so that the gate criteria reflect what the third-party sandbox outage makes achievable

Rewriting criteria and risk levels to match the current results makes the gate unable to fail, which removes the only reason for having one.

Why

A quality gate is a decision point where objective evidence is compared against agreed criteria; its value comes from the criteria being applied as agreed, and from any exception being an explicit, owned decision rather than a silent one. Here two criteria are unmet in substance: the severity 2 defect is not retested, so its closure is unevidenced, and two high risks carry no executed coverage at all, which is the criterion with the largest residual exposure. The manager's job is to report that position with its residual risk and let the accountable stakeholders decide, not to redefine the criteria or to block unilaterally.

Question 5

The syllabus compares sequential (V-model) and iterative (Scrum) lifecycles across aspects such as estimation, testware, roles, tools, monitoring and metrics. Which statement describes the difference in monitoring and reporting between the two?

Sequential reporting is formal and tied to milestones and phase transitions, whereas iterative reporting is continuous and owned largely by the team through short-interval, highly visible measures

Correct answer

Correct. The reporting rhythm follows the lifecycle: phase gates in sequential development, per-iteration and daily visibility in Scrum.

Sequential reporting is continuous and team-owned, whereas iterative reporting is produced by the test manager at each release milestone

This inverts the two lifecycles; continuous team-owned reporting is the iterative pattern, not the sequential one.

Both lifecycles report on the same fixed set of metrics, so only the frequency of the report differs between them

Frequency does differ, but so does the metric set: iterative teams favour velocity and burndown, sequential projects favour phase-based progress and coverage against a baselined plan.

Iterative development reports only on defects found, while sequential development reports only on test cases executed

Neither lifecycle restricts itself to a single measure; both track defects, progress and coverage, differing in formality and cadence rather than in scope.

Why

In a sequential lifecycle, monitoring and reporting are formal and phase-oriented, typically produced by the test manager at defined milestones. In an iterative lifecycle they are continuous and largely team-owned, visible on information radiators such as burndown charts and reviewed at short intervals.

Question 6

Within risk-based testing the syllabus defines the level of a product risk as a combination of two factors. Which pair, and what does the resulting level primarily drive?

Likelihood of occurrence and impact of occurrence; the level drives how early and how intensively the affected area is tested

Correct answer

Correct. This is the definition given in the syllabus, and the prioritisation heuristic it supports is that higher risk means earlier and deeper testing.

Likelihood of occurrence and cost of the corrective fix; the level drives which defects are accepted into the release

Fix cost influences defect triage decisions but is not a factor in the risk level itself, which is assessed before the defects exist.

Impact of occurrence and requirements coverage already achieved; the level drives how many additional test cases are designed

Achieved coverage is a measure of test progress, not a determinant of a product risk's level; the risk exists independently of how much testing has been done.

Defect density observed in the component and the experience of the assigned tester; the level drives who is allocated to the area

Both are real inputs to planning and to skills allocation, and defect density can inform likelihood, but neither is a defining factor of risk level.

Why

Risk level is the combination of the likelihood of the risk occurring and the impact if it does. The resulting level drives the depth and the timing of testing: the higher the risk level, the earlier and the more intensively the associated areas are tested.

Question 7

A test manager scores four product risks on a five-point scale for likelihood…

A test manager scores four product risks on a five-point scale for likelihood and impact and computes risk exposure as likelihood multiplied by impact. Risk A: likelihood 2, impact 5. Risk B: likelihood 4, impact 3. Risk C: likelihood 5, impact 2. Risk D: likelihood 3, impact 4. Only enough effort exists to test two areas deeply in the first week. Applying the syllabus rule that higher risk is tested earlier and more intensively, which two risks are addressed first, and what tie-break does the syllabus support when exposures are equal?

Risks B and D, both at exposure 12; where exposures tie, the risk with the higher impact is taken first, which puts D ahead of B

Correct answer

Correct. 4 × 3 and 3 × 4 both give 12, and resolving the tie on impact places D (impact 4) ahead of B (impact 3).

Risks A and C, both at exposure 10; where exposures tie, the risk with the higher likelihood is taken first, which puts C ahead of A

The arithmetic on the tie-break is applied to the wrong pair: 10 is the lower exposure, so A and C are the two risks that wait rather than the two addressed first.

Risks A and D, because a risk with the maximum score on either factor outranks any risk whose scores are both mid-range

Treating a maximum on a single factor as decisive abandons the multiplicative definition of exposure; A scores 10 and therefore ranks below both B and D.

Risks C and B, because likelihood is weighted more heavily than impact when test effort has to be rationed under a deadline

The syllabus does not weight likelihood above impact when effort is constrained; exposure remains the product of the two factors regardless of the deadline.

Why

The exposures are A = 10, B = 12, C = 10 and D = 12. B and D are tied at the top with 12 and are addressed first. Where exposures are equal, the syllabus supports resolving the tie on impact, because a high-impact failure is the more damaging outcome and impact is the factor stakeholders can least tolerate; likelihood can also be reduced by other means, whereas impact usually cannot.

Question 8

A regulated insurance system has 30 product risks spread over six subsystems…

A regulated insurance system has 30 product risks spread over six subsystems. Two subsystems carry almost all of the high risks; the other four carry medium and low risks only. The release date is fixed, the test window is short, and the sponsor has said that the release will not be stopped for medium-risk findings but will be stopped for any high-risk failure. Analysing this context, which execution order does the syllabus support, and why?

Depth-first: test the two high-risk subsystems thoroughly before touching the rest, because the only findings that can change the release decision are concentrated there

Correct answer

Correct. When the consequences and the go/no-go criterion are concentrated in a few areas, depth-first buys the most decision-relevant information in a short window.

Breadth-first: run a shallow pass over all six subsystems first, because an even picture of the whole system is what a fixed release date requires

Breadth-first is the better fit when risk is evenly distributed; here it spends scarce time on four subsystems whose findings cannot stop the release.

Breadth-first, on the grounds that the regulated context obliges the test manager to demonstrate coverage of every subsystem before deepening any single one

Regulation typically constrains which techniques and evidence are required at a given integrity level, but the syllabus does not derive a breadth-first ordering from regulation as such.

Depth-first, but starting with the four medium-risk and low-risk subsystems so that the high-risk areas are tested last against the most stable build

This delays the findings that carry the stop-the-release consequence to the point where there is least time to act on them, which reverses the purpose of risk-based ordering.

Why

The syllabus contrasts depth-first execution, which takes the highest-risk areas and tests them thoroughly before moving on, with breadth-first execution, which spreads a thin layer of testing across all areas before deepening any of them. Depth-first suits a context where the consequences are concentrated in a few areas and the stop-the-release decision hinges on those areas; breadth-first suits a context where risk is evenly spread or where an early overall picture matters more than depth anywhere.

Question 9

At the end of a risk-based test effort, 11 of 14 high risks have been fully…

At the end of a risk-based test effort, 11 of 14 high risks have been fully covered by passing tests, 2 are covered by tests that pass but exercise only the main flow, and 1 could not be tested because its interface partner never became available. The steering group asks for a one-page status. Analysing what risk-based testing is supposed to produce as its output, what should the report lead with?

The residual risk position: one high risk untested and two covered only on the main flow, each described with the exposure that remains and what would reduce it

Correct answer

Correct. Risk-based testing reports in terms of residual risk, which is precisely the information a steering group needs to accept or reject the release.

The pass rate: 13 of 14 high risks exercised by tests that passed, expressed as a percentage of the risk register

A pass rate hides the two shallow coverages behind the same figure as the eleven thorough ones, so the steering group cannot see the difference in exposure.

The volume of work completed: number of test cases executed, defects raised and defects closed over the test window

These are effort and progress measures. They demonstrate that testing happened without telling the reader what risk remains.

The root cause of the environment problem that prevented the untested risk from being covered, with the corrective action agreed for the next release

This belongs in lessons learned during test completion; it explains why the gap exists but does not state the exposure the steering group must decide on.

Why

The distinctive reporting output of risk-based testing is the residual risk level: which risks remain, at what level, after the testing that was performed. Counts of tests and defects describe the activity; they do not tell a steering group what exposure it is being asked to accept. The untested risk and the two partially covered ones are the residual exposure and belong at the front of the report.

Question 10

The syllabus divides product risk analysis techniques into heavyweight and lightweight approaches. Which option correctly assigns four named techniques to those two groups?

Heavyweight: Failure Mode and Effect Analysis, fault tree analysis. Lightweight: PRISMA, PRAM

Correct answer

Correct. FMEA and fault tree analysis are the heavyweight analytical techniques; PRISMA and PRAM are the named lightweight approaches.

Heavyweight: PRISMA, SST. Lightweight: Failure Mode and Effect Analysis, hazard analysis

The two groups are swapped: PRISMA and SST are the lightweight approaches, while FMEA and hazard analysis carry the heavier analytical overhead.

Heavyweight: cost of exposure, PRAM. Lightweight: fault tree analysis, SST

Cost of exposure is correctly placed but PRAM is a lightweight technique, and fault tree analysis is heavyweight rather than lightweight.

Heavyweight: hazard analysis, risk workshops. Lightweight: brainstorming, checklists

Risk workshops, brainstorming and checklists are techniques for identifying risks, not the assessment approaches this classification refers to.

Why

Heavyweight techniques named in the syllabus include hazard analysis, cost of exposure, Failure Mode and Effect Analysis, and fault tree analysis. Lightweight techniques include SST (Craig and Jaskiel), PRAM (Black) and PRISMA (van Veenendaal).

Question 11

A team has run risk-based testing on four consecutive releases. Each time, the same risk register is reused with only minor edits, and participants approve it quickly because "we did this last time". Which of the difficulties named in the syllabus does this describe?

Deja vu — the previous analysis is carried forward rather than genuinely reassessed, so changes in the product and its context go unnoticed

Correct answer

Correct. Deja vu is the named difficulty where a risk analysis is recycled and the review becomes a formality.

Keen beginnings — the risk analysis is performed enthusiastically at the start of a project and then not maintained as the project proceeds

Keen beginnings describes enthusiasm decaying within one project; here the analysis is repeated across releases but without fresh thought.

Stakeholder churn — the participants in the risk analysis change between sessions, so the assessments lose consistency over time

Stakeholder churn concerns changing participants; in this case the same people take part and the problem is the lack of reassessment.

Difficulty in assessing the risk level — participants cannot agree on likelihood and impact values, so the register drifts towards mid-range scores

That difficulty is about the assessment itself being hard; here the scores are accepted without dispute because nobody re-examines them.

Why

The syllabus lists several difficulties with risk-based testing, including difficulty in assessing the risk level, keen beginnings (initial enthusiasm that fades), deja vu (reusing a previous risk analysis without genuine reassessment), key risks being missed, and stakeholder churn.

Question 12

A product risk has been identified: a third-party currency conversion service…

A product risk has been identified: a third-party currency conversion service may return stale rates during a provider outage, causing incorrect invoice totals. Likelihood is assessed as low, impact as very high. The service cannot be exercised realistically in the test environment, the provider offers a contractual penalty covering financial loss from stale rates, and the development team can add a rate-age check that rejects any rate older than an agreed threshold. Analysing the treatment options in the syllabus, which combination is the most appropriate response?

Mitigate by building and testing the rate-age check, rely on the contract to transfer the residual financial consequence, and record the remaining exposure as accepted by the sponsor

Correct answer

Correct. It combines the mitigation that testing can actually verify, transfers the consequence that testing cannot remove, and makes the leftover exposure an explicit decision.

Accept the risk without further action, since the likelihood is low and the contractual penalty already compensates the organisation for any financial loss

Acceptance may be reasonable for low-impact risks, but a very high impact combined with an available and testable design mitigation makes doing nothing hard to defend.

Treat the risk purely through testing by building a stub that returns stale rates, and raise the likelihood score so that the area receives the most execution effort

The stub is worth building, but inflating the likelihood score to force effort corrupts the risk register, and testing alone leaves the financial consequence untreated.

Transfer the risk entirely to the provider through the contractual penalty and remove it from the risk register, since the organisation no longer bears the financial loss

Transfer moves the financial consequence, not the reputational and operational effects, and a transferred risk stays on the register with its residual level recorded.

Why

The syllabus presents testing as the principal risk mitigation measure but sets it alongside contingency plans, risk transfer and risk acceptance. Where the risk cannot be exercised realistically, mitigation by design plus a contingency, combined with transfer of the financial consequence, addresses the exposure better than testing alone. Accepting the risk outright ignores a very high impact, and simulating the outage in test is still worth doing but does not by itself cover the provider's real behaviour.

Question 13

An organisation maintains a test policy, an organizational test strategy and a project test strategy. Which description matches the project test strategy as the syllabus defines it?

It tailors the organisation's generic approach to the circumstances of one specific project and is the principal output of test planning for that project

Correct answer

Correct. The project test strategy is project-specific, derived from the organizational strategy, and produced as the main planning output.

It states the organisation's overall objectives for testing and the value testing is expected to deliver to the business

That is the test policy, which sits above both strategies and expresses intent rather than approach.

It describes the generic test approach to be applied across all projects in the organisation, independently of any one project's context

That is the organizational test strategy; the project test strategy is what adapts it to a particular project.

It lists the test cases, schedules and environment bookings agreed for each test level within the project

Those details belong in test plans at the release, level or iteration layer; the strategy sets the approach that those plans then implement.

Why

The test policy states why the organisation tests and what it expects from testing. The organizational test strategy describes the generic approach applied across projects. The project test strategy, corresponding to the test strategy of ISO/IEC/IEEE 29119-3, tailors that generic approach to one project and is the main output of test planning.

Question 14

A test manager is choosing a test approach for a new project: a hospital…

A test manager is choosing a test approach for a new project: a hospital medication-dosing module, developed in two-week iterations by a team of four, integrating with two legacy pharmacy systems whose interfaces are documented but not available for testing until week ten, with no anonymised production data yet released by the client's data protection office. Analysing this against the factors the syllabus says must be considered when defining a test approach, which factor is currently the binding constraint on the approach?

Interfaces with other systems together with availability of test data — both are unavailable for most of the schedule, so the approach must be built around stubs, simulators and synthetic data until week ten

Correct answer

Correct. These are the two factors whose current state removes options from the approach, so they shape it more than the others do.

The SDLC model — two-week iterations require the approach to be defined per iteration rather than once for the project, which is what constrains the choice here

The iterative model shapes cadence and planning layers, but it does not remove any technique from the table; the missing interfaces and data do.

Test resources — a team of four is small for a safety-related module, so the approach must be scaled down to what four people can execute

Team size influences how much can be done, but nothing in the scenario indicates the team is undersized relative to the module; the availability problems are stated explicitly.

The domain — medication dosing is safety-related, so the approach is fixed by the regulatory regime and the remaining factors do not materially change it

The domain does raise the required rigour, but the syllabus treats all seven factors as inputs to be analysed together rather than one overriding the rest.

Why

The syllabus lists seven factors to analyse when defining the test approach: the domain, organizational goals, project goals and type, test resources, the SDLC model, interfaces with other systems, and the availability of test data. Here the domain and the SDLC model are known and workable; what actually blocks progress is that neither the interfacing systems nor realistic test data will be available for most of the project, which forces the approach towards simulation, stubs and synthetic data early on.

Question 15

A draft test objective reads: "Improve the quality of the checkout flow as much as possible before the release." Applying the S.M.A.R.T. criteria the syllabus prescribes for test objectives and exit criteria, which rewrite is acceptable?

"By the code freeze on 14 March, execute all 62 checkout regression tests, with no open severity 1 or 2 defects and requirements coverage of the checkout flow at 95% or above."

Correct answer

Correct. Scope, measures, thresholds and a date are all present, and the targets are achievable against a defined test set.

"Before the release, test the checkout flow thoroughly enough that the business stakeholders are confident in its quality and sign off without reservations."

Confidence and thoroughness cannot be measured against a threshold, and "before the release" is not a fixed point, so the statement fails the measurable and time-bound criteria.

"By the code freeze on 14 March, find and fix every remaining defect in the checkout flow so that no defects of any severity are left in the released code."

It is specific, measurable and time-bound, but zero remaining defects is not achievable and cannot be demonstrated, so it fails the achievable criterion.

"Increase automated test coverage of the whole application from 41% to 80% by the code freeze on 14 March, to raise confidence in the checkout flow."

The measure and the date are sound, but the target covers the whole application rather than the checkout flow, so the objective is not relevant to the stated goal.

Why

S.M.A.R.T. objectives are specific, measurable, achievable, relevant and time-bound. The original statement fails on measurability, achievability and time-bounding. A usable rewrite names the scope, states a measurable target with a threshold, and fixes a date.

Question 16

A programme uses several layers of test plan: release, master, level, quality-characteristic and iteration plans. What relationship between these layers does the syllabus describe?

They form a hierarchy: broader plans set objectives and constraints that the narrower plans refine for a level, a quality characteristic or a single iteration

Correct answer

Correct. The layers are nested, and each lower layer elaborates the frame set above it rather than replacing it.

They are independent documents, each owned by a different role, and an organisation adopts exactly one of the layers for a given project

The layers coexist on larger efforts; treating them as mutually exclusive alternatives misses the point of the hierarchy.

They are successive versions of one plan, so the iteration plan supersedes the level plan, which in turn supersedes the master plan

Later plans do not supersede earlier ones; a valid iteration plan operates within the level and master plans that remain in force.

They apply to different lifecycles: master and level plans belong to sequential projects and iteration plans belong to Agile projects only

Iterative programmes frequently keep a release or master plan alongside iteration plans, so the split by lifecycle does not hold.

Why

The plans form a hierarchy in which broader plans set the frame for narrower ones. A master or release plan covers the whole effort, level plans cover a single test level, quality-characteristic plans cover a characteristic such as performance or security across levels, and iteration plans cover one iteration. Lower layers refine the higher ones rather than contradicting them.

Question 17

The IDEAL model is used to structure test process improvement. What do its five phases stand for, and which well-known cycle does the syllabus relate it to?

Initiating, Diagnosing, Establishing, Acting, Learning — related to the Plan-Do-Check-Act cycle

Correct answer

Correct. These are the five IDEAL phases, and the syllabus presents the model as a relative of PDCA.

Identifying, Designing, Executing, Analysing, Leveraging — related to the Goal-Question-Metric approach

The phase names are invented, and GQM is a measurement approach used inside analytical improvement rather than the cycle IDEAL is related to.

Initiating, Defining, Estimating, Assessing, Learning — related to the five maturity levels of TMMi

Two phase names are wrong, and TMMi is a model-based improvement framework, not the cycle that IDEAL corresponds to.

Inspecting, Diagnosing, Escalating, Adjusting, Logging — related to the retrospective process for iterative teams

These phase names do not appear in the model, and retrospectives are one analytical improvement technique rather than IDEAL's counterpart cycle.

Why

IDEAL stands for Initiating, Diagnosing, Establishing, Acting and Learning. The syllabus relates it to the Plan-Do-Check-Act cycle, of which it is a relative: both are iterative improvement frameworks that end by feeding what was learned back into the next cycle.

Question 18

TMMi and TPI NEXT are both model-based test process improvement frameworks. Which statement correctly distinguishes their structures?

TMMi stages the organisation on five maturity levels built from process areas; TPI NEXT assesses sixteen key areas individually, each against four levels, and plots them on a maturity matrix

Correct answer

Correct. TMMi gives one staged organisational rating, while TPI NEXT produces a per-key-area profile.

TMMi assesses sixteen key areas against checkpoints; TPI NEXT stages the organisation on five maturity levels that complement CMMI

The two models are swapped: the sixteen key areas belong to TPI NEXT and the five staged levels to TMMi.

Both models produce a single organisational maturity rating on a five-level scale; they differ only in the number of process areas assessed at each level

TPI NEXT deliberately produces a profile across key areas at four levels rather than one overall rating, so the difference is structural.

TMMi is an analytical-based improvement approach driven by root cause analysis; TPI NEXT is a model-based approach driven by a reference maturity model

Both are model-based. Root cause analysis and metric analysis belong to the analytical-based category alongside GQM.

Why

TMMi defines five maturity levels made up of process areas and is designed to complement CMMI. TPI NEXT works from sixteen key areas, each of which is assessed against four maturity levels using checkpoints, with the results plotted on a maturity matrix.

Question 19

A test manager is facilitating a retrospective after a difficult release. The…

A test manager is facilitating a retrospective after a difficult release. The team has already collected data on what happened and has produced a list of eleven candidate improvements. Following the five-step retrospective process described in the syllabus, what is the next step, and what makes it succeed?

Decide on improvement actions — select a small number of the eleven candidates, give each an owner and a target, and defer the rest so the chosen ones actually get done

Correct answer

Correct. Deciding on actions follows deriving improvements, and the decision has to be selective and owned to survive contact with the next release.

Close the retrospective — record all eleven candidates in the improvement backlog and thank the team, leaving prioritisation to the next planning session

This skips the decision step; a list handed on without selection or ownership rarely produces change, which is why the syllabus makes deciding a distinct step.

Collect data — go back and gather metrics on defect counts and effort so that the eleven candidates can be ranked on evidence rather than opinion

Data collection precedes deriving improvements and has already happened; returning to it now stalls the session rather than advancing it.

Repeat the introduction — restate the blame-free ground rules before the difficult items are discussed, since the release went badly and tempers are likely to rise

The blame-free frame is set at the introduction and upheld throughout, but it is not a step the process returns to in place of deciding on actions.

Why

The five steps are: introduction (setting a blame-free frame), collect data, derive improvements, decide on improvement actions, and close the retrospective. With improvements already derived, the next step is deciding which actions to take: selecting a small number, assigning ownership and making them concrete enough to be followed up.

Question 20

Over three releases, defects escaping to production have risen from 8 to 14 to…

Over three releases, defects escaping to production have risen from 8 to 14 to 23, while the number of tests executed has stayed roughly constant and the team composition has not changed. The test manager wants an analytical-based improvement that will identify why escapes are rising rather than simply adding more tests. Analysing the techniques the syllabus places under analytical-based improvement, which approach fits, and what does it produce?

Run root cause analysis on the escaped defects using an Ishikawa diagram to group causes, then use Goal-Question-Metric to define the measures that will show whether the resulting actions work

Correct answer

Correct. Both techniques are analytical-based: the first explains the trend by cause category, the second makes the improvement measurable.

Commission a TMMi assessment to establish the organisation's maturity level and adopt the process areas prescribed for the next level up

TMMi is a model-based rather than analytical-based approach, and a maturity assessment answers a different question than why these particular defects escaped.

Hold a retrospective at the end of the next release so the team can discuss the escape trend and propose improvements from their own experience

Retrospectives are a legitimate improvement technique in the syllabus, but they rest on participants' recollection rather than on analysing the defect data that is already available.

Apply TPI NEXT checkpoints to the sixteen key areas and improve the key areas that score lowest on the maturity matrix

This is again model-based improvement; the lowest-scoring key areas may have nothing to do with the specific mechanism producing the escapes.

Why

Analytical-based improvement includes root cause analysis, typically supported by an Ishikawa or fishbone diagram, and the analysis of measures, metrics and indicators, often organised through Goal-Question-Metric. Root cause analysis on the escaped defects identifies the categories of cause behind the trend, and GQM then defines the measures that will show whether the chosen actions work.

Question 21

The syllabus classifies test tools into three business types and names four factors that influence the decision on which to adopt. Which option lists the three business types correctly?

Commercial tools, open-source tools and custom tools developed in-house

Correct answer

Correct. This is the three-way classification by business type on which the adoption decision is framed.

Test management tools, test execution tools and static analysis tools

This classifies tools by the activity they support, which is a different taxonomy from the business-type one used for the adoption decision.

Cloud-hosted tools, on-premise tools and hybrid deployments of the two

Deployment model affects cost and security considerations but is not the classification the syllabus uses for business type.

Vendor-supported tools, community-supported tools and tools with no support arrangement

Support arrangements follow from the business type rather than defining it, and custom in-house tools do not fit any of these three labels cleanly.

Why

The three business types of test tool are commercial tools, open-source tools and custom tools built in-house. The four decision factors are regulations and security, financial aspects, stakeholder requirements, and the existing software landscape.

Question 22

A test manager evaluates a commercial test automation tool over a three-year…

A test manager evaluates a commercial test automation tool over a three-year horizon. Licences cost 60,000 per year. Initial tool selection, installation and framework build cost 90,000 once. Training the eight testers costs 24,000 once, with 6,000 per year thereafter for new joiners. Maintaining the automated suite is estimated at 40,000 per year. During the first year the two most experienced testers will spend roughly 30% of their time on the tool rather than on exploratory testing of the highest-risk areas. Analysing this against the cost categories the syllabus defines, which statement is correct?

Non-recurring costs are 114,000, recurring costs are 106,000 per year, and the exploratory testing the two senior testers will not perform is an opportunity cost that must be stated even though it is not invoiced

Correct answer

Correct. 90,000 plus 24,000 is non-recurring; 60,000 plus 6,000 plus 40,000 is recurring; and forgone high-risk exploratory testing is the textbook opportunity cost.

Non-recurring costs are 90,000, recurring costs are 106,000 per year, and the senior testers' redirected time is a recurring cost because their salaries continue to be paid throughout

Initial training is non-recurring and belongs with the 90,000, and the point of an opportunity cost is the value forgone, not the salary that is paid either way.

Non-recurring costs are 114,000, recurring costs are 318,000, and the redirected senior tester time should be excluded because it carries no additional cash outflow

318,000 is the three-year total rather than the annual recurring figure, and excluding opportunity costs is precisely the omission that makes tool business cases over-optimistic.

All 432,000 over three years is a recurring cost, since the organisation must keep paying to retain the automation capability once it has been built

Collapsing the categories loses the distinction the analysis depends on: selection, framework build and initial training are paid once and do not repeat in years two and three.

Why

The syllabus distinguishes non-recurring costs (incurred once, such as selection, installation, framework build and initial training), recurring costs (incurred repeatedly, such as licences, maintenance and ongoing training) and opportunity costs (the value of what is given up, here the exploratory testing the two senior testers will not perform). Non-recurring costs total 114,000; recurring costs total 106,000 per year, or 318,000 over three years; and the lost exploratory testing is an opportunity cost that belongs in the case even though no invoice records it.

Question 23

The syllabus describes a test tool lifecycle and two distinct roles associated with tools. Which option pairs the lifecycle phases with the roles correctly?

Acquisition, support and maintenance, evolution, retirement — with the tool owner accountable across the lifecycle and the tool administrator running day-to-day operation

Correct answer

Correct. These are the four phases named in the syllabus, and it separates ownership from administration in exactly this way.

Selection, piloting, roll-out, decommissioning — with the tool administrator accountable across the lifecycle and the tool owner running day-to-day operation

Piloting and roll-out are steps within acquisition and introduction, and the two roles' responsibilities are the wrong way round.

Acquisition, proof of concept, support and maintenance, retirement — with a single tool owner performing both the accountable and the administrative work

Proof of concept is a good practice during introduction rather than a lifecycle phase, evolution is missing, and the syllabus keeps the two roles distinct.

Evaluation, acquisition, evolution, replacement — with the tool owner responsible for licences and the tool administrator responsible for the business case

Support and maintenance is omitted, and the business case sits with the tool owner rather than with the administrator.

Why

The tool lifecycle runs Acquisition, Support and maintenance, Evolution, and Retirement. The tool owner is accountable for the tool across that lifecycle, including the business case and the decision to evolve or retire it, while the tool administrator handles day-to-day operation such as configuration, access and upgrades.

Question 24

A test manager wants to report on the value delivered by the defect management tool and by the test execution tool. Which TWO of the following are tool metrics the syllabus associates with these tools?

Defect detection percentage — the share of defects detected by testing relative to all defects found, including those found after release

Correct answer

Correct. It is one of the named tool metrics and shows how effective the supported detection activity is.

Defect lead time — the elapsed time a defect takes to progress from being reported to reaching a terminal status

Correct answer

Correct. Lead time is a named tool metric and reflects how well the defect workflow supported by the tool is functioning.

Number of licences purchased compared with the number of named users currently registered in the tool

Licence utilisation is useful for cost control but describes procurement rather than the contribution the tool makes to testing.

Ratio of testers to developers on the project during the period in which the tool was in use

This is a staffing ratio that would be the same with or without the tool, so it cannot evidence the tool's contribution.

Total number of screens in the application under test that the tool is technically capable of automating

Technical reach describes the tool's applicability, assessed before adoption, rather than a metric of the value it delivers in use.

Why

Among the tool metrics the syllabus names are defect detection percentage, which expresses the proportion of defects found by testing before release, and defect lead time, which measures how long a defect takes to move through its lifecycle. Licence utilisation and headcount ratios are administrative facts rather than the metrics the syllabus lists for evaluating tool contribution.

Question 25

Halfway through a six-week system test, the plan expected 480 of 800 tests…

Halfway through a six-week system test, the plan expected 480 of 800 tests executed; 470 have been executed, which is close to plan. However, 130 of the executed tests are blocked on a single defective login service, and all of them sit in the two subsystems carrying the highest product risks. Applying test monitoring and control as the syllabus describes it, which control directive follows from these measurements?

Escalate the login service defect for priority resolution and reschedule effort so the highest-risk subsystems regain coverage once it is fixed, because the deviation is in risk coverage rather than in the execution count

Correct answer

Correct. The directive addresses the cause of the blockage and restores coverage where the exposure is greatest, which is what the measurements actually reveal.

Report progress as on plan, since 470 executed against 480 planned is within normal variance, and revisit the blocked tests at the end of the test window

Reporting on the headline count hides the fact that the executed total includes 130 blocked tests in the highest-risk areas, which is the deviation that matters.

Move the 130 blocked tests out of scope for this release and record the two subsystems as covered by the tests that did run, so the schedule is protected

Descoping tests in the highest-risk subsystems increases residual risk silently; monitoring and control exists to surface such a trade-off, not to conceal it.

Extend the test window by two weeks immediately, since a third of the planned execution is blocked and the original schedule can no longer be met

Extending the schedule is one possible outcome, but issuing it before the login defect is escalated treats the symptom and may prove unnecessary once the blocker clears.

Why

Monitoring compares measurements against the plan; control turns a detected deviation into a directive. The execution count looks healthy, but the blocked tests are concentrated in the highest-risk areas, so the real deviation is in risk coverage rather than in throughput. The directive should escalate the login service defect for priority fix and, meanwhile, redirect effort so that risk coverage recovers.

Question 26

A company runs a hybrid model: the customer-facing web front end is delivered by…

A company runs a hybrid model: the customer-facing web front end is delivered by three Scrum teams on a two-week cadence, while the core banking back end follows a V-model with quarterly releases and formal phase approvals. Front-end stories increasingly depend on back-end changes that will not be available until the next quarterly release. Analysing this against what the syllabus says about test management in a hybrid context, which measure best addresses the situation?

Synchronise planning across the two cadences so cross-boundary dependencies are identified a quarter ahead, and provide simulated back-end interfaces so front-end teams can test against agreed contracts before the real change ships

Correct answer

Correct. It manages the boundary between the lifecycles, which is where the hybrid challenges arise, without demanding that either side abandon its cadence.

Move the three front-end teams onto the quarterly release cadence so that both parts of the organisation plan, test and report on the same schedule

Uniformity removes the mismatch by discarding the benefit of the faster cadence, and the syllabus treats hybrid working as something to manage rather than to eliminate by decree.

Adopt the V-model documentation and phase-approval standards across the front-end teams, since a common set of testware is what makes hybrid reporting comparable

Comparable reporting is a real hybrid challenge, but imposing the heavier documentation regime on iterative teams addresses the paperwork rather than the dependency that is blocking them.

Have the front-end teams defer any story with a back-end dependency to the sprint following the quarterly release, keeping each team's work inside a single lifecycle

This keeps the lifecycles separate but pushes a growing backlog of dependent stories into one crowded sprint, which concentrates rather than reduces the risk.

Why

The syllabus describes hybrid development and names challenges that follow from mixing cadences: differing testware and documentation expectations, differing metrics and reporting, and dependencies that cross the boundary between the two lifecycles. The management response is to make the boundary explicit — synchronising planning across the two cadences and using service virtualisation or stubs so the faster teams are not gated by the slower one — rather than forcing one lifecycle onto the other.

Chapter 2 · Managing the Product — 15 questions

Question 27

The syllabus groups test metrics into three categories. Which option names them and gives a correct example of each?

Project metrics, for example test execution progress against plan; product metrics, for example defect density in a component; process metrics, for example the defect detection percentage of a test level

Correct answer

Correct. Each example belongs to the category named: progress to project, an attribute of the test object to product, and the capability of an activity to process.

Project metrics, for example defect density in a component; product metrics, for example test execution progress against plan; process metrics, for example the number of testers assigned

The first two examples are swapped, and headcount is a resourcing fact rather than a measure of process capability.

Coverage metrics, for example statement coverage; defect metrics, for example open defects by severity; effort metrics, for example person hours consumed

These are three legitimate metric groups used in reporting, but they are not the project, product and process classification the syllabus sets out.

Leading metrics, for example planned test cases; lagging metrics, for example defects found in production; neutral metrics, for example environment uptime

Leading and lagging is a useful distinction for interpreting trends, but it is not the three-category classification defined in this chapter.

Why

The three categories are project metrics, which measure progress towards planned completion; product metrics, which measure attributes of the test object such as defect density or coverage; and process metrics, which measure the capability and efficiency of the development and test processes, such as defect detection percentage or the cost of finding a defect.

Question 28

A test progress report distinguishes tests that are planned, implemented, executed, passed, failed, blocked and skipped. What does the syllabus say the blocked and skipped categories add that the passed and failed counts cannot show?

They show how much planned coverage was not obtained and for what reason — an external impediment in the case of blocked tests, a deliberate decision in the case of skipped tests

Correct answer

Correct. Both categories represent missing information rather than a verdict, and separating them tells the reader whether the gap was forced or chosen.

They show how severe the defects found so far are, since a blocked test usually indicates a defect of higher severity than one that merely failed

A blocker often is severe, but the category records that the test could not run rather than grading the severity of anything.

They show the efficiency of the test team, because blocked and skipped tests represent effort that was spent without producing a result

Blocked tests are usually not executed at all, so they do not represent consumed execution effort, and neither category is a productivity measure.

They show which requirements have been covered, since a skipped test leaves its requirement uncovered while a blocked test still contributes partial coverage

A blocked test contributes no coverage either, and requirements coverage is reported through its own metric rather than inferred from these two categories.

Why

Passed and failed describe outcomes of tests that ran. Blocked tests could not run because something outside the test prevented it, such as a defect or an unavailable environment; skipped tests were deliberately not run. Reporting them separately shows how much of the planned coverage was never obtained and why, which a pass/fail split conceals.

Question 29

A weekly report shows 42 open defects, unchanged from last week, and a steering…

A weekly report shows 42 open defects, unchanged from last week, and a steering group member concludes that defect management has stalled. Looking behind the snapshot, the test manager sees that 31 defects were closed and 31 new ones were raised during the week, that the new ones cluster in a module integrated three days ago, and that the average age of the open set has fallen from 19 days to 11. Analysing snapshot against trend as the syllabus requires, what is the correct interpretation?

Defect management is working faster than before — the falling average age shows quicker resolution — and the stable open count reflects an inflow concentrated in the newly integrated module, which is where attention now belongs

Correct answer

Correct. The snapshot is flat because inflow and outflow happen to match; the trends show both improved throughput and a specific new source of defects.

The steering group member is right — an open count that does not fall over a full week means resolution is not keeping pace with the rate at which defects are being reported

Closures matched the inflow exactly, so resolution is keeping pace; the flat count is the arithmetic result of equal flows, not evidence of a backlog growing.

The report should be replaced with a single trend line of open defects over time, since snapshots of the open count mislead readers and add nothing to a trend chart

Snapshots have a legitimate role in reporting the current position; the syllabus asks for them to be read alongside trends rather than discarded.

The falling average age is the concerning figure — it suggests defects are being closed prematurely, which typically shows up later as a rise in re-opened reports

Premature closure is a real phenomenon, but it is evidenced by the re-opened rate, which is not reported here; age alone does not support that reading.

Why

The syllabus distinguishes a snapshot, which gives the position at a moment, from a trend, which shows direction over time; a single figure can be stable while the underlying flow changes substantially. Here the constant open count masks a high turnover, a falling average age indicating faster resolution, and an inflow concentrated in a newly integrated module.

Question 30

A test manager must advise whether to stop testing. The evidence: requirements…

A test manager must advise whether to stop testing. The evidence: requirements coverage 96%, risk coverage 100% of high risks and 78% of medium risks, defect discovery rate down from 14 per day to 2 per day over the last five days, three open defects all of low severity, and effort consumed at 118% of the estimate. Analysing how the syllabus says metrics should be combined for this decision, which conclusion is best supported?

The combined picture supports recommending a stop, with the 22% of medium risks not yet covered reported explicitly as residual exposure for the stakeholders to accept

Correct answer

Correct. Coverage, discovery rate and severity converge, and the one genuine gap is surfaced rather than averaged away.

Testing should stop because effort has already reached 118% of the estimate, which is the clearest signal that the value of further testing has fallen below its cost

Budget consumption says nothing about the quality of the product; using it as the stopping signal is exactly the single-metric reasoning the syllabus warns against.

Testing should continue until medium-risk coverage also reaches 100%, since a risk-based approach is only complete when every identified risk has been covered

Risk-based testing deliberately allocates less effort to lower risks; requiring full coverage of every level would remove the prioritisation the approach exists to provide.

Testing should continue because a falling defect discovery rate is more likely to indicate tester fatigue or exhausted test data than a genuine improvement in quality

That alternative explanation is worth checking, but here it is contradicted by the coverage figures and the low severity of what remains open.

Why

The syllabus warns that no single metric supports a stop-testing decision and that metrics must be combined and read against the agreed exit criteria. Here coverage of high risks is complete, discovery has flattened, and the remaining defects are low severity — a converging picture. The effort overrun is a project fact that does not itself indicate product quality, and the medium-risk gap is the one item that should be reported as residual exposure.

Question 31

The syllabus lists coverage as one of the metric groups used in test reporting, and names several bases against which coverage can be measured. Which TWO of the following are coverage bases the syllabus identifies?

Risk coverage — the proportion of identified product risks exercised by at least one executed test

Correct answer

Correct. Risk coverage is named explicitly and is the coverage basis most closely tied to a risk-based approach.

Code coverage — measured as statements, branches, paths or conditions exercised by the tests

Correct answer

Correct. The syllabus names code coverage and lists these four sub-measures within it.

Effort coverage — the proportion of the estimated test effort that has been consumed to date

Consumed effort belongs to the costs and effort metric group; it measures spend rather than what has been exercised.

Defect coverage — the proportion of reported defects that have reached a terminal status in the workflow

This describes defect resolution progress, which is reported under defect metrics rather than as a coverage basis.

Environment coverage — the proportion of scheduled test environment hours that were actually available

Environment availability is a constraint tracked during monitoring and control; the syllabus does not list it as a coverage basis.

Why

Coverage may be measured against requirements, against risks, and against code, where code coverage itself subdivides into statements, branches, paths and conditions. Effort consumed and defect counts are reported in their own metric groups and are not coverage bases.

Question 32

A test manager estimates the effort for a migration test cycle using three-point…

A test manager estimates the effort for a migration test cycle using three-point estimation. The optimistic estimate is 18 person-days, the most likely estimate is 27 person-days, and the pessimistic estimate is 54 person-days. Applying the weighted three-point formula the syllabus describes, what is the resulting estimate, and what does the spread indicate?

30 person-days; the wide optimistic-to-pessimistic spread signals high estimation uncertainty, which should be reported with the figure rather than hidden behind it

Correct answer

Correct. (18 + 108 + 54) / 6 = 30, and the three-fold spread is exactly the uncertainty the technique is meant to expose.

33 person-days; the wide spread signals high estimation uncertainty, which should be reported with the figure rather than hidden behind it

33 is the simple average of the three values. The weighted formula gives the most likely value four times the weight of the others, which yields 30.

27 person-days; the most likely value is retained because the optimistic and pessimistic values cancel each other out around it

They do not cancel: the pessimistic value lies 27 days above the most likely one while the optimistic lies only 9 below, so the weighted result is pulled upwards.

36 person-days; the pessimistic value carries the heaviest weight because migration work is the category most prone to overrun

The formula weights the most likely value most heavily regardless of the type of work; no variant weights the pessimistic value above it.

Why

Three-point estimation weights the most likely value four times against one weighting each for the optimistic and pessimistic values: (18 + 4 × 27 + 54) / 6 = (18 + 108 + 54) / 6 = 180 / 6 = 30 person-days. The wide gap between optimistic and pessimistic indicates high uncertainty, which the syllabus treats as a signal that the estimation error is large and should be communicated alongside the figure.

Question 33

The syllabus organises the factors that may influence test effort into five groups. Which option lists them?

Product, development process, people, test results, and test context

Correct answer

Correct. These are the five groups named in the chapter on test estimation.

Product, technology, tooling, budget, and delivery deadline

Tooling and budget are considerations within the context and process groups, but this is not the five-group classification the syllabus defines.

Effort, time, cost, quality, and scope

These are the dimensions being traded off in the time-cost-quality triangle, not the factors that influence how much effort testing will take.

Requirements volatility, team seniority, defect density, environment stability, and regulatory load

Each of these is a real influence, but they are individual factors drawn from several groups rather than the grouping itself.

Why

The five groups of factors influencing test effort are product factors, development process factors, people factors, test results factors, and test context factors. Effort, time and cost are what is being estimated; they are related through the time-cost-quality triangle rather than being factor groups.

Question 34

A test manager must estimate two efforts in the same programme. Effort A is the…

A test manager must estimate two efforts in the same programme. Effort A is the twelfth annual regression cycle of a stable reporting product: five years of effort and defect data exist, the scope is well understood, and the technology has not changed. Effort B is the first release of a machine-learning recommendation engine: no historical data exists, the architecture is still being decided, two domain experts are available for half a day, and the estimate is needed by Friday. Analysing the selection factors the syllabus gives for estimation techniques, which pairing is appropriate?

Metric-based estimation for A, because five years of comparable data exist and the work is well understood; expert-based estimation for B, because there is no data to extrapolate from and the novelty is high

Correct answer

Correct. This applies the syllabus heuristic directly: data availability and low complexity favour metric-based, novelty and absent data favour expert-based.

Expert-based estimation for A, because experienced testers know the regression cycle better than any historical figure does; metric-based estimation for B, using industry benchmark data for machine-learning projects

This reverses the heuristic and substitutes external benchmarks for organisational data, which the syllabus does not treat as an equivalent basis.

Metric-based estimation for both, since a consistent estimation technique across a programme is what makes the two efforts comparable when they are reported together

Comparability is desirable but cannot be bought by applying a data-driven technique where no relevant data exists; the resulting figure for B would be false precision.

Expert-based estimation for both, with a full Wideband Delphi round for each, since expert judgement is the more accurate technique whenever domain experts can be convened

Expert-based is not inherently more accurate, and a full Delphi round is hard to reconcile with two experts available for half a day and a Friday deadline.

Why

The syllabus offers a printed heuristic: low complexity with available historical data favours metric-based estimation, while high complexity and novelty favour expert-based estimation. It also matches the expert technique to the lifecycle — Wideband Delphi in sequential contexts, planning poker in Agile ones — and notes that expert availability and time constraints are themselves selection factors.

Question 35

A team is designing its defect workflow. The proposal has the statuses OPEN, IN…

A team is designing its defect workflow. The proposal has the statuses OPEN, IN PROGRESS, RESOLVED, CLOSED, REJECTED, RE-OPENED and DEFERRED, with CLOSED and REJECTED both terminal, and the same engineer permitted to move a report from IN PROGRESS to RESOLVED and then from RESOLVED to CLOSED. Applying the good practices the syllabus lists for defect workflows, which change is required?

Require a different role to make the transition from RESOLVED to CLOSED, and route REJECTED into the single terminal status rather than treating it as a second endpoint

Correct answer

Correct. It fixes both breaches: no one confirms their own fix, and the workflow ends at one place so completion is unambiguous.

Remove DEFERRED and RE-OPENED, because a workflow with seven statuses is too heavy for a team to maintain and only the core four are needed

The syllabus does not cap the number of statuses; DEFERRED and RE-OPENED carry information that the core four cannot express.

Make RESOLVED terminal as well, so that a report reaching a fix is recorded as complete without waiting for retest to be scheduled

This adds a third terminal status and would mark defects complete before the fix has been confirmed by retesting.

Allow only one outgoing transition from each non-terminal status, so the path a defect takes through the workflow is predictable and easy to report on

The good practice is the opposite: non-terminal statuses should offer more than one outgoing transition, since a report in progress may be fixed, rejected or deferred.

Why

Among the good practices the syllabus lists are: the workflow should have a single terminal status; adjacent statuses should be handled by different roles so that no one confirms their own work; and non-terminal statuses should have more than one outgoing transition. Here the same engineer closing what they resolved breaches the role-separation practice, and having two terminal statuses breaches the single-terminal-status practice.

Question 36

During defect management the syllabus distinguishes a failure, an anomaly, a defect report and a false-negative result. Which description of a false-negative result is correct?

A test passes even though a defect is present in the test object, so the defect goes unreported

Correct answer

Correct. The result is negative for a defect while a defect exists, which is what makes it false.

A test reports a failure that is later traced to faulty test data rather than to a defect in the test object

This describes a false-positive result: a failure is signalled where the test object is sound.

A defect report is rejected by the development team because the described behaviour matches the specification

That is a rejected defect report, which is a workflow outcome rather than a property of the test result.

An anomaly is observed but no defect report is raised because the tester cannot reproduce the behaviour a second time

This describes an unreproducible anomaly; the test did detect something, so the result was not negative.

Why

A false-negative result is one in which a test does not detect a defect that is present — the test passes although the product is faulty. It is distinct from a false-positive, where a test reports a failure that turns out not to be caused by a defect in the test object, for example a problem in the test data or environment.

Question 37

A test manager reviews a defect report that contains only a title and the sentence "Totals wrong on the invoice screen after the last build." Applying the syllabus guidance on the information a defect report must carry, which revision makes it usable?

Add a description with the steps to reproduce, the expected and actual totals, and assign both a severity and a priority so the report can be triaged

Correct answer

Correct. These are the reporter-supplied fields the syllabus requires, and together they let someone else reproduce and rank the problem.

Add the identifier, the reporter's name and the date and time the report was raised, since a report without traceability information cannot be tracked

These fields are generated by the defect management tool rather than entered by the reporter, so adding them manually does not fix what is missing.

Add a severity value only, and let priority be derived automatically from severity so that triage effort is reduced

Severity and priority are separate judgements — technical consequence and business urgency — and deriving one from the other collapses that distinction.

Add every field the tool offers, including root cause, phase of introduction and estimated fix effort, so the report supports later statistical analysis

Root cause and phase of introduction are valuable when they are used, but the syllabus advises collecting only what will be used, and most of these cannot be known at reporting time.

Why

The syllabus names title, description including steps to reproduce, severity and priority as the fields that must be supplied by the reporter, alongside fields the tool generates automatically such as identifier, reporter and timestamp. It also advises collecting only information that will actually be used, so adding fields nobody consumes is not an improvement.

Question 38

What is the purpose of a defect taxonomy as the syllabus describes it?

It provides a categorised list of defect types so that defects can be classified consistently, revealing recurring categories that inform process improvement and future test design

Correct answer

Correct. The value of the taxonomy lies in consistent classification, which turns individual reports into a pattern that can be acted on.

It provides an agreed mapping from defect severity to defect priority so that triage decisions are made consistently across teams

That would be a triage policy. A taxonomy classifies what kind of defect it is, not how urgently it should be fixed.

It provides the sequence of statuses a defect report passes through, so that every team uses the same defect workflow

The sequence of statuses is the defect workflow, a separate concept described alongside the taxonomy.

It provides a list of the defects known to remain open at release, so that support staff can recognise them when customers report symptoms

That is a known-issues list produced at release; a taxonomy is a classification scheme that exists independently of any one release.

Why

A defect taxonomy is a categorised list of defect types. Classifying defects against it allows patterns to be seen across releases and projects — which categories recur, where they originate — and this supports both process improvement and the design of tests targeting the categories that occur most.

Question 39

Analysis of a year of defect data shows: 61% of defects classified as…

Analysis of a year of defect data shows: 61% of defects classified as requirements defects were first detected during system testing rather than during requirements review; 44% of all defects are concentrated in two of the seventeen components; and the proportion of defect reports later marked as duplicates has risen from 4% to 17%. Analysing these three findings against the uses of defect statistics the syllabus describes, which set of conclusions is best supported?

A phase containment failure in requirements review, a defect cluster in two components that should attract more test effort, and a decline in the quality of defect reporting indicated by the duplicate rate

Correct answer

Correct. Each finding maps onto a distinct documented use of defect statistics, and the duplicate rate is read as a reporting-quality indicator rather than a product signal.

A phase containment failure in requirements review, a defect cluster in two components, and a decline in the quality of debugging indicated by the duplicate rate

The first two are right, but debugging quality is assessed through the re-opened rate; duplicates reflect how reports are raised and searched, not how fixes are made.

An effective system test level catching requirements defects, a well-targeted test effort concentrated on two components, and improved tester diligence shown by more reports being raised

Reading a containment failure as a success reverses the point of phase containment analysis, and duplicates represent rework rather than diligence.

A defect cluster spread across requirements activity, a phase containment failure in the two concentrated components, and a defect taxonomy that needs additional categories

The cluster and containment findings are attached to the wrong observations, and nothing in the data indicates that the taxonomy lacks categories.

Why

The syllabus describes using defect statistics for phase containment analysis, for defect cluster analysis, and for assessing the quality of debugging and of defect reporting. Requirements defects escaping to system testing is a phase containment failure pointing at the review process. Concentration in two components is a defect cluster, which argues for redirecting test effort there. A rising duplicate rate is one of the named indicators of defect report quality, not of product quality.

Question 40

In an Agile team, a tester finds a defect in a user story that is still in…

In an Agile team, a tester finds a defect in a user story that is still in progress within the current iteration and fixes it with the developer at the desk within twenty minutes. The syllabus discusses when a defect report should nonetheless be created in Agile contexts. What is the position it takes?

No report is needed for this case, but a report is still created where the defect affects previously delivered functionality, cannot be fixed within the iteration, or must be recorded for regulatory or organisational reasons

Correct answer

Correct. The trigger is whether the report serves a purpose beyond the immediate conversation, which is how the syllabus frames the decision.

A report is required for every defect found by a tester, since defect statistics lose their value as soon as some categories of defect are left out of the data

Consistency in the data matters, but the syllabus explicitly allows in-iteration defects on work in progress to go unreported where nothing depends on the record.

No report is ever needed in an Agile team, because unresolved items move to the product backlog as user stories and the backlog replaces defect tracking

The backlog does receive unresolved items, but it does not remove the cases where the syllabus says a defect report should still be raised.

A report is required whenever the fix takes longer than the time-box the team has agreed for informal fixes, which is the single criterion the syllabus applies

Duration is not the criterion; the syllabus lists several situations, most of which concern the scope and traceability of the defect rather than how long the fix takes.

Why

In Agile teams a defect found and fixed inside the iteration on work still in progress often does not warrant a formal report, because the report's coordination purpose is served by the conversation. The syllabus nevertheless names situations in which a report is created regardless — for example where the defect is in previously delivered functionality, where it cannot be fixed within the iteration and moves to the product backlog, or where regulation or the organisation requires a record.

Question 41

Development and test disagree repeatedly on the priority of defects: developers…

Development and test disagree repeatedly on the priority of defects: developers close reports as "working as designed" while testers re-open them, and the same arguments recur in the daily stand-up. Applying the mechanisms the syllabus describes for defect management, what should the test manager put in place?

A triage meeting with representatives from development, test and the business, which decides severity, priority and disposition for disputed reports on an agreed cadence

Correct answer

Correct. Triage is the named mechanism for reaching a cross-role decision on disposition, which is exactly what is being argued about.

A rule that only the reporting tester may close a defect report, so that a developer cannot dispose of a report the tester still considers valid

Giving one side the final word suppresses the disagreement rather than resolving it, and it conflicts with the practice of separating roles across adjacent statuses.

A defect taxonomy covering the disputed categories, so that classification determines disposition and the argument no longer needs to be had

A taxonomy classifies the kind of defect; it does not decide whether a given report is valid or how urgently it should be addressed.

A weekly report of re-opened defects sent to the programme manager, so that the pattern becomes visible to someone with the authority to intervene

The metric makes the problem visible and is worth having, but visibility alone does not supply the forum in which the disposition is actually agreed.

Why

The syllabus describes the defect management committee, which sets and maintains the defect management process, and the triage meeting, in which representatives of the relevant parties agree the severity, priority and disposition of reported defects. A defect manager may be appointed to own the process. Recurring disputes over disposition are what the triage mechanism exists to resolve.

Chapter 3 · Managing the Team — 9 questions

Question 42

The syllabus describes four areas of competence for members of a test team and notes which of them ISTQB certification primarily develops. Which option is correct?

Professional, methodological, social and personal competence, with ISTQB certification primarily developing professional competence

Correct answer

Correct. These are the four areas named, and the syllabus is explicit that certification addresses the professional one.

Professional, methodological, social and personal competence, with ISTQB certification primarily developing methodological competence

The four areas are right but the attribution is not: certification builds the body of testing knowledge, which the syllabus places under professional competence.

Technical, analytical, interpersonal and leadership competence, with ISTQB certification primarily developing technical competence

These labels resemble the real ones but are not the four areas the syllabus cites from the competence literature it references.

Domain, tooling, process and communication competence, with ISTQB certification primarily developing process competence

Domain and tooling knowledge are elements within professional competence rather than areas in their own right in this model.

Why

The four areas of competence are professional competence, methodological competence, social competence and personal competence. ISTQB certification primarily develops professional competence — the domain and testing knowledge itself — while the other three are developed largely through experience, coaching and team work.

Question 43

A test manager is staffing a team for a telemedicine platform: two-week…

A test manager is staffing a team for a telemedicine platform: two-week iterations, a microservice architecture with heavy use of message queues, strict clinical safety requirements, and a customer who reviews test documentation formally. The available people are: a tester with eight years of clinical domain knowledge but no automation experience; an automation engineer strong in API and message-queue testing with no domain background; and a junior tester with good written communication and six months of experience. Analysing the mapping of skills to test process activities that the syllabus requires, which allocation is best supported?

Domain expert on risk analysis and test analysis for the clinical scenarios; automation engineer on test implementation against the services and queues; junior tester on test documentation and execution support, paired with the domain expert

Correct answer

Correct. Each person is placed on the activities their competence actually drives, and the junior tester is developed through pairing rather than left unsupported.

Automation engineer on risk analysis and test analysis, since the architecture is where the technical risks live; domain expert on manual execution; junior tester on building the automation framework

Clinical risk analysis needs domain judgement, and asking a tester with six months of experience to build the framework inverts the skill mapping entirely.

Rotate all three through every activity in each iteration, so that the team becomes cross-functional and no single point of knowledge remains in a safety-related product

Reducing key-person dependency is a legitimate aim, but full rotation from the outset places uncertified judgement on clinical risk analysis in a safety-related context.

Domain expert on risk analysis and test analysis; automation engineer on test implementation; junior tester assigned to a training course for the whole first iteration to close the experience gap

The first two placements are sound, but removing a third of the team from a two-week iteration for formal training ignores the on-the-job development the syllabus also lists.

Why

The syllabus asks for skills to be mapped onto the activities of the test process, with the context drivers — domain, architecture and technologies, and the SDLC model — taken into account. Here the domain expert is most valuable where clinical judgement determines what to test, the automation engineer where the architecture determines how it is exercised, and the junior tester where documentation quality matters and supervision is available.

Question 44

A newly formed test team has moved past its polite first weeks and is now arguing openly about test approach, tool choice and who decides what. Using the team development model the syllabus cites, which phase is this, and what is the manager's task in it?

Storming — the manager surfaces and mediates the conflicts so they resolve into agreed ways of working, rather than suppressing the disagreement

Correct answer

Correct. Storming is the expected conflict phase, and moving through it is what allows norming to follow.

Forming — the manager provides clear direction and structure, because the team is still orienting itself and has not yet built working relationships

Forming is the polite orientation phase the team has already passed; the open argument is the marker that it has moved on.

Norming — the manager steps back and lets the team consolidate the working agreements it has already reached about approach and tooling

Norming follows storming and is characterised by settled agreements; here the agreements are precisely what is being contested.

Adjourning — the manager focuses on capturing lessons learned, because visible conflict at this stage indicates the team is close to disbanding

Adjourning is the disbanding phase at the end of a team's life and has nothing to do with early conflict over approach.

Why

The syllabus uses Tuckman's model: forming, storming, norming, performing and adjourning. Storming is the phase in which conflict over roles, approach and influence surfaces. The manager's task is to let the disagreements be worked through and steered towards agreed ways of working, rather than suppressing them or treating the team as broken.

Question 45

The syllabus draws on Herzberg's motivation-hygiene theory to distinguish motivators from hygiene factors in a test team. Which TWO of the following does that theory classify as motivators rather than hygiene factors?

Recognition of a tester's contribution by the team and by stakeholders outside it

Correct answer

Correct. Recognition arises from the work itself and is one of the motivators named in the model.

Responsibility and autonomy over how a tester's own area of work is approached

Correct answer

Correct. Responsibility and autonomy are motivators, since they change the nature of the work rather than its surroundings.

Remuneration and the terms on which it is reviewed

Herzberg places remuneration among the hygiene factors: inadequate pay causes dissatisfaction, but improving it does not produce lasting motivation.

The personnel policy the organisation applies to the team

Personnel policy surrounds the work rather than forming part of it, which places it among the hygiene factors.

Interpersonal relationships between team members and with neighbouring teams

Relationships are listed as a hygiene factor: poor ones demotivate, but good ones are treated as a baseline condition rather than a motivator.

Why

In Herzberg's model, motivators arise from the work itself: recognition, responsibility and autonomy, meaningful tasks, and advancement. Hygiene factors surround the work: remuneration, personnel policy, working conditions, job security and interpersonal relationships. Their absence causes dissatisfaction, but their presence does not by itself motivate.

Question 46

The syllabus names a skills matrix as one instrument for assessing a test team, alongside Belbin's Team Roles, DISG and PCM. What distinguishes the skills matrix from the other three?

The skills matrix records what competencies each person holds and at what level, exposing gaps and single points of knowledge; the other three describe behavioural or personality preferences within a team

Correct answer

Correct. One instrument maps capability, the others map working and interaction styles.

The skills matrix describes how team members prefer to interact, while Belbin, DISG and PCM each measure the technical competencies required for a given test activity

This reverses the two: the three named models are behavioural, and none of them measures technical competency.

The skills matrix is applied to individuals while Belbin, DISG and PCM are applied only to the team as a whole and produce no individual results

All four are applied to individuals; the models are then used to reason about the composition of the team.

The skills matrix is used during recruitment only, whereas Belbin, DISG and PCM are used for ongoing performance appraisal of existing staff

The skills matrix is maintained continuously to guide development and allocation, and the behavioural models are not appraisal instruments.

Why

A skills matrix records which competencies each team member holds and at what level, so gaps and single points of knowledge become visible. Belbin's Team Roles, DISG and PCM are behavioural or personality models used to understand how people prefer to work and interact within a team, which is a different question from what they can do.

Question 47

An organisation records the following per-defect figures: defect prevention…

An organisation records the following per-defect figures: defect prevention costs 150, appraisal costs 450, internal failure costs 250, external failure costs 3,600. Applying the formula the syllabus gives for the average saving achieved by finding a defect before release rather than after it, what is the result?

2,900 — the external failure cost less the sum of appraisal and internal failure costs

Correct answer

Correct. 3,600 − (450 + 250) = 2,900, which is the formula applied exactly as printed.

2,750 — the external failure cost less the sum of prevention, appraisal and internal failure costs

Prevention costs are incurred whether or not a given defect occurs and are not subtracted in this formula, which is why the printed version omits them.

3,350 — the external failure cost less the internal failure cost, since appraisal is a fixed overhead independent of any individual defect

Appraisal cost per defect is exactly the cost of finding it, so leaving it out overstates the saving that early detection produces.

4,450 — the sum of all four cost categories, representing the full cost the organisation avoids by preventing the defect

Adding the categories gives the total cost of quality per defect, not the saving from finding one earlier rather than later.

Why

The syllabus gives Average Savings per Defect = Average External Failure Costs − (Average Appraisal Costs + Average Internal Failure Costs). Here that is 3,600 − (450 + 250) = 3,600 − 700 = 2,900. Prevention costs are part of the total cost of quality but are not part of this particular formula.

Question 48

A finance director proposes cutting the test budget by a third, arguing that the…

A finance director proposes cutting the test budget by a third, arguing that the last two releases went out with no production incidents and that testing therefore produced no measurable return. Analysing this against the cost-benefit view of testing the syllabus sets out, which response is best supported?

Quantify the defects found before release and the external failure costs they would have carried, and present the clean releases as the outcome the appraisal spend bought rather than as evidence it was unnecessary

Correct answer

Correct. It converts an invisible benefit into a figure and directly addresses the inference the finance director has drawn.

Accept the reduction for one release and measure the resulting production incidents, so that the value of the removed testing is demonstrated with evidence rather than argument

Proving the point by allowing failures reaches customers first and treats external failure cost — the most expensive category — as an acceptable experiment.

Argue that the cost of quality cannot be quantified for testing, and defend the budget on the qualitative grounds of reputation, customer trust and regulatory standing

The qualitative benefits are real and worth raising, but the syllabus provides explicit formulas for the quantitative side, so conceding that it cannot be measured gives away the stronger argument.

Redirect the whole remaining budget into defect prevention activities such as reviews and static analysis, since prevention costs are the lowest of the four cost of quality categories

Prevention is cheaper per defect, but shifting the entire budget away from appraisal removes the detection that produced the clean releases in the first place.

Why

The syllabus asks the test manager to argue the cost-benefit relationship of testing in both qualitative and quantitative terms, and notes that the benefit of testing appears as failures that did not happen — which is invisible unless it is expressed as avoided external failure costs. The absence of incidents is evidence of the appraisal activity working, not of its being unnecessary; the Boehm curve makes the same point about the rising cost of late detection.

Question 49

The cost of quality model used in CTAL-TM divides costs into four categories. Which option lists them, and which classification does a defect found during system testing and fixed before release fall into?

Defect prevention, appraisal, internal failure and external failure costs — the rework of fixing that defect before release is an internal failure cost

Correct answer

Correct. Internal failure covers rework from defects caught before the product reaches the customer.

Defect prevention, appraisal, internal failure and external failure costs — the rework of fixing that defect before release is an appraisal cost

The categories are right, but appraisal covers the effort of evaluating the product; the repair effort that follows is internal failure cost.

Planning, detection, correction and warranty costs — the rework of fixing that defect before release is a correction cost

These labels describe the same idea loosely but are not the four categories the syllabus names, and the model it cites uses the prevention-appraisal-failure structure.

Direct, indirect, fixed and variable quality costs — the rework of fixing that defect before release is a direct variable cost

This is a general accounting classification rather than the cost of quality model applied in this chapter.

Why

The four categories are defect prevention costs, appraisal costs, internal failure costs and external failure costs. A defect found during testing and repaired before release generates internal failure costs — the rework arising from a failure that the customer never experienced. The activity of finding it is appraisal cost; the repair is internal failure cost.

Question 50

A skills matrix shows that only one tester in a team of six can perform…

A skills matrix shows that only one tester in a team of six can perform performance test analysis, and that person is leaving in four months. Budget for external training is available but the next public course runs in five months. Applying the skill development approaches the syllabus lists, what should the test manager arrange?

Have the departing expert mentor two colleagues on live performance analysis work over the coming months, supported by self-study, and book the course afterwards to consolidate what they have learned

Correct answer

Correct. Mentoring and training on the job transfer the knowledge inside the window that actually exists, and the course then reinforces it.

Book two testers onto the public course in five months and accept a two-month gap in performance test analysis capability, since formal training is the most reliable transfer of knowledge

It leaves the capability gap open and forgoes the one chance to learn directly from the person who holds the knowledge.

Ask the departing expert to write a detailed handover document covering the performance analysis approach, so the knowledge remains available to whoever needs it later

Documentation is a useful supplement, but the syllabus treats competence as developed through practice and interaction; a document alone does not create an analyst.

Outsource performance test analysis to a specialist supplier for the next year, and revisit whether the skill needs to exist inside the team at all

Buying the capability is a valid commercial option, but it answers a sourcing question rather than the skill development question the matrix has raised.

Why

The syllabus lists training and education, self-study, peer learning, mentoring and coaching, and training on the job as ways to develop competence. When a formal course cannot be scheduled in time, the workable route is to have the departing expert mentor one or two colleagues on real work before leaving, supported by self-study, with formal training used afterwards to consolidate.