ISTQB Advanced (CTAL-TM v3.0) Mock Exam #1 — Questions & Answers
Every question in this mock exam, with the correct answer marked and a written rationale behind each option below — for reading and review, not a timed run.
Chapter 1 · Managing the Test Activities — 26 questions
CTAL-TM v3.0 restructures the test process around three management activities aligned with ISO/IEC/IEEE 29119-2, rather than the seven-activity view used at Foundation Level. Which set of three activities does the syllabus use?
Test planning; test monitoring and control; test completion
Correct. These are the three management activities named in the syllabus and mapped onto ISO/IEC/IEEE 29119-2's dynamic test process.
Test planning; test analysis and design; test execution and reporting
Analysis, design and execution are activities the test manager plans and monitors, but the syllabus classifies them as test process work rather than as management activities.
Test strategy definition; test implementation; test closure reporting
Strategy definition sits inside planning and closure reporting inside completion, so this list splits and renames the real grouping rather than reproducing it.
Risk identification; risk analysis; risk mitigation planning
These are the steps of risk management, which the syllabus treats as an input to planning and control, not as the top-level structure of the test process.
The syllabus groups the test manager's work into test planning, test monitoring and control, and test completion. The remaining Foundation activities (analysis, design, implementation, execution) are the day-to-day work being managed, not management activities in their own right.
A release has passed its exit criteria and the test manager is closing out the test effort. Which TWO of the following are described in the syllabus as test completion activities?
Archiving the testware and handing it over to the group that will maintain the product
Correct. Archiving and handover of testware is one of the named completion activities, so the assets remain usable for maintenance testing.
Restoring the test environment to an agreed state so it can serve the next effort
Correct. Restoring or decommissioning the test environment is explicitly listed among the test completion activities.
Comparing actual test progress against the planned progress and issuing control directives
This is test monitoring and control: it happens while testing is running so that deviations can still be corrected, not after the effort is closed.
Identifying product risks with stakeholders and agreeing which risk treatment applies to each
Product risk identification and treatment selection is a planning activity, performed before the risks in question have been tested.
Assessing test readiness and confirming that entry criteria for the next test level are met
Test readiness assessment is part of monitoring and control, gating the start of a level rather than closing the effort down.
Test completion covers producing the test completion report, archiving and handing over testware, restoring the test environment, and capturing lessons learned. Measuring progress against the plan and issuing control directives belong to monitoring and control, which runs during execution.
A test manager maps stakeholders onto the power-interest matrix. The chief…
A test manager maps stakeholders onto the power-interest matrix. The chief financial officer controls the test budget and can cancel the automation programme, but has shown no interest in the test approach and does not attend test reviews. Which stakeholder category does the syllabus assign to this position?
Latent — high influence over the test effort combined with currently low interest in it
Correct. Latents hold enough power to affect the effort but are not engaged, so the manager's task is to keep them satisfied and watch for their interest rising.
Promoter — high influence over the test effort combined with high interest in it
Promoters are actively engaged as well as powerful; this stakeholder attends nothing and has expressed no view on the approach.
Defender — low influence over the test effort combined with high interest in it
Defenders care about testing but cannot direct it. Here the situation is reversed: the stakeholder can cancel the programme outright.
Apathetic — low influence over the test effort combined with low interest in it
The low-interest half fits, but budget authority and the ability to cancel the programme are exactly what distinguishes high power from low.
The power-interest matrix in the syllabus names four categories: Promoters (high power, high interest), Latents (high power, low interest), Defenders (low power, high interest) and Apathetics (low power, low interest). High influence combined with low interest is the Latent quadrant.
A payments platform runs a quality gate between system testing and user…
A payments platform runs a quality gate between system testing and user acceptance testing. The agreed gate criteria are: no open defects of severity 1 or 2, requirements coverage at or above 95%, and all high product risks covered by at least one executed test. Two days before the gate the test manager has: zero severity 1 defects, one open severity 2 defect in the refund flow with a fix merged but not yet retested, requirements coverage at 97%, and two of the eleven high product risks covered only by tests that are currently blocked by an unavailable third-party sandbox. The programme manager asks to pass the gate now and "catch up during UAT". Analysing the situation against the purpose of a quality gate, what is the most defensible response from the test manager?
Report that two criteria are unmet — an unretested severity 2 fix and two high risks with no executed coverage — quantify the residual risk each represents for UAT, and put the exception decision to the accountable stakeholders in writing
Correct. The gate's evidence is compared against the agreed criteria, the shortfall is expressed as residual risk, and the waiver becomes an explicit decision owned by the stakeholders who carry that risk.
Pass the gate on the strength of the coverage figure of 97% and the absence of severity 1 defects, and record the sandbox problem in the test completion report at the end of UAT
Meeting one criterion does not compensate for another, and deferring the record of two uncovered high risks to a report written after UAT removes the stakeholders' chance to act on it.
Refuse to pass the gate until every criterion is met, since the criteria were agreed with the stakeholders and the test manager is accountable for enforcing them
The test manager evaluates and reports against the criteria, but the decision to accept residual risk sits with the accountable stakeholders; taking it away turns a business trade-off into a unilateral one.
Relax the coverage criterion to 90% and lower the two affected risks from high to medium, so that the gate criteria reflect what the third-party sandbox outage makes achievable
Rewriting criteria and risk levels to match the current results makes the gate unable to fail, which removes the only reason for having one.
A quality gate is a decision point where objective evidence is compared against agreed criteria; its value comes from the criteria being applied as agreed, and from any exception being an explicit, owned decision rather than a silent one. Here two criteria are unmet in substance: the severity 2 defect is not retested, so its closure is unevidenced, and two high risks carry no executed coverage at all, which is the criterion with the largest residual exposure. The manager's job is to report that position with its residual risk and let the accountable stakeholders decide, not to redefine the criteria or to block unilaterally.
The syllabus compares sequential (V-model) and iterative (Scrum) lifecycles across aspects such as estimation, testware, roles, tools, monitoring and metrics. Which statement describes the difference in monitoring and reporting between the two?
Sequential reporting is formal and tied to milestones and phase transitions, whereas iterative reporting is continuous and owned largely by the team through short-interval, highly visible measures
Correct. The reporting rhythm follows the lifecycle: phase gates in sequential development, per-iteration and daily visibility in Scrum.
Sequential reporting is continuous and team-owned, whereas iterative reporting is produced by the test manager at each release milestone
This inverts the two lifecycles; continuous team-owned reporting is the iterative pattern, not the sequential one.
Both lifecycles report on the same fixed set of metrics, so only the frequency of the report differs between them
Frequency does differ, but so does the metric set: iterative teams favour velocity and burndown, sequential projects favour phase-based progress and coverage against a baselined plan.
Iterative development reports only on defects found, while sequential development reports only on test cases executed
Neither lifecycle restricts itself to a single measure; both track defects, progress and coverage, differing in formality and cadence rather than in scope.
In a sequential lifecycle, monitoring and reporting are formal and phase-oriented, typically produced by the test manager at defined milestones. In an iterative lifecycle they are continuous and largely team-owned, visible on information radiators such as burndown charts and reviewed at short intervals.
Within risk-based testing the syllabus defines the level of a product risk as a combination of two factors. Which pair, and what does the resulting level primarily drive?
Likelihood of occurrence and impact of occurrence; the level drives how early and how intensively the affected area is tested
Correct. This is the definition given in the syllabus, and the prioritisation heuristic it supports is that higher risk means earlier and deeper testing.
Likelihood of occurrence and cost of the corrective fix; the level drives which defects are accepted into the release
Fix cost influences defect triage decisions but is not a factor in the risk level itself, which is assessed before the defects exist.
Impact of occurrence and requirements coverage already achieved; the level drives how many additional test cases are designed
Achieved coverage is a measure of test progress, not a determinant of a product risk's level; the risk exists independently of how much testing has been done.
Defect density observed in the component and the experience of the assigned tester; the level drives who is allocated to the area
Both are real inputs to planning and to skills allocation, and defect density can inform likelihood, but neither is a defining factor of risk level.
Risk level is the combination of the likelihood of the risk occurring and the impact if it does. The resulting level drives the depth and the timing of testing: the higher the risk level, the earlier and the more intensively the associated areas are tested.
A test manager scores four product risks on a five-point scale for likelihood…
A test manager scores four product risks on a five-point scale for likelihood and impact and computes risk exposure as likelihood multiplied by impact. Risk A: likelihood 2, impact 5. Risk B: likelihood 4, impact 3. Risk C: likelihood 5, impact 2. Risk D: likelihood 3, impact 4. Only enough effort exists to test two areas deeply in the first week. Applying the syllabus rule that higher risk is tested earlier and more intensively, which two risks are addressed first, and what tie-break does the syllabus support when exposures are equal?
Risks B and D, both at exposure 12; where exposures tie, the risk with the higher impact is taken first, which puts D ahead of B
Correct. 4 × 3 and 3 × 4 both give 12, and resolving the tie on impact places D (impact 4) ahead of B (impact 3).
Risks A and C, both at exposure 10; where exposures tie, the risk with the higher likelihood is taken first, which puts C ahead of A
The arithmetic on the tie-break is applied to the wrong pair: 10 is the lower exposure, so A and C are the two risks that wait rather than the two addressed first.
Risks A and D, because a risk with the maximum score on either factor outranks any risk whose scores are both mid-range
Treating a maximum on a single factor as decisive abandons the multiplicative definition of exposure; A scores 10 and therefore ranks below both B and D.
Risks C and B, because likelihood is weighted more heavily than impact when test effort has to be rationed under a deadline
The syllabus does not weight likelihood above impact when effort is constrained; exposure remains the product of the two factors regardless of the deadline.
The exposures are A = 10, B = 12, C = 10 and D = 12. B and D are tied at the top with 12 and are addressed first. Where exposures are equal, the syllabus supports resolving the tie on impact, because a high-impact failure is the more damaging outcome and impact is the factor stakeholders can least tolerate; likelihood can also be reduced by other means, whereas impact usually cannot.
A regulated insurance system has 30 product risks spread over six subsystems…
A regulated insurance system has 30 product risks spread over six subsystems. Two subsystems carry almost all of the high risks; the other four carry medium and low risks only. The release date is fixed, the test window is short, and the sponsor has said that the release will not be stopped for medium-risk findings but will be stopped for any high-risk failure. Analysing this context, which execution order does the syllabus support, and why?
Depth-first: test the two high-risk subsystems thoroughly before touching the rest, because the only findings that can change the release decision are concentrated there
Correct. When the consequences and the go/no-go criterion are concentrated in a few areas, depth-first buys the most decision-relevant information in a short window.
Breadth-first: run a shallow pass over all six subsystems first, because an even picture of the whole system is what a fixed release date requires
Breadth-first is the better fit when risk is evenly distributed; here it spends scarce time on four subsystems whose findings cannot stop the release.
Breadth-first, on the grounds that the regulated context obliges the test manager to demonstrate coverage of every subsystem before deepening any single one
Regulation typically constrains which techniques and evidence are required at a given integrity level, but the syllabus does not derive a breadth-first ordering from regulation as such.
Depth-first, but starting with the four medium-risk and low-risk subsystems so that the high-risk areas are tested last against the most stable build
This delays the findings that carry the stop-the-release consequence to the point where there is least time to act on them, which reverses the purpose of risk-based ordering.
The syllabus contrasts depth-first execution, which takes the highest-risk areas and tests them thoroughly before moving on, with breadth-first execution, which spreads a thin layer of testing across all areas before deepening any of them. Depth-first suits a context where the consequences are concentrated in a few areas and the stop-the-release decision hinges on those areas; breadth-first suits a context where risk is evenly spread or where an early overall picture matters more than depth anywhere.
At the end of a risk-based test effort, 11 of 14 high risks have been fully…
At the end of a risk-based test effort, 11 of 14 high risks have been fully covered by passing tests, 2 are covered by tests that pass but exercise only the main flow, and 1 could not be tested because its interface partner never became available. The steering group asks for a one-page status. Analysing what risk-based testing is supposed to produce as its output, what should the report lead with?
The residual risk position: one high risk untested and two covered only on the main flow, each described with the exposure that remains and what would reduce it
Correct. Risk-based testing reports in terms of residual risk, which is precisely the information a steering group needs to accept or reject the release.
The pass rate: 13 of 14 high risks exercised by tests that passed, expressed as a percentage of the risk register
A pass rate hides the two shallow coverages behind the same figure as the eleven thorough ones, so the steering group cannot see the difference in exposure.
The volume of work completed: number of test cases executed, defects raised and defects closed over the test window
These are effort and progress measures. They demonstrate that testing happened without telling the reader what risk remains.
The root cause of the environment problem that prevented the untested risk from being covered, with the corrective action agreed for the next release
This belongs in lessons learned during test completion; it explains why the gap exists but does not state the exposure the steering group must decide on.
The distinctive reporting output of risk-based testing is the residual risk level: which risks remain, at what level, after the testing that was performed. Counts of tests and defects describe the activity; they do not tell a steering group what exposure it is being asked to accept. The untested risk and the two partially covered ones are the residual exposure and belong at the front of the report.
The syllabus divides product risk analysis techniques into heavyweight and lightweight approaches. Which option correctly assigns four named techniques to those two groups?
Heavyweight: Failure Mode and Effect Analysis, fault tree analysis. Lightweight: PRISMA, PRAM
Correct. FMEA and fault tree analysis are the heavyweight analytical techniques; PRISMA and PRAM are the named lightweight approaches.
Heavyweight: PRISMA, SST. Lightweight: Failure Mode and Effect Analysis, hazard analysis
The two groups are swapped: PRISMA and SST are the lightweight approaches, while FMEA and hazard analysis carry the heavier analytical overhead.
Heavyweight: cost of exposure, PRAM. Lightweight: fault tree analysis, SST
Cost of exposure is correctly placed but PRAM is a lightweight technique, and fault tree analysis is heavyweight rather than lightweight.
Heavyweight: hazard analysis, risk workshops. Lightweight: brainstorming, checklists
Risk workshops, brainstorming and checklists are techniques for identifying risks, not the assessment approaches this classification refers to.
Heavyweight techniques named in the syllabus include hazard analysis, cost of exposure, Failure Mode and Effect Analysis, and fault tree analysis. Lightweight techniques include SST (Craig and Jaskiel), PRAM (Black) and PRISMA (van Veenendaal).
A team has run risk-based testing on four consecutive releases. Each time, the same risk register is reused with only minor edits, and participants approve it quickly because "we did this last time". Which of the difficulties named in the syllabus does this describe?
Deja vu — the previous analysis is carried forward rather than genuinely reassessed, so changes in the product and its context go unnoticed
Correct. Deja vu is the named difficulty where a risk analysis is recycled and the review becomes a formality.
Keen beginnings — the risk analysis is performed enthusiastically at the start of a project and then not maintained as the project proceeds
Keen beginnings describes enthusiasm decaying within one project; here the analysis is repeated across releases but without fresh thought.
Stakeholder churn — the participants in the risk analysis change between sessions, so the assessments lose consistency over time
Stakeholder churn concerns changing participants; in this case the same people take part and the problem is the lack of reassessment.
Difficulty in assessing the risk level — participants cannot agree on likelihood and impact values, so the register drifts towards mid-range scores
That difficulty is about the assessment itself being hard; here the scores are accepted without dispute because nobody re-examines them.
The syllabus lists several difficulties with risk-based testing, including difficulty in assessing the risk level, keen beginnings (initial enthusiasm that fades), deja vu (reusing a previous risk analysis without genuine reassessment), key risks being missed, and stakeholder churn.
A product risk has been identified: a third-party currency conversion service…
A product risk has been identified: a third-party currency conversion service may return stale rates during a provider outage, causing incorrect invoice totals. Likelihood is assessed as low, impact as very high. The service cannot be exercised realistically in the test environment, the provider offers a contractual penalty covering financial loss from stale rates, and the development team can add a rate-age check that rejects any rate older than an agreed threshold. Analysing the treatment options in the syllabus, which combination is the most appropriate response?
Mitigate by building and testing the rate-age check, rely on the contract to transfer the residual financial consequence, and record the remaining exposure as accepted by the sponsor
Correct. It combines the mitigation that testing can actually verify, transfers the consequence that testing cannot remove, and makes the leftover exposure an explicit decision.
Accept the risk without further action, since the likelihood is low and the contractual penalty already compensates the organisation for any financial loss
Acceptance may be reasonable for low-impact risks, but a very high impact combined with an available and testable design mitigation makes doing nothing hard to defend.
Treat the risk purely through testing by building a stub that returns stale rates, and raise the likelihood score so that the area receives the most execution effort
The stub is worth building, but inflating the likelihood score to force effort corrupts the risk register, and testing alone leaves the financial consequence untreated.
Transfer the risk entirely to the provider through the contractual penalty and remove it from the risk register, since the organisation no longer bears the financial loss
Transfer moves the financial consequence, not the reputational and operational effects, and a transferred risk stays on the register with its residual level recorded.
The syllabus presents testing as the principal risk mitigation measure but sets it alongside contingency plans, risk transfer and risk acceptance. Where the risk cannot be exercised realistically, mitigation by design plus a contingency, combined with transfer of the financial consequence, addresses the exposure better than testing alone. Accepting the risk outright ignores a very high impact, and simulating the outage in test is still worth doing but does not by itself cover the provider's real behaviour.
An organisation maintains a test policy, an organizational test strategy and a project test strategy. Which description matches the project test strategy as the syllabus defines it?
It tailors the organisation's generic approach to the circumstances of one specific project and is the principal output of test planning for that project
Correct. The project test strategy is project-specific, derived from the organizational strategy, and produced as the main planning output.
It states the organisation's overall objectives for testing and the value testing is expected to deliver to the business
That is the test policy, which sits above both strategies and expresses intent rather than approach.
It describes the generic test approach to be applied across all projects in the organisation, independently of any one project's context
That is the organizational test strategy; the project test strategy is what adapts it to a particular project.
It lists the test cases, schedules and environment bookings agreed for each test level within the project
Those details belong in test plans at the release, level or iteration layer; the strategy sets the approach that those plans then implement.
The test policy states why the organisation tests and what it expects from testing. The organizational test strategy describes the generic approach applied across projects. The project test strategy, corresponding to the test strategy of ISO/IEC/IEEE 29119-3, tailors that generic approach to one project and is the main output of test planning.
A test manager is choosing a test approach for a new project: a hospital…
A test manager is choosing a test approach for a new project: a hospital medication-dosing module, developed in two-week iterations by a team of four, integrating with two legacy pharmacy systems whose interfaces are documented but not available for testing until week ten, with no anonymised production data yet released by the client's data protection office. Analysing this against the factors the syllabus says must be considered when defining a test approach, which factor is currently the binding constraint on the approach?
Interfaces with other systems together with availability of test data — both are unavailable for most of the schedule, so the approach must be built around stubs, simulators and synthetic data until week ten
Correct. These are the two factors whose current state removes options from the approach, so they shape it more than the others do.
The SDLC model — two-week iterations require the approach to be defined per iteration rather than once for the project, which is what constrains the choice here
The iterative model shapes cadence and planning layers, but it does not remove any technique from the table; the missing interfaces and data do.
Test resources — a team of four is small for a safety-related module, so the approach must be scaled down to what four people can execute
Team size influences how much can be done, but nothing in the scenario indicates the team is undersized relative to the module; the availability problems are stated explicitly.
The domain — medication dosing is safety-related, so the approach is fixed by the regulatory regime and the remaining factors do not materially change it
The domain does raise the required rigour, but the syllabus treats all seven factors as inputs to be analysed together rather than one overriding the rest.
The syllabus lists seven factors to analyse when defining the test approach: the domain, organizational goals, project goals and type, test resources, the SDLC model, interfaces with other systems, and the availability of test data. Here the domain and the SDLC model are known and workable; what actually blocks progress is that neither the interfacing systems nor realistic test data will be available for most of the project, which forces the approach towards simulation, stubs and synthetic data early on.
A draft test objective reads: "Improve the quality of the checkout flow as much as possible before the release." Applying the S.M.A.R.T. criteria the syllabus prescribes for test objectives and exit criteria, which rewrite is acceptable?
"By the code freeze on 14 March, execute all 62 checkout regression tests, with no open severity 1 or 2 defects and requirements coverage of the checkout flow at 95% or above."
Correct. Scope, measures, thresholds and a date are all present, and the targets are achievable against a defined test set.
"Before the release, test the checkout flow thoroughly enough that the business stakeholders are confident in its quality and sign off without reservations."
Confidence and thoroughness cannot be measured against a threshold, and "before the release" is not a fixed point, so the statement fails the measurable and time-bound criteria.
"By the code freeze on 14 March, find and fix every remaining defect in the checkout flow so that no defects of any severity are left in the released code."
It is specific, measurable and time-bound, but zero remaining defects is not achievable and cannot be demonstrated, so it fails the achievable criterion.
"Increase automated test coverage of the whole application from 41% to 80% by the code freeze on 14 March, to raise confidence in the checkout flow."
The measure and the date are sound, but the target covers the whole application rather than the checkout flow, so the objective is not relevant to the stated goal.
S.M.A.R.T. objectives are specific, measurable, achievable, relevant and time-bound. The original statement fails on measurability, achievability and time-bounding. A usable rewrite names the scope, states a measurable target with a threshold, and fixes a date.
A programme uses several layers of test plan: release, master, level, quality-characteristic and iteration plans. What relationship between these layers does the syllabus describe?
They form a hierarchy: broader plans set objectives and constraints that the narrower plans refine for a level, a quality characteristic or a single iteration
Correct. The layers are nested, and each lower layer elaborates the frame set above it rather than replacing it.
They are independent documents, each owned by a different role, and an organisation adopts exactly one of the layers for a given project
The layers coexist on larger efforts; treating them as mutually exclusive alternatives misses the point of the hierarchy.
They are successive versions of one plan, so the iteration plan supersedes the level plan, which in turn supersedes the master plan
Later plans do not supersede earlier ones; a valid iteration plan operates within the level and master plans that remain in force.
They apply to different lifecycles: master and level plans belong to sequential projects and iteration plans belong to Agile projects only
Iterative programmes frequently keep a release or master plan alongside iteration plans, so the split by lifecycle does not hold.
The plans form a hierarchy in which broader plans set the frame for narrower ones. A master or release plan covers the whole effort, level plans cover a single test level, quality-characteristic plans cover a characteristic such as performance or security across levels, and iteration plans cover one iteration. Lower layers refine the higher ones rather than contradicting them.
The IDEAL model is used to structure test process improvement. What do its five phases stand for, and which well-known cycle does the syllabus relate it to?
Initiating, Diagnosing, Establishing, Acting, Learning — related to the Plan-Do-Check-Act cycle
Correct. These are the five IDEAL phases, and the syllabus presents the model as a relative of PDCA.
Identifying, Designing, Executing, Analysing, Leveraging — related to the Goal-Question-Metric approach
The phase names are invented, and GQM is a measurement approach used inside analytical improvement rather than the cycle IDEAL is related to.
Initiating, Defining, Estimating, Assessing, Learning — related to the five maturity levels of TMMi
Two phase names are wrong, and TMMi is a model-based improvement framework, not the cycle that IDEAL corresponds to.
Inspecting, Diagnosing, Escalating, Adjusting, Logging — related to the retrospective process for iterative teams
These phase names do not appear in the model, and retrospectives are one analytical improvement technique rather than IDEAL's counterpart cycle.
IDEAL stands for Initiating, Diagnosing, Establishing, Acting and Learning. The syllabus relates it to the Plan-Do-Check-Act cycle, of which it is a relative: both are iterative improvement frameworks that end by feeding what was learned back into the next cycle.
TMMi and TPI NEXT are both model-based test process improvement frameworks. Which statement correctly distinguishes their structures?
TMMi stages the organisation on five maturity levels built from process areas; TPI NEXT assesses sixteen key areas individually, each against four levels, and plots them on a maturity matrix
Correct. TMMi gives one staged organisational rating, while TPI NEXT produces a per-key-area profile.
TMMi assesses sixteen key areas against checkpoints; TPI NEXT stages the organisation on five maturity levels that complement CMMI
The two models are swapped: the sixteen key areas belong to TPI NEXT and the five staged levels to TMMi.
Both models produce a single organisational maturity rating on a five-level scale; they differ only in the number of process areas assessed at each level
TPI NEXT deliberately produces a profile across key areas at four levels rather than one overall rating, so the difference is structural.
TMMi is an analytical-based improvement approach driven by root cause analysis; TPI NEXT is a model-based approach driven by a reference maturity model
Both are model-based. Root cause analysis and metric analysis belong to the analytical-based category alongside GQM.
TMMi defines five maturity levels made up of process areas and is designed to complement CMMI. TPI NEXT works from sixteen key areas, each of which is assessed against four maturity levels using checkpoints, with the results plotted on a maturity matrix.
A test manager is facilitating a retrospective after a difficult release. The…
A test manager is facilitating a retrospective after a difficult release. The team has already collected data on what happened and has produced a list of eleven candidate improvements. Following the five-step retrospective process described in the syllabus, what is the next step, and what makes it succeed?
Decide on improvement actions — select a small number of the eleven candidates, give each an owner and a target, and defer the rest so the chosen ones actually get done
Correct. Deciding on actions follows deriving improvements, and the decision has to be selective and owned to survive contact with the next release.
Close the retrospective — record all eleven candidates in the improvement backlog and thank the team, leaving prioritisation to the next planning session
This skips the decision step; a list handed on without selection or ownership rarely produces change, which is why the syllabus makes deciding a distinct step.
Collect data — go back and gather metrics on defect counts and effort so that the eleven candidates can be ranked on evidence rather than opinion
Data collection precedes deriving improvements and has already happened; returning to it now stalls the session rather than advancing it.
Repeat the introduction — restate the blame-free ground rules before the difficult items are discussed, since the release went badly and tempers are likely to rise
The blame-free frame is set at the introduction and upheld throughout, but it is not a step the process returns to in place of deciding on actions.
The five steps are: introduction (setting a blame-free frame), collect data, derive improvements, decide on improvement actions, and close the retrospective. With improvements already derived, the next step is deciding which actions to take: selecting a small number, assigning ownership and making them concrete enough to be followed up.
Over three releases, defects escaping to production have risen from 8 to 14 to…
Over three releases, defects escaping to production have risen from 8 to 14 to 23, while the number of tests executed has stayed roughly constant and the team composition has not changed. The test manager wants an analytical-based improvement that will identify why escapes are rising rather than simply adding more tests. Analysing the techniques the syllabus places under analytical-based improvement, which approach fits, and what does it produce?
Run root cause analysis on the escaped defects using an Ishikawa diagram to group causes, then use Goal-Question-Metric to define the measures that will show whether the resulting actions work
Correct. Both techniques are analytical-based: the first explains the trend by cause category, the second makes the improvement measurable.
Commission a TMMi assessment to establish the organisation's maturity level and adopt the process areas prescribed for the next level up
TMMi is a model-based rather than analytical-based approach, and a maturity assessment answers a different question than why these particular defects escaped.
Hold a retrospective at the end of the next release so the team can discuss the escape trend and propose improvements from their own experience
Retrospectives are a legitimate improvement technique in the syllabus, but they rest on participants' recollection rather than on analysing the defect data that is already available.
Apply TPI NEXT checkpoints to the sixteen key areas and improve the key areas that score lowest on the maturity matrix
This is again model-based improvement; the lowest-scoring key areas may have nothing to do with the specific mechanism producing the escapes.
Analytical-based improvement includes root cause analysis, typically supported by an Ishikawa or fishbone diagram, and the analysis of measures, metrics and indicators, often organised through Goal-Question-Metric. Root cause analysis on the escaped defects identifies the categories of cause behind the trend, and GQM then defines the measures that will show whether the chosen actions work.
The syllabus classifies test tools into three business types and names four factors that influence the decision on which to adopt. Which option lists the three business types correctly?
Commercial tools, open-source tools and custom tools developed in-house
Correct. This is the three-way classification by business type on which the adoption decision is framed.
Test management tools, test execution tools and static analysis tools
This classifies tools by the activity they support, which is a different taxonomy from the business-type one used for the adoption decision.
Cloud-hosted tools, on-premise tools and hybrid deployments of the two
Deployment model affects cost and security considerations but is not the classification the syllabus uses for business type.
Vendor-supported tools, community-supported tools and tools with no support arrangement
Support arrangements follow from the business type rather than defining it, and custom in-house tools do not fit any of these three labels cleanly.
The three business types of test tool are commercial tools, open-source tools and custom tools built in-house. The four decision factors are regulations and security, financial aspects, stakeholder requirements, and the existing software landscape.
A test manager evaluates a commercial test automation tool over a three-year…
A test manager evaluates a commercial test automation tool over a three-year horizon. Licences cost 60,000 per year. Initial tool selection, installation and framework build cost 90,000 once. Training the eight testers costs 24,000 once, with 6,000 per year thereafter for new joiners. Maintaining the automated suite is estimated at 40,000 per year. During the first year the two most experienced testers will spend roughly 30% of their time on the tool rather than on exploratory testing of the highest-risk areas. Analysing this against the cost categories the syllabus defines, which statement is correct?
Non-recurring costs are 114,000, recurring costs are 106,000 per year, and the exploratory testing the two senior testers will not perform is an opportunity cost that must be stated even though it is not invoiced
Correct. 90,000 plus 24,000 is non-recurring; 60,000 plus 6,000 plus 40,000 is recurring; and forgone high-risk exploratory testing is the textbook opportunity cost.
Non-recurring costs are 90,000, recurring costs are 106,000 per year, and the senior testers' redirected time is a recurring cost because their salaries continue to be paid throughout
Initial training is non-recurring and belongs with the 90,000, and the point of an opportunity cost is the value forgone, not the salary that is paid either way.
Non-recurring costs are 114,000, recurring costs are 318,000, and the redirected senior tester time should be excluded because it carries no additional cash outflow
318,000 is the three-year total rather than the annual recurring figure, and excluding opportunity costs is precisely the omission that makes tool business cases over-optimistic.
All 432,000 over three years is a recurring cost, since the organisation must keep paying to retain the automation capability once it has been built
Collapsing the categories loses the distinction the analysis depends on: selection, framework build and initial training are paid once and do not repeat in years two and three.
The syllabus distinguishes non-recurring costs (incurred once, such as selection, installation, framework build and initial training), recurring costs (incurred repeatedly, such as licences, maintenance and ongoing training) and opportunity costs (the value of what is given up, here the exploratory testing the two senior testers will not perform). Non-recurring costs total 114,000; recurring costs total 106,000 per year, or 318,000 over three years; and the lost exploratory testing is an opportunity cost that belongs in the case even though no invoice records it.
The syllabus describes a test tool lifecycle and two distinct roles associated with tools. Which option pairs the lifecycle phases with the roles correctly?
Acquisition, support and maintenance, evolution, retirement — with the tool owner accountable across the lifecycle and the tool administrator running day-to-day operation
Correct. These are the four phases named in the syllabus, and it separates ownership from administration in exactly this way.
Selection, piloting, roll-out, decommissioning — with the tool administrator accountable across the lifecycle and the tool owner running day-to-day operation
Piloting and roll-out are steps within acquisition and introduction, and the two roles' responsibilities are the wrong way round.
Acquisition, proof of concept, support and maintenance, retirement — with a single tool owner performing both the accountable and the administrative work
Proof of concept is a good practice during introduction rather than a lifecycle phase, evolution is missing, and the syllabus keeps the two roles distinct.
Evaluation, acquisition, evolution, replacement — with the tool owner responsible for licences and the tool administrator responsible for the business case
Support and maintenance is omitted, and the business case sits with the tool owner rather than with the administrator.
The tool lifecycle runs Acquisition, Support and maintenance, Evolution, and Retirement. The tool owner is accountable for the tool across that lifecycle, including the business case and the decision to evolve or retire it, while the tool administrator handles day-to-day operation such as configuration, access and upgrades.
A test manager wants to report on the value delivered by the defect management tool and by the test execution tool. Which TWO of the following are tool metrics the syllabus associates with these tools?
Defect detection percentage — the share of defects detected by testing relative to all defects found, including those found after release
Correct. It is one of the named tool metrics and shows how effective the supported detection activity is.
Defect lead time — the elapsed time a defect takes to progress from being reported to reaching a terminal status
Correct. Lead time is a named tool metric and reflects how well the defect workflow supported by the tool is functioning.
Number of licences purchased compared with the number of named users currently registered in the tool
Licence utilisation is useful for cost control but describes procurement rather than the contribution the tool makes to testing.
Ratio of testers to developers on the project during the period in which the tool was in use
This is a staffing ratio that would be the same with or without the tool, so it cannot evidence the tool's contribution.
Total number of screens in the application under test that the tool is technically capable of automating
Technical reach describes the tool's applicability, assessed before adoption, rather than a metric of the value it delivers in use.
Among the tool metrics the syllabus names are defect detection percentage, which expresses the proportion of defects found by testing before release, and defect lead time, which measures how long a defect takes to move through its lifecycle. Licence utilisation and headcount ratios are administrative facts rather than the metrics the syllabus lists for evaluating tool contribution.
Halfway through a six-week system test, the plan expected 480 of 800 tests…
Halfway through a six-week system test, the plan expected 480 of 800 tests executed; 470 have been executed, which is close to plan. However, 130 of the executed tests are blocked on a single defective login service, and all of them sit in the two subsystems carrying the highest product risks. Applying test monitoring and control as the syllabus describes it, which control directive follows from these measurements?
Escalate the login service defect for priority resolution and reschedule effort so the highest-risk subsystems regain coverage once it is fixed, because the deviation is in risk coverage rather than in the execution count
Correct. The directive addresses the cause of the blockage and restores coverage where the exposure is greatest, which is what the measurements actually reveal.
Report progress as on plan, since 470 executed against 480 planned is within normal variance, and revisit the blocked tests at the end of the test window
Reporting on the headline count hides the fact that the executed total includes 130 blocked tests in the highest-risk areas, which is the deviation that matters.
Move the 130 blocked tests out of scope for this release and record the two subsystems as covered by the tests that did run, so the schedule is protected
Descoping tests in the highest-risk subsystems increases residual risk silently; monitoring and control exists to surface such a trade-off, not to conceal it.
Extend the test window by two weeks immediately, since a third of the planned execution is blocked and the original schedule can no longer be met
Extending the schedule is one possible outcome, but issuing it before the login defect is escalated treats the symptom and may prove unnecessary once the blocker clears.
Monitoring compares measurements against the plan; control turns a detected deviation into a directive. The execution count looks healthy, but the blocked tests are concentrated in the highest-risk areas, so the real deviation is in risk coverage rather than in throughput. The directive should escalate the login service defect for priority fix and, meanwhile, redirect effort so that risk coverage recovers.
A company runs a hybrid model: the customer-facing web front end is delivered by…
A company runs a hybrid model: the customer-facing web front end is delivered by three Scrum teams on a two-week cadence, while the core banking back end follows a V-model with quarterly releases and formal phase approvals. Front-end stories increasingly depend on back-end changes that will not be available until the next quarterly release. Analysing this against what the syllabus says about test management in a hybrid context, which measure best addresses the situation?
Synchronise planning across the two cadences so cross-boundary dependencies are identified a quarter ahead, and provide simulated back-end interfaces so front-end teams can test against agreed contracts before the real change ships
Correct. It manages the boundary between the lifecycles, which is where the hybrid challenges arise, without demanding that either side abandon its cadence.
Move the three front-end teams onto the quarterly release cadence so that both parts of the organisation plan, test and report on the same schedule
Uniformity removes the mismatch by discarding the benefit of the faster cadence, and the syllabus treats hybrid working as something to manage rather than to eliminate by decree.
Adopt the V-model documentation and phase-approval standards across the front-end teams, since a common set of testware is what makes hybrid reporting comparable
Comparable reporting is a real hybrid challenge, but imposing the heavier documentation regime on iterative teams addresses the paperwork rather than the dependency that is blocking them.
Have the front-end teams defer any story with a back-end dependency to the sprint following the quarterly release, keeping each team's work inside a single lifecycle
This keeps the lifecycles separate but pushes a growing backlog of dependent stories into one crowded sprint, which concentrates rather than reduces the risk.
The syllabus describes hybrid development and names challenges that follow from mixing cadences: differing testware and documentation expectations, differing metrics and reporting, and dependencies that cross the boundary between the two lifecycles. The management response is to make the boundary explicit — synchronising planning across the two cadences and using service virtualisation or stubs so the faster teams are not gated by the slower one — rather than forcing one lifecycle onto the other.
Chapter 2 · Managing the Product — 15 questions
The syllabus groups test metrics into three categories. Which option names them and gives a correct example of each?
Project metrics, for example test execution progress against plan; product metrics, for example defect density in a component; process metrics, for example the defect detection percentage of a test level
Correct. Each example belongs to the category named: progress to project, an attribute of the test object to product, and the capability of an activity to process.
Project metrics, for example defect density in a component; product metrics, for example test execution progress against plan; process metrics, for example the number of testers assigned
The first two examples are swapped, and headcount is a resourcing fact rather than a measure of process capability.
Coverage metrics, for example statement coverage; defect metrics, for example open defects by severity; effort metrics, for example person hours consumed
These are three legitimate metric groups used in reporting, but they are not the project, product and process classification the syllabus sets out.
Leading metrics, for example planned test cases; lagging metrics, for example defects found in production; neutral metrics, for example environment uptime
Leading and lagging is a useful distinction for interpreting trends, but it is not the three-category classification defined in this chapter.
The three categories are project metrics, which measure progress towards planned completion; product metrics, which measure attributes of the test object such as defect density or coverage; and process metrics, which measure the capability and efficiency of the development and test processes, such as defect detection percentage or the cost of finding a defect.
A test progress report distinguishes tests that are planned, implemented, executed, passed, failed, blocked and skipped. What does the syllabus say the blocked and skipped categories add that the passed and failed counts cannot show?
They show how much planned coverage was not obtained and for what reason — an external impediment in the case of blocked tests, a deliberate decision in the case of skipped tests
Correct. Both categories represent missing information rather than a verdict, and separating them tells the reader whether the gap was forced or chosen.
They show how severe the defects found so far are, since a blocked test usually indicates a defect of higher severity than one that merely failed
A blocker often is severe, but the category records that the test could not run rather than grading the severity of anything.
They show the efficiency of the test team, because blocked and skipped tests represent effort that was spent without producing a result
Blocked tests are usually not executed at all, so they do not represent consumed execution effort, and neither category is a productivity measure.
They show which requirements have been covered, since a skipped test leaves its requirement uncovered while a blocked test still contributes partial coverage
A blocked test contributes no coverage either, and requirements coverage is reported through its own metric rather than inferred from these two categories.
Passed and failed describe outcomes of tests that ran. Blocked tests could not run because something outside the test prevented it, such as a defect or an unavailable environment; skipped tests were deliberately not run. Reporting them separately shows how much of the planned coverage was never obtained and why, which a pass/fail split conceals.
A weekly report shows 42 open defects, unchanged from last week, and a steering…
A weekly report shows 42 open defects, unchanged from last week, and a steering group member concludes that defect management has stalled. Looking behind the snapshot, the test manager sees that 31 defects were closed and 31 new ones were raised during the week, that the new ones cluster in a module integrated three days ago, and that the average age of the open set has fallen from 19 days to 11. Analysing snapshot against trend as the syllabus requires, what is the correct interpretation?
Defect management is working faster than before — the falling average age shows quicker resolution — and the stable open count reflects an inflow concentrated in the newly integrated module, which is where attention now belongs
Correct. The snapshot is flat because inflow and outflow happen to match; the trends show both improved throughput and a specific new source of defects.
The steering group member is right — an open count that does not fall over a full week means resolution is not keeping pace with the rate at which defects are being reported
Closures matched the inflow exactly, so resolution is keeping pace; the flat count is the arithmetic result of equal flows, not evidence of a backlog growing.
The report should be replaced with a single trend line of open defects over time, since snapshots of the open count mislead readers and add nothing to a trend chart
Snapshots have a legitimate role in reporting the current position; the syllabus asks for them to be read alongside trends rather than discarded.
The falling average age is the concerning figure — it suggests defects are being closed prematurely, which typically shows up later as a rise in re-opened reports
Premature closure is a real phenomenon, but it is evidenced by the re-opened rate, which is not reported here; age alone does not support that reading.
The syllabus distinguishes a snapshot, which gives the position at a moment, from a trend, which shows direction over time; a single figure can be stable while the underlying flow changes substantially. Here the constant open count masks a high turnover, a falling average age indicating faster resolution, and an inflow concentrated in a newly integrated module.
A test manager must advise whether to stop testing. The evidence: requirements…
A test manager must advise whether to stop testing. The evidence: requirements coverage 96%, risk coverage 100% of high risks and 78% of medium risks, defect discovery rate down from 14 per day to 2 per day over the last five days, three open defects all of low severity, and effort consumed at 118% of the estimate. Analysing how the syllabus says metrics should be combined for this decision, which conclusion is best supported?
The combined picture supports recommending a stop, with the 22% of medium risks not yet covered reported explicitly as residual exposure for the stakeholders to accept
Correct. Coverage, discovery rate and severity converge, and the one genuine gap is surfaced rather than averaged away.
Testing should stop because effort has already reached 118% of the estimate, which is the clearest signal that the value of further testing has fallen below its cost
Budget consumption says nothing about the quality of the product; using it as the stopping signal is exactly the single-metric reasoning the syllabus warns against.
Testing should continue until medium-risk coverage also reaches 100%, since a risk-based approach is only complete when every identified risk has been covered
Risk-based testing deliberately allocates less effort to lower risks; requiring full coverage of every level would remove the prioritisation the approach exists to provide.
Testing should continue because a falling defect discovery rate is more likely to indicate tester fatigue or exhausted test data than a genuine improvement in quality
That alternative explanation is worth checking, but here it is contradicted by the coverage figures and the low severity of what remains open.
The syllabus warns that no single metric supports a stop-testing decision and that metrics must be combined and read against the agreed exit criteria. Here coverage of high risks is complete, discovery has flattened, and the remaining defects are low severity — a converging picture. The effort overrun is a project fact that does not itself indicate product quality, and the medium-risk gap is the one item that should be reported as residual exposure.
The syllabus lists coverage as one of the metric groups used in test reporting, and names several bases against which coverage can be measured. Which TWO of the following are coverage bases the syllabus identifies?
Risk coverage — the proportion of identified product risks exercised by at least one executed test
Correct. Risk coverage is named explicitly and is the coverage basis most closely tied to a risk-based approach.
Code coverage — measured as statements, branches, paths or conditions exercised by the tests
Correct. The syllabus names code coverage and lists these four sub-measures within it.
Effort coverage — the proportion of the estimated test effort that has been consumed to date
Consumed effort belongs to the costs and effort metric group; it measures spend rather than what has been exercised.
Defect coverage — the proportion of reported defects that have reached a terminal status in the workflow
This describes defect resolution progress, which is reported under defect metrics rather than as a coverage basis.
Environment coverage — the proportion of scheduled test environment hours that were actually available
Environment availability is a constraint tracked during monitoring and control; the syllabus does not list it as a coverage basis.
Coverage may be measured against requirements, against risks, and against code, where code coverage itself subdivides into statements, branches, paths and conditions. Effort consumed and defect counts are reported in their own metric groups and are not coverage bases.
A test manager estimates the effort for a migration test cycle using three-point…
A test manager estimates the effort for a migration test cycle using three-point estimation. The optimistic estimate is 18 person-days, the most likely estimate is 27 person-days, and the pessimistic estimate is 54 person-days. Applying the weighted three-point formula the syllabus describes, what is the resulting estimate, and what does the spread indicate?
30 person-days; the wide optimistic-to-pessimistic spread signals high estimation uncertainty, which should be reported with the figure rather than hidden behind it
Correct. (18 + 108 + 54) / 6 = 30, and the three-fold spread is exactly the uncertainty the technique is meant to expose.
33 person-days; the wide spread signals high estimation uncertainty, which should be reported with the figure rather than hidden behind it
33 is the simple average of the three values. The weighted formula gives the most likely value four times the weight of the others, which yields 30.
27 person-days; the most likely value is retained because the optimistic and pessimistic values cancel each other out around it
They do not cancel: the pessimistic value lies 27 days above the most likely one while the optimistic lies only 9 below, so the weighted result is pulled upwards.
36 person-days; the pessimistic value carries the heaviest weight because migration work is the category most prone to overrun
The formula weights the most likely value most heavily regardless of the type of work; no variant weights the pessimistic value above it.
Three-point estimation weights the most likely value four times against one weighting each for the optimistic and pessimistic values: (18 + 4 × 27 + 54) / 6 = (18 + 108 + 54) / 6 = 180 / 6 = 30 person-days. The wide gap between optimistic and pessimistic indicates high uncertainty, which the syllabus treats as a signal that the estimation error is large and should be communicated alongside the figure.
The syllabus organises the factors that may influence test effort into five groups. Which option lists them?
Product, development process, people, test results, and test context
Correct. These are the five groups named in the chapter on test estimation.
Product, technology, tooling, budget, and delivery deadline
Tooling and budget are considerations within the context and process groups, but this is not the five-group classification the syllabus defines.
Effort, time, cost, quality, and scope
These are the dimensions being traded off in the time-cost-quality triangle, not the factors that influence how much effort testing will take.
Requirements volatility, team seniority, defect density, environment stability, and regulatory load
Each of these is a real influence, but they are individual factors drawn from several groups rather than the grouping itself.
The five groups of factors influencing test effort are product factors, development process factors, people factors, test results factors, and test context factors. Effort, time and cost are what is being estimated; they are related through the time-cost-quality triangle rather than being factor groups.
A test manager must estimate two efforts in the same programme. Effort A is the…
A test manager must estimate two efforts in the same programme. Effort A is the twelfth annual regression cycle of a stable reporting product: five years of effort and defect data exist, the scope is well understood, and the technology has not changed. Effort B is the first release of a machine-learning recommendation engine: no historical data exists, the architecture is still being decided, two domain experts are available for half a day, and the estimate is needed by Friday. Analysing the selection factors the syllabus gives for estimation techniques, which pairing is appropriate?
Metric-based estimation for A, because five years of comparable data exist and the work is well understood; expert-based estimation for B, because there is no data to extrapolate from and the novelty is high
Correct. This applies the syllabus heuristic directly: data availability and low complexity favour metric-based, novelty and absent data favour expert-based.
Expert-based estimation for A, because experienced testers know the regression cycle better than any historical figure does; metric-based estimation for B, using industry benchmark data for machine-learning projects
This reverses the heuristic and substitutes external benchmarks for organisational data, which the syllabus does not treat as an equivalent basis.
Metric-based estimation for both, since a consistent estimation technique across a programme is what makes the two efforts comparable when they are reported together
Comparability is desirable but cannot be bought by applying a data-driven technique where no relevant data exists; the resulting figure for B would be false precision.
Expert-based estimation for both, with a full Wideband Delphi round for each, since expert judgement is the more accurate technique whenever domain experts can be convened
Expert-based is not inherently more accurate, and a full Delphi round is hard to reconcile with two experts available for half a day and a Friday deadline.
The syllabus offers a printed heuristic: low complexity with available historical data favours metric-based estimation, while high complexity and novelty favour expert-based estimation. It also matches the expert technique to the lifecycle — Wideband Delphi in sequential contexts, planning poker in Agile ones — and notes that expert availability and time constraints are themselves selection factors.
A team is designing its defect workflow. The proposal has the statuses OPEN, IN…
A team is designing its defect workflow. The proposal has the statuses OPEN, IN PROGRESS, RESOLVED, CLOSED, REJECTED, RE-OPENED and DEFERRED, with CLOSED and REJECTED both terminal, and the same engineer permitted to move a report from IN PROGRESS to RESOLVED and then from RESOLVED to CLOSED. Applying the good practices the syllabus lists for defect workflows, which change is required?
Require a different role to make the transition from RESOLVED to CLOSED, and route REJECTED into the single terminal status rather than treating it as a second endpoint
Correct. It fixes both breaches: no one confirms their own fix, and the workflow ends at one place so completion is unambiguous.
Remove DEFERRED and RE-OPENED, because a workflow with seven statuses is too heavy for a team to maintain and only the core four are needed
The syllabus does not cap the number of statuses; DEFERRED and RE-OPENED carry information that the core four cannot express.
Make RESOLVED terminal as well, so that a report reaching a fix is recorded as complete without waiting for retest to be scheduled
This adds a third terminal status and would mark defects complete before the fix has been confirmed by retesting.
Allow only one outgoing transition from each non-terminal status, so the path a defect takes through the workflow is predictable and easy to report on
The good practice is the opposite: non-terminal statuses should offer more than one outgoing transition, since a report in progress may be fixed, rejected or deferred.
Among the good practices the syllabus lists are: the workflow should have a single terminal status; adjacent statuses should be handled by different roles so that no one confirms their own work; and non-terminal statuses should have more than one outgoing transition. Here the same engineer closing what they resolved breaches the role-separation practice, and having two terminal statuses breaches the single-terminal-status practice.
During defect management the syllabus distinguishes a failure, an anomaly, a defect report and a false-negative result. Which description of a false-negative result is correct?
A test passes even though a defect is present in the test object, so the defect goes unreported
Correct. The result is negative for a defect while a defect exists, which is what makes it false.
A test reports a failure that is later traced to faulty test data rather than to a defect in the test object
This describes a false-positive result: a failure is signalled where the test object is sound.
A defect report is rejected by the development team because the described behaviour matches the specification
That is a rejected defect report, which is a workflow outcome rather than a property of the test result.
An anomaly is observed but no defect report is raised because the tester cannot reproduce the behaviour a second time
This describes an unreproducible anomaly; the test did detect something, so the result was not negative.
A false-negative result is one in which a test does not detect a defect that is present — the test passes although the product is faulty. It is distinct from a false-positive, where a test reports a failure that turns out not to be caused by a defect in the test object, for example a problem in the test data or environment.
A test manager reviews a defect report that contains only a title and the sentence "Totals wrong on the invoice screen after the last build." Applying the syllabus guidance on the information a defect report must carry, which revision makes it usable?
Add a description with the steps to reproduce, the expected and actual totals, and assign both a severity and a priority so the report can be triaged
Correct. These are the reporter-supplied fields the syllabus requires, and together they let someone else reproduce and rank the problem.
Add the identifier, the reporter's name and the date and time the report was raised, since a report without traceability information cannot be tracked
These fields are generated by the defect management tool rather than entered by the reporter, so adding them manually does not fix what is missing.
Add a severity value only, and let priority be derived automatically from severity so that triage effort is reduced
Severity and priority are separate judgements — technical consequence and business urgency — and deriving one from the other collapses that distinction.
Add every field the tool offers, including root cause, phase of introduction and estimated fix effort, so the report supports later statistical analysis
Root cause and phase of introduction are valuable when they are used, but the syllabus advises collecting only what will be used, and most of these cannot be known at reporting time.
The syllabus names title, description including steps to reproduce, severity and priority as the fields that must be supplied by the reporter, alongside fields the tool generates automatically such as identifier, reporter and timestamp. It also advises collecting only information that will actually be used, so adding fields nobody consumes is not an improvement.
What is the purpose of a defect taxonomy as the syllabus describes it?
It provides a categorised list of defect types so that defects can be classified consistently, revealing recurring categories that inform process improvement and future test design
Correct. The value of the taxonomy lies in consistent classification, which turns individual reports into a pattern that can be acted on.
It provides an agreed mapping from defect severity to defect priority so that triage decisions are made consistently across teams
That would be a triage policy. A taxonomy classifies what kind of defect it is, not how urgently it should be fixed.
It provides the sequence of statuses a defect report passes through, so that every team uses the same defect workflow
The sequence of statuses is the defect workflow, a separate concept described alongside the taxonomy.
It provides a list of the defects known to remain open at release, so that support staff can recognise them when customers report symptoms
That is a known-issues list produced at release; a taxonomy is a classification scheme that exists independently of any one release.
A defect taxonomy is a categorised list of defect types. Classifying defects against it allows patterns to be seen across releases and projects — which categories recur, where they originate — and this supports both process improvement and the design of tests targeting the categories that occur most.
Analysis of a year of defect data shows: 61% of defects classified as…
Analysis of a year of defect data shows: 61% of defects classified as requirements defects were first detected during system testing rather than during requirements review; 44% of all defects are concentrated in two of the seventeen components; and the proportion of defect reports later marked as duplicates has risen from 4% to 17%. Analysing these three findings against the uses of defect statistics the syllabus describes, which set of conclusions is best supported?
A phase containment failure in requirements review, a defect cluster in two components that should attract more test effort, and a decline in the quality of defect reporting indicated by the duplicate rate
Correct. Each finding maps onto a distinct documented use of defect statistics, and the duplicate rate is read as a reporting-quality indicator rather than a product signal.
A phase containment failure in requirements review, a defect cluster in two components, and a decline in the quality of debugging indicated by the duplicate rate
The first two are right, but debugging quality is assessed through the re-opened rate; duplicates reflect how reports are raised and searched, not how fixes are made.
An effective system test level catching requirements defects, a well-targeted test effort concentrated on two components, and improved tester diligence shown by more reports being raised
Reading a containment failure as a success reverses the point of phase containment analysis, and duplicates represent rework rather than diligence.
A defect cluster spread across requirements activity, a phase containment failure in the two concentrated components, and a defect taxonomy that needs additional categories
The cluster and containment findings are attached to the wrong observations, and nothing in the data indicates that the taxonomy lacks categories.
The syllabus describes using defect statistics for phase containment analysis, for defect cluster analysis, and for assessing the quality of debugging and of defect reporting. Requirements defects escaping to system testing is a phase containment failure pointing at the review process. Concentration in two components is a defect cluster, which argues for redirecting test effort there. A rising duplicate rate is one of the named indicators of defect report quality, not of product quality.
In an Agile team, a tester finds a defect in a user story that is still in…
In an Agile team, a tester finds a defect in a user story that is still in progress within the current iteration and fixes it with the developer at the desk within twenty minutes. The syllabus discusses when a defect report should nonetheless be created in Agile contexts. What is the position it takes?
No report is needed for this case, but a report is still created where the defect affects previously delivered functionality, cannot be fixed within the iteration, or must be recorded for regulatory or organisational reasons
Correct. The trigger is whether the report serves a purpose beyond the immediate conversation, which is how the syllabus frames the decision.
A report is required for every defect found by a tester, since defect statistics lose their value as soon as some categories of defect are left out of the data
Consistency in the data matters, but the syllabus explicitly allows in-iteration defects on work in progress to go unreported where nothing depends on the record.
No report is ever needed in an Agile team, because unresolved items move to the product backlog as user stories and the backlog replaces defect tracking
The backlog does receive unresolved items, but it does not remove the cases where the syllabus says a defect report should still be raised.
A report is required whenever the fix takes longer than the time-box the team has agreed for informal fixes, which is the single criterion the syllabus applies
Duration is not the criterion; the syllabus lists several situations, most of which concern the scope and traceability of the defect rather than how long the fix takes.
In Agile teams a defect found and fixed inside the iteration on work still in progress often does not warrant a formal report, because the report's coordination purpose is served by the conversation. The syllabus nevertheless names situations in which a report is created regardless — for example where the defect is in previously delivered functionality, where it cannot be fixed within the iteration and moves to the product backlog, or where regulation or the organisation requires a record.
Development and test disagree repeatedly on the priority of defects: developers…
Development and test disagree repeatedly on the priority of defects: developers close reports as "working as designed" while testers re-open them, and the same arguments recur in the daily stand-up. Applying the mechanisms the syllabus describes for defect management, what should the test manager put in place?
A triage meeting with representatives from development, test and the business, which decides severity, priority and disposition for disputed reports on an agreed cadence
Correct. Triage is the named mechanism for reaching a cross-role decision on disposition, which is exactly what is being argued about.
A rule that only the reporting tester may close a defect report, so that a developer cannot dispose of a report the tester still considers valid
Giving one side the final word suppresses the disagreement rather than resolving it, and it conflicts with the practice of separating roles across adjacent statuses.
A defect taxonomy covering the disputed categories, so that classification determines disposition and the argument no longer needs to be had
A taxonomy classifies the kind of defect; it does not decide whether a given report is valid or how urgently it should be addressed.
A weekly report of re-opened defects sent to the programme manager, so that the pattern becomes visible to someone with the authority to intervene
The metric makes the problem visible and is worth having, but visibility alone does not supply the forum in which the disposition is actually agreed.
The syllabus describes the defect management committee, which sets and maintains the defect management process, and the triage meeting, in which representatives of the relevant parties agree the severity, priority and disposition of reported defects. A defect manager may be appointed to own the process. Recurring disputes over disposition are what the triage mechanism exists to resolve.
Chapter 3 · Managing the Team — 9 questions
The syllabus describes four areas of competence for members of a test team and notes which of them ISTQB certification primarily develops. Which option is correct?
Professional, methodological, social and personal competence, with ISTQB certification primarily developing professional competence
Correct. These are the four areas named, and the syllabus is explicit that certification addresses the professional one.
Professional, methodological, social and personal competence, with ISTQB certification primarily developing methodological competence
The four areas are right but the attribution is not: certification builds the body of testing knowledge, which the syllabus places under professional competence.
Technical, analytical, interpersonal and leadership competence, with ISTQB certification primarily developing technical competence
These labels resemble the real ones but are not the four areas the syllabus cites from the competence literature it references.
Domain, tooling, process and communication competence, with ISTQB certification primarily developing process competence
Domain and tooling knowledge are elements within professional competence rather than areas in their own right in this model.
The four areas of competence are professional competence, methodological competence, social competence and personal competence. ISTQB certification primarily develops professional competence — the domain and testing knowledge itself — while the other three are developed largely through experience, coaching and team work.
A test manager is staffing a team for a telemedicine platform: two-week…
A test manager is staffing a team for a telemedicine platform: two-week iterations, a microservice architecture with heavy use of message queues, strict clinical safety requirements, and a customer who reviews test documentation formally. The available people are: a tester with eight years of clinical domain knowledge but no automation experience; an automation engineer strong in API and message-queue testing with no domain background; and a junior tester with good written communication and six months of experience. Analysing the mapping of skills to test process activities that the syllabus requires, which allocation is best supported?
Domain expert on risk analysis and test analysis for the clinical scenarios; automation engineer on test implementation against the services and queues; junior tester on test documentation and execution support, paired with the domain expert
Correct. Each person is placed on the activities their competence actually drives, and the junior tester is developed through pairing rather than left unsupported.
Automation engineer on risk analysis and test analysis, since the architecture is where the technical risks live; domain expert on manual execution; junior tester on building the automation framework
Clinical risk analysis needs domain judgement, and asking a tester with six months of experience to build the framework inverts the skill mapping entirely.
Rotate all three through every activity in each iteration, so that the team becomes cross-functional and no single point of knowledge remains in a safety-related product
Reducing key-person dependency is a legitimate aim, but full rotation from the outset places uncertified judgement on clinical risk analysis in a safety-related context.
Domain expert on risk analysis and test analysis; automation engineer on test implementation; junior tester assigned to a training course for the whole first iteration to close the experience gap
The first two placements are sound, but removing a third of the team from a two-week iteration for formal training ignores the on-the-job development the syllabus also lists.
The syllabus asks for skills to be mapped onto the activities of the test process, with the context drivers — domain, architecture and technologies, and the SDLC model — taken into account. Here the domain expert is most valuable where clinical judgement determines what to test, the automation engineer where the architecture determines how it is exercised, and the junior tester where documentation quality matters and supervision is available.
A newly formed test team has moved past its polite first weeks and is now arguing openly about test approach, tool choice and who decides what. Using the team development model the syllabus cites, which phase is this, and what is the manager's task in it?
Storming — the manager surfaces and mediates the conflicts so they resolve into agreed ways of working, rather than suppressing the disagreement
Correct. Storming is the expected conflict phase, and moving through it is what allows norming to follow.
Forming — the manager provides clear direction and structure, because the team is still orienting itself and has not yet built working relationships
Forming is the polite orientation phase the team has already passed; the open argument is the marker that it has moved on.
Norming — the manager steps back and lets the team consolidate the working agreements it has already reached about approach and tooling
Norming follows storming and is characterised by settled agreements; here the agreements are precisely what is being contested.
Adjourning — the manager focuses on capturing lessons learned, because visible conflict at this stage indicates the team is close to disbanding
Adjourning is the disbanding phase at the end of a team's life and has nothing to do with early conflict over approach.
The syllabus uses Tuckman's model: forming, storming, norming, performing and adjourning. Storming is the phase in which conflict over roles, approach and influence surfaces. The manager's task is to let the disagreements be worked through and steered towards agreed ways of working, rather than suppressing them or treating the team as broken.
The syllabus draws on Herzberg's motivation-hygiene theory to distinguish motivators from hygiene factors in a test team. Which TWO of the following does that theory classify as motivators rather than hygiene factors?
Recognition of a tester's contribution by the team and by stakeholders outside it
Correct. Recognition arises from the work itself and is one of the motivators named in the model.
Responsibility and autonomy over how a tester's own area of work is approached
Correct. Responsibility and autonomy are motivators, since they change the nature of the work rather than its surroundings.
Remuneration and the terms on which it is reviewed
Herzberg places remuneration among the hygiene factors: inadequate pay causes dissatisfaction, but improving it does not produce lasting motivation.
The personnel policy the organisation applies to the team
Personnel policy surrounds the work rather than forming part of it, which places it among the hygiene factors.
Interpersonal relationships between team members and with neighbouring teams
Relationships are listed as a hygiene factor: poor ones demotivate, but good ones are treated as a baseline condition rather than a motivator.
In Herzberg's model, motivators arise from the work itself: recognition, responsibility and autonomy, meaningful tasks, and advancement. Hygiene factors surround the work: remuneration, personnel policy, working conditions, job security and interpersonal relationships. Their absence causes dissatisfaction, but their presence does not by itself motivate.
The syllabus names a skills matrix as one instrument for assessing a test team, alongside Belbin's Team Roles, DISG and PCM. What distinguishes the skills matrix from the other three?
The skills matrix records what competencies each person holds and at what level, exposing gaps and single points of knowledge; the other three describe behavioural or personality preferences within a team
Correct. One instrument maps capability, the others map working and interaction styles.
The skills matrix describes how team members prefer to interact, while Belbin, DISG and PCM each measure the technical competencies required for a given test activity
This reverses the two: the three named models are behavioural, and none of them measures technical competency.
The skills matrix is applied to individuals while Belbin, DISG and PCM are applied only to the team as a whole and produce no individual results
All four are applied to individuals; the models are then used to reason about the composition of the team.
The skills matrix is used during recruitment only, whereas Belbin, DISG and PCM are used for ongoing performance appraisal of existing staff
The skills matrix is maintained continuously to guide development and allocation, and the behavioural models are not appraisal instruments.
A skills matrix records which competencies each team member holds and at what level, so gaps and single points of knowledge become visible. Belbin's Team Roles, DISG and PCM are behavioural or personality models used to understand how people prefer to work and interact within a team, which is a different question from what they can do.
An organisation records the following per-defect figures: defect prevention…
An organisation records the following per-defect figures: defect prevention costs 150, appraisal costs 450, internal failure costs 250, external failure costs 3,600. Applying the formula the syllabus gives for the average saving achieved by finding a defect before release rather than after it, what is the result?
2,900 — the external failure cost less the sum of appraisal and internal failure costs
Correct. 3,600 − (450 + 250) = 2,900, which is the formula applied exactly as printed.
2,750 — the external failure cost less the sum of prevention, appraisal and internal failure costs
Prevention costs are incurred whether or not a given defect occurs and are not subtracted in this formula, which is why the printed version omits them.
3,350 — the external failure cost less the internal failure cost, since appraisal is a fixed overhead independent of any individual defect
Appraisal cost per defect is exactly the cost of finding it, so leaving it out overstates the saving that early detection produces.
4,450 — the sum of all four cost categories, representing the full cost the organisation avoids by preventing the defect
Adding the categories gives the total cost of quality per defect, not the saving from finding one earlier rather than later.
The syllabus gives Average Savings per Defect = Average External Failure Costs − (Average Appraisal Costs + Average Internal Failure Costs). Here that is 3,600 − (450 + 250) = 3,600 − 700 = 2,900. Prevention costs are part of the total cost of quality but are not part of this particular formula.
A finance director proposes cutting the test budget by a third, arguing that the…
A finance director proposes cutting the test budget by a third, arguing that the last two releases went out with no production incidents and that testing therefore produced no measurable return. Analysing this against the cost-benefit view of testing the syllabus sets out, which response is best supported?
Quantify the defects found before release and the external failure costs they would have carried, and present the clean releases as the outcome the appraisal spend bought rather than as evidence it was unnecessary
Correct. It converts an invisible benefit into a figure and directly addresses the inference the finance director has drawn.
Accept the reduction for one release and measure the resulting production incidents, so that the value of the removed testing is demonstrated with evidence rather than argument
Proving the point by allowing failures reaches customers first and treats external failure cost — the most expensive category — as an acceptable experiment.
Argue that the cost of quality cannot be quantified for testing, and defend the budget on the qualitative grounds of reputation, customer trust and regulatory standing
The qualitative benefits are real and worth raising, but the syllabus provides explicit formulas for the quantitative side, so conceding that it cannot be measured gives away the stronger argument.
Redirect the whole remaining budget into defect prevention activities such as reviews and static analysis, since prevention costs are the lowest of the four cost of quality categories
Prevention is cheaper per defect, but shifting the entire budget away from appraisal removes the detection that produced the clean releases in the first place.
The syllabus asks the test manager to argue the cost-benefit relationship of testing in both qualitative and quantitative terms, and notes that the benefit of testing appears as failures that did not happen — which is invisible unless it is expressed as avoided external failure costs. The absence of incidents is evidence of the appraisal activity working, not of its being unnecessary; the Boehm curve makes the same point about the rising cost of late detection.
The cost of quality model used in CTAL-TM divides costs into four categories. Which option lists them, and which classification does a defect found during system testing and fixed before release fall into?
Defect prevention, appraisal, internal failure and external failure costs — the rework of fixing that defect before release is an internal failure cost
Correct. Internal failure covers rework from defects caught before the product reaches the customer.
Defect prevention, appraisal, internal failure and external failure costs — the rework of fixing that defect before release is an appraisal cost
The categories are right, but appraisal covers the effort of evaluating the product; the repair effort that follows is internal failure cost.
Planning, detection, correction and warranty costs — the rework of fixing that defect before release is a correction cost
These labels describe the same idea loosely but are not the four categories the syllabus names, and the model it cites uses the prevention-appraisal-failure structure.
Direct, indirect, fixed and variable quality costs — the rework of fixing that defect before release is a direct variable cost
This is a general accounting classification rather than the cost of quality model applied in this chapter.
The four categories are defect prevention costs, appraisal costs, internal failure costs and external failure costs. A defect found during testing and repaired before release generates internal failure costs — the rework arising from a failure that the customer never experienced. The activity of finding it is appraisal cost; the repair is internal failure cost.
A skills matrix shows that only one tester in a team of six can perform…
A skills matrix shows that only one tester in a team of six can perform performance test analysis, and that person is leaving in four months. Budget for external training is available but the next public course runs in five months. Applying the skill development approaches the syllabus lists, what should the test manager arrange?
Have the departing expert mentor two colleagues on live performance analysis work over the coming months, supported by self-study, and book the course afterwards to consolidate what they have learned
Correct. Mentoring and training on the job transfer the knowledge inside the window that actually exists, and the course then reinforces it.
Book two testers onto the public course in five months and accept a two-month gap in performance test analysis capability, since formal training is the most reliable transfer of knowledge
It leaves the capability gap open and forgoes the one chance to learn directly from the person who holds the knowledge.
Ask the departing expert to write a detailed handover document covering the performance analysis approach, so the knowledge remains available to whoever needs it later
Documentation is a useful supplement, but the syllabus treats competence as developed through practice and interaction; a document alone does not create an analyst.
Outsource performance test analysis to a specialist supplier for the next year, and revisit whether the skill needs to exist inside the team at all
Buying the capability is a valid commercial option, but it answers a sourcing question rather than the skill development question the matrix has raised.
The syllabus lists training and education, self-study, peer learning, mentoring and coaching, and training on the job as ways to develop competence. When a formal course cannot be scheduled in time, the workable route is to have the departing expert mentor one or two colleagues on real work before leaving, supported by self-study, with formal training used afterwards to consolidate.