ISTQB Specialist (CT-PT) Mock Exam #4 — Questions & Answers
Every question in this mock exam, with the correct answer marked and a written rationale behind each option below — for reading and review, not a timed run.
Chapter 1 · Basic Concepts — 6 questions
Which statement best describes capacity testing as a type of performance testing?
It applies a workload that rises to an extreme level in a very short time and then falls away again, so that both the behaviour at the peak and the recovery afterwards can be observed.
Incorrect — this describes spike testing. Spike testing is concerned with sudden surges and with whether the system returns to normal service once the surge has passed, not with establishing a sustainable maximum.
It keeps a representative workload running over an extended period so that slow resource exhaustion, such as a memory leak or uncontrolled table growth, becomes visible.
Incorrect — this describes endurance testing. Endurance testing looks for degradation that appears with the passage of time under load, and the volume applied is usually a normal one.
It establishes the maximum number of concurrent users or transactions that the system can sustain while the performance objectives agreed with stakeholders are still being met.
Correct. Capacity testing determines the upper limit at which the system still satisfies its stated performance requirements, which is exactly what capacity planning takes as its input.
It checks whether adding processing, memory or network resources raises the achievable workload roughly in proportion to the amount of resource that was added.
Incorrect — this describes scalability testing. Scalability testing examines the relationship between added resources and added capability, rather than the absolute limit of the current configuration.
Capacity testing answers the question of how much the system can take: it establishes the maximum volume of users or transactions that can be sustained while the agreed performance objectives are still met, which is the figure capacity planning needs. Spike, endurance and scalability testing each answer a different question — behaviour during and after a short extreme burst, behaviour over a long period at normal volume, and whether added resources translate into added capability.
A national railway operator has agreed with the business that its online…
A national railway operator has agreed with the business that its online ticketing service must handle the documented busy-hour volume of 4,000 completed bookings per hour, with seat-selection responses under 2 seconds for 90% of requests. Before the summer timetable goes live, the test team wants to confirm that the service meets exactly this expected demand. Which type of performance testing matches that objective?
Load testing, because the objective is to confirm that the agreed response-time and throughput targets are met at the workload the service is expected to carry in production rather than beyond it.
Correct. The documented busy-hour volume is the anticipated workload, and the question being asked is whether the agreed objectives hold at precisely that level.
Stress testing, because the busy hour of the summer timetable is the heaviest demand that the ticketing service will see anywhere in its operating year, and the agreed targets still have to hold there.
Incorrect — stress testing deliberately applies a workload at or beyond the anticipated maximum, or runs with reduced resources. A documented and expected volume does not become a stress condition by being the busiest expected hour.
Capacity testing, because the figure of 4,000 bookings per hour is the capacity that the ticketing service has been required to provide, and the run has to confirm that this capacity is available.
Incorrect — capacity testing searches for the maximum volume the system can still sustain within its objectives. Here the volume is already fixed by the business and the test only has to confirm the objectives are met at it.
Endurance testing, because the summer timetable will hold the ticketing service at this level of passenger demand for several consecutive months without any quiet period in which to recover.
Incorrect — endurance testing applies a workload over an extended continuous period to reveal effects such as memory leaks. A busy hour that recurs daily is not the test objective described in the scenario.
Load testing evaluates system behaviour under an anticipated, realistic workload — here the documented busy-hour volume agreed with the business. Stress testing would deliberately go at or beyond that volume or reduce available resources; capacity testing would search for the maximum the system can sustain; endurance testing would run a normal workload for an extended period to expose degradation over time.
Before any system-level performance test is scheduled, a development team profiles a single pricing component in isolation, driving it directly with a generated request stream and recording CPU time and allocation counts per call. Which statement describes this activity correctly?
It is a static performance testing activity, because the component is examined in isolation rather than as a part of the fully integrated system under test.
Incorrect — static testing in performance testing means examining work products without executing them, for example reviewing a database schema or a stored procedure. Code is being executed here, so the activity is dynamic.
It is dynamic performance testing at component level, a recognised test level at which profiling can expose inefficient code well before system testing begins.
Correct. The syllabus lists unit or component testing with profiling among the dynamic performance testing levels, alongside component integration, system, system integration and acceptance testing.
It is a load generation technique rather than a test level, since the requests are produced by a generator instead of being captured from a real communication protocol.
Incorrect — how the load is produced and the test level at which it is applied are two separate questions. Generating requests directly against a component does not stop the activity being a component-level dynamic test.
It falls outside performance testing, because performance efficiency becomes measurable at system level, where the real infrastructure and realistic data volumes are present.
Incorrect — the syllabus states that performance testing may be executed on a complete system or on components of it, and component-level profiling is given as an example of early dynamic performance testing.
The syllabus separates static testing types in performance testing — reviews of requirements, database schemas and queries, architecture and critical code — from dynamic ones. Dynamic performance testing can be applied at several test levels, and unit or component testing with profiling is explicitly listed among them. Performance testing may be carried out on a complete system or on components of it.
A regional ferry operator is arguing about how far to take the performance testing of its new booking engine. Two of the statements made in the discussion are consistent with the principles of performance testing set out in the syllabus. Which two? (Choose two.)
Every transaction the booking engine offers ought to be scripted, so that no part of the system is left unmeasured when the results are presented to the board.
Incorrect — scripts are derived from the operational profiles, which concentrate on the transactions that carry the volume and the risk. Full coverage of operations is not a performance testing principle.
The results have to be presented so that the people receiving them can understand the figures and place them beside measurements taken earlier in the project.
Correct. Understandability and comparability of results is a stated principle: a measurement nobody can interpret or place against a baseline supports no decision.
Measurement ought to begin once the booking engine has been deployed to production, where the real traffic and the real data volumes are finally present.
Incorrect — production monitoring is valuable and the syllabus recommends it, but it comes in addition to testing beforehand rather than in place of it.
The testing ought to be carried out by a team independent of the developers, so that the figures cannot be shaped by the people whose work is being measured.
Incorrect — independence is a general testing consideration rather than one of the performance testing principles, and the syllabus expects developers to profile their own components early.
Whatever approach is chosen has to be achievable within the budget, the schedule and the resources that the project actually has available to it for the work.
Correct. Feasibility within project constraints is a stated principle: an approach the project cannot fund or fit into its timeline produces no measurements at all.
Among the principles the syllabus states are that results have to be understandable and comparable to the people who receive them, and that the approach has to be feasible within the project's budget, schedule and resources. Scripting everything, deferring measurement to production and insisting on an independent test team are project decisions that may or may not be sensible, but none of them is stated as a principle.
A container terminal is replacing its terminal operating system. The…
A container terminal is replacing its terminal operating system. The crane-scheduling flow that has to be performance tested has no user interface of its own: it is driven by other systems across a documented REST interface, and the supplier publishes the message formats. The test team has to produce the load. Which load generation approach fits this situation best?
Capture the traffic exchanged between the existing systems during a busy shift and replay those recorded protocol exchanges back against the new terminal operating system.
Incorrect — replaying captured communication is the approach for interfaces that are not documented or not directly reachable, and it carries substantial correlation work. Here the interface is published and can be driven directly.
Generate the load through the published REST interface, building the requests from the documented message formats and the volumes taken from the operational profile.
Correct. Load generation through the API suits a component with no user interface of its own, and it is repeatable and capable of the volumes the scheduling flow has to carry.
Drive the load through the user interfaces of the systems that call the crane-scheduling flow, so that the whole chain the terminal operators work with is exercised.
Incorrect — driving through the user interface measures end-to-end behaviour but pulls unrelated systems into the result and scales poorly to machine-to-machine volumes.
Ask terminal staff across several shifts to use the system at an agreed time, so that the load arrives from real operators on real workstations and real network paths.
Incorrect — load generation by crowds gives realistic conditions but is hard to repeat and cannot reach the volumes a machine interface has to sustain.
The syllabus names four principal approaches to generating load: through the user interface, through crowds of real people, through the API, and by replaying captured communication protocols. When a documented machine interface exists and the component has no user interface, generating directly through the API gives repeatable, high-volume load without the correlation work that replaying captured traffic brings.
A university library's archive search returns its results in 9 to 12 seconds even when a single user runs one query against an otherwise idle test system, and the measured times barely move when 200 virtual users are added. Which performance efficiency failure mode does this pattern point to?
Resource saturation under moderate to heavy load, where a pool, a device or a thread set reaches its limit and response times climb as more users arrive.
Incorrect — saturation shows itself as a response time that is acceptable at low load and deteriorates as the load grows. Here the single-user figure is already the problem.
Degradation over time, where response times worsen gradually while the system remains available, through effects such as memory leaks or uncontrolled table growth.
Incorrect — degradation over time is identified by comparing measurements taken hours apart under the same load. The scenario compares one user against two hundred, not early against late.
Inelegant handling of errors at the limits of capacity, where the system stops serving requests cleanly once its resource pools and queues have been exhausted.
Incorrect — that failure mode is recognised by errors, dropped sessions or raw stack traces at the limit. The scenario reports slow but successful responses at both load levels.
Slow response under all levels of load, which points at the design or implementation of the archive search rather than at a shortage of any resource on the server.
Correct. A cost that is already present with one user on an idle system and does not grow with load belongs to the operation itself, so profiling the query is the next step.
The syllabus distinguishes slow response under all levels of load from slow response under moderate to heavy load. A system that is already slow with one user on an idle machine, and does not get much slower as load grows, is not running out of a resource: the cost is in the design or implementation of the operation itself, for example a query without a usable index.
Chapter 2 · Performance Measurement Fundamentals — 4 questions
Which of the following is a metric of the technical environment, as that category of performance metrics is described in the syllabus?
The efficiency of the claims-handling business process, expressed as the proportion of cases closed inside the agreed window.
Incorrect — business process efficiency is a business environment metric. It describes an outcome the organisation cares about rather than the state of the machinery.
The elapsed time the service needs to start up and reach the point at which it accepts its first request.
Incorrect — start-up timing belongs to the operational environment category, together with backup, shutdown, resumption and restore.
The amount of network bandwidth consumed by the application servers while the scripted transactions were executing.
Correct. Network bandwidth consumption is named among the technical environment metrics, alongside processor, memory, disk and input/output measurements.
The timing of usage, meaning the hours of the day at which the organisation's users place the heaviest demand on the system.
Incorrect — timing of usage sits in the business environment category, next to concurrency of usage and business process efficiency.
The syllabus groups performance metrics into three categories. Technical environment metrics describe the machinery and the traffic it carries — response times, processor, memory, disk and network consumption, input/output rates and the throughput of key transactions. Business environment metrics describe outcomes and usage patterns such as business process efficiency and the timing of usage, and operational environment metrics cover start-up, backup, shutdown, resumption, restore and alerting.
While preparing a performance test of a prescription-fulfilment system, a tester proposes switching on every counter the monitoring tool offers, on the grounds that more data can only help. The test manager disagrees and asks for a shorter list. Which reasoning matches the syllabus?
Metrics should be derived from the goals and from the questions those goals raise, so that each metric collected answers something a stakeholder actually asked.
Correct. This is the Goal-Question-Metric approach the syllabus names for metric selection, and it also keeps collection light enough to limit the probe effect.
Metrics should be limited to what the load generation tool reports by itself, because readings taken from other sources cannot be placed on a common timeline with the transactions.
Incorrect — the syllabus names performance monitoring tools and log analysis tools as metric sources alongside the test tool, and expects their clocks to be synchronised so the readings can be correlated.
Metrics should repeat the categories used in the previous project's report, so that the two sets of results can be placed side by side and compared without further work.
Incorrect — comparability matters, but it follows from the goals of this test rather than replacing them. A metric that answers no question here is not worth collecting because another report once contained it.
Metrics should cover every resource on every host in the environment, since the cost of collecting them is negligible once the monitoring agent has been installed.
Incorrect — the syllabus warns about the probe effect: detailed collection consumes resources on the system under test and can change the very figures being measured.
The syllabus recommends the Goal-Question-Metric approach for selecting performance metrics: start from the goals, derive the questions those goals raise, and collect the metrics that answer them. Collecting everything available is also discouraged because the act of collecting data can change what is being measured, which the syllabus calls the probe effect.
A legal e-discovery platform shows intermittent slow responses that the load generator records only as elapsed time. The team decides to bring in log analysis tools as an additional source of performance metrics. Which two kinds of finding does the syllabus associate with that source? (Choose two.)
Database deadlocks recorded by the database engine while concurrent transactions competed for the same rows.
Correct. Deadlocks are named in the syllabus as one of the findings log analysis tools surface, and they explain response times that resource counters leave unexplained.
Processor and memory utilisation sampled on each of the application hosts at a fixed interval throughout the whole run.
Incorrect — resource utilisation sampling is what performance monitoring tools provide. Logs may mention resource pressure indirectly, but the sampled counters come from the monitoring source.
Timeout and exception entries written by the application when a downstream call failed to return in time.
Correct. SQL exceptions and timeouts are listed among the findings log analysis delivers, and they often identify the component behind an intermittent slow response.
Transaction response times measured at the moment each scripted request was issued by the load generation tool.
Incorrect — these come from the performance test tool itself. They are the measurements the team already has and which prompted the search for a second source.
The number of virtual users that completed their script compared with the number that were started at the outset.
Incorrect — virtual user status is reported by the load generation tool, and the syllabus treats it as the first figure to examine when results are analysed.
The syllabus names three key sources of performance metrics: performance test tools, performance monitoring tools and log analysis tools. Log analysis is the source that surfaces what the application and the database wrote down while they were struggling — deadlocks, SQL exceptions and timeouts — which neither a resource counter nor an elapsed-time measurement reveals on its own.
A pharmacy fulfilment service returns its order-confirmation screen in 2.4…
A pharmacy fulfilment service returns its order-confirmation screen in 2.4 seconds under the planned load. The team asks the test manager whether that counts as a pass or a fail. No response-time figure appears anywhere in the requirements or in the contract. Which statement reflects the syllabus position on this situation?
It is a failure, because 2.4 seconds sits above the two-second figure that is widely quoted across the industry as the expectation for an interactive web transaction.
Incorrect — the syllabus mentions a rule of thumb for web responsiveness as context, but a figure circulating in the industry is not an agreed acceptance criterion for this service.
Performance testing has no clear test oracle, so the figure has to be judged against criteria agreed with the stakeholders or against a recorded baseline.
Correct. The syllabus makes exactly this point: the measurement itself cannot tell you whether it is good, so the missing criterion is the item to resolve with the stakeholders.
It is a pass, because no stated requirement has been breached and a measurement cannot be said to fail against a criterion that was never written down anywhere.
Incorrect — the absence of a criterion makes the result uninterpretable rather than acceptable. Reporting an unverified figure as a pass hides the gap instead of closing it.
It cannot be interpreted until the resource utilisation figures are known, because a response time that comes with low utilisation is acceptable by definition.
Incorrect — the syllabus warns of the opposite: utilisation below 75% can accompany response times an order of magnitude worse than required, so low utilisation proves nothing on its own.
The syllabus states that performance testing usually has no clear test oracle: a measured figure does not announce whether it is acceptable. The verdict comes from criteria agreed with stakeholders, from service level agreements, or from a baseline the figure can be compared against. Rules of thumb such as the seven-second expectation for web pages give context, not a pass mark.
Chapter 3 · Performance Testing in the Software Lifecycle — 7 questions
The syllabus maps performance testing onto the activities of the ISTQB test process. Which of the following belongs to the planning activity of a performance test?
Deciding which test environments, test data and tools the campaign will need, and identifying the performance risks the test has to address.
Correct. These are exactly the planning concerns the syllabus lists for performance testing, and they drive everything the later activities can and cannot do.
Working through the documents and volume forecasts already available in order to establish which performance objectives can be derived from them.
Incorrect — this is test analysis. It consumes the plan rather than producing it, and it identifies what can be tested from the available basis.
Building modular test cases that will later be recombined into the scenarios described by the operational profiles for the system.
Incorrect — this is test design. The syllabus recommends modular building blocks precisely so that scenarios can be assembled from them later.
Comparing the measured figures against the agreed criteria and recommending the changes that the stakeholders will be asked to fund.
Incorrect — this belongs to analysing results and reporting, which happens after execution and produces the recommendations section of the report.
Planning for a performance test establishes what the test will need and what it has to address: the test environments, the test data, the tools, and the performance risks to be covered. Deriving objectives from the available documents is test analysis. Building reusable modular test cases is design, and comparing measurements against criteria belongs to the completion and reporting work.
A shipping line is procuring a new freight documentation system under a sequential, phase-gated contract. How does the syllabus describe the handling of performance requirements in a sequential lifecycle?
Performance work is deferred to a dedicated iteration late in the project, where it can be run against a build that is already feature-complete.
Incorrect — a dedicated iteration is how the syllabus describes one option within iterative and Agile lifecycles, not the sequential arrangement described here.
Performance criteria are stated as acceptance criteria early on and then verified progressively across the phases, rather than at the end alone.
Correct. The syllabus ties performance criteria to the acceptance criteria of a sequential project and expects verification to accompany each phase.
Performance requirements are written as non-functional user stories and pulled into whichever phase still has capacity left in its schedule.
Incorrect — non-functional user stories are the iterative and Agile practice the syllabus describes, and pulling work by free capacity is not a phase-gated approach.
Performance testing is treated as part of acceptance testing, because earlier phases cannot yet produce a build that behaves like production.
Incorrect — the syllabus expects earlier verification through static reviews and component-level measurement, even though the production-like build arrives late.
In a sequential lifecycle the syllabus expects performance criteria to be stated as acceptance criteria early, during requirements work, and then verified progressively as the phases deliver verifiable artefacts — architecture reviews, component measurements, integration runs — rather than being left to a single test at the end when changes are at their most expensive.
A research institute runs genome-analysis batches across 60 compute nodes that all read from and write to one shared storage cluster, coordinated by a central scheduler. Which category of architectural performance risk does the syllabus assign to this arrangement?
Virtualised systems, where the resources a guest appears to have are not the resources it receives, because other guests on the same host compete for them.
Incorrect — the virtualisation risk is about contention invisible from inside the guest. Nothing in the scenario says the compute nodes are guests sharing a hypervisor.
Client-server systems, where connection speed, congestion and the network equipment between client and server dominate the times that end users observe.
Incorrect — the client-server category concerns the path between a user's client and the server. The batches described here have no interactive client leg at all.
Distributed systems, where work is spread over many nodes and the components they all share, such as storage and the scheduler, can bound the whole job.
Correct. The syllabus treats distributed architectures as their own risk category precisely because the shared dependencies, not the individual nodes, usually set the limit.
Dynamic or cloud-based systems, where self-scaling rules alter the resources available during a run and so become a test object in their own right.
Incorrect — that category applies where capacity changes automatically during the test. The institute's node count is fixed, so the scaling rules are not in play.
Distributed systems spread the work across many nodes, so the performance of the whole job is governed not only by the nodes themselves but by the components every node depends on. Here the shared storage cluster and the scheduler are candidate bottlenecks that no single-node measurement will reveal.
Which sequence describes the way the syllabus expects performance risks for a system under test to be handled?
Identify the risks, assess their likelihood and impact, decide on mitigation, and then state the residual risk that remains once the mitigation is in place.
Correct. This is the four-step process the syllabus sets out, and the residual-risk statement is what the stakeholders are asked to accept or reject.
Measure the current behaviour, compare it with the baseline, report the deviation, and leave the stakeholders to decide whether anything should be done.
Incorrect — this describes trend reporting against a baseline. It skips the identification and assessment work that decides which risks are worth measuring in the first place.
Collect the service level agreements, derive acceptance criteria from them, test against those criteria, and record the outcome of each one as pass or fail.
Incorrect — deriving criteria from service level agreements is part of test analysis. Risks can exist where no service level agreement was ever written.
Rank the architectural categories by exposure, test the highest-ranked category first, and stop once the remaining categories share a common component.
Incorrect — the architectural categories help identify candidate risks, but the syllabus does not offer this ranking rule or this stopping condition.
The syllabus describes a four-step treatment of performance risks: identify them, assess their likelihood and impact, decide on and apply mitigation, and then express what remains as residual risk. Stating the residual risk quantitatively, for example as the percentage of users still expected to exceed a latency threshold, is what lets stakeholders decide whether to accept it.
The final run of a performance test campaign for a motorway toll-collection system has been reported and accepted by the stakeholders. Which work belongs to the completion activity for that campaign?
Agreeing with the stakeholders which transactions the next campaign will cover and what the acceptance criteria for each of them are going to be.
Incorrect — this is planning work for the next campaign. It may follow completion in time, but the syllabus assigns it to a different activity.
Archiving the scripts, test data and measurement sets so a later campaign can repeat the runs, and recording the lessons learned from this one.
Correct. Preserving the testware and the measurements is what makes a future comparison possible, and capturing lessons learned is part of closing the campaign.
Adjusting the generated load and the hardware configuration so that the remaining runs of the campaign can still reach the objectives that were agreed.
Incorrect — these are control measures, which belong to test monitoring and control while the campaign is still running.
Correlating the response times with the resource counters in order to locate the point at which the system under test began to degrade.
Incorrect — this is part of analysing results, and it happens before the report that the stakeholders have already accepted.
Test completion gathers what the campaign produced so that it can be reused and learned from: scripts, test data, configurations and measurement sets are archived so a later campaign can repeat the runs and compare against them, and the lessons learned are recorded. Control measures belong to monitoring and control, criteria agreement to planning, and correlation of counters to analysing results.
A hotel group is buying a reservation platform as a service. The supplier…
A hotel group is buying a reservation platform as a service. The supplier supplies a benchmark report: 12,000 searches per minute at a median of 240 ms, produced on the supplier's reference configuration against a synthetic catalogue of 500 properties, with all availability served from the platform's own data. The group operates 4,100 properties, and roughly 70% of its traffic is rate-and-availability requests that fan out to three external channel managers. The programme manager proposes accepting the benchmark as evidence that performance is adequate. What is the strongest objection?
The benchmark used a far smaller catalogue and a different transaction mix, and it excludes the external channel-manager calls that dominate the group's traffic.
Correct. The acquirer's own profiles and volumes are what the test has to reproduce, and the two differences named here are the ones that change the result most.
The benchmark reports searches per minute, whereas the group's own requirements express throughput per second, so the two figures cannot be compared directly.
Incorrect — that is a unit conversion and nothing more. Dividing by sixty does not change what the benchmark measured or on which data it was measured.
The benchmark was executed by the supplier rather than by an independent party, so the figures cannot be relied on until someone reproduces them under supervision.
Incorrect — supervision would not fix the mismatch. A supplier-run benchmark on a representative workload would be far more useful than an independent one on the wrong catalogue.
The benchmark does not state the think times used between searches, so the number of virtual users behind the stated throughput cannot be reconstructed from it.
Incorrect — the missing think time is a genuine gap in the report, but throughput is the figure that defines the load, and the group could ask for it separately.
For commercial off-the-shelf software and for services bought in, the syllabus places the performance testing responsibility on the acquiring organisation, because only the acquirer knows its own operational profiles, data volumes and integrations. The supplier's benchmark was produced with a catalogue roughly eight times smaller and with the transaction type that dominates the group's traffic absent altogether, so it does not describe the group's workload.
A video-conferencing platform runs on a cloud deployment that adds application…
A video-conferencing platform runs on a cloud deployment that adds application instances whenever average processor use stays above 65% for three minutes. A forty-minute run at the target load reports a mean meeting-join time of 1.0 s and a 90th percentile of 1.5 s, both inside the 3 s objective, and the team recommends release. The run chart tells a different story: join time climbs steadily over the first four minutes, reaching 4.8 s, then falls back to about 0.8 s once two further instances come up and stays there. What does the evidence actually support?
The objective is met, because the aggregate figures cover the whole of the run including the minutes before the additional instances became available to serve requests.
Incorrect — averaging across a transient is exactly what the syllabus warns against. A four-minute breach is invisible in a forty-minute mean yet entirely visible to the users living through it.
The run points to a memory leak in the signalling service, since join time rose steadily throughout the opening minutes of the run before the platform eventually recovered.
Incorrect — a leak produces degradation that does not reverse on its own. Join time fell back as soon as capacity was added, which points at a capacity shortfall instead.
The aggregates hide a four-minute window in which join time reached 4.8 s, so everyone meeting during the scale-out saw times well past the objective.
Correct. The scaling rule reacts three minutes after the load arrives, and that lag is the defect the run has actually exposed.
The run demonstrates that the scaling rule works correctly, because join time returned to its starting level once the two additional instances had finished starting up.
Incorrect — one scale-out event shows the rule can fire, not that it fires early enough. Four minutes of breached objective is the cost of the delay it introduces.
The syllabus warns that aggregate figures taken across a transient period mislead, and that transient states matter in exactly the situations where many users arrive at once. Four minutes out of forty is too small a share to move the mean or the 90th percentile, so the aggregates pass while every user who joined during the scale-out waited well beyond the objective. The finding is the scaling lag itself, not the headline figure.
Chapter 4 · Performance Testing Tasks — 20 questions
During planning for a charity donation platform, the only performance requirement the sponsor has stated is that the donation page must feel fast during the Christmas appeal. Which reformulation turns this into an acceptance criterion the test can actually be executed against?
The donation page must respond more quickly than the pages of the two comparable charities that the marketing team monitors from week to week.
Incorrect — the target moves whenever someone else changes their site, and it cannot be evaluated inside a controlled test environment at all.
The donation page must not drive processor use on the web tier above 70% at any point while the Christmas appeal campaign is running.
Incorrect — this is a legitimate technical objective for the system owner, but it does not express what the sponsor means, and low utilisation can coexist with slow responses.
90% of donation submissions must complete within 3 seconds at the appeal peak of 1,200 submissions per hour, measured at the web server.
Correct. Metric, threshold, percentile, load level and measurement point are all present, so a run either satisfies this statement or it does not.
The donation page must perform at least as well during the Christmas appeal as the platform did throughout the previous year's spring appeal.
Incorrect — a baseline comparison is useful, but no figure from the spring appeal is quoted and the two campaigns carry different volumes, so the statement cannot be evaluated.
An acceptance criterion in a performance test plan has to name the metric, the threshold, the proportion of transactions the threshold applies to, the load at which it holds, and the point of measurement. The syllabus also separates user-based objectives, which express what a user experiences, from technical objectives such as resource utilisation, which belong to the system owner.
A test analyst is reviewing the think times written into the scripts for a library catalogue service. Which two statements about think time are correct? (Choose two.)
Think time separates the time the system spends responding from the time a simulated user spends reading or typing between requests.
Correct. Transaction response time plus think time gives elapsed time, and keeping the two apart is what makes the response-time figure meaningful.
Think time is counted by the load generator inside the transaction response time, so shortening it brings the reported response time down.
Incorrect — think time falls outside the timed transaction. Shortening it raises the load the scenario applies, which may worsen response time rather than improve it.
For a fixed number of virtual users, changing the think time changes the throughput the scenario actually delivers to the system under test.
Correct. Throughput equals the virtual user count divided by processing time plus think time, so think time is the lever used to reach a target throughput.
Think time should be set to one fixed value across all virtual users, so that transactions arrive at the system in a repeatable pattern.
Incorrect — a single fixed value makes the virtual users march in step and produces arrival bursts that no real population generates. Varied think times are preferred.
Think time is applied once at the start of each virtual user's script, so that the users do not all begin their first transaction at the same moment.
Incorrect — staggering the start of virtual users is the job of the ramp-up in the load profile. Think time recurs between the steps inside the script.
Think time represents the pause a real user takes between requests. It sits outside the transaction response time and together with it makes up the elapsed time. Because system throughput for a closed scenario is the number of virtual users divided by the sum of processing time and think time, adjusting think time is the usual way of tuning a fixed population of virtual users to a target throughput. The syllabus favours varied think times over a single fixed value, which produces an unrealistic lock-step arrival pattern.
A print and publishing workflow is delivered to its users as a published desktop…
A print and publishing workflow is delivered to its users as a published desktop over a remote-access protocol. No application traffic reaches the user's own machine directly: the client receives screen updates and sends keyboard and pointer events. Which protocol family does the syllabus associate with generating load against a deployment like this?
Database protocols such as ODBC and JDBC, which carry the queries the publishing workflow issues against its own catalogue.
Incorrect — those protocols run between the application and its database, inside the hosted environment. The load has to be applied where the users actually connect.
Web service protocols such as SOAP and REST, which carry the structured messages exchanged between the components of the workflow.
Incorrect — the scenario states that no application traffic of this kind reaches the user's machine, so there is nothing of the sort to record or replay at the client.
Remote access protocols such as Citrix ICA and remote terminal emulation, which carry the screen updates and the input events.
Correct. The syllabus names remote access among the protocol families, and a published desktop is exactly the deployment it refers to.
Messaging and integration protocols such as MQSeries and WSCL, which carry asynchronous traffic between back-end services.
Incorrect — these belong to service-oriented integration between back-end components, not to the session between a user's client and a published desktop.
The syllabus lists the protocol families a performance test tool may have to speak, grouped by what they carry: database access, web and web services, networking and mail, mobile, service-oriented integration, and remote access. A published desktop is driven through remote access protocols such as Citrix ICA or remote terminal emulation, and tool compatibility with that family becomes the decisive constraint.
A script is being built for a national archive search service. A user signs in once at the start of a session and then runs many searches before leaving. The team wants both the sign-in time and the search time to appear in the results. Where should the sign-in step be placed in the script, and why?
In the initialization section with a timer of its own, because it happens once per session while the searches are the part that repeats.
Correct. Placing it in initialization reproduces real usage, and the timer keeps the sign-in duration visible in the results.
In the main repeatable section with a timer around it, so that each iteration produces a fresh sign-in measurement next to its search measurement.
Incorrect — that makes every virtual user sign in once per search, which inflates authentication load far beyond anything the real population generates.
In the clean-up section alongside the sign-out step, so that the whole session boundary is measured as a single unit at the end of the script.
Incorrect — clean-up runs after the main section, so sign-in placed there would follow the searches it is supposed to precede.
Outside the script as a preparation step run before the scenario starts, so that the sign-in load does not reach the system under test at all.
Incorrect — authentication is part of the load a real population applies, and excluding it would leave the sign-in time the team asked for unmeasured.
The syllabus describes a three-part script structure: an initialization section that runs once, a main section that is repeated, and a clean-up section that runs once at the end. Steps are placed according to how often a real user performs them, and timers are put around the operations whose duration is to be reported. Sign-in happens once per session, so it belongs in initialization, with its own timer so that its duration is still measured.
The results of a performance test on a regional bus ticketing service have to be presented twice: once to the operations board that funds the service, and once to the infrastructure team that runs it. Which approach does the syllabus recommend?
Report the complete measurement set unchanged to both audiences, since showing different figures to different groups risks the two versions disagreeing later on.
Incorrect — presenting the same figures in different terms is not the same as presenting different figures. An undigested measurement set leaves the board without the outcome it has to decide on.
Report the same underlying results twice over, in business terms for the operations board and in transaction and resource terms for the infrastructure team.
Correct. The syllabus distinguishes business-focused from technology-focused stakeholders and expects the reporting to be adapted to each.
Report the business figures to the board and leave the infrastructure team to pull the resource counters out of the monitoring tool for themselves afterwards.
Incorrect — the counters have to be correlated with the transactions on a common timeline, which is work the test produces rather than something a reader reconstructs.
Report the resource figures to both audiences, since utilisation is the measurement that decides whether additional hardware has to be purchased.
Incorrect — utilisation is a technical objective. The board's decision rests on the business outcome, and low utilisation can accompany unacceptable response times.
The syllabus is explicit that communication with business-focused stakeholders differs from communication with technology-focused stakeholders. The underlying measurement set is the same, but the board needs it expressed as business outcomes — journeys sold per hour, proportion of purchases inside the target — while the infrastructure team needs transaction timings and resource counters.
A test analyst is writing the load profile for a seasonal ticketing surge at an events venue. Which three items does a load profile specify? (Choose three.)
The number of virtual users executing the operational profiles at each point in time across the whole of the run.
Correct. This is the core of a load profile: the population over time, expressed in virtual users.
The shape of the ramp-up and of the ramp-down, including any stepped increases applied along the way to the target.
Correct. The syllabus names ramp-up, ramp-down, steps and predefined distributions as the ways a load profile shapes the population.
The proportion in which the individual business transactions occur within a single ticket buyer's session at the venue.
Incorrect — that mix is the operational profile. The load profile says how many users run those profiles and when, not what each user does.
How long the load is to be held at the target level once the ramp-up phase has been completed in full.
Correct. The duration at the target level is what produces the steady state in which the results are analysed.
The thresholds that the measured response times are going to be judged against once the whole run has been completed.
Incorrect — those are the acceptance criteria, recorded in their own section of the performance test plan rather than in the load profile.
A load profile describes the number of virtual users executing the operational profiles over time: how the population grows, how long it is held at the target level, and how it is brought back down. The mix of business transactions within a session belongs to the operational profile, and the thresholds the results are judged against belong to the acceptance criteria.
A draft acceptance criterion for a car-park payment system reads: no payment confirmation may take longer than 2 seconds. The test manager asks for it to be restated using a percentile instead. Why does the syllabus favour that form?
Because a percentile sets a less demanding target than an absolute limit, which keeps the criterion reachable on a scaled-down test environment.
Incorrect — the aim is a criterion that expresses the real requirement, not an easier one. A percentile can be set at a stricter threshold than the absolute limit it replaces.
Because requiring every single transaction to conform is not recommended: isolated outliers are expected and would fail an otherwise acceptable system.
Correct. The syllabus advises against demanding full conformance and reports response time through percentiles for exactly this reason.
Because load generation tools report percentiles directly, whereas an absolute maximum has to be derived by hand from the raw measurement set.
Incorrect — the maximum is among the figures a tool reports as standard, alongside minimum, average and percentile. Availability of the number is not the issue.
Because a percentile excludes the failed transactions from the calculation, while the maximum is distorted by transactions that never completed at all.
Incorrect — a percentile does not filter anything out by itself. Failed transactions have to be dealt with separately, by examining virtual user status before any timing figure is trusted.
The syllabus states that requiring 100% conformance to a response-time target is not recommended, and that response times are normally reported as minimum, maximum, average and a percentile, commonly the 90th. A handful of outliers is expected in any run, and an absolute limit turns each of them into a failure of the whole system.
An agricultural fleet telematics platform exposes about 140 distinct operations. The schedule allows scripts to be built for roughly a dozen of them. On what basis does the syllabus expect the test analyst to choose which operations are scripted?
On the basis of which operations the development team expects to be slow, since those are the ones a performance test has a chance of exposing.
Incorrect — a test built on expectations confirms them rather than challenging them, and it leaves the high-volume operations nobody suspected unmeasured.
On the basis of which operations are the easiest to script reliably, so that the limited time available yields working scripts rather than half-finished ones.
Incorrect — scripting effort is a real constraint, but letting it choose the content means the scenario stops representing the production workload.
On the basis of the operational profile: the operations carrying the business volume and those whose failure matters most to the stakeholders.
Correct. Volume determines the load the system actually carries, and business criticality determines where a performance failure does the most damage.
On the basis of code coverage, selecting the operations that between them exercise the largest proportion of the server-side code paths.
Incorrect — coverage is a structural criterion from functional testing. A rarely used operation touching much code contributes almost nothing to the load.
Scripts are derived from the operational profiles, which are built from personas and roles and populated from interviews and production logs. The operations that carry the business volume dominate the load the system actually sees, and the ones whose failure hurts most carry the risk. Ease of scripting, developer intuition and code coverage are not the selection criteria the syllabus gives.
A tester analysing the results of a run against a mapping and navigation tile service has confirmed that every virtual user completed its script without error. Following the order of analysis the syllabus sets out, which figure is examined next, and in what form?
Hits per second, as an average across the run, because it shows how much work the servers were asked to carry out.
Incorrect — hits per second comes later in the order. It describes the request volume rather than what the users experienced.
Network throughput in bits per second, plotted over time, because it shows whether the link between the generators and the system saturated.
Incorrect — network throughput is examined further down the list, after the transaction figures have established whether there is a problem to explain.
Transaction response time, reported as minimum, maximum and average together with a percentile, which is usually the 90th.
Correct. This is the next step in the stated order, and the percentile is what the acceptance criteria are normally expressed against.
The mix of HTTP response codes, counted by code, because redirects and cached responses change how much work each request represents.
Incorrect — the response code mix is useful context, but the syllabus places it near the end of the analysis order rather than immediately after virtual user status.
The syllabus puts virtual user status first, because failures distort everything that follows. Once that is clean, transaction response time is the next figure, reported as minimum, maximum and average together with a percentile, commonly the 90th. Transactions per second, transaction failures, hits per second, network throughput and the HTTP response code mix follow after that.
A performance test report on a broadcast ad-serving platform is going to be used as the baseline for next year's campaign. Which three facts must the report record so that a later run can be compared against it in a meaningful way? (Choose three.)
The configuration of the test environment, including the ways in which it differed from production at the time of the run.
Correct. Performance is a non-linear function of the environment, so a measurement is only interpretable alongside the configuration that produced it.
The load profile that was applied, including the ramp, the steps and the time the load was held at the target level.
Correct. A different ramp or a different hold time produces different figures from the same system, so the profile belongs with the result.
The volume and the state of the test data the run was executed against, and how that data set was produced.
Correct. Data volume drives index behaviour and query cost, so a comparison against a differently sized data set measures the data rather than the change.
The defects raised against the system under test during the campaign and the status each of them currently holds in the tracker.
Incorrect — useful project information, but a defect list does not let anyone decide whether two sets of measurements were taken under the same conditions.
The resource utilisation thresholds the infrastructure team applies to its own production alerting rules and escalation policy.
Incorrect — production alerting thresholds belong to operations. They may inform the acceptance criteria, but they are not a condition of the run.
Comparability is one of the stated principles, and a baseline is only usable if the conditions that produced it are written down. The environment configuration and its differences from production, the load profile that was applied, and the volume and state of the test data all change the measurements directly, so a later run can only be compared once they are known to match or the differences are understood.
A pharmacy chain allows performance tests only inside a two-hour maintenance…
A pharmacy chain allows performance tests only inside a two-hour maintenance window that runs from 01:00 to 03:00. Ten minutes at the start of the window are reserved for resetting the environment and restoring the test database, and five minutes at the end are reserved for collecting logs and measurement files. The load profile calls for a ramp-up in six steps of five minutes each, followed by the target load being held, followed by a ramp-down lasting fifteen minutes. How long is the steady-state period that this window leaves, and when does it begin?
75 minutes, running from 01:40 until 02:55 when the window closes for log collection.
Incorrect — this leaves no room for the ramp-down. The fifteen minutes it occupies have to come out of the usable time before the log collection starts.
65 minutes, running from 01:40 until 02:45 when the ramp-down has to begin.
Incorrect — this forgets the five minutes reserved at the end. The run has to be finished by 02:55, so the ramp-down starts at 02:40 rather than 02:45.
50 minutes, running from 01:50 until 02:40 when the ramp-down has to begin.
Incorrect — the ramp-up is six steps of five minutes, which is 30 minutes from 01:10 and therefore ends at 01:40 rather than 01:50.
60 minutes, running from 01:40 until 02:40 when the ramp-down has to begin.
Correct. Usable time 01:10 to 02:55, minus 30 minutes of ramp-up and 15 minutes of ramp-down, leaves exactly one hour of steady state.
The usable test time runs from 01:10 to 02:55, which is 105 minutes. The ramp-up of six five-minute steps takes 30 minutes and finishes at 01:40, and the ramp-down of 15 minutes has to finish by 02:55, so it starts at 02:40. The steady state is therefore 01:40 to 02:40, exactly 60 minutes. This is the only part of the run the syllabus expects the analysis to be based on.
A ferry operator's booking system handles, during its documented busy hour…
A ferry operator's booking system handles, during its documented busy hour, 2,400 availability searches, 600 booking confirmations and 1,000 check-in look-ups. The scenario will be driven by 80 virtual users, and the team has decided to allocate them across the three scripts in proportion to the busy-hour transaction volumes, so that the mix the system receives matches the operational profile. How many virtual users are assigned to each script?
48 virtual users on availability search, 12 on booking confirmation and 20 on check-in look-up.
Correct. The shares are 60%, 15% and 25% of 4,000 transactions, and applying them to 80 virtual users gives exactly these figures.
60 virtual users on availability search, 15 on booking confirmation and 25 on check-in look-up.
Incorrect — these are the percentage shares used as if the population were 100 virtual users. The scenario has 80, so each figure has to be scaled.
27 virtual users on availability search, 27 on booking confirmation and 26 on check-in look-up.
Incorrect — this splits the population evenly across the three scripts and ignores the operational profile, which is what defines the mix.
44 virtual users on availability search, 16 on booking confirmation and 20 on check-in look-up.
Incorrect — the check-in share is right but the other two are not. 2,400 of 4,000 is 60%, which is 48 of 80, not 44.
The busy hour carries 4,000 transactions in total. Availability searches are 2,400 of 4,000, which is 60%; confirmations are 600 of 4,000, which is 15%; check-in look-ups are 1,000 of 4,000, which is 25%. Applying those shares to 80 virtual users gives 48, 12 and 20. Allocating in proportion to volume is what keeps the applied mix faithful to the operational profile.
A national vehicle-inspection register currently holds 18.4 million inspection…
A national vehicle-inspection register currently holds 18.4 million inspection records and grows by a steady 310,000 records each month. The replacement system goes live in four months' time, and the performance test environment has to represent the register as it will stand 18 months after go-live, because that is the horizon the capacity plan covers. Assuming the growth rate holds, how many records should be loaded into the test database?
About 19.6 million records, which is today's volume plus the growth expected before the system goes live.
Incorrect — this covers only the four months to go-live. The capacity plan runs for eighteen months after that, and the test has to represent the end of it.
About 24.0 million records, which is today's volume plus eighteen months of growth at the stated rate.
Incorrect — this omits the four months between now and go-live. The eighteen months are counted from go-live, not from today.
About 30.5 million records, which is today's volume plus the growth expected over the next 39 months.
Incorrect — 39 months is roughly double the horizon. Oversizing the data set makes the environment slower than the system will ever be and produces a pessimistic result.
About 25.2 million records, which is today's volume plus the whole 22 months of growth at the rate given.
Correct. Four months to go-live plus eighteen months of operation gives 22 months, and 22 at 310,000 adds 6.82 million to the 18.4 million already held.
The horizon is four months to go-live plus eighteen months of operation, which is 22 months of growth from today. 22 months at 310,000 records is 6.82 million, so the register will hold about 25.2 million records. Sizing the test data to the horizon matters because data volume drives index behaviour and query cost, and a test against today's volume says nothing about the system at the end of the capacity plan.
During a two-hour run against a document-archiving service, the Java heap still…
During a two-hour run against a document-archiving service, the Java heap still occupied immediately after each full garbage collection rose steadily from 820 MB at the start to 1,180 MB at the end. The heap is capped at 4,096 MB. The team has to decide how long the endurance run should be in order to show the effect reaching the limit. Assuming the rate of growth continues unchanged, roughly how long after the start of the run would the heap be exhausted?
About 9 hours, so a working day of continuous load would be enough to reach the limit.
Incorrect — this treats the 360 MB as an hourly figure. It was measured across two hours, so the rate is 180 MB per hour.
About 18 hours, so the endurance run has to be far longer than the two hours used here.
Correct. 3,276 MB of headroom at 180 MB per hour is roughly 18 hours, which is the duration the run would need to reach the cap.
About 16 hours, so an overnight run would be close to reaching the limit but might not pass it.
Incorrect — this measures the headroom from the end-of-run level of 1,180 MB. The elapsed time is counted from the start of the run, where the heap held 820 MB.
About 23 hours, so a full day of continuous load would be needed before the limit is reached.
Incorrect — this divides the whole 4,096 MB cap by the growth rate and ignores the 820 MB that was already occupied when the run began.
The occupied heap grew by 360 MB in two hours, which is 180 MB per hour. The headroom from the starting level is 4,096 minus 820, which is 3,276 MB. At 180 MB per hour that gives about 18.2 hours. The practical consequence is that a two-hour run is far too short: endurance testing has to run long enough for the slow effect it targets to become visible.
A team compares two runs of the same scenario against a toll-road account…
A team compares two runs of the same scenario against a toll-road account service. Run A, three months ago, produced a 90th percentile of 1.2 s. Run B, executed this week after an indexing change intended to improve matters, produced 2.1 s. The team concludes that the change made things worse and plans to revert it. Checking the environment records, you find that Run A executed against a database restored from a six-month-old extract holding 4.1 million accounts, while Run B ran against a fresh extract holding 11.8 million. What is the sound conclusion?
The change made matters worse: the 90th percentile almost doubled between the two runs, and the indexing change is the only thing the team altered on purpose.
Incorrect — intent is not the test of comparability. The data set changed too, whether or not anyone decided to change it, and it moves the figure on its own.
The change made matters better: a data set nearly three times larger answering in under twice the time shows that the new index is doing the work asked of it.
Incorrect — this is an appealing inference but an unsupported one. Nothing says the relationship between volume and response time is linear in this system.
The comparison holds, because both of the runs used the same scenario, the same load profile and the same environment configuration from start to finish.
Incorrect — the state and volume of the test data are part of the conditions of a run, as much as the hardware and the profile are.
The two runs are not comparable: the data volume differs, so the measurement has to be repeated on one data set before anything is said about the change.
Correct. Restoring the same extract for both runs isolates the indexing change, and until that is done the difference has at least two candidate causes.
A baseline is only usable when the conditions that produced it are reproduced. Data volume drives index selectivity, cache hit rates and query cost directly, so two runs against data sets differing by a factor of nearly three measure the data as much as the code. Nothing can be concluded about the indexing change until both runs are executed against the same data set.
An IoT building-management platform fails its load test: response time for the…
An IoT building-management platform fails its load test: response time for the set-zone-temperature command rises from 0.4 s at low load to 6.1 s at the target load. Processor, memory, disk and network counters are collected on the web tier, the application tier and the database, and none of them exceeds 45% at any point. The command passes through a message broker between the application tier and the device gateway. The broker is run by another team and is not instrumented in this environment. Which three conclusions follow from this evidence? (Choose three.)
The counters collected rule out a resource constraint, so the cause has to lie in the application code rather than in any component's capacity.
Incorrect — the counters cover three of the four components in the path. A resource constraint inside the uninstrumented broker, such as an exhausted queue or thread pool, remains entirely possible.
The uninstrumented broker is the one component this evidence cannot clear, and it has to be brought into the measurement set.
Correct. Diagnosis needs coverage of the whole path, and the single gap in that coverage sits squarely between the tiers whose timings diverge.
Low utilisation on the instrumented tiers is consistent with those tiers waiting, so it is not evidence that the system is healthy.
Correct. The syllabus makes exactly this point: a component that spends its time blocked shows little resource consumption while serving users slowly.
The run should be repeated at a higher load so that the bottleneck finally becomes visible in the counters already being collected.
Incorrect — raising the load on an instrument set that misses the suspect component produces a worse symptom and the same blind spot.
Until the broker is measured, the report can state the symptom and the measurement gap, but cannot name a cause or recommend a change.
Correct. Recommending work on a component that has not been shown to be responsible is how effort gets spent in the wrong place.
Low utilisation across the instrumented components is consistent with those components waiting on something else, which is precisely what the syllabus warns about when it notes that utilisation below 75% can accompany response times an order of magnitude worse than required. The one component with no measurements is the one the evidence cannot clear, and until it is instrumented the report can describe the symptom and the gap but not name a cause.
After enlarging the connection pool of a wholesale pharmacy ordering service, a…
After enlarging the connection pool of a wholesale pharmacy ordering service, a team executes the scenario once and obtains a 90th percentile of 1.84 s, against 1.97 s from the run immediately before the change. They recommend rolling the change out. Looking further back, the three runs executed before the change, all under the same conditions, gave 90th percentiles of 1.97 s, 1.78 s and 2.06 s. What does the evidence actually support?
The change is an improvement of roughly 7%, which is modest but entirely consistent with the direction the connection-pool tuning was intended to produce.
Incorrect — the 7% is measured against one earlier run that happened to be the slowest but one. Against the fastest earlier run, the same change looks like a slight loss.
The change should be rejected outright, because 1.84 s still sits above the best figure of 1.78 s that was recorded before the pool was enlarged at all.
Incorrect — this makes the same mistake in the other direction, treating a single run from the spread as if it were the system's true value.
The comparison should be made against the average of the three earlier runs, which is 1.94 s, and the improvement reported as being about 5% in total.
Incorrect — averaging the baseline helps, but one post-change run still has no spread of its own, so the difference cannot be separated from variation.
The single post-change figure falls inside the spread of the three earlier runs, so the scenario has to be repeated before any improvement is claimed.
Correct. Several post-change runs would show whether the distribution has moved, which is the question the team is really asking.
Reproducibility is one of the stated principles of performance testing, and it cuts both ways: a single measurement cannot be separated from the natural variation between runs. The earlier runs span 1.78 s to 2.06 s, and the post-change figure of 1.84 s sits comfortably inside that spread, so it is indistinguishable from another run of the unchanged system.
A property-conveyancing case-management system is tested in an environment that…
A property-conveyancing case-management system is tested in an environment that reproduces production in every respect but one: production runs a read replica that serves all reporting and search queries, while the test environment serves them from the single primary database. The run shows search response times inside target, and the team records the environment as production-equivalent. Which two statements are correct? (Choose two.)
Because the primary carried both the write and the read load during the test, the measured search times transfer to production unchanged.
Incorrect — carrying more load in test may make the figures conservative for this one metric, but it does not make them transferable, and production adds replication work the test never applied.
The missing replica makes the test optimistic, because the single primary avoids the network hop that production places between the tiers.
Incorrect — the replica removes read work from the primary in production, so if anything the test configuration is the harder one for search. The direction stated here is the wrong way round.
The run says nothing at all about replication lag, which in production can return search results that are not yet fully current.
Correct. Replication is absent from the test configuration altogether, so neither its delay nor its cost on the primary has been measured.
The search measurements should be discarded, because a result from a non-equivalent environment carries no usable information.
Incorrect — the syllabus expects environments to differ and the differences to be stated, not the results to be thrown away. The measurement still bounds the primary's behaviour.
The environment is not production-equivalent, and the difference belongs in the report rather than being described as equivalence.
Correct. Recording how the environment differs is what allows a reader to judge how far the figures can be carried, which is the point of the environment section.
The syllabus expects differences between the test environment and production to be recorded, because performance is a non-linear function of the environment. Serving searches from the primary rather than a replica changes where the read load lands and removes replication entirely from the picture, so the run cannot speak to replication lag or to the capacity of the replica, and the environment cannot honestly be called equivalent.
A payments reconciliation service receives files from a clearing network at a…
A payments reconciliation service receives files from a clearing network at a peak rate the business states as 95 files per minute. Over a two-hour run at that arrival rate, 11,400 files were submitted and 9,300 were processed. The mean processing time of a completed file stayed flat at 540 ms from the first minute to the last. The inbound queue depth, plotted across the run, rose steadily from zero to about 2,100 files. The team reports that processing time is well inside target. What does the evidence show?
The service is below the required rate, falling behind by roughly 17 files a minute, and the flat processing time hides the waiting because it measures service time alone.
Correct. The 2,100-file gap over 120 minutes is the shortfall, and it will keep growing for as long as the peak lasts.
The service meets its requirement, since the mean processing time of 540 ms is comfortably inside the target and did not degrade at any point across the two hours of the run.
Incorrect — a steady service time with a growing queue is the signature of a system that cannot keep up. End-to-end latency is rising even though service time is not.
The queue growth points to a defect in the queue manager rather than in the service, because the processing time of the files that did complete did not change at all during the run.
Incorrect — the queue is doing its job, which is to hold work that has arrived and not yet been served. The shortfall is in the rate at which files are consumed.
The run is inconclusive because the arrival rate was imposed by the load generator rather than being produced by the clearing network itself under real operating conditions.
Incorrect — reproducing the stated arrival rate is exactly what an open-system test should do. The generator applied the agreed peak and the result is interpretable.
This is an open system: arrivals are set by the clearing network and do not wait for the service to finish. Submitted minus processed is 2,100 files, which matches the queue growth exactly, so the service is running about 17.5 files per minute short of the arrival rate. The flat processing time is not reassurance: it measures only the service time of a file once it is picked up, and says nothing about how long the file waited in the queue first.
A run against a season-ticket renewal portal reports 1,184 failed transactions…
A run against a season-ticket renewal portal reports 1,184 failed transactions out of 148,000 executed, which is 0.8%. The plan allows up to 1%, so the team records the run as a pass. Broken down by script, the failures are: browse 0 of 61,000, select seat 0 of 44,000, pay 1,184 of 21,000, confirm 0 of 22,000. What is the correct reading of this result?
The run is a pass: the overall failure rate sits inside the tolerance the plan set, and that tolerance was agreed with the stakeholders before the campaign began.
Incorrect — the tolerance was agreed against a figure nobody expected to be concentrated in one place. An aggregate inside tolerance can still hide a transaction type that is failing badly.
The run is a fail because the tolerance written into the plan was meant for read-only operations and cannot reasonably be applied to a payment step at all.
Incorrect — nothing in the scenario restricts the tolerance in that way, and inventing a scope for it afterwards is not how the result should be argued.
The run cannot be read at all until it has been repeated, since 0.8% sits within the normal run-to-run variation for a ticketing portal of this size.
Incorrect — variation would scatter failures across scripts. A pattern this concentrated is a finding in itself and does not need a second run to be taken seriously.
The aggregate hides the real figure: about one in eighteen payment attempts failed, and the timings for that script are distorted by those failures too.
Correct. The failure rate has to be read per transaction type, and the payment timings cannot be trusted until the failures are explained and removed.
An aggregate failure rate averages across transactions that have nothing in common. Here every failure falls on one script: 1,184 of 21,000 payment attempts is about 5.6%, roughly one in eighteen. The syllabus also warns that failed transactions distort the timing figures, because a transaction that fails usually finishes faster than one that succeeds, which inflates throughput and flatters response time for that script.
Chapter 5 · Tools — 3 questions
A national census agency has to run a week-long performance campaign that peaks at 30,000 virtual users driven from four countries. Which considerations belong to the scalability factor of tool suitability, as the syllabus defines it?
Whether the tool records and replays the protocols the census application uses, whether it reaches the agency's own external components, and which platforms it is able to run on.
Incorrect — protocols, interfaces to external components and supported platforms make up the compatibility factor.
How many concurrent virtual users the tool supports, what the licence permits, how the generator machines are configured, and how many points of presence it can drive from.
Correct. These are the four considerations the syllabus groups under scalability, and all of them bind at 30,000 users across four countries.
Whether the counters the tool collects are sufficient on their own, whether they can be supplemented from elsewhere, and whether they can be correlated with the transactions executed.
Incorrect — these are the questions the monitoring factor asks, and they concern what the tool can tell you rather than how far it stretches.
What level of technical knowledge the tool demands of the people who are going to operate it, and what becomes of the results when that level of knowledge is not present.
Incorrect — this is the understandability factor, which the syllabus describes as the most frequently underestimated of the four.
The syllabus names four factors for judging tool suitability: compatibility, scalability, understandability and monitoring. Scalability covers how many concurrent virtual users the tool can drive, what the licence permits, how the generator machines have to be configured to reach that number, and whether load can be generated from several points of presence.
A municipal utility is choosing a performance test tool. Its testers have no scripting background, and there is no budget to bring in a specialist. Which two considerations belong to the understandability factor of tool suitability? (Choose two.)
How many virtual users a single generator machine can drive before a second machine has to be brought in.
Incorrect — generator capacity and machine configuration sit under the scalability factor.
What level of technical knowledge the tool expects from the people who are going to operate it day to day.
Correct. This is the core question the understandability factor puts, and it is decisive for a team with no scripting background.
Whether the counters the tool gathers can be supplemented from elsewhere when they turn out to be insufficient.
Incorrect — sufficiency of the counters and the ability to supplement them are questions the monitoring factor asks.
What happens to the validity of the results when the tool is operated by people without that level of knowledge.
Correct. The syllabus notes that insufficiently skilled testers produce incorrect results, which is why this factor is so often underestimated.
Whether load can be generated from more than one point of presence so that regional network paths are represented.
Incorrect — generating from several points of presence is part of the scalability factor, and it concerns reach rather than the skill the tool demands.
Understandability asks what level of technical knowledge a tool demands of the people who will actually use it, and what the consequence is when that level is absent. The syllabus singles it out as the factor most often underestimated, because testers without the required skill produce results that look complete and are wrong. Generator capacity, points of presence and counter coverage belong to the scalability and monitoring factors instead.
A tax software vendor needs load testing twice a year: a four-week campaign…
A tax software vendor needs load testing twice a year: a four-week campaign before each filing season, peaking at 9,000 virtual users, plus a 200-user check that runs against every nightly build all year round. Tool A is commercial, sold as an annual per-seat subscription with a 10,000-user ceiling, and integrates with the build pipeline; finance has said the subscription cannot be justified for eight weeks of use. Tool B is a cloud service billed per virtual-user-hour with no ceiling, driven from a web console, with no pipeline integration. Tool C is open source and free, supports the protocols in use and runs from the command line, but reaching 9,000 users would need about a dozen generator machines the vendor would have to supply and maintain. Which recommendation is best supported by the suitability factors?
Use Tool A for both needs: it covers the seasonal peak and the pipeline, and one tool means one set of scripts to build and maintain.
Incorrect — a single toolchain is genuinely attractive, but finance has ruled the annual subscription out. A recommendation the project cannot fund is not a feasible approach.
Use Tool B for both needs: it has no user ceiling, so peak and nightly run are both covered with no generator machines to look after.
Incorrect — Tool B has no pipeline integration, and a nightly check that someone has to start from a web console will not survive contact with a build schedule.
Use Tool C for both needs: it is free, supports the protocols, and the generator machines can be powered down between the two campaigns.
Incorrect — the machines still have to be bought, configured and maintained for two short peaks a year, which is the cost the cloud model exists to avoid.
Use Tool C for the nightly 200-user check, where command-line operation fits the pipeline, and Tool B for the two seasonal campaigns.
Correct. Each need gets the acquisition model that suits its shape, and the scripting effort is duplicated only for the small nightly scenario.
The syllabus describes three ways of acquiring tools — commercial per seat or site, cloud pay-as-you-go, and open source — and judges tools on compatibility, scalability, understandability and monitoring, within the principle that the approach has to be feasible for the project. Two different needs sit here: a nightly, low-volume run that has to be driven by the pipeline, and a short, very large seasonal peak. A command-line open-source tool fits the first, and pay-as-you-go cloud capacity fits the second without carrying a year-round licence or a dozen idle machines.