Certified Tester Performance Testing (CT-PT) — Chapter 2: Performance Measurement Fundamentals

4 of 40 questions on the real exam12 practice questions available

Chapter 2 is about measurement itself: which metrics are typically collected and how they group into technical, business and operational environment categories, how metrics are selected using Goal-Question-Metric, why results are aggregated, the three key sources metrics come from, and what a performance test result can and cannot tell you. Like chapter 1 it stays at recall and understanding level.

It is the shortest chapter on the exam and the one that makes the analysis questions later answerable. If you cannot say which category "orders per hour" belongs to, or articulate why a number can be true and misleading at once, the diagnostic questions in the tasks chapter have nothing to stand on.

Where candidates lose these points:

  • Sorting metrics into the wrong environment. Response time per transaction, resource utilisation, network latency, I/O rate, idle and busy threads and batch processing time are technical. Business process efficiency, orders per hour, SLA compliance, and the concurrency and timing of usage are business. Start-up, shutdown, backups, restore from backup, resumption and alerts are operational.

  • Expecting a clear test oracle. Performance testing usually has none — there is often no single expected value to compare a result against. That is precisely why a baseline and comparison over time carry more weight here than any absolute figure.

  • Trusting raw numbers. The syllabus offers its own example: utilisation below 75% while response time is an order of magnitude worse than target. Comfortable-looking utilisation is not evidence of health, and a question will offer it as the reassuring answer.

  • Ignoring the probe effect. Data collection must not perturb the system under test. Verbose per-call tracing written to disk changes the thing being measured, and the resulting numbers describe the instrumented system, not the real one.

  • Treating the three sources as interchangeable. Performance test tools, performance monitoring tools (which also run in production and can reveal denial-of-service activity) and log analysis tools each see different things. Deadlocks, SQL exceptions and internal time-outs surface in the logs, not in the load tool's summary.

  • Choosing metrics bottom-up. Goal-Question-Metric works from a stated goal, through the questions that would show the goal is met, to the metrics that answer them. Taking whatever the tool reports by default is the behaviour GQM exists to prevent — and one of the named project risks is having no baseline at all, so trends cannot be seen and judgement stays subjective.

This chapter is covered most heavily (4 questions) in CT-PT — Mock Exam 1 (Performance Testing Tasks, Load Profiles and Metrics), CT-PT — Mock Exam 2 (Resource Saturation & Bottleneck Diagnosis), CT-PT — Mock Exam 3 (Distributed Systems, Garbage Collection and Cluster Analysis)