Performance testing as a service: scope, timeline, price drivers and the report

It is a Tuesday in mid-September. In six weeks a campaign goes live that marketing has spent more on than on the entire previous quarter, and the online shop or customer portal will have to take several times its normal traffic. In the board meeting someone asks the question that ends up on your desk as the tech lead: “Will the system hold?” You have no honest answer, because nobody has ever checked where the limit is. The team is busy building campaign features, so you decide to outsource the performance testing.

A week later you have three quotes. The first: €2,500, “load test up to 5,000 users, report”. The second: €9,500, four weeks, scenarios, load, stress and soak tests, bottleneck analysis, a confirmation run. The third: €30,000, three months, “end-to-end performance engineering programme”. Each sounds reasonable, and none of them can be compared with the others, because each prices something different.

This article is for you if that is where you are. We do not explain from scratch what load testing is; other articles do that. Here we deal with the purchase: what actually belongs in the scope of the service, how the timeline runs, what drives the price, what the report must contain and what to put in the order, so that in a few weeks you have an answer for the board rather than a file of charts.

What are you actually buying when you order performance testing as a service?

You are buying an answer to a business question, not a number of virtual users. The distinction sounds trivial, but it explains why quotes differ tenfold. The €2,500 quote sells traffic generation. The €9,500 quote sells a finding on whether the system meets defined criteria and what must change if it does not. The €30,000 quote sells a change in how the team works. Three different products with the same name.

A well-defined performance testing service has five components. The first is the workload model: which operations users perform, in what proportions, with what think time and at what peak. The second is scripts and test data: automated scenarios that reproduce the model, with data varied enough that you are not just testing the cache. The third is test runs with application and infrastructure monitoring. The fourth is analysis: why response times rise, where the bottleneck is, what is cause and what is symptom. The fifth is the report and recommendations, and in better engagements a confirmation run after fixes.

The cheap option usually contains only the third component in a simplified form. The vendor records a few journeys, runs traffic, hands over a response time chart and states that “at 3,000 users the average response time exceeded 2 seconds”. That sentence does not tell you whether the campaign will succeed, because nobody knows whether 3,000 virtual users is a lot or a little compared with real traffic, or why the time went up.

So the first step before comparing quotes is to write down which decision the tests should support. “Do we go into the campaign without scaling the infrastructure?”, “Is the new release slower than the previous one?”, “How much traffic can the system take before we start losing orders?” These are three different engagements, with different scope and different prices. For the methodology itself, see our guide Load testing in 2026: the complete guide from baseline to production.

Which tests do you need: load, stress or soak?

Each test type answers a different question and has a different cost, so name them explicitly in the order. The Grafana k6 documentation (Load test types) organises them in a way that works well in a vendor conversation regardless of the tool.

A smoke test checks at minimal load that the scripts work and the system responds correctly. It lasts seconds or minutes. It answers no business question, but it is a precondition for trusting every other test. If a quote does not mention it, ask how the vendor validates its scripts.

A load test (average-load) measures behaviour at expected traffic, usually for tens of minutes. It is the core pre-campaign test: at the forecast peak, do response times and error rates stay within the criteria?

A stress test goes well above expectations. It answers what happens if the forecast turns out too low: does the system slow down gracefully or fall over, and does it recover when traffic drops? Closely related is the breakpoint test, which raises load gradually to the capacity limit. That one tells you how much headroom you have.

A spike test checks a sudden, short burst, for example after a newsletter goes to the whole list or a TV ad airs. This is often where auto-scaling that reacts too slowly shows up.

A soak test holds typical load for many hours. It finds what an hour-long test will not: memory leaks, growing queues, filling logs, database connections that never return to the pool. It is expensive in calendar time and infrastructure, but for systems that run for weeks without a restart it can be the most important test of all.

Practical advice: do not order the “full package” just because it sounds safe. Before a campaign with a one-day peak you need load, stress and spike. Before migrating a system that runs for months, soak is key. Before a new release, a comparison against the previous version at the same load is often enough. Choosing the tests is the first line where you consciously save money or overpay.

How do you turn a business goal into pass criteria?

Without pass criteria a performance test cannot fail, and so it cannot prove anything. A criterion has three parts: a metric, a threshold and conditions. “The product page must be fast” is not a criterion. “For 95% of requests to the product page, server response time stays under 800 ms at 1,200 requests per minute with an error rate below 0.5%” is a criterion.

Note three things in that sentence. First, a percentile, not an average. Averages hide problems: if 90% of users get a response in 200 ms and 10% wait 8 seconds, the average looks acceptable while one customer in ten leaves. Criteria are usually stated at the 95th or 99th percentile. Second, throughput expressed in business operations, not “users”. The number of virtual users depends on think time in the script, so the same number of VUs can mean completely different traffic. Third, error rate, because a system that answers quickly with an HTTP 503 formally has excellent response times.

Where do the thresholds come from? Ideally from data: logs and analytics from previous peaks, the marketing forecast converted into operations, contractual commitments to your own customers. For the browser layer, Google’s public thresholds are a useful reference: according to web.dev, a good score is LCP up to 2.5 s, INP up to 200 ms and CLS up to 0.1, measured at the 75th percentile of page loads. Load tests mostly measure the server, though, so you have to set API and back-end thresholds yourself, starting from how much of those 2.5 seconds the server may consume.

A good vendor starts with a workshop where you jointly convert the business goal into a workload model and criteria. If a quote contains a ready-made number of users before anyone has asked about your traffic, the vendor is selling a template. And if you cannot give a forecast, do not pretend: say so and order a breakpoint test. The answer will then be “the system sustains X operations per minute”, and the decision whether that is enough stays with you.

What does a typical engagement look like week by week?

A one-off engagement for one application with a handful of key journeys usually takes two to five calendar weeks. The plan below is a typical run, not a standard; we give it so you can ask a vendor where their plan differs and why.

Week 1: goals, workload model, environment. A workshop with you and someone who knows production traffic. Agreeing pass criteria, choosing 3–8 scenarios that account for most traffic or revenue, reviewing the test environment and monitoring, listing test data needs. By the end of the week you should have a short test plan to approve. It is the cheapest week of the engagement and the one that most often decides its value.

Week 2: scripts and data. Writing scripts, parameterisation (different users, products, baskets), correlating dynamic values such as session tokens, preparing test data, a smoke test. Complex authentication, unusual protocols or heavy browser-side logic make this stage grow.

Week 3: runs and analysis. A baseline run, the load test, the stress test, possibly a spike test. The soak test usually runs overnight or at the weekend. Analysis happens in parallel: correlating response times with CPU, memory, connection pools, database queries and queues. The vendor should be in constant contact with your team, because some hypotheses can only be confirmed with access to the configuration.

Week 4: report, fixes, confirmation run. A draft report, a walkthrough, fixes by your team, a repeat run and a final report. Without the confirmation run you do not know whether the fix worked or just moved the bottleneck somewhere else.

What stretches the timeline? Almost never the testing itself. Most often it is an environment that is not ready, test data that cannot be used because of GDPR, no access to monitoring, a firewall blocking traffic from the load generators and nobody available to change configuration during the tests. If the campaign is six weeks away, start by booking an environment window, not by choosing a vendor.

Which tool should you pick: k6, JMeter or Gatling, and is it your call at all?

The tool matters less than quotes suggest, but it affects three things that concern you: the cost of maintaining scripts after the engagement, how easily tests fit into CI/CD and who on your team will be able to read the scripts.

Apache JMeter is a mature open-source tool written in Java. As the project site points out, it works at protocol level, not as a browser, and supports HTTP/HTTPS, SOAP and REST, JDBC, JMS, LDAP and FTP among others. Its graphical test plan editor suits teams where tests are prepared by non-programmers, and systems with protocols other than HTTP. Test plans are stored as XML, which makes code review of changes harder.

Grafana k6 is an open-source tool where scenarios are written in JavaScript (k6 documentation). Scripts are ordinary code: easy to version, review in pull requests and run in a CI pipeline. It fits teams where developers are expected to co-write and maintain the tests.

Gatling lets you write tests in Java or in JavaScript and TypeScript (Gatling documentation), with a community and an enterprise edition. It produces readable HTML reports and is a natural choice for JVM teams.

Then there are commercial tools such as LoadRunner, common in large organisations with existing licences and many protocols. ARDURA’s software testing services page lists JMeter, Gatling and LoadRunner as performance testing tools.

Is it your call? Partly. Do not let a vendor pick the tool only because it suits them, if the scripts are supposed to stay with you. Ask who will maintain the scripts after the engagement, what language your team writes in and whether the tests must run in CI. If the answers are “us”, “TypeScript” and “yes”, k6 or Gatling in JS/TS will be cheaper to maintain. If you are testing a JMS-based messaging system and a database over JDBC, JMeter may be the more sensible choice. What matters most is that the scripts live in a repository you own, not in the vendor’s tooling.

What drives the price of performance testing?

The price of the service is essentially specialist days multiplied by the day rate, plus any load-generation infrastructure and licences. The rate varies less than you might think; the estimate is moved by the number of days. The table below collects the factors that move it and the questions you can use to check them on your side before sending a request for proposal.

Factor	What reduces the days	What increases the days	Check question
Number and complexity of scenarios	3–5 HTTP/REST journeys, simple login	10+ journeys, SSO, multi-step processes, WebSocket, non-HTTP protocols	Which journeys account for 80% of traffic or revenue?
Test types	Load test and comparison with the previous release	Plus stress, spike, breakpoint and a soak lasting hours	Which business question does each test answer?
Environment readiness	Dedicated environment architecturally close to production, ready on day 1	Shared environment, built during the engagement, night-time test windows	Who makes the environment available exclusively, and when?
Test data	Ready, anonymised data at realistic volume	Data must be generated or anonymised, database empty	Does the test database have production-like volume?
Monitoring	APM and infrastructure metrics already in place	No monitoring, it has to be installed	Can you see database query times and pool usage today?
Depth of analysis	Measurement against criteria, no diagnosis	Root-cause diagnosis, profiling, code- and configuration-level recommendations	Do you need to know “whether” or also “why”?
Confirmation run	None, the report ends the engagement	One or two rounds after fixes	Who will implement fixes, and when?
Traffic scale and infrastructure	Traffic from a few machines	Generators in several cloud regions, commercial licences	Must traffic come from multiple locations?
Handover	Scripts in the client’s repository, short walkthrough	CI/CD integration, documentation, team training	Will the tests be run after the engagement?

One conclusion stands out: several of the largest factors sit on your side. Environment, data, monitoring and the availability of people who can make fixes can double the number of days without any change to the test scope. The vendor cannot control them, so a sensible vendor either prices them with a buffer or writes them into the quote as assumptions. The second is better for you, because you can see what you are paying for.

Load-generation infrastructure and licences are often a separate line. Open-source tools carry no licence fee, and the machines that generate traffic are billed at the cloud provider’s prices. Ask for this line to be separated in the quote, with a note on whether you pay it directly or through the vendor and at what mark-up. For spreading the budget across several test cycles a year, see Performance testing budget: how to plan and optimise.

What does it cost in practice, and how can you estimate the range yourself?

We will not give you a single price list, because there is no honest price list for performance testing; there is a rate and a number of days. You can, however, calculate the range yourself before you see the first quote, and that lets you recognise a quote that is unrealistically cheap or inflated.

Start with the rate. In ARDURA’s pricing, a senior QA engineer (test automation, SDET) costs €335–465 per day and an architect or lead €465–595 per day, net, for an 8-hour day, all-inclusive. For comparison, ARDURA’s pricing page puts senior rates at roughly 35–55% below the average German IT freelance rate. Performance testing is rarely done by juniors, because it requires understanding architecture, databases and infrastructure, so take the senior level as your starting point.

Then estimate the days. The ranges below are our estimate for a typical engagement, not a market fact or a price list; the factors in the table above move them.

• Basic scope (3–5 HTTP/REST scenarios, environment and monitoring ready, load and stress test, report without deep diagnosis): about 8–12 days. At the senior rate, roughly €2,700–5,600.
• Typical scope before a major event (5–8 scenarios, data preparation, load, stress, spike and one soak, bottleneck diagnosis, one confirmation run): about 15–25 days, roughly €5,000–11,600.
• Extended scope (several systems and integrations, non-HTTP protocols, an environment to build, several fix rounds, CI/CD integration and team training): 30 days or more. At that point it is more sensible to talk about a specialist embedded in your team than about a one-off engagement.

Back to the three quotes from the introduction. The €2,500 quote is about 5–7 days at a senior rate. That covers recording a few journeys and a test run, but not a workload model, data and diagnosis. It is not a scam, just a different product. The €9,500 quote falls within the typical scope, and it is worth checking whether the assumptions are written down. The €30,000 quote is 65–90 days; it only makes sense if you genuinely want to build a performance engineering practice in your organisation, not answer one question before a campaign.

A final point on comparing: ask for quotes in days, broken down by stage. A lump sum without a breakdown makes comparison and negotiation impossible, and a breakdown immediately shows whether someone has planned one day for analysis or five.

What do you need to prepare on your side to avoid overpaying?

The cheapest days are the ones the vendor does not spend waiting. Most delays and extra charges in performance engagements come from gaps on the buyer’s side, not from vendor mistakes. Before you sign the order, prepare seven things.

A traffic forecast in operations. Not “100,000 users”, but how many product page views, searches, add-to-basket actions and payments per minute at peak. If you have data from a previous peak, go through it with marketing and multiply by the campaign forecast.

An exclusive environment in an agreed window. A test on an environment where someone is simultaneously running regression tests or deploying a new release produces results you will throw away. Book the window, and if the test environment differs from production, write the differences down.

Test data at realistic volume. A database with a thousand products behaves differently from one with a million. If production data contains personal data, prepare the anonymisation in advance; it is a frequent cause of week-long delays under GDPR.

Monitoring. Without application and infrastructure metrics the vendor can only tell you it is slow. With them, they can tell you why. If you have no APM tool, agree before the tests what the vendor will use to collect application metrics (transaction tracing, memory, threads, database query times) and who installs it on the test environment.

Network access. Agree with your administrators which addresses the traffic will come from, and disable or configure rate limits, WAF rules and bot protection that would otherwise block the generators in the first minute.

A person on your side with permissions. Someone who can resize a connection pool, restart a service or add an index during the tests. Without them every hypothesis waits a day.

A decision on who makes the fixes. A confirmation run only makes sense if your team has time for fixes between rounds. Plan that into the sprint before the engagement starts.

What should a performance test report contain?

The report is the only lasting product of the engagement, so agree its contents in the order rather than seeing them for the first time at the end. A good report reads on two levels: the board gets the answer to the business question on page one, the team gets the detail it needs to act.

Executive summary. A one-sentence answer (“at the forecast peak the system meets the criteria for four of five scenarios; payment exceeds the threshold at 70% of forecast”), the three biggest risks and a recommendation: go, go after fixes, or do not go without scaling.

Test conditions. The environment and how it differs from production, application version, workload model, scenarios, data, run durations. Without this section the results cannot be repeated or compared six months later.

Results against criteria. A table: scenario, metric, threshold, result, pass or fail. Percentiles (p50, p95, p99), throughput, error rate broken down by type.

Time-correlated charts. Response times, requests and errors on one timeline with CPU, memory, database connections and queue lengths. This correlation is what separates cause from symptom.

Bottlenecks with a probable cause and priority. Not “the database is slow”, but “query X in the basket service does a full table scan, its time grows linearly with the number of products; adding index Y should remove the problem; high priority because it blocks payment”.

Recommendations with an effort estimate. What to change in code, configuration and infrastructure, in what order and roughly at what cost.

Artefacts. Test scripts, configuration, input data and raw results in your repository. If the vendor keeps the scripts, you pay for writing them again with every future test. If a test goes badly just before launch, see An application performance crisis? Your first 5 steps to regaining control.

What should you ask a vendor before signing?

Seven questions are enough to separate a vendor who sells an answer from a vendor who sells traffic.

How will you build the workload model? A good answer starts with your data: logs, analytics, forecast. A bad answer starts with a number of users.

What pass criteria do you propose, and where do they come from? Look for percentiles, throughput in operations and error rate. If the vendor proposes no criteria, the test cannot fail.

Who does the analysis, and what experience do they have with our stack? Performance testing a Java application on an application server, a serverless cloud system and a PHP application on shared hosting are three different skill sets. Ask to speak to the person who will analyse the results, not just to sales.

How do you make sure the load generator is not the bottleneck? A generator short of CPU or network bandwidth measures its own limits, not your system. The vendor should monitor its own machines too.

What is in the report? Ask for an anonymised sample or a table of contents and compare it with the list in the previous section.

Who owns the scripts, and in what tool will they be written? The answer should be: you do, in your repository, in a tool your team can maintain.

What assumptions does the price rest on, and what happens if they are not met? Good quotes list assumptions (environment ready on day X, data, monitoring access) and explain how waiting days are billed. No assumptions means the risk is hidden in the price or will come back as a change request.

What should go into the contract or order?

One-off performance testing is usually billed in one of two ways: a fixed price for a defined scope, or time and materials with a budget cap. Both are fine, provided the order contains a few specific provisions.

Goal and pass criteria. Write down the business question and the criteria, even if they are refined in week one. That gives “test execution” a measurable meaning.

Test scope and number of rounds. Test types, number of scenarios, maximum soak duration, number of confirmation rounds. Without this, every extra round becomes a negotiation.

Assumptions and buyer obligations. Dates for environment, data and access, a named contact, the test window. An honest clause on how downtime caused on your side is billed protects both parties.

Deliverables and ownership. A report with defined contents, plus scripts, configuration and raw results delivered to your repository. Under most EU jurisdictions, make sure the contract transfers or licenses the rights to scripts and documentation explicitly; in Germany and Austria, where copyright itself cannot be assigned, that means an exclusive, unrestricted right of use. “Handing over the files” alone does not grant rights.

Security and data. Access rules for environments, no personal data without a data processing agreement under GDPR, how data is stored and deleted after the engagement, an NDA. If the test touches production, add a maintenance window, a rollback plan and a person authorised to stop the test.

Acceptance criterion. Acceptance follows delivery of the report and artefacts as scoped, not the system “meeting the criteria”. The vendor is responsible for sound measurement and analysis, not for the performance of your code. That is a fair split, and writing it down avoids a dispute when the result is bad.

When is a one-off test not enough, and what should you do instead?

A one-off test is a snapshot of the system on one day. If the team ships changes every week, a month later that snapshot is out of date. There are three situations where you should consider something beyond another engagement before the next campaign.

Performance regularly degrades between releases. If every major release brings surprises, you need performance tests in the CI/CD pipeline: a short comparison test on every merge to the main branch, a longer test before release and a periodic soak. Scripts from a one-off engagement are a good starting point, as long as they were written in a tool that can run automatically.

The system has contractual SLAs with customers. When response time is written into contracts, pre-launch tests are not enough; you also need continuous production performance monitoring to catch degradation before customers notice. We discuss that difference in “Proactive vs. reactive QA: why continuous performance monitoring pays off”.

Nobody on the team owns performance. Then each engagement will rediscover the same problems. A more sensible solution is often a QA engineer with a performance specialism embedded in the team for a few months, who builds the tests, teaches the team to maintain them and leaves once the practice sticks. At ARDURA this is done through staff augmentation of QA engineers, billed at a day rate, with a one-month minimum and 30 days’ notice under the terms on the pricing page.

The reverse also applies: if the system is small, traffic is predictable and the infrastructure scales by itself, one load test a year and good monitoring may be entirely sufficient. Do not buy a performance engineering programme if your question fits into one engagement.

What are the key takeaways?

Performance testing as a service means buying an answer to a business question. Quotes differ tenfold because they price different products under the same name: traffic generation, measurement against criteria with diagnosis, or building a practice. Before you compare prices, write down which decision the tests are meant to support.

Remember six things. First, choose test types for the question: load for expected traffic, stress and spike for surprises, soak for systems that never stop. Second, state pass criteria as percentiles, throughput in business operations and error rate. Third, a typical engagement takes two to five weeks, and the timeline is stretched mainly by environment, data and monitoring access on your side. Fourth, the price is days times rate; at a senior QA rate of €335–465 per day, a typical 15–25 day engagement is an estimated €5,000–11,600. Fifth, the report must answer the question on page one and identify causes, and the scripts must stay in your repository. Sixth, if performance degrades between releases, build the tests into your CI/CD pipeline instead of buying one-off engagements.

You will find more on testing and quality in our Quality Assurance hub. If you are facing a campaign, a migration or a launch and need an answer to “will the system hold?”, see how we run performance testing as part of our software testing services or book a short scoping call. After the call you will know which tests you need, how many days they will take and what to prepare on your side.