Need testing support? Check our Quality Assurance services.

See also

Cloud computing, undoubtedly one of the most transformative technologies of recent decades, has revolutionized the fundamental way modern organizations design, build, deploy and manage software and entire IT infrastructures. Unparalleled resource flexibility, virtually unlimited scalability, instant access to a wide range of advanced, off-the-shelf services, and an attractive, cost-effective pay-only-for-actual-use (pay-as-you-go) model have led an increasing number of companies, both large corporations and dynamic startups, to move their existing applications and systems to cloud environments or, as is becoming increasingly popular, build them from the ground up in the innovative cloud-native model. Cloud-native applications are specifically designed and optimized to take full advantage of the unique potential and peculiarities of the cloud environment, often relying on paradigms such as containerization (e.g. Docker, Kubernetes), microservices architecture or serverless computing (serverless) functions. This profound technological and architectural transformation brings huge, undeniable business benefits, such as accelerated innovation, increased operational agility and global reach. At the same time, however, it also poses new, often highly complex and unique challenges to the entire quality assurance (QA) process and software testing strategies and techniques. Traditional, long-established testing methods, developed for relatively static, predictable and fully controlled local server (on-premise) environments, often prove insufficient, ineffective or simply inadequate in the dynamic, highly distributed, ephemeral and often highly complex world of modern cloud applications.

“96% of organizations are either using or evaluating Kubernetes, making it the de facto standard for container orchestration.”

— CNCF, CNCF Annual Survey 2023 | Source

At ARDURA Consulting, with extensive, long-standing experience in designing, developing, deploying and testing advanced cloud applications for our clients in various sectors, we understand these specific challenges and nuances very well. Over the years, we have developed and continuously refined specialized, comprehensive testing strategies that allow us to effectively and efficiently ensure the highest quality, reliability, security and performance of cloud-based solutions. Our approach is based on a deep understanding of the unique characteristics of cloud environments, such as the dynamic and ephemeral nature of infrastructure, the need to verify continuous scalability and flexibility, complex security issues in a shared responsibility model, the need to consciously manage the cost of testing processes, and the specifics of testing interactions with a wide range of managed cloud services. We believe that only a holistic, integrated and proactive approach to testing, closely aligned with DevOps processes and using modern tools and techniques, can guarantee the success of cloud-native applications. In this article, we will take a closer look at the key differences and challenges in testing cloud-native applications, and outline the strategies we at ARDURA Consulting use to effectively address them and deliver superior solutions to our clients.

Dynamic cloud infrastructure: New paradigms for test environments

One of the first and most fundamental challenges QA teams face when testing cloud applications is the dynamic, programmable and often ephemeral (short-term) nature of the cloud infrastructure itself. Unlike traditional physical servers and static on-premise environments, cloud resources - such as virtual machines, containers, Kubernetes clusters, databases or network services - can be created, configured, scaled and removed fully automatically, often in a matter of minutes, using specialized tools and approaches referred to as Infrastructure as Code (IaC). Technologies such as Terraform, AWS CloudFormation, Azure Resource Manager (ARM) templates or Bicep allow the entire infrastructure to be defined and managed as code, ensuring reproducibility, versioning and full automation. For testing processes, this means a revolutionary change: test environments are no longer static, long-maintained entities, but can be dynamically, on-demand invoked to run specific sets of tests (e.g., for each new build, each branch of code or each regression cycle) and then automatically deleted when they are completed. Such a model, often referred to as “ephemeral environments,” brings huge benefits in terms of speed, consistency and cost optimization, but at the same time requires QA teams and test engineers to acquire entirely new skills and competencies. They must not only understand the basics of cloud platforms, but also have knowledge of IaC tools, the ability to write scripts that automate the creation and configuration of test environments, and the ability to tightly integrate test processes with continuous integration and continuous delivery (CI/CD) pipelines that manage the entire lifecycle of both the application and its infrastructure. At ARDURA Consulting, our QA specialists work closely, on a day-to-day basis, with experienced DevOps engineers to ensure that testing processes are fully integrated with infrastructure management automation to deliver system quality feedback quickly and reliably. Of course, dynamic environments also bring some challenges, such as managing application state between tests, ensuring test data persistence, or avoiding test instability (flakiness) resulting from environmental issues, but with the right strategies and tools, we can effectively minimize them.

Testing for the cloud: Verifying scalability, flexibility and performancennanother key aspect that must be thoroughly addressed when testing cloud-native applications is the need for thorough verification of their ability to scale and flexibly adapt to changing loads, which is one of the main fundamental advantages and promises of cloud technology. A modern cloud application must be able to smoothly and efficiently handle both small, daily user traffic and sudden, often multiple and hard-to-predict load spikes, such as during intensive marketing campaigns, seasonal sales peaks, or in response to unexpected events. This requires QA teams to conduct advanced, comprehensive performance tests that are not limited to simple load tests, but cover a broad spectrum of scenarios. These include load testing, which verifies the system’s behavior under an expected, typical load; stress testing, which tests the limits of the system’s endurance and its behavior under extreme conditions; soak testing or endurance testing, which checks the system’s stability and performance under prolonged, constant load; spike testing, which analyzes the system’s response to sudden, abrupt changes in load; and, of particular importance in the context of the cloud, dedicated scalability testing. The latter are aimed at precisely verifying the effectiveness and correctness of the automatic scaling (auto-scaling) mechanisms offered by a given cloud provider, both in terms of up-scaling (scale-out) and down-scaling (scale-in). At ARDURA Consulting, we use specialized, often also cloud-based, performance testing platforms and tools such as k6, Gatling, JMeter (often deployed in a distributed configuration in the cloud) or dedicated services offered by the cloud providers themselves (e.g. Azure Load Testing, AWS Distributed Load Testing) to perform these complex tasks. They allow us to realistically generate traffic from different geographic locations, simulate thousands or even millions of virtual users at the same time, and precisely monitor key performance indicators (KPIs) such as response times, throughput, resource utilization or error rates. As a result, we are able to make sure that our clients’ application maintains the required stability, performance and adequate response times under even the most demanding workloads, taking full advantage of the cloud’s flexibility potential. Our methodology includes establishing reliable baseline scenarios, defining realistic load profiles and, where possible, incorporating performance testing as a permanent part of the CI/CD pipeline (so-called continuous performance testing).

Security in the Cloud: ARDURA Consulting’s shared responsibility and comprehensive testing strategies

Also extremely important, and often a source of confusion and potential problems, are the specific challenges of ensuring application and data security in a cloud environment. Although the largest public cloud providers (AWS, Azure, GCP) invest huge resources in securing their global infrastructure (what is referred to as “cloud” security - security of the cloud), the security of the applications themselves, data, operating systems, network configurations and identity management running *i

  • the cloud (so-called “security in the cloud” - security *i
  • the cloud) is always the responsibility of the customer and its software provider. This is known as the Shared Responsibility Model, the precise understanding and consistent adherence to which is absolutely key to ensuring overall security. Security testing in a cloud environment must therefore encompass not only the application itself and its code (which is, of course, essential and implemented through SAST, DAST or penetration testing, among others), but also, to an equally large extent, **the correctness of the configuration and security of the individual cloud services used by the application **. This includes such critical areas as Identity and Access Management (IAM) systems, where we verify the application of the principle of least privilege, the proper configuration of roles and access policies, and the enforcing of strong authentication (e.g. MFA). We also test the configuration of network security groups (security groups) and access control lists (ACLs) to ensure proper network segmentation and protection against unauthorized traffic. We check the configuration of firewalls and Web Application Firewalls (WAFs), verifying their effectiveness in blocking known attack vectors. We also attach great importance to verifying data encryption mechanisms, both at rest, for example for storage services such as S3, EBS or databases, and in transit, by checking the correct configuration of TLS/SSL protocols and certificate management. At ARDURA Consulting, we use specialized Cloud Security Posture Management tools (CSPM) for automatic cloud configuration scanning, such as built-in security centers offered by cloud providers (e.g. AWS Security Hub, Azure Security Center) or third-party solutions that help identify potential misconfigurations, security standard violations and policy non-compliance. We also perform dedicated security testing to verify the system’s resilience to attacks specific to cloud environments, such as attacks on cloud APIs, vulnerabilities in serverless functions, unsecured containers or configuration errors in storage services. All these activities are carried out in strict accordance with the principles of the shared responsibility model and DevSecOps best practices, integrating security testing at every stage of the software development lifecycle.

Optimize testing costs in the cloud: Efficiency without compromising quality

An aspect of testing in a cloud environment that is often overlooked at the planning stage, but extremely important in practice, is the conscious and proactive management of costs generated by testing processes. Ruing extensive, automated regression test suites or, most importantly, advanced performance tests on a large, production scale in a cloud environment can, if not properly controlled, unexpectedly generate very high, often surprising bills for consumed computing resources, storage, data transfer or specialized cloud services. That’s why at ARDURA Consulting, when we design our testing strategies for cloud-native applications, we always do so with ongoing cost optimization in mind, but without sacrificing the required level of quality and test coverage. We use a number of proven techniques and approaches to do so. We try, where possible and reasonable, to use smaller but still representative and production-reflective test environments, rather than always working on full-sized copies of the production environment. We actively use techniques such as “mocking” or “stubbing” (service virtualization) of expensive or hard-to-reach external services and some cloud services, especially at earlier stages of testing (e.g., component or integration testing). We also place an extremely high priority on fully automating the process of removing no longer used test resources and entire environments, using IaC scripts and appropriate steps in CI/CD pipelines, as well as implementing mechanisms for scheduled shutdowns of development and test environments during off-hours. We also consistently follow the practice of tagging all cloud resources used for testing, which allows us to accurately track costs, allocate them to specific test activities or teams, and identify areas with the greatest potential for optimization. We continuously monitor the costs generated by testing processes, using tools provided by cloud platforms (e.g. AWS Cost Explorer, Azure Cost Management), set appropriate budgets and alerts, and constantly look for ways to further rationalize them, but always in such a way that it does not come at the expense of reducing the quality, security or reliability of the tests performed. Our goal is to find the optimal balance between testing comprehensiveness and cost-effectiveness.

Managed Services Interaction Testing: Contracts, Resilience and Advanced Chaos Engineering

Cloud-native applications very often, even by definition, rely on extensive use of a variety of off-the-shelf managed services provided directly by the cloud platform. These include, for example, managed databases (such as AWS RDS, Azure SQL Database, Google Cloud SQL), message queues (e.g. AWS SQS, Azure Service Bus), object storage services (AWS S3, Azure Blob Storage), streaming data processing platforms, or the increasingly popular serverless functions (AWS Lambda, Azure Functions, Google Cloud Functions). Testing applications that rely so heavily on these external, managed components requires a specific, dedicated approach. The fundamental principle here is that we do not test the internal functionality of the cloud service itself - as we assume that the cloud provider (e.g. AWS, Microsoft, Google) bears full responsibility for its quality, reliability and security, which is usually specified in the relevant SLAs. Our task as a QA team, on the other hand, is to thoroughly and comprehensively test the interaction of our application with that cloud service. We need to verify that our application correctly communicates with the service, that it correctly interprets the data and error codes it returns, and, crucially, that it can elegantly and securely handle any errors, delays or even temporary unavailability of this external service, minimizing the negative impact on the end user. At ARDURA Consulting, we use advanced techniques for this purpose, such as Contract Testing, for example, using tools such as Pact. It allows us to verify the compatibility of interfaces (API contracts) between our application and cloud services (or other microservices), even if these components evolve and are deployed independently. We also implement comprehensive system resilience testing strategies (Resilience Testing), which include controlled simulation of various types of external service failures (e.g., by using fault injection tools - fault injection, manipulation of network traffic, or creation of special service mock-ups to simulate their unavailability or incorrect operation) and thoroughly checking how our application handles them, whether mechanisms such as retries, circuit breakers or fallback strategies work correctly. In more advanced scenarios, for mission-critical systems, we also use the principles and techniques of Chaos Engineering, popularized by Netflix, among others. This involves the deliberate, controlled introduction of various types of failures and disruptions into the test (and sometimes even production, to a very limited extent) environment to proactively identify potential system weaknesses, bottlenecks and unforeseen points of failure before they cause real problems. Tools such as Chaos Monkey, Gremlin and AWS Fault Injection Simulator help us realize such experiments, leading to building much more resilient and reliable cloud systems.

In summary, effective and comprehensive testing of cloud applications, especially those designed in a cloud-native model, requires QA teams and the IT organization as a whole to undergo a significant paradigm shift, acquire new specialized skills, implement cutting-edge tools and adopt innovative testing strategies that go far beyond traditional, well-known approaches. At ARDURA Consulting, we are acutely aware of these growing challenges and are constantly investing in developing the competencies of our QA professionals, as well as in researching and implementing the latest, most effective techniques and tools for testing cloud applications. Our comprehensive approach focuses on tight integration of testing processes with modern DevOps and CI/CD practices, intelligent test automation at all levels where it brings the most measurable value, strategic and conscious management of costs generated by testing processes, and thorough, in-depth verification of all key non-functional aspects of the application. These include, first and foremost, scalability, performance, security, availability and fault tolerance - everything that, in today’s dynamic and extremely demanding technology world, determines the real success and value of an application running in a cloud computing environment. Our goal is to provide solutions that not only meet functional expectations, but are also robust, secure and ready for future challenges.

Are you planning to migrate your applications to the cloud or build new solutions in a cloud-native model? Want to make sure your quality assurance process can keep up with the specifics and challenges of the cloud environment? Contact the QA experts at ARDURA Consulting. We will share our experience in testing cloud applications and help you develop a QA strategy that will guarantee the success of your cloud projects.

Feel free to contact us