Non-functional Testing - Load Testing

Non-Functional Testing

Functional Testing: verifies the operational execution of a software.

• unit, integration, and system testing
• typically starts before non-functional testing

Non-functional Testing: tests aspects of the software other than its functionality

• making usre interests of the end-users are respected
• vital to add market value to the produt
• is a blackbox testing
• must be measurable (no place for subjective characterizations)
• can be as important as functional requirements.
• You can also format them in the same "user story" cards

There are many types:

Performance Testing determines how a system performs in terms of responsiveness and stability uner a particular workload/condition.

• This includes load test, stress, spike, soak, and configuration testing.

Recovery Testing asks can the application recover from a crash or a failure? Security Testing asks how secure is the application? Usability Testing asks is the user interface of the application intuitive and user-friendly enough? Accessibility Testing asks can disabled people use the application too? Internationalization/localization Testing asks can people with different langauges and regional peculiarities use the application? Documentation Testing aer documents/user manual provided with the application easy to understand?

Load Testing

"Modeling the expected usage of a software by simulating multiple users accessing the software concurrently"

Load Testing asks how does the application behave when too many users access it concurrently?

• Goal: improve performance bottlenecks and ensure smooth futioning
• Examples: transfer many documents at once to a printer, transfer multipel files to and from disk, test web portals with thousands on concurrent users.
• According to a survey, 75% of users said if a site crashed or it was slow, they would leave the site. 50% of users said they would shop else where

Client is the application that runs on the end-user computer that sends "request" to the server. E.g., the browser

The server is the application that receives requests from the clients and contains the logic to send the appropriate "response" back to the client. The server usually has an API and often includes a database (which will persistently store the data of the application).

How to load test

Load testing typically involves simulating the load.

1. create a pertinent scenario
2. script the scenario as a test
3. creation of virtual user
4. having (virtual) user replay the test
5. analyze test results

Load test metrics:

• response metrics:
  • average response time: mean of every roundtrip request/response cycle
  • peak response time (PRS): longest response time (0.1s is "instant", 1.0s is acceptable, 5+s is "user will avoid the site")
  • error rate: percentage of probelm requests compared to all requests
• volume metrics:
  • concurrent users: number of (virtual) users active at any point of time
  • request per seconds (RPS): number of requests being sent to the target server in 1 second
  • throughput: meausrement of bandwidth consumed during the test (kbps or num request/time)

Tools for load testing:

• commercial: loadninja, loadview, stresStimulus, etc
• open source: apache jmeter, locust, gatling, etc.