Functional or Non-functional this is the question

As a developers we want all the time to write new code and new functionality.

Even while we’re doing tests and writing unit tests or component tests we’re always think positive. Thinking how the flow should look like and imitate this activity.

This is OK for 80% of the flows however the evil is in the details.

While going to ensure app especially SaaS app for production readiness you should consider negative scenarios.

Negative scenarios split in minimum to two sections:

  1. Negative functional scenarios
  2. Non functional scenarios

Non functional scenarios and tests

These type of tests contains the following:

  1. Requirement for non functional readiness –
    Expected load, stress scenarios, unexpected infrastructure and application behavior.
  2. Scalability tests
  3. Availability tests

While understanding the needs and the extreme use cases that the system should function under it’s much easier to write testing scenarios.

Imitating such extreme use cases might be harder to implement and in most cases it’s contain relatively large hardware and human resources allocation.

For such tests there are bunch of tools, free and commercial that can help you imitate such load.

One of the famous tools is JMETER 

JMETER can imitate APIs calls and can sample web activity easily.

To simulate High availability and fail-over scenarios you’ll need to use your own scripts that will simulate proactive crashes of application and OS.

Another unique approach that can help you a log it entering hooks into your code in such a way you could fine-grain the tests to function and method level.

To enable this you can use Aspect Oriented Programming in general and AspectJ to implement it in Java.

Another important action here is to measure the system from resources perspective during the tests and also to test functional behavior and auditing of data correction along the tests.

As rule of thumb such tests might take longer to run and manage since you would like to simulate stability activity of the system.

In extreme cases such test can take 14 days to run and to get fair results.

In agile and CI methodology on mature software such tests should run continuously and enable test of smooth upgrade as well.

 

We’ll discuss upgrade procedures and best practices in one of our upcoming posts.

Enjoy your week,

Moshe

Posted in Uncategorized.

Leave a Reply

Your email address will not be published. Required fields are marked *