Albert Gareev

How to Avoid Test Flow Hard-Coding And Dramatically Reduce Code Maintenance Cost in Functional Test Automation

Bookmark and Share Bookmark and Share

Many times looking at the busy screen together with Business Analyst I used to discuss dozens of manual Test Cases they execute to cover the functionalities of this single Window or Dialog. Business Logic seemed to be quite similar but different enough to produce a multiple Test Logic flows

So the dilemma I had was about either to follow manual test requirements and create multiple scripts setting maintenance trap or to limit number of Test Cases to automate but reduce testing coverage as a result. Both ways were still accepted as reasonable by QA Manager but reducing coverage was really against my QA nature.

Looking at the screen alone I was storming the simple concept: the Application still has just a single screen providing all the functionalities. If one program can do it why another program requires multiple implementations? While being different on steps to perform the all Test Cases have the same entry point and major PASS/FAIL condition.

The solution I came to I’m going to demonstrate using MS Windows Calculator as the example.

 

 

Manual Test Cases

1. A + B = C
Test Data and Steps.
Input. A = 2; B = 3.
Expected result. C = 5.

2. A – B = C
Test Data and Steps.
Input. A = 2; B = 3.
Expected result. C = -1.

3. A * B = C
Test Data and Steps.
Input. A = 2; B = 3.
Expected result. C = 6.

4. A / B = C
Test Data and Steps.
Input. A = 6; B = 3.
Expected result. C = 2.

Recorded Scripts (Hard-coded Data and Logic)

TC1.
Click “2”;Click “+”;Click “3”;Click “=”.
Read value C.
Verify C is equal 5.

TC2.
Click “2”;Click “-”;Click “3”;Click “=”.
Read value C.
Verify C is equal -1.

TC3.
Click “2”;Click “*”;Click “3”;Click “=”.
Read value C.
Verify C is equal 6.

TC4.
Click “6”;Click “/”;Click “3”;Click “=”.
Read value C.
Verify C is equal 2.

Parameterized Scripts (Parameterized Data / Hard-coded Logic)

Reusable Function

1: Input Number. Description: Clicks on number buttons in accordance to the number passed-in. No return value.

Reusable Function 2: Compare Number.

Description: Compare 2 passed-in numbers. Return TRUE if equal FALSE otherwise.

Scripts

TC1.
Input Number A;
Click “+”;
Input Number B;
Click “=”.
Read value as C-Actual;
Compare Number C-Actual, C-Expected.

TC2.
Input Number A;
Click “-”;
Input Number B;
Click “=”;
Read value as C-Actual;
Compare Number C-Actual, C-Expected.

TC3.
Input Number A;
Click “*”;
Input Number B;
Click “=”;
Read value as C-Actual;
Compare Number C-Actual, C-Expected.

TC4.
Input Number A;
Click “/”;
Input Number B;
Click “=”;
Read value as C-Actual;
Compare Number C-Actual, C-Expected

Data Table

Test Case A B C-Expected
TC1 2 3 5
TC2 2 3 -1
TC3 2 3 6
TC4 6 3 2

Analysis

Parameterization of the values obviously extends Test Coverage within the same Test Case group. Whether it’s 2 + 3 or 200 + 300 same script would cover all the combination provided.

However the logic is still hard-coded as a sequence of steps to perform. Code duplication rate for the scripts exceeds 75%.

Data-Driven Script (Data as Input and Test Flow control)

Reusable Functions

Reusable Function 3: Input Operator.

Description: Clicks on the operator button in accordance to the operator passed-in. Skips if the argument is empty. No return value.

Script

Input Number A;
Input Operator O1; Input Operator O2; Input Operator O3; Input Operator O4;
Input Number B;
Click “=”.
Read value as C-Actual;
Compare Number C-Actual, C-Expected.

Data Table

Test Case A O1 O2 O3 O4 B C-Expected
TC1
2
+
3
5
TC2
2
-
-1
3
-1

TC3

2

 

 

*

 

3

6

TC4

6

 

 

 

/

3

2

Analysis

Four separate scripts were converted to a single one.

Amount of the code was decreased to 37.5 % of the initial one without losing the initial coverage.

Manual Test Cases (Extended Coverage)

Maybe it was initially planned or due to a Production incident Test Coverage was requested to be extended with the following Test Cases.

A1 + A2 + A3 + … + An = C

A1 – B2 + A3 – B4 + … + An – … – Bn = C

Iterative Data-Driven Script

Reusable Function

Reusable Function 4: Get Operation

Description: Based on the parameter passed-in (Operation Name) returns the following record:

O1 O2 O3 04 B

Script

Input Number A0;
Begin loop through comma-separated list of Operations
Get Operation;
Input Operator O1; Input Operator O2;
Input Operator O3; Input Operator O4;
Input Number B;
End Loop If no more Operations in the list
Click “=”.
Read value as C-Actual;
Compare Number C-Actual, C-Expected.


Primary Data Table

Test Case A0 Operations C-Expected

TC1.1

2

ADD.01

5

TC1.2

2

ADD.02

3

TC2.1

2

SUB.01

-1

TC2.2

2

SUB.02

-3

TC3.1

2

MUL.01

6

TC3.2

2

MUL.02

8

TC4.1

6

DIV.01

2

TC4.2

6

DIV.02

1

TC5.1

2

ADD.01, ADD.02

6

TC6.1

2

ADD.01, SUB.01

2

TC7.1

2

ADD.01, ADD.02,
SUB.02

1

TC8.1

2

ADD.01, MUL.01,
SUB.01

3

TC9.1

2

MUL.01,SUB.01,
ADD.02

4

 

Secondary Data Table

Operations

O1

O2

O3

O4

B

ADD.01

+

 

 

 

3

ADD.02

+

 

 

 

1

SUB.01

 

-

 

 

3

SUB.02

 

-

 

 

5

MUL.01

 

 

*

 

3

MUL.02

 

 

*

 

4

DIV.01

 

 

 

/

3

DIV.02

 

 

 

/

6

Analysis

Single script covers the required Addition/Subtraction Test Cases and provides coverage for the all possible combinations of Addition/Subtraction/Multiplication/Division.

3 extra lines of the code were added to the script.

Development of the additional reusable function was required.

Within the same type/class of Test Cases Test Logic creation/modification does not require code change and could be performed by any user with basic Excel skills.

Conclusion

In fact, what single Business Program does could be covered with the single Testing Program while manual testing requires a set of Test Cases. Utilizing a simple trick “skip step if no data” testing script produces test flow combinations covering different manual testing sequences. There are still mandatory prerequisites: common GUI, same entry point, same major PASS/FAIL conditions. The code becomes extremely compact and that saves a lot of time on the maintenance phase. This approach also reduces data maintenance efforts as tester may leave empty any optional inputs, as well as reuse secondary data rows. The coverage is getting extended even beyond the required level.

P.S. The real Business Window I had to cover contained about 50 Input fields on 3 tabs. With 2 independent loops, same fields could be populated undefined (from 1 to 100) number of times. Submitted within a loop data were displayed on the on-screen table as records. The Application responded to an input dynamically showing/hiding and enabling/disabling co-dependant Input and Message fields, as well as changing default values and contents of dropdown-lists.

The total number of Manual Test Cases exceeding 100 was successfully covered with a single Iterative Data-Driven script.

.

About the author

Albert Gareev holds Master’s degree in IT, and has been working for over 12 years in Software Development and Quality Assurance. Passionate for QA Automation he successfully accomplished multiple large scale projects in the past 5 years, implementing Test Architectures with WinRunner and Quick Test Pro. He is currently working in a technical leading role coordinating in-house and off-shore automation projects while staying a hands-on and innovative professional.

Review Comments

Praveen Udupa said,
May 11, 2009, 7:38am
 
Well, superb explanation. Started with basic stuffs but landed at the professional level.
Covered the tough topics in a decent and simple way. I will be expecting more and more articles in future also...
 

Durga Prasad said,
May 12, 2009, 3:28am
 
very nice concept... expecting more articles on this(related to automation)
 

Wayne Richards said,
May 14, 2009, 11:56am
 
As a software QA professional, I realize the premise you define. However, given the calculator functions (as a QA professional), I can not condone the obvious. Other math functions visible and available (i.e. Memory, deletion, square root. percentages, etc.) are ignored.
Again, the pragmatist in me suggests that without specific requirements stating that "only those basic math functions will be written", I have to test all functions. A never-ending dillemma for QA professionals - yes. A more "reasonable" prerequisite (requirements)from stakeholders prior to your hard work - the "real timesaver". Pardon my old school, fundamentalist mindset, but I was taught (and request) stakeholders define their use before writiing one piece of code.
I agree with your approach, but I can readily see your effort (increased 3 - 4 times easily) as you "started your work" without inquiring the most burning, critical path issue that will save you (and the project) time and money. In today's business, the QA resource must be proactive and gain all the valid information (from stakeholders, Users, etc.) available - that is where the real time is saved.
PS - Please read (or take his course) Software Testing and Quality Assurance by Ross Collard - version 3.2 UC Berkley.
 

deepak said,
May 18, 2009, 1:41pm
 
plz send me interview question for testing
 

 

Comments:
Name:
E-mail:
Enter the text shown :
 
 

 


Return to Weekly Article