QAGuild

User Acceptance Testing (UAT) is a very important phase of the software testing cycle. It ensures that the system requirements meet the business needs. It also allows for any issues that are not errors to be resolved before the system goes live. The end users are able to preview how the system will affect the work processes and take measures to accommodate the system. Successful UAT requires good understanding of
1) System Functionality
2) Business Process and Procedures.

UAT is conducted to ensure that a new system works as expected or any changes to an existing are fully functional and do not affect existing functionality and business processes and procedures.

It is important that testing is undertaken in a separate environment to keep live processing unaffected and that the test environment replicates the live environment

Real processing scenarios can then be used on this test environment to ensure that the new system changes work in the way they are supposed to and all other areas of the system remain unaffected.

UAT has many benefits. Firstly, it helps to identify and resolve errors before they reach the live environment and checks that requirements meet business needs. It provides hands on experience of how changes affect work processes. It allows time for any issues (not errors) to be ironed out. The amount and type of training needed for the end users can be identified during this phase and provides the business resources to gain a bigger picture and become business experts on that change.

The Test Matrix is a mechanism for ensuring that every variable is covered without duplication. Normally this will take the form of a spreadsheet where all the variables will be listed across the top and for each scenario listed down the side tick where appropriate.

The ‘Scenario’ is a brief description of what the test is about. For example the scenario may be ‘Entering details of an Insurance claim’.

The test plan is a master document outlining the requirements, activities, resources, documentation and schedules to be completed. Some form of test plan should be developed prior to the User Acceptance Testing. The key reasons for developing test plans are:
Preparation: To assure that all reasonable aspects of running a test have been considered.
Communication and Training: To train those who need to assist with the test.
Effectiveness: To provide a mechanism for outlining test needs, limitations (listing assumptions), and justification for purposes of setting expectations, acquiring resources, investigating unexpected results, assuring normalcy and effectiveness.
Legal and Regulatory Prudence: To enable replication and protection of discoveries made, to mitigate potential litigation costs from use of those discoveries, and to help provide evidence to show regulatory bodies of efficacy.
Scripts

The ‘Test Scripts’ detail all the steps that need to be followed in order to satisfy the scenario. It will also detail the expected outcomes.

The data for each ‘Scenario’ is simply a list of all the variables needed to satisfy the test, e.g. Hospital, Specialist, condition, treatment, dates, place of service etc. It is important that the data is accurate so that we can document the expected outcome of the test. Once we have reached the data preparation stage we go back to the test matrix to review what has been completed. If there are any additions or amendments we start round the cycle again.

This is performed by IT with support from the business. It is never as in depth as UAT but covers areas of change where any errors can be fixed before the changes reaches UAT. This phase tests the main system functionality not business processes. Sign off is obtained prior to going into UAT

Performed within the business area with the assistance and guidance from the IT department. It is an opportunity to perform in depth testing of all changes. Tests covered in test scripts/matrixes are executed and these are normally linked to end-to-end business processes. This phase will ensure that the system is fit for day-to-day use.

This test type is performed as part of UAT and tests the areas not involved in the changes to ensure that these remain unaffected by the changes. The key areas of functionality are tested. Test packs are normally created and can be re-used for future tests.

Usually done by IT with support from the business. Recreates ‘real life’ user volumes. Establishes system response times

Can be done by both IT and business testers. Involves testing two changes in different test environments. Can create re-work when both are merged into one system, as they will require regression tests to ensure they do not adversely affect one another.

Done by both IT and Testers. Checks that all linked areas of a system work as expected. Done throughout testing and finally on the implementation weekend

A defect is where the actual result does not match the expected results. Defects are also referred to as Faults or Bugs. Defects can be caused by a variety of factors - these include: IT/Programming errors, Business Information/Missed Data, Business Information/Management Information, Business Process/Procedure, Usability, etc.Test Analysts need to provide as much detail as possible when an error is found to ensure that the Development team has all the relevant facts when investigating the error.Business process/procedure and usability will not tend to be errors but may become training or a business issue. Decisions on these will require business input and all faults will be logged.

Defects need a severity level allocated to them to indicate how urgently they require fixing and this is usually based on the overall impact that the error has. Severity Levels are:

1 = Critical. Large system component or area of functionality is inoperable. Testing cannot continue. Cannot go live with an error of this type.

2 = Major. Medium system component or area of functionality is inoperable. Testing can continue. Should not go live with fault of this type.

3 = Minor. Minor system component or area of functionality is inoperable. Workaround available/testing can continue. Could go live with this type.

4 = Cosmetic. Does not affect any area of functionality. A ‘nice to have’ fix rather than a ‘must have’.

A Defect Form is completed for each error found. All raised errors are held centrally on an error log. The UAT Coordinator maintains the error log. The error forms get sent to the Development team to address.

When the defect form is returned from Development indicating that the fault is fixed, a re-test will need to be performed. Re-testing involves replicating the test scenario that originally caused the defect and checking that it now produces the expected result. Also testing any areas around the fix to ensure that previously correct functions remain unaffected. The original defect may have caused an area of functionality to be inaccessible. When the fix has been successfully tested it will open up the opportunity to start testing those areas. If the defect fails the re-test then the defect form will need to be returned and the defect log updated. If the defect passes the re-test, then the defect form can be filed and the defect log updated.

Action points when there is a change to process When a testing issue becomes a requirement to change a process, the following steps would need to be taken:
1) Obtain Business input
2) Business write new procedure
3) Procedure is verified by the business owner.

All documentation/plans can be subject to change. At any stage of the test phase, adjustments can be made and the new plan circulated.

• Used as a guide for how UAT activity fits into the overall project


• Displays project tasks in an easy to view timeline


• Each Project/change will have its own plan


• Created and maintained by the UAT Coordinator

• Displays project tasks and responsibilities alongside a timeline


• Each Project/change will have own plan


• Created/maintained by the UAT Coordinator

• Lists responsibilities and tests against each day


• Maintained by Test team


• Batches are included on plan


• Data refreshes are plotted


• Created by UAT Coordinator

• Recording passes and fails on documentation supplied


• Sharing business knowledge within the testing team


• Change business champion on return to business team


• Executing Re-tests

• Assisting with production of test matrixes/scripts


• Working from test scripts to a low level


• Communicating all errors to UAT Coordinator


• Process and Procedure validation


• Participating in team meetings


• Communicating any business functions not covered in tests scripts


• Helping to identify data required for tests

• Coordinate UAT on behalf of the business


• Establish links between systems and risks involved


• Manage teams of permanent and seconded testers


• Planning Resources


• Training of testers


• Monitor day-to-day testing


• Liaise with other business areas


• Facilitate update meetings


• Monitoring of the logged errors


• Provide feedback to testers

• Puts together the initial requirements of change


• Monitors progress of project


• Co-ordinates whole of project


• Involved throughout project life cycle

• Responsible for identifying and resolving business impacts/training etc


• Makes business decisions


• Ensures project products / benefits are delivered


• Approves and signs off IT and business requirements

• Putting the change into the test environment


• Fixing any faults raised by testing


• Coding the system change


• Liaising with test team


• Batch running


• Performance testing

• Liaise with IT throughout project/change


• Support business team during testing


• Supply us with new system/change