Continuing the blog series, Foundational Data Warehouse, we will be taking a closer look into the next foundational building block which is the Testing phase. The primary objective of this phase is to ensure the product meets the Business Requirements as defined during the Analysis Phase. You should have completed your Unit testing during the Design phase. This includes field level testing and the program level testing (insert, update, deletes., error processing). Once the basics are covered you are ready to begin the Testing phase.
To enter the Testing phase coding should be completed and the users interface frozen. Defect tracking is a crucial activity that helps track problems as well as determine the quality of the product. The status and ownership of all defects should be tracked. Frequent meetings to discuss the defects amongst the team members will be key to the success of this phase. The following is a chronological list that define the order in which the testing should be done. Don’t get to hung up on the distinctions between the different type of test. You really need to communicate amongst the team what the focus of the test should be and the intent and end result of each of the different types of test. Again, defect tracking and frequent meetings are key to success in this phase.
Integration Testing – This type of testing generally comes after Unit testing. Integration testing focuses on validating that multiple subcomponents of the system work correctly together (integrate together). In many cases there will be multiple developers working on different subcomponents within a project the main object is that the integrated components are functioning properly and the as a group.
Regression Testing – Tests the Foundation to ensure recently introduced changes have not had a negative impact on the existing systems. The focus of Regression testing is to have an automated bucket of test-cases that encompass the entire foundation to ensure you did not impact existing programs or data flow.
System Test – Focuses on the testing that simulate an end-to-end scenario of how the customer will use the system being built. This is where you want to focus on specific business scenarios to ensure the behavior of the application meets the expected requirements. This is my favorite type of testing, I like to think of it as trying to break the system, you should be focusing on both functional and non-functional requirements. You need to build detailed test cases that encompass any functionally that could have been missed.
Performance Test – This type of testing focuses on the system robustness by stressing the system with high volumes of data. It also focuses on speed and stability, helping to uncover bottlenecks and establish a baseline for future batch patterns before moving to production. If you’re lucky your company will have a separate database that simulates production that can be used for performance testing.
User Acceptance Test (UAT) – This type of testing is generally completed by the end-users to help ensure the requirements are being fulfilled. UAT should be performed with production like data (hopefully in performance environment) and focus on end-users performing scenarios that will be required during their normal daily usage. This will help ensure the application performance meets end-users needs. The goal is that by the end of UAT a representative of the end-users will formally accept the application and sign off on approval.
When you come out of the Testing phase you should have little to no known bugs and feel confident that the product meets all the end-user requirements, doesn’t impact existing systems, and the performs is acceptably.
Integration Testing Complete: The BI/ETL Developers generally creates these document. This is the start of incorporating the dependencies of the newly design system and validating that the data flows correctly once you have integrated them together as a whole.
Regression Testing Complete: The Solution Lead or someone who has a good understanding of the current batch processing will create the regression testing documentation. Try to make these reusable over the releases so that you will only have to add the respective modifications to the regression testing. By the end of this phase you should be able to provide a diagram of the data movement strategy and how the new/modified objects fit into the current batch process. Also, you should have automated test cases setup that continue to monitor the entire batch process and ensures that the newly added programs didn’t impact any existing objects.
System Test Cases Complete: This document is usually created by the Business Analyst or Solution Lead that are familiar with the business processes. In many cases you may have a script that runs daily to ensure the expected results are continually being met and will send out alerts of any test case failures.
Performance Testing Complete: The Solution Lead generally creates these document focusing on scalability and stress testing the systems. Be sure to document any indexes that need to be added to help with performance to help ensure that any modifications don’t impact other systems. This should also include documentation that help monitor batch cycle, current processes, etc.
User Acceptance Test Plan Complete: This document is usually created as a partnership between the end-user and the Business Analyst or some other Lead that has a good understanding of the agreed upon requirements. This is generally the businesses first look at the product you will be producing and need to make sure to have the business test thoroughly enough that they feel confident in the product they will be receiving. Test cases tend to be non-technical. By the end of UAT you should have the end-users sign off of the finished product that will be deployed to production.
In conclusion, the testing phase is a very important part of the project lifecycle. There should be a lot of communication taking place amongst the team members. By the end of this phase the end-user should have a clear vision of what is being delivered and feel confident in the end result. You should be ready for code freeze and feel confident moving into the next phase leading into my next blog, Implementation and Warranty.