Quality Assurance and Quality Control
Being a Test Professionals we should know about QA and QC. What we are doing is actually Quality Control related stuff and what people involved in bringing up CMMI, ISO standards are Quality Assurance. Quality Assurance is more likely related to maintain the process in any given Project or Programme. Quality Control is nothing but giving the right product by verifying the requirements are covered and working as expected.
We know there are multiple levels of testing methods are spoken in the testing world and we have the methodology by which we need to execute them like System Test, Integration Test etc., and Methodology like Water fall, Iterative etc.,
Let me explain what I know and aware of Quality Assurance:
There 3 different roles who will be responsible for assuring the process in any projects.
- PQA – Project Quality Analyst
- CC – Configuration Controller
- DPA – Defect Prevention Analyst
Project Quality Analyst – PQA role
A person who involved in this role needs to know the pre-defined industry standards of that Organization.
PQA’s Roles and Responsibilities
- Documents Naming Convention met as per the Industry Standard
- Names of who prepared , reviewed , approved the deliverables
- Reviews happened across all the Customer facing documents
- Review defects are found, fixed, verified and captured for Metrics
- Checking whether all the deliverables are kept in a Common place where the stake holders can access
- Checking all the necessary deliverables are prepared by the project team
- Checking the actual project delivery date and the date mentioned in the documents are same
- Checking the Stake Holders names, document owner names are mentioned correctly in all customer deliverables
- Differentiating the Customer facing deliverables and Internal audit specific deliverables are as per the industry standards
- Verifying the Entry Criteria and Exit Criteria of any Levels in SDLC are met and collecting the proofs for the same
- PQA’s will be involved in all levels of SDLC
Business Analyst Teams will have separate sets of Deliverables like Business Requirement documents, Business Understanding Documents, Requirement Traceability documents etc.,
- Development teams will have separate sets of Deliverables like High Level Design, Low Level Design , Functional Specifications etc.,
- Testing teams will have separate sets of documents like Test Plans, Test Conditions
The PQA should validate all the documents that supposed to be delivered to the Clients and maintain for internal audits
CC – Configuration Controller
Configuration Controller who controls the versions and the placement of the documents in tools like VSS – Microsoft Virtual Source Safe or Documentum etc.,
Configuration Controller Roles and Responsibilities
- CC’s are responsible of Creating the folder structures in VSS or Documentum
Like, in any Projects the following folders will be created to maintain the projects deliverables
- Project Kick off
- Minutes of Meeting
- Review Log
1.1. High Level design
1.2. Low Level Design
1.3. Issue Log
1.1 Unit Testing
1.2 System Testing
1.3 System Integration Testing
1.4 User Acceptance Testing
CC’s will have the Admin rights to Grant and Revoke access to folders.
Developers should not have access to the folders related to Testing and vice versa
- CC’s will maintain the Check in and Check out of the documents that goes into VSS
- CC’s will maintain the relevant documents are kept in corresponding folders in VSS
DPA – Defect Prevention Analyst
Defect Prevent Analysts will maintain the defects across the SDLC. For any raised defects the work flow should be maintained. Proper comments for those defects should be given when they are created. All high severity defects should be fixed from one Phase to next phase to being with.
As testers when we raise defects we need to concentrate on applying Defect Cause and Defect Type in any Defect Management tool. This will help DPA’s to classify the defects and comes up prevention tips.
Defect Cause – What is the root cause of the defect that is,
- Is the defect caused because of the Upstream Source Data or Test Data
- Is the defect caused because of Incomplete of Missing requirements
- Is the defect caused because the Inconsistent Requirement
- Is the defect caused because of the Code discrepancy
- If you find any anomalies in any Documents then raise the defects to Artefacts
- If any of your defects leads to Changes in the Requirement then please raise them as Change Request – CR can be on the Actual Business Requirement or on Design Changes.
Defect Type – Classifications of the Defects that is,
- Is the defect related to Data Error
- Is the defect related to Application Code
- Is the defect related to Work Request
- Is the defect related to Change Request
- Is the defect related to Deployment
- Is the defect related to Specification
- Is the defect related to Artefact
- Is the defect related to Production
- Is the defect related to Process
- Is the defect related to Environment
- Is the defect related to Specification
- Is the defect related to Requirements
- Is the defect related to Reviews
DPA’s most prominent work is to prepare the CAPA – “Corrective Analysis and Preventive Actions”
DPA roles and Responsibilities
- DPA’s will collect the Metrics related to Defects in a periodic manner – Weekly, Monthly or Ad-hoc
- DPA’s will collect the defects by Defect classifications like in a given period of time how many defects are raised in Reviews, Code, Requirement Change and collects the cause of the defects
- Then using the metrics that was retrieved from any Defect Management tools, they design the Fish Bone diagram by filling the necessary details
- Using the Statistical tools like Minitab, they calculate the Defect Density of the defects from each phase
- Then they will create the Prevention actions on the areas where the Defect Density is above to the threshold limits.
Suppose if your Organization has the Defect Density threshold limit as 0.5 and your Defects under Review defect type is more than 0.5 then they will ask the Reviewers to do their work better to minimize the review defects at any levels of SDLC.
Cheers – Asik
HP Quality Center is quality management software offered from the HP Software Division of Hewlett-Packard with many capabilities acquired from Mercury Interactive Corporation. HP Quality Center offers software quality assurance, including requirements management, test management and business process testing for IT and application environments. HP Quality Center is a component of the HP Application Life cycle Management software solution set. (Taken from Wiki)
I have been working in QC right from version 9.0 to 11.0. There are so many key functionalities added to the new version.
I hope most of them worked in Quality Centre as a Management Tool. Here I will explain for those who want to learn QC and its modules.
QC is a web tool, user can access from anywhere but only in INTRANET (specific to your company we). You cannot access outside of your company. Projects will be created and by QC Admins and the access to the projects will be given by QC Admins as per your Role. Quality Center has so many functionalities and a user cannot access all of them and can use most of them based on the access given.
- If you are a Test Analyst, you will not have access to delete any entity that’s created folder level.
- If you are a Test Analyst you will not have access to move the defects from Analysis status to retest status.
- If you are a Developer you will only have access to move the defects from Analysis status to retest status.
- If you are a Test Lead / Dev Lead , you will have most of the access
- If you are a Test Manager you will have access to all the functionalities existing in Dashboard module
Logging into Quality Center:
- Use the URL (Web Link)
- Login Name – Enter your User Id (Mostly Windows Login user Name)
- Password – Enter your password (You will be entitled to a Default password @ the time of user id creation and you can change the password by Tools à Customize à Change Password.
- After entering User Name and Password and Check the Authentication of your user name and password by clicking Authentication button.
- After Clicking it, it will show the Domain and Project Name that is assigned to you, If you are assigned to more than one project the use the drop down and select the Project that you wanted to work.
- You can change the project from the QC Home Screen (Top Left Corner) Change project.
Module 1à Dashboard Module:
In Dashboard Module user can see two types of View, Analysis and Dashboard View. Analysis view can be used to get the reports for the particular Test Sets, Defects from particular Test Sets and Reports from particular project. Dashboard views will be used by the Test Managers, and they can create a report across programme level. In each View you will have two folders, Public and Private. If you create any Reports under Private then it’s visible to the creator only but if you create a report in Public then it’s visible to all the users who are accessed to the Domain and the Project.
1. Analysis View
In this user can create a Graphical Report or Standard Report. There is a button (+), if you click you can see
Graph Wizard, New Graph, New project Report, New Excel Report and New Standard Report. Click on the type of the report that you wanted and choose where you wanted to post the report either in Private or in Public.
Now I am using New Graph functionality.
- Click on New Graph (Under Private section)
- Upon clicking you will get a New Window where you need to select1.
- Entity (Defects, Tests etc.,)
- Graph Type (Progress , Age, Summary, Trend)
- Graph Name (Project Name – User Specific)
- After creating the Test Report, you can see the Project Name that you have created under the Private Section. (Left Side of the tool)
- Click on the Project then check your right hand side, you can see 3 tabs, Details, Configuration and View.
- Details tab – the use can’t edit anything, its auto populates the values that we had given when created the project.
- Configuration tab – this is the place that we need set up what we need in the report.
- X- Axis – Choose from the drop down what you need to see in the X- Axis
- Y- Axis – Choose from the drop down what you need to see in the Y- Axis
- Grouped by – Choose the entity by which the X-Axis and Y-Axis needs to be grouped by
- And you can see a button Filter (Funnel Symbol) click on that then you will get a Pop up window where you need to set the target folder from where you get the data for the Graph.
- In that window you will have Filter and Cross filter tabs. Choose Cross Filter tab and you can see different sections like Defects, Test Sets, Requirements etc,
- Choose the Entity that you need, right now I want to show to my lead how many test cases that I have executed over a period of time so I will choose Test Set section.
- Under Test Set section you will have 3 radio button, click on the radio button that shows next to None radio button.
- At the end of the text box you will have (…) 3 dots. Click on it. It will take you to the Test Lab where you can see all the Test Sets created under a given domain.
- Select the Test Set that you want from the Test Set tree.
- And Click Ok now Click on View you can see the Graphical representation of the data that you have in your Test Lab.
- By Clicking Data Grid on View tab you can see the data in Numbers.
Module 2 > Management Module:
Management module will allow the Test Managers or Test Leads to set up the Cycle Start and Cycle End date to the given Test Set in Test Lab. A user can’t access a Test Set after the Cycle End date. This module can give the data to drive the Test Metrics related to Effort.
Module 3 > Requirements Module:
Business requirements are captured in this module. You can add the actual business scenarios or you can add the test conditions as requirements. You can add requirements by simple clicking the Add New Requirement button or you can upload using the Excel upload add in manager.
By capturing the Testable requirements we can achieve the requirement traceability. Suppose if you are testing a log in screen then you will set up a test condition as
‘Verify the Login Screen is working as expected by giving valid user name and password’
This test conditions can be furthermore explained in detail steps under Test Plan module.
The key points that we need to maintain when setting up the Test Conditions are
- We need to set the Priority to each test conditions
- We need to write the meaning full test condition name
- We need to choose the correct requirement type from the drop down box (Right top most corner)
- Author name will be auto populated from your Login credential (if I logged in then my name will be populated as Author name)
- You can’t delete a created Test Requirement whereas you can Cut and Paste into Recycle Bin folder. (Requirement, Test Plan and Test Lab modules will have the Recycle Bin to keep the rubbish contents)
- The created Test Conditions will be mapped to Test Cases from the Test Plan module. If you have not mapped then you can see Direct Cover Status as Not Covered, if you have mapped but not executed then you can see it as Not Completed, you have executed the Test Case if the test case is Passed or Failed then you can see it in Direct Cover Status.
Module 4 > Testing
Under Testing module we will have below mentioned tabs
Test Resources – This tab will be used for QTP to keep the Scripts
Test Plan – Marinating all detailed steps involved in particular Test Conditions
Test Lab – We will pull the test cases under Test Lab for Execution
We can simply add a Test Case using New Test button or we can write the test cases in spread sheet and upload it into QC using QC Excel Add in Manager.
The key points that we need to maintain when setting up the Test Plan are:
- Click on the New Test button , you can see a window opened
- Enter a Valid test name, mostly it should match with the name that we had given for test conditions
- Select the type of the test that we are performing (right top most corner)
- If you see any sections are marked as Red then it’s a mandatory column and you need to enter the value.
- Select the SDLC phase from drop down box like System Test, E2E Test, Regression Test etc
- Select the Priority from the drop down box, this priority should match with the priority that we had given for Test Conditions.
- Select the Test Type from drop down , what kind of testing that you are doing, like Functional or Non Functional etc.,
- Capability mostly Do Not Know
- Select the Application that you are going to work in and this will be pre-defined by the QC Admin team
- When you are creating the Test Case please keep the Reviewed question as Not Reviewed and assign the Reviewer name.
- Once you save the Test Case the Auto Email will be sent to the Reviewer.
- Now reviewer will be reviewing the test case and will set the status as Reviewed.
- Under the Description Section we need to enter the following details, these details will be pre-defined by the QC Admin team and will be populated to all the resources associated to the domain.
- We have set up the Test Case, now we need to map this Test Case to the Test Conditions. To do that please Click on the Test Case name from the left side pan so that you can see the below mentioned tabs on your right side
- Summary – All the details related to that test cases (what we had given when creating) will be populated
- Design Steps – will have the Details steps that will be performed on the Test Conditions
- Parameters – the data that needs to be passed for Automation Frameworks
- Test Configuration – auto populates the values
- Attachments – if you like to add any documents that related to this particular scenario you can add under this tab
- Requirement Coverage – from this tab we can search the Test Requirement (test conditions) that related to the test case and drag and drop using Select Requirement button
- Linked Defects – Defects can be attached to Test Case or to Test Steps. So the attached Defects will be shown under this tab
- Dependency – This tab is used for Automation tests, where you need to create dependency between different modules.
PS: After mapping the Test Conditions to Test Cases please go to Requirement module and check the Direct Cover Status, if the Status is Covered then leave it else Refresh it reflect the status.
We have set up the Test Conditions and Test Cases and linked them now we need to pull them into Test Lab and make them available to Test Execution.
Before pulling the Test Cases into Test Lab, please create a Test Set folder using New Test Set button. By Clicking it you will be getting a window where we need to enter the logical test set name.
Now we have created the Test Set and pull the test cases into test lab by going to Execution Grid tab in the Test Lab module and Click Select Tests this will take you to Test Plan module where you browse the test cases that you wanted to move.
Pull all the test cases that you wanted. All the modules in QC you will have the Select Columns button to customize the details that you wanted to display in the Screen, so play across the needed columns and make them available in your screen.
The key points that we need to maintain when setting up the Test Lab are:
- Running the Test Case
- You have Run button to execute the test case. In Run we will have Run With Manual Runner and Continue Manual Run.
- By simply clicking Run will lead you to Run with Manual Runner option. Now you can see the test steps that are written in Test Plan module.
- To pass the Test Case, we can click CTL+P to fail the Test Case CTL+F. Please attach a Test Evidence for each Test Case that you are executing.
- Do not execute the test cases more than one, if you do then it will create an Instance for each time that you are executing this will annoy your test reports
- If you kept a Test Step as Not Completed for any reason @ first time of execution and you wanted to execute that particular test step then choose Continue Manual Run under RUN tab.
- If you are failing a Step then you can create a Defect from Test Plan module.
- After failing the Defect, go to Linked Defects tab and Click Add (+) button it will take you to defect module from there you can create defect and it automatically linked to that particular step.
- If you are keeping a Test Case as Not Applicable then you need to attach a Evidence why this test cases are chosen as Not Applicable
- If you are keeping a Test Case as Deferred then you need to attach a Evidence why this test cases are chosen as Deferred
Module 5 > Defects
Defects can be added from Test Plan module or from Defect Module. If a Defect is attached to a test cases then please do link the test case to this defect. You can raise an Orphan defects without linking them to any test cases.
Go to defect module and click Add Defect button then you will get a window where we need to input the defect details.
- Summary – Brief description about defects
- There are few columns will be auto populated as per the QC configuration
- When we raise the Defects it will be in New Status as per the Defect Life Cycle
- Defect Type – We need to select the defect type from the drop down like Application Code, Requirement Defect etc.,
- Discovery Phase – We need to select on which phase the defect is Injected like System Test, UAT, E2E etc.,
And there will be more than 10 mandatory fields that we need to enter as per the project specific details.
- Description – Testers needs to give the detailed description about the defect in Description section
Test User Name/ID
Steps to Replicate
Test Data Reference
Test Case Reference [Test Case Name (Step number)]
Defect Life Cycle in QC
- New – (When the defect is created)
- Analysis – (When the defect is moved to Developers)
- Fix – (When the fix is given by the developers)
- Deploy –(when the developers deployed the defect fixed code into test environment)
- Retest – (When the code is ready for retesting)
- Closed – (when the defect is retested and closed by the testers – once you closed the defect we can’t modify the defect.
Note: User can’t jump from one status to another status by by-passing one status in between.
Setting up the Priority and Severity to the Defects:
Do you remember we have kept Priority to a Test Requirement when we have created in Requirement module? I have raised a Defect that related to the Test Case which was set to Priority Low then keep Defect priority as Low.
Cheers – Asik
Hi Guys ,
So we are ready with Test Environments, Test Plans, Test Cases to begin with Static Testing.
When we design our test cases we need to write test cases and test conditions right from Source validation till the Reports testing.
1. Test cases to validated the Source Data .
What you wanted to do with source data ? Why it is important ? Why do we need to know the importance of the source Data ?
Let me explain you with normal Scenario!!!
You wanted to purchase a Mobile so what you will do , start analyzing the markets with different channels like [Suggestions from friends] ,[Surfing in Internet], [Checking in news Papers Advertisements], [Checking @ Outlets]. In this you will not get the same details from all of these channels. So we will note down all the data retrieved from different channels and choose what you need !!!
Like the above scenario, a customer who approaches an Organization will go through multiple channels and share multiple information but at the end what you need for your Reports will be Designed , Developed , Tested and Extracted by Us 🙂
As far as I know, if you are 90% OK with the source data then there are high possibility of not finding even a single defect @ any of the test phases unless care less mistakes or code defects found by developers. The defects leaked by Developers can be easily fixed because ETL code is under our control. If a defect is leaked because of the data then it is a huge impact to the projects because we need to go back to the source system and inform them to fix !!!!
What all you need to verify in Source data ?
Just think you are intermediate between a seller (Source) and a target (Seller) . Do you buy defective product from your buyer? Does your buyer buy defective products from you ? No 😦 so please ensure you are buying a non defective product and a selling a 90% defect free product to your customer.
Coming to Business Intelligence 😉
1.The data that we need will be sent to our address (Unix Landing Directory) tester should verify the data has arrived to the correct address (Landing Directory).
2.Our source system will tell us on what Format they are sending the files. So the tester should verify the Name of the file is as what source said to us.
3.Date component in the file name is critical for us, this date is the date when the file is extracted for us.
4.The Source data will be sent to use is different formats , mostly it will be .DAT (data) or .DLY (daily) files.
5.Along with DATA files they will send CTL (Empty file) to make sure all the files arrived -as Acknowledgement copy 🙂
6.Now you validated the address (Landing Directory) and File names.
7. Now open the file you will see Numbers, Alpha-Numeric, Alpha characters separated by [,] or [|] – you need to verify how its separated
8.Most of the files will have 3 sections – Header , Detail and Trailer records.Testers should verify these sections are retrieved as per the specifications.
9.As a tester you should be interested on the data how it is Organized as per the Specification? the order of the column is matching with the order given in Specification ? Data types of the columns are matching with the data types mention in the Specification?
Order of the column:
In the spec you have this Order |Name|Place|Number|DOB| but in the file you are getting as |Name|DOB|Number|Place| —Wohooo its a Defect 🙂
Number of the Columns :
In the spec you have this Order |Name|Place|Number|DOB| but in the file you are getting as |Name|DOB|Place| —Wohooo its a Defect 🙂
Data Type of the Columns:
In the spec you have this Order |Name as VARCHAR |Place as VARCHAR |Number as INTEGER |DOB as DATE | but in the file you are getting as |Name as VARCHAR |Place as VARCHAR |Number as CHAR|DOB as DATE | Wohooo its a Defect 🙂
File name specification :
What are the file name, how it should be named, and extension of the file (.DAT), business time stamp?
File name should be as per the Spec, because developers mention the exact file name in their Data Stage File watched job. If the file name is not as per the Spec then the Job fail due to the mismatch. The time stamp filed (Date + Time) should be validated, example the Spec says the Date should be DDMMYYYY if source is sending the date as MMDDYYYY then our Data Stage job fails due to Invalid Date.
File landing location :
Each project has specific location to X-Com the files from Source System. Testers need to verify the file is arrived
to the correct location.
(Note: If tester cannot able to view the file due to File size, he can view the file using Unix Commands in PUTTY
File Header specification :
In general every file must have Header and the header record will contain when the file is processed. The testers
should aware of the Date in External file name and the date in the Header details.
i) Date in External file name: The date when the file is X-Com’d
ii) Date in Header record: The date when the file is processed.
Suppose if SAP system is extracting the file on 3/12/2013 and X-Com’ing on 4/12/2013, then External file name is
4/12/2013 and Header date is 3/12/2013. The Header is very important because Developer will use the Header
date as EFFECTIVE Date for Data loads.
File detailed records specification:
Detailed records section of the file contains the actual data that need to be loaded into Warehouse. Detail records
are separated by Pipe delimiter in general and it may change from source to source. Testers should be very
conscious about detailed records.
|File Position||Column Name||Key Column||Null / Not Null||Data Type||Filed Length|
The testers should verify:
1.The columns are in the correct order as per the Spec
2.If the columns are mentioned as Key then source should send us only one record for that combination, if
find more than one record after loading into the Staging tables then raise defect
3.If the columns are mentioned as Not Null in the spec then we should not raise Nulls (||) from the source,
suppose if we get NULLs , our Data Stage job fails due to NULLs presents in Not Null Columns , now
raise defect to Source system
4.Data type verification is the critical stuff, if a column is defined as INT then the source should send INT only
if not please raise defect to source system.
5. Filed length should be as per the spec, if the source sends less then what spec says then its OK, if it
exceeds then please raise defects to source system, because it will fail our Data Stage load jobs.
File Trailer record specification:
Trailers records will tell you how many records in the file. And the count could be including Header and Trailer. If
the file is related to transactional data then we might have a Hash total count column.
The tester should use the Trailer record count to verify the reconciliation between source file to staging table data.
So now we have validated our source 🙂 @ my next blog we are going to see how to deal with the Requirement documents for execution.
Thanks – Asik