Black-Box Vs White-Box Testing: Key Differences
There are many testing types, depending on the criteria on which we categorize them - we can have functional vs non-functional testing, manual testing vs automation, scripted vs exploratory, and black-box vs white-box testing.
With black-box and white-box testing, the distinction comes from whether or not the tester is concerned with the internal structure of the AUT (application under test). Both software testing types are used to increase test and code coverage, which helps identify the most critical defects in the system.
In this post, I’ll walk you through both types of testing, explaining what they are, the key differences between them, how they are used, as well as their pros and cons.
Need expert help selecting the right tool?
With one-on-one help, we guide you to your top software options. Narrow down your software search & make a confident choice.
What Is Black-Box Testing?
Black-box testing, sometimes called behavioral testing, is a software testing type where the tester does not have access to the source code of the system that's being tested. It’s usually performed by the quality assurance team and does not necessarily require advanced technical skills, such as programming.
In black-box testing, the test cases are written based on the inputs and outputs of the AUT, as defined in the requirements specifications.
The most notable black-box testing techniques are:
- Equivalence partitioning: where the inputs are categorized into classes (or partitions) of equivalence, meaning every value inside a class will generate the same output. Only one test case per equivalence class is necessary for good test coverage.
Example: If a field accepts integer values between 1 and 10, then the valid class contains all numbers between 1 and 10, and the two invalid classes are numbers smaller than 1, and numbers higher than 10. This means that 3 total test cases are enough to cover all possible classes.
- Boundary-value analysis: where the extreme input values, which are more prone to generate defects, are tested.
Example: Using the same field as above, the valid boundaries are 1 and 10, and the invalid are 0 and 11.
- Decision table testing: where the relationships between inputs and outputs are displayed in a tabular format. The table typically has columns for conditions, and rows for the different combinations. Each row should have a corresponding test case.
Example: This technique works on more complex scenarios. Let’s say we have a bank loan application with the following conditions:
| Conditions | Age equal to or over 25 | Income equal to or over 50000 USD | Outcome |
|---|---|---|---|
| 1 | True | True | Approve lawn |
| 2 | True | False | Reject loan |
| 3 | False | False | Refer to a manager |
| 4 | False | False | Reject loan |
We have four possible combinations, so four test cases need to be executed.
- State-transition testing: verifies the behavior of a system as it transitions between different states.
Example: Let’s say you are testing a basic bug tracker. The statuses diagram is:
The available transitions are:
- From New to In Progress
- From In Progress to In Testing
- From In Testing to Closed
- From In testing back In Progress
For complete coverage, you need to make sure that each of the transitions and statuses is tested at least once.
- Experience-based testing, such as exploratory testing. This type of testing means executing tests based on the tester’s experience with the system or with similar systems, and knowledge of the app’s behavior.
Example: Imagine you are testing a new social media application. Your goal is to explore the application to find any defects or issues that need to be addressed.
During exploratory testing, you might perform the following actions:
- Sign up for a new account
- Upload a profile picture
- Write a post
- View your own profile
- Search for friends
- Send a friend request
- Accept a friend request
- Like and comment on a post
Using exploratory testing, you would perform these actions in an unstructured manner without a pre-determined plan. The focus would be on finding defects or areas of improvement in the application.
If you want to learn more about exploratory testing, I found Elisabeth Hendrickson’s book, Explore It!, really useful.
Pros & Cons of Black Box Testing
Of course, there are benefits, as well as downsides, to performing black-box testing.
Pros:
- It is more time-consuming
- The control over test cases is limited
- Some specific scenarios can be difficult to test
- Limited information on the root cause of failure
- May miss certain errors
- Limited ability to test for performance and scalability
Cons:
- Preparing the test environment and executing the tests can be more time-consuming
- The control over test cases is limited
- Some specific scenarios can be difficult to test
- Limited information on the root cause of failure
- May miss certain errors
- Limited ability to test for performance and scalability
When to Use Black-Box Testing?
Black-box testing can be used at any level of testing. However, it is better to be used at a higher level, and leave the lower level testing to white-box testing. This means that, while it can work to test at the unit level, the black-box methodology is more appropriate for system testing and acceptance testing.
Black-box techniques can be used for functional tests, as well as for non-functional (such as performance testing, usability, and accessibility).
They should also be applied to newly implemented functionalities, or existing test cases identified through these techniques can be executed during regression testing.
What Is White-Box Testing?
White-Box (sometimes known as clear box testing, glass box testing, code-based testing, or structural testing) is a testing method that focuses on the internal workings of the UAT.
White-box testing approaches:
- Statement coverage: all the code statements (code lines) are executed at least once at the source code level. The formula to calculate the coverage is:
Statement coverage = (Number of executed statements / Total number of statements in source code) * 100
Example: Let’s say we have the following code:
if(condition1 or condition2)) {
print(“test 1 OK”)
}
else {
if(condition3) {
print(“test 2 OK”)
}
}
- For full statement coverage, you have to go through each line of code at least once. This means that a number of tests are necessary:
- condition1=true condition2=false, which will print “test 1 OK”
- condition1=false, condition2=false and condition3=true, which will print “test 2 OK”.
- Branch coverage: in this technique, the test scenarios cover all branches of the control flow graph. Each condition’s possible true and false outputs are covered at least once. The formula used for the branch coverage is:
- Branch coverage = (Number of executed branches / Total number of branches in the code) * 100
- Example: For the same code, while the statement coverage is 100%, not all possible branches are covered. You will need an additional test where:
- condition1=false, condition2=false, condition3=false - this way, we also cover the false path in the second if. Nothing should be printed out in this test case,
- Condition coverage: a comprehensive technique where all the paths are tested. It ensures that every application path is covered by at least one test. It’s particularly useful for complex apps. Example: For the code above, we need one more test for full condition coverage:
- condition1=false, condition1=true, which will have the same outcome as the first test, but covers a different condition to reach the result.
Pros & Cons of White-Box Testing
Let’s take a look at the advantages and disadvantages of white-box testing.
Pros:
- Can find errors early in the software development life cycle
- Tests the internal code structure.
- Tests code coverage and logic.
- Improves knowledge of the code base.
- Can be used in performance and scalability testing.
Cons:
- Works better at a lower level of testing
- Requires good knowledge of the system’s programming language
- Limited testing from the end-user perspective
- May overlook real-world scenarios
When to Use White-Box Testing
White-box tests are best suited for lower levels, such as unit and integration testing. This helps identify errors and defects early in the development process. Executing these tests after every deployment is a good idea, especially when working in a CI/CD environment.
White-box testing can be used to test functionality, but it can also be used to uncover vulnerabilities in the system - something that would be hard to achieve with black-box testing techniques.
Black-Box vs White-Box Testing: Summary
Let’s see what the key differences between black-box vs white-box testing are:
| Black-box testing | White-box testing |
| No knowledge of the internal workings is needed. | Is based on a good understanding of the system’s code. |
| Mostly performed by the QA team. | Usually done by the developers. |
| Focuses on the behavior of the system. | Focuses on the logic and the implementation of the software. |
| Techniques include: Equivalence partitioning Boundary value analysis Decision table State-transition | Techniques include: Statement coverage Branch coverage Condition coverage |
| Scenarios can be executed manually or automated. | Is usually done through automation testing. |
| Better suited for higher levels of testing. | Works best at the lower testing levels. |
| Tests from the end-users' perspective. | Tests from a technical perspective. |
But let’s not forget that while both approaches have pros and cons, we should use both in our testing process, to have good test and code coverage, and to find the most important defects.
Conclusions
Black-box and white-box testing are two different approaches, and they all work for different needs during the development process. While white-box testing is mostly done by developers and applies to lower-level testing, and black-box testing is done by the QA team at higher levels, they work best when used together.
If you liked this article, subscribe to the newsletter to receive all the newly published articles about testing and quality assurance!
Need expert help selecting the right Testing Software?
We’ve joined up with Crozdesk.com to give all our readers (yes, you!) access to Crozdesk’s software advisors. Just use the form below to share your needs, and they will contact you at no cost or commitment. You will then be matched and connected to a shortlist of vendors that best fit your company, and you can access exclusive software discounts!
