Skip to main content

There are many testing types, depending on the criteria on which we categorize them - we can have functional vs non-functional testing, manual testing vs automation, scripted vs exploratory, and black-box vs white-box testing.

With black-box and white-box testing, the distinction comes from whether or not the tester is concerned with the internal structure of the AUT (application under test). Both software testing types are used to increase test and code coverage, which helps identify the most critical defects in the system.

In this post, I’ll walk you through both types of testing, explaining what they are, the key differences between them, how they are used, as well as their pros and cons. 

What Is Black-Box Testing?

Black-box testing, sometimes called behavioral testing, is a software testing type where the tester does not have access to the source code of the system that's being tested. It’s usually performed by the quality assurance team and does not necessarily require advanced technical skills, such as programming.  

In black-box testing, the test cases are written based on the inputs and outputs of the AUT, as defined in the requirements specifications.

The most notable black-box testing techniques are:

  • Equivalence partitioning: where the inputs are categorized into classes (or partitions) of equivalence, meaning every value inside a class will generate the same output. Only one test case per equivalence class is necessary for good test coverage.

Example: If a field accepts integer values between 1 and 10, then the valid class contains all numbers between 1 and 10, and the two invalid classes are numbers smaller than 1, and numbers higher than 10. This means that 3 total test cases are enough to cover all possible classes. 

  • Boundary-value analysis: where the extreme input values, which are more prone to generate defects, are tested. 

Example: Using the same field as above, the valid boundaries are 1 and 10, and the invalid are 0 and 11. 

  • Decision table testing: where the relationships between inputs and outputs are displayed in a tabular format. The table typically has columns for conditions, and rows for the different combinations. Each row should have a corresponding test case. 

Example: This technique works on more complex scenarios. Let’s say we have a bank loan application with the following conditions:

ConditionsAge equal to or over 25Income equal to or over 50000 USDOutcome
1TrueTrueApprove lawn
2TrueFalseReject loan
3FalseFalseRefer to a manager
4FalseFalseReject loan

We have four possible combinations, so four test cases need to be executed.

  • State-transition testing: verifies the behavior of a system as it transitions between different states.  

Example: Let’s say you are testing a basic bug tracker. The statuses diagram is:

The available transitions are:

  • From New to In Progress
  • From In Progress to In Testing
  • From In Testing to Closed
  • From In testing back In Progress

For complete coverage, you need to make sure that each of the transitions and statuses is tested at least once.

  • Experience-based testing, such as exploratory testing. This type of testing means executing tests based on the tester’s experience with the system or with similar systems, and knowledge of the app’s behavior. 

Example: Imagine you are testing a new social media application. Your goal is to explore the application to find any defects or issues that need to be addressed. 

During exploratory testing, you might perform the following actions:

  • Sign up for a new account
  • Upload a profile picture
  • Write a post
  • View your own profile
  • Search for friends
  • Send a friend request
  • Accept a friend request
  • Like and comment on a post

Using exploratory testing, you would perform these actions in an unstructured manner without a pre-determined plan. The focus would be on finding defects or areas of improvement in the application.  

If you want to learn more about exploratory testing, I found Elisabeth Hendrickson’s book, Explore It!, really useful.

Pros & Cons of Black Box Testing

Of course, there are benefits, as well as downsides, to performing black-box testing.

Pros:

  • It is more time-consuming
  • The control over test cases is limited
  • Some specific scenarios can be difficult to test
  • Limited information on the root cause of failure
  • May miss certain errors
  • Limited ability to test for performance and scalability

Cons:

  • Preparing the test environment and executing the tests can be more time-consuming
  • The control over test cases is limited
  • Some specific scenarios can be difficult to test
  • Limited information on the root cause of failure
  • May miss certain errors
  • Limited ability to test for performance and scalability
Discover how to deliver better software and systems in rapidly scaling environments.

Discover how to deliver better software and systems in rapidly scaling environments.

  • By submitting this form you agree to receive our newsletter and occasional emails related to the CTO. You can unsubscribe at anytime. For more details, review our Privacy Policy. We're protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
  • This field is for validation purposes and should be left unchanged.

When to Use Black-Box Testing?

Black-box testing can be used at any level of testing. However, it is better to be used at a higher level, and leave the lower level testing to white-box testing. This means that, while it can work to test at the unit level, the black-box methodology is more appropriate for system testing and acceptance testing.   

Black-box techniques can be used for functional tests, as well as for non-functional (such as performance testing, usability, and accessibility). 

They should also be applied to newly implemented functionalities, or existing test cases identified through these techniques can be executed during regression testing.

What Is White-Box Testing?

White-Box (sometimes known as clear box testing, glass box testing, code-based testing, or structural testing) is a testing method that focuses on the internal workings of the UAT.  

White-box testing approaches:

  • Statement coverage: all the code statements (code lines) are executed at least once at the source code level. The formula to calculate the coverage is:

Statement coverage = (Number of executed statements / Total number of statements in source code) * 100

Example: Let’s say we have the following code:

 if(condition1 or condition2)) {
print(“test 1 OK”)
}
else {
if(condition3) {
print(“test 2 OK”)
}
}
  • For full statement coverage, you have to go through each line of code at least once. This means that a number of tests are necessary:
    • condition1=true condition2=false,  which will print “test 1 OK”
    • condition1=false, condition2=false and condition3=true, which will print “test 2 OK”.
  • Branch coverage: in this technique, the test scenarios cover all branches of the control flow graph. Each condition’s possible true and false outputs are covered at least once. The formula used for the branch coverage is:
  • Branch coverage = (Number of executed branches / Total number of branches in the code) * 100 
  • Example: For the same code, while the statement coverage is 100%, not all possible branches are covered. You will need an additional test where:
    • condition1=false, condition2=false, condition3=false - this way, we also cover the false path in the second if. Nothing should be printed out in this test case,
  • Condition coverage: a comprehensive technique where all the paths are tested. It ensures that every application path is covered by at least one test. It’s particularly useful for complex apps. Example: For the code above, we need one more test for full condition coverage:
    • condition1=false, condition1=true, which will have the same outcome as the first test, but covers a different condition to reach the result.

Pros & Cons of White-Box Testing

Let’s take a look at the advantages and disadvantages of white-box testing.

Pros:

  • Can find errors early in the software development life cycle
  • Tests the internal code structure.
  • Tests code coverage and logic.
  • Improves knowledge of the code base.
  • Can be used in performance and scalability testing.

Cons:

  • Works better at a lower level of testing 
  • Requires good knowledge of the system’s programming language
  • Limited testing from the end-user perspective
  • May overlook real-world scenarios

When to Use White-Box Testing 

White-box tests are best suited for lower levels, such as unit and integration testing. This helps identify errors and defects early in the development process. Executing these tests after every deployment is a good idea, especially when working in a CI/CD environment

White-box testing can be used to test functionality, but it can also be used to uncover vulnerabilities in the system - something that would be hard to achieve with black-box testing techniques.

Black-Box vs White-Box Testing: Summary 

Let’s see what the key differences between black-box vs white-box testing are:

Black-box testingWhite-box testing
No knowledge of the internal workings is needed.Is based on a good understanding of the system’s code.
Mostly performed by the QA team.Usually done by the developers.
Focuses on the behavior of the system.Focuses on the logic and the implementation of the software.
Techniques include:
Equivalence partitioning
Boundary value analysis
Decision table
State-transition
Techniques include:
Statement coverage
Branch coverage
Condition coverage
Scenarios can be executed manually or automated.Is usually done through automation testing.
Better suited for higher levels of testing.Works best at the lower testing levels.
Tests from the end-users' perspective.Tests from a technical perspective.

But let’s not forget that while both approaches have pros and cons, we should use both in our testing process, to have good test and code coverage, and to find the most important defects.

Conclusions

Black-box and white-box testing are two different approaches, and they all work for different needs during the development process. While white-box testing is mostly done by developers and applies to lower-level testing, and black-box testing is done by the QA team at higher levels, they work best when used together.

If you liked this article, subscribe to the newsletter to receive all the newly published articles about testing and quality assurance!

Need expert help selecting the right Testing Software?

We’ve joined up with Crozdesk.com to give all our readers (yes, you!) access to Crozdesk’s software advisors. Just use the form below to share your needs, and they will contact you at no cost or commitment. You will then be matched and connected to a shortlist of vendors that best fit your company, and you can access exclusive software discounts!

Andreea Draniceanu

Hi there! My name is Andreea, I’m a software test engineer based in Romania. I’ve been in the software industry for over 10 years. Currently my main focus is UI test automation with C#, but I love exploring all QA-related areas 😊