For anyone involved in software development, the software testing process is the most important phase of the project. Software testing is practically unavoidable especially for software designed to run on platforms which may be vulnerable to attacks by criminals, or software which may need to be installed on a wide range of devices. Also, software which will be sold must be of appropriate quality before any distribution goes on. In fact, for most software companies, the risks involved in sending out an untested software just cannot be entertained, so, much time and resources have been invested into the software testing process. There are different types of software testing techniques involved in the process of detecting every possible error to avoid any embarrassment at the end when the software is in distribution.
Software testing is aimed at determining certain parameters;
ü If the software meets the requirements which served as the design guide
ü If it achieves the kind of result that its developers would desire.
ü If it can be installed and run in all its intended environments or platforms
ü If it responds properly to all kinds of inputs.
ü If it performs its functions within an acceptable time period, and
ü If it is sufficiently usable.
Testing ensures the correct functioning of the software in testing under certain specific conditions, and not necessarily under all circumstances. Hence testing is necessary for building of applications, ensuring the correct functioning of the application in its designed platforms.
Testing for each of these prerequisites requires different types of software testing techniques. Here we will explore several of the best strategies which are in use today for software testing and show how they can help you in your software testing process. Remember testing is not the same thing debugging. Testing is the location of faults in the software meanwhile debugging is the process of removing those faults. Here we shall be looking at the different testing methods, testing levels, testing types, testing processes and more.
Software testing methods
Now to cover some software testing basics, we have two main ways of classifying the software testing approaches available to software Engineers; the Static or the dynamic testing methods and the box methods.
Static versus Dynamic software testing.
Basically, all testing methods can either be described as static or dynamic. Static testing involves testing the application without actually simulating it, so to speak. This involves inspecting and checking the program code, a sort of proofreading and requires skill and sufficient know how. What is checked using such methods is limited as code that adds up well on sight just may not behave as expected when executed. Hence the static and the dynamic testing go hand in gloves to give a more excellent result.
Dynamic testing involves executing the software itself, under certain conditions, with different inputs and test cases, in order to study the program’s reaction and mode of operation. Dynamic testing is possible before the program is a hundred percent completed to test particular sections of the software and ensure the smooth operation of certain functions and modules of the software. Typical techniques for dynamic testing include using stubs/drivers or executing the software from a debugger environment.
Static testing is limited to verification, whereas dynamic testing also includes validation. Together they help improve software quality and provide assurance of functionality.
The box view of testing
All the types of testing in software engineering can be classified on the basis of this test. This classification describes the point of view of a test engineer when designing test cases for testing simulations. The main two include the white box testing and the black box testing.
White-box testing (also known as glass box testing, clear box testing, transparent box testing and structural testing, by seeing the source code) tests internal workings of a program, checking the proper functionality of the structures, as opposed to the view which an end-user receives while running the software. In white-box testing an internal perspective of the system, as well as programming skills, are used to design test cases. The test involves choosing inputs to exercise paths through the code and determine the appropriate outputs. In this case, the tester has an expectation for every set of input, which if not produced would imply the presence of a problem.
White-box testing, even though it can be applied at varied levels of the software testing process including the unit, integration and system levels, it is usually done at the unit level. It can test paths within a unit, paths between units during integration, and test paths between subsystems during a system–level test. Though this method of test design can uncover many errors or problems, it might miss errors related to unimplemented parts of the specification or missing requirements.
Several techniques are used in white-box testing including:
API testing: The testing of the application by the use of public and private APIs (application programming interfaces)
Code coverage: This involves different tests to satisfy some criteria of code coverage (the test designer can create tests to cause all statements in the program to be executed at least once for example). Code coverage allows the software engineers to examine different parts of a system that are rarely tested and ensures that the most important function points have been tested.
Code coverage as a software metric can be reported as a percentage for:
· Function coverage, which reports on functions executed
· Statement coverage, which reports on the number of lines executed to complete the test
· Decision coverage, which reports on whether both the True and the False branch of a given test has been executed
Fault injection methods: this is the intentional introduction of faults to check the efficacy of testing strategies, to ensure that the testing is effective.
Mutation testing methods
Static testing methods
Black-box testing method is software testing basics of the first order, and it is taken from the black-box concept of experimentation, where one has a device that only gives the user access to the operation of the device, not to the components and makeup of it. Here the testers are only interested in, and perform their tests based on what the software is supposed to do, not how it does it. Testing methods under the Black-box include: boundary value analysis, equivalence partitioning, all-pairs testing, state transition tables, decision table testing, model-based testing, use case testing, fuzz testing, exploratory testing and specification-based testing.
Specification-based testing is mainly used to test the functionality of software according to the applicable requirements. This level of testing usually requires thorough test cases such as truth tables to be provided to the tester, who then can simply verify that for a given input or series of inputs, the output behavior is either true or false compared to the expected values specified in the test cases provided. Test cases are built around specifications and requirements, that is, what the application is supposed to do. It uses external descriptions of the software, including specifications, requirements, and designs to derive test cases.
Specification-based testing may be necessary to ensure correct functionality, but it is insufficient to guard against complex situations or situations addressing high security risks.
One advantage of the black box technique is that no actual programming knowledge is required, so this is practically software testing basics. Whatever biases the programmers may have had, the tester likely has a different set and may emphasize different areas of functionality. On the other hand, black-box testing has disadvantage in that it focuses on the outside of the software, given that with such an approach, very critical parts of the code could be left untested.
Also, because they do not examine the source code, there are situations when a tester writes many test cases to check something that could have been tested by only one test case, thereby reducing efficiency or leaves some parts of the program untested. Now, though this test method can be applied on all the levels of software development, it is usually limited to the highest levels.
Grey-box testing involves having knowledge of internal data structures and algorithms for purposes of designing tests, while executing those tests at the user, or black-box level. The tester is not required to have complete access to the software's source code. The distinction between the black-box and Grey-box is particularly important when conducting integration testing between two modules of code written by different developers, where only the interfaces are exposed for test.
By knowing the underlying concepts of how the software works, the tester can make better-informed testing choices while testing the software from outside. Typically, a Grey-box tester will be permitted to set up an isolated testing environment with activities such as seeding a database. The tester makes changes to the database and then finds out using normal black-box techniques to check if the expected changes have occurred. Grey-box testing implements intelligent test scenarios, based on limited information. This will particularly apply to data type handling, exception handling, etc.
Testing levels are the levels of development of the software at which tests can be performed, which include four main recognized levels; unit testing, integration testing, component interface testing, and system testing. The types of testing in software engineering are frequently grouped by the level in which they are added in the software development process, or by the level of specificity of the test, looking to the exact objectives of it.
Unit level testing include tests that are performed to verify the functionality of a specific section of code, usually at the function level, if the code does what it is supposed to do. Unlike the other types of testing in software engineering, these types of tests are progressively written by developers as they work from segment to segment on the code using the white-box style, to ensure that the specific function is working as expected. A function might have multiple tests, to explore different decision points in the code and ensure that each individual function does just what it should.
It is not actually sufficient that we perform only unit testing because unit testing focuses only on the working of the unit, not considering if the unit works well with other units just like they should. This simply means that each unit or code segment may actually work properly, but we find errors at a higher level due to mismatch of inter-unit communication, in the case where units which should work together as one main software lack synchronization and proper coordination.
Unit testing which is one of the most common types of software testing techniques is a software development process that involving a synchronized application of a wide spectrum of issue prevention and detection strategies in order to reduce software development risks, time, and costs. It is performed by the software development team engineer during the construction phase of the software development life cycle. The main aim of Unit testing is to eliminate construction errors even before code is promoted to additional testing; this strategy is intended to increase the quality of the resulting software as well as the efficiency of the total development process.
Depending on the different types of software for software development, unit testing might include static code analysis, peer code reviews, data-flow analysis, metrics analysis, code coverage analysis amongst other software testing methods.
Integration testing refers to the types of software testing techniques that seeks to verify the interfaces between software modules with the original software design. There are two ways modules could be combined: in an iterative way or all at once. Of course, for the purpose of detecting and correcting errors, the iterative method is considered a better practice since it eases the location of interface issues and they can also quickly be fixed.
Integration testing goes beyond the work of the unit testing to expose any defects in the interfaces connecting modules and in the interactions between integrated components or modules. The method is that increasingly larger groups of tested software components corresponding to elements of the architectural design are integrated and tested until the software works as a system.
The aim of system testing is to test a completely integrated system to verify that the system meets its requirements; functions according to design, looks right and functions within the appropriate time limit. Here, the tests would be limited to the end user’s experience. A system test for example, might involve testing a login interface, creating / editing an entry, sending or printing results, followed by summary processing or deletion (or archiving) of entries, then log-off. Depending on the different types of software in consideration, the software system test would definitely vary to include the actual expected day-to-day functioning of the entire system.
Operational acceptance testing
Operational acceptance testing is used to conduct operational readiness or the pre-release of a product, service or system as part of the quality control management system. OAT is a common type of non-functional software testing, used mainly in software development and software maintenance projects. This type of testing focuses on the functional readiness of the system to be supported, or to become part of the production environment. Hence, the focus of this test is compatibility with other software, and it’s ability to coexist in its installed environment. Functional testing within OAT is limited to those tests which are required to verify the non-functional aspects of the system.
In addition, the software testing should ensure the working portability of the system, as well as that the software is working as expected, does not also damage or partially corrupt its operating environment or cause other processes within that environment to become inoperative.
Several types of software testing techniques have been developed to address issues that have been developed in the process of searching for and eliminating errors in software engineering. We will list and explain a couple of them, so that you can determine which of these software testing techniques will work best for you.
This is definitely software testing basics. An installation test checks that the system is installable and properly working at an actual customer's hardware. Among the types of testing in software engineering, this is a must do test, which cannot be overlooked as it covers the compatibility of the software with the actual environment designed for it.
In software engineering, a common cause of software failure is the lack of its compatibility with other application software, older or backward operating systems (different Operating systems, or older versions of the same), or target environments that differ greatly from the original such as a complete change of an application which was designed for a desktop environment, now required to work on the web, to be used perhaps on the cloud. Such problems usually are generated because the programs or software are designed only with the latest system in mind, forgetting that the end users may still as yet be using outdated hardware or operating systems
Smoke and sanity testing
The Sanity test determines whether it is reasonable or even necessary to proceed with further testing. Smoke testing consists in minimal trials to operate the software, designed to check whether there are any basic problems that will prevent it from working at all. If this test shows negative results, then it means the build is not well done. Hence, such tests can be used as build verification test, to validate a complete build as ready for testing.
Regression testing is focused on finding faults which usually come up after a major code change has occurred. Specifically, it seeks to expose any software regressions, such as degraded or lost features, including old bugs that may have come back. Regressions generally occur whenever some software functionality that was previously working correctly, begins malfunctioning or stops working as intended. Typically, regressions happen as unintended consequences of program changes, when the newly developed module of the software collides with the previously existing code.
Common methods of regression testing include re-doing previous test cases and checking if previously fixed faults have re-emerged. Therefore, the coverage and intensity of testing depends on the phase in the software development process and the associated risks of the added features. You may have to do a complete testing, for changes added, late release or deemed to be risky, or do a shallow test, consisting of positive tests on each feature, if the changes are early–release or deemed to be of low risk.
Of all the types of testing in software engineering, regression testing is typically the largest test effort in most commercial software development due to numerous details in prior software features, and even new software could be developed while using some old test cases to test parts of the new design to ensure prior functionality is still supported.
Alpha and Beta Testing
Alpha testing is the simulated or actual operational testing of a software by an independent test team among the developers. The software is used by the design team, to use it and evaluate it from the viewpoint of a user, so that any glitches that a user might face by using the software in the long run might be discovered. The faults can then be handled before the software goes to Beta testing which normally comes after alpha testing and is a form of external user acceptance testing.
Versions of the software called Beta versions can be made available to a small group of subscribed individuals or even to the open public to increase the feedback field to the maximum, in order to benefit from the comments and remarks, and especially the faults that will be discovered as they use it. The users at this stage benefit from the free use of the more advanced or new software, while the developers benefit from a form of free testing. Forums are usually hosted for the purpose of collecting results of these users, in an attempt to improve the software quality.
Aside from all this, testers use several methods and procedures to make sure that the final software is up to standard such as testing its limits and what it can handle. So much is improved during the phases of testing which may not have been improved any other way. Several software testing basics have been shown here, and other methods and strategies are in use, and being improved daily.