|
The basic purpose of software testing is to ensure that a
software system is free of significant defects and that the system possesses
the required functionality to meet its intended objectives. There are a variety of
types of testing that can be used to certify that a system is truly ready
to be used by its target audience, of which the most important is what is
known as system testing.
I'll provide a whirlwind overview of a typical system testing process
as it applies to web sites and web-based applications. Then I'll describe several other
types of testing that can be used in conjunction with system testing to ensure the
creation of quality software systems.
System Testing
System testing needs to be organized into a systematic and comprehensive
process in order to be effective. For this reason, system testing usually
requires a detailed plan, a blueprint of how testing will be done. Such a
blueprint is known as a System Test Plan, and is a formal document that defines
in excruciating detail how system testing will be conducted.
The plan typically consists of a variety of test cases, or scenarios,
which are specifically designed to exercise the functionality of the
system in significant ways. A System Test Plan should have the following
characteristics:
Testing of Typical Operations: Test cases must provide
detailed coverage of the typical operations that will be performed by
users. This ensures that testing will cover the majority of activities
that users will engage in while using the system. Exercising of Boundary Conditions: Some test cases should be
deliberately designed to test software boundary conditions. For example,
a test case may be created to test conditions such as the user executing
a search with no criteria, leaving a field blank in a form, etc. Repeatability: The test cases must be repeatable. All software
changes should be subjected to system testing before being placed into
production. This means that the System Test Plan will be executed
numerous times, first as the initial system goes into production, and
later after defect fixes and additional enhancements go into
production.
System testing is an iterative process, requiring a number of
cycles to ensure the defect-free status of a sizable and complex system. In a
testing cycle, all of the test cases in the System Test Plan are executed
against the software (preferably by people other than the original
developers, who are too close to the system to be entirely objective).
Any defects revealed during testing should be carefully tracked.
The defects should b assigned to individual development staff members for
resolution. This means that a round of fixes generally follows a test
cycle. A new edition of the system software incorporating the various
defect fixes will then be created, and subsequently subjected to the next
cycle of testing.
This process generally repeats until appropriate decision makers
determine that the system should go into production or, if already in
production for a limited number of users (for example, in Beta mode), that it should
be made available to a larger pool of users. Alternately, system testing may continue
until the clock runs out, and the system is pushed out the door, ready or not (Yes,
this isn't what's supposed to happen...but it does. Trust me.).
Other Types of Testing
Organized system testing can be vital to the success of sizable
development efforts, and typically results in the final certification that
a system is ready to be placed into production. But system testing alone
cannot ensure a successful and viable project outcome. There are several
other types of testing that can also be useful, particularly for large
systems.
Unit Testing: Developers typically perform unit
testing upon completion of various software elements. In other words,
developers should have done some ad hoc testing on their own software
before submitting it to system testing.
Warning: One of the often fatal mistakes for projects
is to force developers to push software into system testing that is not
ready. Usability Testing: Usability testing consists
of exposing the system software to focus groups of typical users. In effect,
real users are asked to try out a Beta (or pre-Beta) version of the software.
They are asked to perform typical operations using the system. After using
the system, users are then questioned to elicit feedback concerning ease
of use, opinions on the site organization, usefulness of help
information, etc.
Warning: So what if the software works correctly? If the
users can't figure out how to use it, the project's a bust. Test your system
with real users. Test early, and test often. Link Testing: Any web-based application will have
links to other web-based resources, whether part of the same site or on other
web sites. These links need to be tested to ensure that none of the links
are "dead", i.e. - to ensure that they point to an accessible
destination.
Many links may be exercised in the course of a normal
test cycle of the overall System Test Plan. However, link testing is not
actually the focus of system testing. Therefore, link testing is an organized
and comprehensive effort to test all links accessible through the system.
Link testing is typically automated, either using commercial testing
software or custom scripts.
Warning: Dead links make web sites seem
unprofessional and out-of-date. Editorial Review: System testing typically focuses
on software functionality. An editorial review focuses on content. Some
editorial issues may surface during system testing, i.e. - a tester may report
typos in the text of a web page. An editorial review is a concerted effort
by someone with writing ability (which does not usually include
developers) to ensure that the text elements of the system are free of
typos, display excellent grammar, flow well and exhibit the proper tone
for the system.
Warning: Typos on a web site are as bad, if not worse,
than dead links. At least with dead links, there's a chance that the user
won't click on it and therefore won't notice it. Data Quality Assurance: Many systems have important
data components, such as information publishing systems or systems that
dynamically provide access to information in a database. Any time that a
large amount of content is manipulated in order to make it available
through an information publishing system, there will be significant data
issues.
An organized and concerted effort will be required to
ensure that all content is tested thoroughly to verify that it is free of
typos, possesses all required data elements, is organized properly for
indexing and has the appropriate structure and formatting.
Warning: Many clients often have unrealistic
assumptions about the ease with which large amounts of information can
be published to the web. For a large-scale system, Data Quality Assurance
can be a considerably more time-consuming and grueling process than system
testing. Stress Testing: Stress testing is an organized
effort to overload the capabilities of the system. Stress testing addresses
issues such as the number of concurrent users that can be handled by the
system, the number of searches that can be executed in a set period of
time, the performance of the system with an inordinately large database,
etc.
Conclusion
Sometimes it seems like the art of testing has been all but
forgotten by many companies, especially dot-coms. How many web sites are there
that could have used a little more usability testing for their shopping cart
features? How many web sites got pushed out the door with no organized
testing at all?
A solid development process encourages developers to engage in unit
testing, before they roll onto the next task on their list. Such a process
follows this up with organized system testing, supplemented by other types
of testing as needed.
|
