Testing

Learning Intentions

Describe, exemplify and implement a comprehensive final test plan to show the functional requirements are met
Identify syntax, execution and logic errors at this level
Describe and exemplify debugging techniques (Dry runs, Trace tables/ tools, Breakpoints, Watchpoints)

Success Criteria

Be able to identify errors and ways of determining where errors happen in code

Testing

Testing is the 4th stage of the software development process.
The purpose of testing is to find bugs and ensure that the program does not fail to run or crash.
Testing attempts to test different areas or ways in which a program could crash, but it is computationally impossible to test all features with all data in every program out there, and therefore testing cannot be exhaustive.

Test plan

A test plan is a document that describe the approach to testing as well as any test data that will be used for testing.
Test plans are usually created by non-development staff (mainly by a test team) using the original requirements. They spend a lot of time checking as many ways to try and make the program break.
At Higher we need to detail a comprehensive test plan.
Comprehensive testing is when testing is thorough and complete, testing everything possible.
In other words, we ensure that all functional requirements of the program are tested.

Test plan

In our test plan we want to make sure we are testing everything.
In this test plan you might test:
- That the program responds to input correctly
- Each process performs as expected
- The program produces the expected result
- Program reads/ writes data to file
- Program is efficient
- Program runs correctly in it’s intended environment

Test data

Test data is the type of data we will use to test our programs.
There are three kinds of test data:
- Normal
- Extreme
- Exceptional

Test data

Normal data: Data that is within the range. Data that will be accepted by the program
Extreme data: Data that is on the boundary of the range. This data should still be accepted by the program/ work as expected.
Exceptional data: Data that is out with the range. This is data that will not be accepted by the program.

In computer science and programming, an exception is similar to an error except that unlike errors, exceptions are normally (but not always) recoverable from. Exceptions tend to change the flow of a program whereas errors tend to crash the program.

Test data

A program enters seven exam marks as a percentage. The program then displays the number of marks in each grade
(Fail = 0-49, C = 50-59, B = 60-69, A = 70-100)

Test tables

A test table for a program to accept numbers between 1 and 60 inclusive is shown. Notice that 1 and 60 are not accepted so the table shows that this test does not pass.

Test tables

Comprehensive testing

Doesn’t always have to be normal, extreme, exceptional.
For example, a program that reads in names from an external file, sorts them by surname then prints into a separate file.
We would test that:
- Data is read from file correctly
- Data is sorted alphabetically by surname
- Sorted data is written to a separate file

As part of the program’s comprehensive test plan each sub-program was tested individually.

Describe one benefit of having a comprehensive test plan.

1 mark

Past Paper Question

A comprehensive test plan ensure that all the functional requirements of a program are tested

As part of the program’s comprehensive test plan each sub-program was tested individually.

Describe one benefit of having a comprehensive test plan.

1 mark

Past Paper Question

A comprehensive test plan ensure that all the functional requirements of a program are tested

Errors

There are three types of errors you need to know about at Higher:
- Syntax
- Logic
- Runtime errors

Syntax errors

Syntax errors occur when the program contains something that it doesn't recognise.
Syntax errors occur when something is written that breaks the syntax rules of the programming language. The program cannot run as a result of a syntax error.
Syntax errors are discovered by the compiler or interpreter rather than during runtime.
Common syntax errors include:
- Misspelling a word
- Forgetting a colon
- Forgetting a bracket
- Forgetting quote marks
- Incorrect indentation

Logic errors

Logic errors occur when the logic or mathematical sums aren't correctly written.
For example:
- average = mark1 + mark2 + mark3 / 3
- is not the same as
- average = (mark1 + mark2 + mark3) / 3
The most common logic error tends to be when using And and Or and getting them muddled up. The other very common cause for logic errors is with > and <, as well as >= and <=.

Runtime errors

Runtime errors occur when the program is running.
They are not often easily identifiable.
Runtime errors will crash the program, but only when (and if) the error is reached.
The most common types of runtime errors are:
- Division by 0
- Not declaring a variable before using it (only in interpreted programs like Python)
- Accessing an array element passed the end of the array

Debugging code

Debugging code can be made much easier using different tools available.
At Higher, you need to know about:
- Dry runs
- Trace tables
- Breakpoints
- Watchpoints

Dry runs

Dry runs are very useful for testing a program. They are done manually by a programmer by stepping through each line of code.
As the program progresses, the programmer attempts to predict what will happen and will document what actually happens.
When carrying out a dry run, the programmer might make use of a trace table to keep track of the data stored in individual variables.

Trace tables

Trace tables are also very useful for debugging a program.
Dry runs often consist of lots of trace tables being formed.
Trace tables keep a record of what a variable is at each line when it changes.

Breakpoints

Breakpoints are perhaps the most useful tool in any IDE.
A breakpoint is a marker that tells the program to halt execution when it is being run through the interpreter.
Breakpoints allow the tester to see exactly what values are in variables.
They also allow the developer to see what gets written to an array or a file on each line (or iteration of a loop)

Watchpoints

Watchpoints are used to stop a program's execution when a condition is met such as a variable reaching a certain value.
They are similar to breakpoints as they allow testers to easily see the value of variables.

Pick out one completed program that you have.
1. Create a trace table for it
2. Using break points, check your values in your trace table
Try the following past paper questions:
2018 – 15(d), 15e (i), (ii), (iii)
2019 – 18(c)
2022 – 6(e), 7d (i)

Task

Plenary

Pick up to 4 key words and provide a description for them.

Watchpoint

Dry run

Logic error

Debug

Breakpoint

Trace table

Runtime error

Normal test data

Extreme test data

Syntax error

Exception

Exceptional test data