Testing Code, Debugging Code, Tracing Code: What's the difference?

Jeff Smith & Steven Feuerstein

May 3, 2007

Executive Summary

Finding and fixing bugs in software usually involves several different processes, including testing, debugging and tracing. All too often, these terms and what they mean to developers, are confused and conflated. In reality, these three activities – test, debug, trace – are all quite different in nature and serve different purposes.

In a nutshell, here is the way to distinguish between these activities:

• Test your code to identify bugs that occur under a certain set of circumstances (a "test case" with specific inputs to arguments in the program).
• Turn on tracing and obtain a usually substantial amount of "raw data" about what the program did as it ran.
• Use your debugger to isolate the specific lines of code that caused the bug (and correspond to the flow indicated by the trace data).

Let's take a closer look at each of these steps.

Testing

When you test a program, you run test code that exercises the program. That test code then either tells you whether or not the test succeeded -- or you have to manually go through the results of the test code and deduce for yourself if it worked.

The more automated you can make the testing process the better. Quest Code Tester for Oracle offers the highest level of test automation in the Oracle tools market. Simply describe the tests you need to run in a graphical interface, and Quest Code Tester then generates the test code, and runs the test, automatically reporting all the results.

A test identifies a problem in your code, but it usually doesn't give you a whole lot more information than that ("You passed in 16 and should have gotten back 12. Instead you got 10."). You then need to figure out what lines of code are causing that problem. Both tracing and debugging can help you do that.

Tracing

When you trace execution of your program, you run that program and as it is running, record or build a trace of information about what was happening inside the program.

In Oracle PL/SQL, the most common (an also the most crude) type of tracing is a call to DBMS_OUTPUT.PUT_LINE, which sends information to the screen after the program has finished running.

It is not at all rare to come across PL/SQL applications that are full of (littered with?) calls to DBMS_OUTPUT.PUT_LINE. I generally avoid ever making a direct call to this built-in. For example, in the Quest Code Tester backend, the development team uses a tracing utility adapted from the Quest CodeGen Utility qd_runtime package, to allow them flexibly turn on and off tracing, and write information to the qu_log table. From there, they can extract the data to display on the screen, fill up a file, etc. Here is a simple example from the Quest Code Tester backend:

 IF qu_runtime.trace_enabled

 THEN

    qu_runtime.trace ('get_attribute_ex datatype guid'

                     , ret_attr.test_element.data_type_guid                   

                     );

 END IF;

Notice that we call the trace_enabled function to determine if tracing is on before calling the trace subprogram. Qu_runtime.trace will, itself, check to see if tracing is enabled, but by that time it will have also evaluated the arguments. To achieve the lowest possible runtime overhead, we call trace_enabled, which simply checks the value of a Boolean flag.

The user can specify when to turn on tracing with a convenient window:

Debugging

Debugging is not the same thing as tracing and it definitely should not be confused with testing. You will generally start a debugging session after your tests have identified a bug. You might have also turned on tracing to get a "dump" of information about the overall flow of the application leading up to the bug.

With this "raw data" (specific test case failure information and trace data), you then need to find the specific lines of code that are causing the bug, and also figure how to change those lines of code to fix the bug.

Given the maturity of the PL/SQL IDE (integrated development environment) market, most serious editors including a visual source code debugger.

For example, in Toad, you click in the "gutter" of the editor next to the line on which you want execution to "break" or pause. You start up your program and it runs to the breakpoint. You can then examine the values of variables, change values (in some debuggers) and then step through the code, line by line, examining the program flow and how data structures are affected.

PL/SQL source code debuggers are incredibly useful and some are very powerful, flexible implementations. Make them a part of your toolset when searching out the causes of bugs in your code!

Summary

To sum it all up, when thinking about identifying and fixing bugs in your code, keep in mind the differences between testing, tracing and debugging:

• Test your code to identify bugs that occur under a certain set of circumstances (a "test case" with specific inputs to arguments in the program).
• Turn on tracing and obtain a usually substantial amount of "raw data" about what the program did as it ran.
• Use your debugger to isolate the specific lines of code that caused the bug (and correspond to the flow indicated by the trace data).

Then use Toad (or another IDE) to fix the bug, and start the cycle over again, until your tests give you a "green light" that you have no identifiable bugs.