EUSES Logoskip to page content Intranet Login »
Home    About EUSES    News & Events    Research    Education    Publications    Resources    People    Contact 

WYSIWYT

Overview

The What You See Is What You Test (WYSIWYT) methodology helps users test their spreadsheets while they're creating them. As a user develops a spreadsheet, he or she can also test that spreadsheet incrementally yet systematically. At any point in the process of developing the spreadsheet, the user can validate any value that he or she notices is correct.

Behind the scenes, these validations are used to measure the quality of testing in terms of a test adequacy criterion. These measurements are then projected to the user via several different visual devices, to help them direct their testing activities.

WYSIWYT has been investigated using both a research spreadsheet language known as Forms/3 (which was used for the example) and using Excel. It has also been investigated to some extent in its fit with dataflow languages and screen-transition languages. Investigations of its fit with other types of end-user programming environments are also underway.



Figure 1 (a)


Figure 1 (b)


Figure 1 (c)


Figure 2

WYSIWYT Example

For example, suppose that a teacher is creating a student grades spreadsheet, as in Figure 1(a). During this process, whenever the teacher notices that a value in a cell is correct, she can check it off ("validate" it). The checkmark provides feedback, and later reminds the teacher that the cell's value has been validated under current inputs. (Empty boxes and question marks in boxes are also possible: both indicate that the cell's value has not been validated under the current inputs. In addition, the question mark indicates that validating the cell would increase testedness.)

A second, more important, result of the teacher's validation action is that the colors of the validated cell's borders become more blue, indicating that data dependencies between the validated cell and cells it references have been exercised in producing the validated values. From the border colors, the teacher is kept informed of which areas of the spreadsheet are tested and to what extent. Thus, in the figure, row 4's Letter cell's border is partially blue (purple), because some of the dependencies ending at that cell have now been tested. Testing results also flow upstream against dataflow to other cells whose formulas have been used in producing a validated value. In our example, all dependencies ending in row 4's Course cell have now been exercised, so that cell's border is now blue.

If the teacher chooses, she can also view dependencies by displaying dataflow arrows between cells or between subexpressions in formulas. In Figure 1(b), she has chosen to view dependencies ending at row 4's Letter cell. These arrows follow the same color scheme as the cell borders.

Suppose that in the process of testing, the teacher notices that row 5's Letter grade ("A") is incorrect. There must be some error in at least one of the teacher's formulas, but which one? The teacher indicates that row 5'2 Letter grade is erroneous by placing an X mark in it. Row 5's Course average is obviously also erroneous, so she X's that one too. As Figure 1(c) shows, both cells now contain pink interiors, but Course is darker than Letter because Course contributed to two incorrect values (its own and Letter's) whereas Letter contributed to only its own. These colorings reflect the likelihood that the cell formulas contain faults, with darker shades reflecting greater likelihood.

Although the teacher need not realize it, the colors that result from placing checkmarks reflect the use of a definition-use test adequacy criterion that tracks the data dependencies between cell formulas caused by references to other cells. Testing a program "perfectly" (well enough to guarantee detecting all faults) in general requires too many inputs; a test adequacy criterion provides a way to distribute testing effort across elements of the program. In the spreadsheet paradigm we say that a cell is fully tested if all its data dependencies have been covered by tests; those cells have their borders painted blue. Cells for which dependencies have not been fully covered have borders ranging from red to various shades of purple. This overall testing process is similar to the process used by professional programmers in "white box" unit testing, in which inputs are applied until some level of code coverage has been achieved. In the spreadsheet environment, however, the process is truly incremental, bearing some similarity to test-driven development approaches. These considerations, along with the testing theory underlying this methodology, are described in detail in this paper.

WYSIWYT Example in Excel

A research prototype of WYSIWYT is also being developed in Excel. See Figure 2.

People


Publications


Patents Issued and Pending

  • A Methodology for Testing Spreadsheets. Inventors: Gregg Rothermel, Margaret Burnett, Lixin Li. Filed 1999 (pending).
  • A Methodology for Testing Spreadsheet Grids. Inventors: Andrei Sheretov, Margaret Burnett, Gregg Rothermel. U. S. Patent Number 06766509, issued July 20, 2004.

WYSIWYT Research Sponsored By

Experimental Software Systems Award 9806821, ITR Award 0082265, and ITR Award 0325273 by the National Science Foundation.

 
Home    About EUSES    News & Events    Research    Education    Publications    Resources    People    Contact 
©2006 EUSES
Last Updated on March 18, 2009
Site Designed and Developed by Htet Htet Aung
Oregon State University University of Nebraska Lincoln University of Cambridge IBM Penn State University HCII of Carnegie Mellon University Drexel University