EUSES Logoskip to page content Intranet Login »
Home    About EUSES    News & Events    Research    Education    Publications    Resources    People    Contact 

Resources on End-User Software Engineering

This page is a collection of materials, resources, and programs about end-user software engineering. When end users develop software, they do more then just program. End-user software engineering considers all aspects of the software life cycle involved in end-user software development.

About End User Programmers and the Software They Design, Program, Test, and Debug

    What are End-User Programmers Doing?

People who write programs -- but not as their primary job function -- are end-user programmers. They have a wide range of formal training in programming; some have none at all, and some have significant backgrounds in programming. Examples of end-user programmers include graphic designers, owners of mom-and-pop businesses, secretaries, and physicists. For one reason or another, they find themselves creating, maintaining, and debugging programs in support of their other goals.

Examples of End-User Programmers:

  • Accounting (spreadsheets)
  • Analysts using MatLab
  • Creating a web page
  • recording macros in Word
  • Automating office tasks
  • Business software (SAP programming)
  • Scientific research and calculation
  • Authoring educational software
  • Creating email filters
  • Musicians configuring synthesizers
  • Mashups
  • Entertainment (e.g., behaviors in The Sims)

    What is a Program?

A set of statements that can be submitted as a unit of some computer system and used to direct the behavior of that system. ~ Oxford Dictionary of Computing

The process of transforming a mental plan of desired actions for a computer into a representation that can be understood by the computer. Jean-Michel Hoc and Anh Nguyen-Xuan (J.-M. Hoc and A. Nguyen-Xuan, "Language Semantics, Mental Models and Analogy," in Psychology of Programming, Computers and People Series, J.-M. Hoc, T. R. G. Green, R. Samuray, and D. J. Gilmore, Eds. London: Academic Press, 1990, 139-156.)

End User Software Engineering Links

  • A collection of end-user software engineering research papers
    Publications by the EUSES Consortium.

  • EUD-Net
    The goal of EUD-NET Network of Excellence is to help the European Commission to prepare a research agenda in the end-user development field for the next framework and to increase contacts among highly-qualified research centers, both academic and industrial, in order to speed-up the production of innovative ideas and approaches.

  • EUSES Empirical Findings
    A listing of the empirical findings about end-user software engineering.

  • Gender HCI
    EUSES Gender HCI project, which also includes a public collection of gender research as applied to software. Information, projects, and publications studying how men and women problem-solve differently in an end-user software engineering setting.

  • Natural Programming
    The Natural Programming Project is developing general principles, methods, and programming language and environment designs that will significantly reduce the amount of learning and effort needed to write programs for people who are not professional programmers. These principles are based on a thorough analysis of previous empirical studies of programmers and new studies designed to discover more natural approaches to programming and end-user software engineering.

  • Visual Language Research Bibliography
    This page is a structured bibliography of papers pertaining to visual language (VL) research. It also contains a brief list of links to other, related resources about visual language research.
  • End-User Software Engineering in Spreadsheets

  • European Spreadsheet Risk Interest Group
    An interest group of academia and industry promoting research regarding the extent and nature of spreadsheet risks, methods of prevention and detection of errors and methods of limiting damage. They bring together researchers and professionals in the areas of business, software engineering and audit to actively seek useful solutions.

  • Ray Panko's Spreadsheet Research Website
    The spreadsheet Research (SSR) website is a repository for research on spreadsheet development, testing, use, and technology.

    The What You See Is What You Test (WYSIWYT) methodology helps users test their spreadsheets while they're creating them. As a user develops a spreadsheet, he or she can also test that spreadsheet incrementally yet systematically. At any point in the process of developing the spreadsheet, the user can validate any value that he or she notices is correct. Behind the scenes, these validations are used to measure the quality of testing in terms of a test adequacy criterion. These measurements are then projected to the user via several different visual devices, to help them direct their testing activities.

  • Incisive
    Incisive has moved some of the WYSIWYT and UCheck research into the commercial realm. The company provides tools for spreadsheet auditing and error detection.
  • End-User Software Engineering in Web Applications

  • The EUSES Web Macro Scenario Corpus, Version 1.0
    Based on observations of browser users, we have compiled various scenarios describing tasks that end users would benefit from automating using web macros. Our analysis of these scenarios yields specific requirements that web macro tools must support if those tools are to be applicable to real-life situations. Our collection of requirements constitutes a benchmark for evaluating new and improved web macro tools. We developed this corpus as a collaboration within the EUSES Consortium, whose aim is to help End Users Shape Effective Software.
  • Other Links

  • End-User Development Disscussion Forum
    A forum dedicated to discussion regarding end-user development

  • End-User Programming Overview
    An invited research presentation at CHI'06 by Brad Myers, Andrew Ko, and Margaret Burnett.

  • End-User Programming Errors Resource
    Print publications related to end-user programmer's errors.

  • End-User Programming
    This page covers the tools, techniques, and theory of people who exploit computer automation for their purposes without becoming professional programmers.

  • Sam Adam's keynote at ICSE'08
    on End-User Programming and End-User Software Engineering.

Conferences and Journals

Here are the currently active conferences and journals most likely to contain papers relevant to end-user software engineering, roughly in order by likelihood. There are other older resources for papers not listed here.


  1. IEEE VL/HCC Symposium
    The IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) is the premier international conference on visual research programming, or any other research aimed at helping users succeed at programming.

  2. ACM CHI
    ACM CHI provides an international, interdisciplinary forum for the exchange of ideas about the field of human-computer interaction (HCI).

    International Conference on Software Engineering is the premier software engineering conference, providing a forum for researchers, practitioners and educators to present and discuss the most recent innovations, trends, experiences and concerns in the field of software engineering.

  4. PPIG
    Psychology of Programming Interest Group, was established in 1987 in order to bring together people from diverse communities to explore common interests in the psychological aspects of programming and in the computational aspects of psychology.

  5. SoftVis
    Software visualization encompasses the development and evaluation of methods for graphically representing different aspects of software, including its structure, its abstract and concrete execution, and its evolution. The goal of this symposium is to provide a forum for researchers from different backgrounds (HCI, software engineering, programming languages, visualization, computer science education) to discuss and present original research on software visualization.

    ISSRE focuses on the theory and practice of Software Reliability Engineering. The conference scope includes techniques and practices to (1) verify and validate software, (2) estimate and predict its dependability, and (3) make it more tolerant/robust to faults.

    The objective of the International Symposium on Empirical Software Engineering is to foster collaboration by providing a forum where researchers and practitioners can report and discuss research results and practical experiences in the area of empirical and experimental software engineering.

  8. ACM UIST Conference
    UIST is the premier forum for innovations in the software and technology of human-computer interfaces.


  1. JVLC
    The Journal of Visual Languages and Computing is a forum for researchers, practitioners, and developers to exchange ideas and results for the advancement of visual languages and its implication to the art of computing. The journal publishes research papers, state-of-the-art surveys, and review articles in all aspects of visual languages, many of which are intended for end-user programmers.

  2. IEEE Transactions on Software Engineering
    A journal interested in well-defined theoretical results and empirical studies that have potential impact on the construction, analysis, or management of software. The scope of this IEEE Transactions ranges from the mechanisms through the development of principles to the application of those principles to specific environments.

  3. ACM Transactions on Software Engineering and Methodology
    The purpose of the ACM Transactions on Software Engineering and Methodology (TOSEM) is to publish original and significant results in all areas of software engineering research.

    This journal is the premier archival journal in the multidisciplinary field of human-computer interaction. The primary emphasis is on results of broad application, but the journal considers original work focused on specific domains, on special requirements, on ethical issues -- the full range of design, development, and use of interactive systems.

Data Sets

Many EUSES research projects include empirical studies. As we produce public-use data sets and similar materials from this work, we will post these below.

    Spreadsheet corpus

We have collected a large sample of spreadsheets (5607 total files, 4499 of which are unique and suitable for automated processing in Excel) that researchers can use to evaluate their methodologies and tools for creating and maintaining spreadsheets. To obtain the corpus, you must be a researcher in the field of software engineering, end-user programming, human-computer interaction, or usability.

Available at

Cite as: Marc Fisher II and Gregg Rothermel. The EUSES Spreadsheet Corpus: A shared resource for supporting experimentation with spreadsheet dependability mechanisms. In Proceedings of the 1st Workshop on End-User Software Engineering, pages 47-51, St. Louis, MO, USA, May 2005.





    A model and supporting system to support validation and reuse of short, human-readable data in end-user programmers' programs. Users create a "tope" to describe rules for recognizing and reformatting a certain kind of data, such as phone numbers, and then associate a tope with spreadsheet cells, web form textfields, web macro variables, or other fields. Values are automatically checked and transformed at runtime. Studies show that end-user programmers can validate data more quickly and accurately than with existing tools.


    WYSIWYT: The What You See Is What You Test

    Helps users test their spreadsheets while they're creating them. As a user develops a spreadsheet, he or she can also test that spreadsheet incrementally yet systematically. At any point in the process of developing the spreadsheet, the user can validate any value that he or she notices is correct.


    A debugging tool that allows programmers to ask "Why did" and "Why didn't" questions about their program's output. Programmers choose from a set of questions generated automatically via static and dynamic analyses, and the tool provides answers in terms of the runtime events that caused or prevented the desired output. In user studies of the Whyline (prototyped in the Alice programming environment), programmers using the Whyline to debug spent a factor of 8 less time debugging the same bugs than programmers without the Whyline.


    The goal of this project is to develop type systems for spreadsheets that (i) offer the advantages of static typing and are (ii) still usable by end users. To this end we have defined a unit system for spreadsheets that allows to reason about the correctness of formulas in concrete terms, which is achieved by using concrete values from the spreadsheet, for example, headers, as types (or units). A particular focus of our work is the flexibility of the unit system, both in terms of error reporting and adaptability of reasoning rules, to achieve a high acceptance among end users.



    Observing that all errors in spreadsheets result from updates or changes applied to the spreadsheet - be it during the creation of a new spreadsheet or while adapting an existing one - an obvious alternative to debugging is to prevent errors by making these update operations safe. We have created an extension to Excel, called Gencel, that is based on the concept of a spreadsheet template, which captures the essential structure of a spreadsheet and all of its future evolutions. Such a template ensures that the spreadsheet can be changed only in the the anticipated ways, so that spreadsheets evolving from templates will provably never contain any reference, range, or type errors. Gencel can help to reduce maintenance costs while at the same time it dramatically increases the level of correctness and reliability of spreadsheets.


    Graphical structured editors for code and data have many benefits over editing raw XML, but they can be difficult and time-consuming to build using modern programming languages. Citrus is a new object-oriented, interpreted language that is designed to simplify the creation of such editors, by providing first-class language support for one-way constraints, custom events and event handlers, and value restrictions and validation. We're using Citrus to implement several novel software development tools.


    A new implementation framework, implemented in Citrus, which enables the creation of a new class of highly visual, highly interactive code editors. Editors built with Barista can offer standard features such as conventional text-editing interaction techniques, immediate feedback about errors and code-completion menus. However, Barista editors can also support drag and drop interaction techniques, new types of embedded tools, and alternative views of code.


    An application framework (written in Java and using the Swing toolkit) that extends the work of the Whyline, enabling the creation of software applications that allow users to ask questions about their data and the application's state.


    Robofox is a web browser extension that enables the automation of repetitive browsing tasks such as extracting information from a web site, integrating data from different web sites, and customizing the appearance of the collected information.


    Many spreadsheet systems allow users to specify units with their data (e.g. 5 lbs.) in order to help users detect errors. Slate allows users to specify the object of measurement (e.g., 5 lbs. of apples), in addition. By intelligently propagating labels representing these objects, Slate helps users identify errors in their spreadsheets that other spreadsheet systems can't.


    Web applications are increasingly prominent in society, serving a wide variety of user needs. Engineers seeking to enhance, test, and maintain these applications must be able to understand and characterize their interfaces. Third-party programmers (professional or end user) wishing to incorporate the data provided by such services into their own applications would also benefit from such characterization when the target site does not provide adequate programmatic interfaces.
    Home    About EUSES    News & Events    Research    Education    Publications    Resources    People    Contact 
    Last Updated on October 12, 2017
    Site Designed and Developed by Htet Htet Aung
    Oregon State University University of Nebraska Lincoln University of Cambridge IBM STEM Academy Penn State University HCII of Carnegie Mellon University Drexel University University of Washington City University London National Instruments University of Tulsa