Robert J. K. Jacob
Abstract
A direct-manipulation user interface presents a set of visual representations on a display and a repertoire of manipulations that can be performed on any of them. Such representations might include screen buttons, scroll bars, spreadsheet cells, or flowchart boxes. Interaction techniques of this kind were first seen in interactive graphics systems; they are now proving effective in user interfaces for applications that are not inherently graphical. Although they are often easy to learn and use, these interfaces are also typically difficult to specify and program clearly.
Examination of direct-manipulation interfaces reveals that they have a coroutine-like structure and, despite their surface appearance, a peculiar, highly moded dialogue. This paper introduces a specification technique for direct-manipulation interfaces based on these observations. In it, each locus of dialogue is described as a separate object with a single-thread state diagram, which can be suspended and resumed, but retains state. The objects are then combined to define the overall user interface as a set of coroutines, rather than inappropriately as a single highly regular state transition diagram. An inheritance mechanism for the interaction objects is provided to avoid repetitiveness in the specifications. A prototype implementation of a user-interface management system based on this approach is described, and example specifications are given.
- 1 ANSON, E. The device model of interaction. Comput. Graph. 16, 3 (July 1982), 107-114. Google Scholar
- 2 BUXTON, W., LAMB, M. R., SHERMAN, D., AND SMITH, K.C. Towards a comprehensive user interface management system. Comput. Graph. 17, 3 (July 1983), 35-42. Google Scholar
- 3 CARDELLI, L., AND PIKE, R. Squeak: A language for communicating with mice. Comput. Graph. 19, 3 {July 1985), 199-204. Google Scholar
- 4 FELDMAN, M. B., AND ROGERS, G.T. Toward the design and development of style-independent interactive systems. In Proceedings of the A CM SIGCHI Human Factors in Computer Systems Conference (Gaithersburg, Md., Mar. 15-17). ACM, New York, 1982, pp. 111-116. Google Scholar
- 5 FOLEY, J. D., AND VAN DAM, A. Fundamentals of Interactive Computer Graphics. Addison- Wesley, Reading, Mass., 1982. Google Scholar
- 6 FOLEY, J. D., AND WALLACE, V.L. The art of graphic man-machine conversation. Proc. IEEE 62, 4 (Apr. 1974), 462-471.Google Scholar
- 7 GOLDBERG, A., AND ROBSON, D. Smalltalk-80: The Language and Its Implementation. Addison- Wesley, Reading, Mass., 1983. Google Scholar
- 8 GREEN, M. The University of Alberta user interface management system. Comput. Graph. 19, 3 (July 1985), 205-213. Google Scholar
- 9 HAVES, P. J. Executable interface definitions using form-based interface abstractions. In Advances in Human-Computer Interaction, H. R. Hartson, Ed. Ablex, Norwood, N.J., 1985, pp. 161-189.Google Scholar
- 10 HEITMEYER, C. L., LANDWEHR, C. E., AND CORNWELL, M.R. The use of quick prototypes in the military message systems project. Softw. Eng. Notes 7, 5 (Dec. 1982), 85-87. Google Scholar
- 11 HOARE, C. A. R. Communicating sequential processes. Commun. ACM 21, 8 (Aug. 1978), 666-677. Google Scholar
- 12 HUERAS, J.F. A formalization of syntax diagrams as k-deterministic language recognizers. M.S. thesis, Computer Science Dept., Univ. of California, Irvine, 1978.Google Scholar
- 13 HUTCHINS, E. L., HOLLAN, J. D., AND NORMAN, D.A. Direct manipulation interfaces. In User Centered System Design: New Perspectives in Human-Computer Interaction, D. A. Norman and S. W. Draper, Eds. Erlbaum, Hillsdale, N.J., 1986.Google Scholar
- 14 JACOB, R. J.K. Executable specifications for a human-computer interface. In Proceedings of the ACM SIGCHI Human Factors in Computer Systems Conference (Boston, Mass., Dec. 12-15). ACM, New York, 1983, pp. 28-34. Google Scholar
- 15 JACOB, R. J. K. Using formal specifications in the design of a human-computer interface. Commun. ACM 26, 4 (Apr. 1983), 259-264. Google Scholar
- 16 JACOB, R. J.K. An executable specification technique for describing human-computer interaction. In Advances in Human-Computer Interaction, H. R. Hartson, Ed. Ablex, Norwood, N.J., i985, pp. 211-242.Google Scholar
- 17 KASIK, D.J. A user interface management system. Comput. Graph. 16, 3 (July 1982), 99-106. Google Scholar
- 18 KIERAS, D., AND POLSON, P.G. A generalized transition network representation for interactive systems. In Proceedings of the A CM SIGCHI Human Factors in Computer Systems Conference (Boston, Mass., Dec. 12-15). ACM, New York, 1983, pp. 103-106. Google Scholar
- 19 LIEBERMAN, H. There's more to menu systems than meets the screen. Comput. Graph. 19, 3 (July 1985), 181-189. Google Scholar
- 20 MYERS, B. A., AND BUXTON, W. Creating highly-interactive and graphical user interfaces by demonstration. Comput. Graph. 20, 4 (Aug. 1986), 249-258. Google Scholar
- 21 OLSEN, D. R., AND DEMPSEY, E.P. SYNGRAPH: A graphical user interface generator. Comput. Graph. 17, 3 (July 1983), 42-50. Google Scholar
- 22 PARNAS, D. L. On the use of transition diagrams in the design of a user interface for an interactive computer system. In Proceedings of the 24th National ACM Conference. ACM, New York, 1969, pp. 379-385. Google Scholar
- 23 REISNER, P. Formal grammar and human factors design of an interactive graphics system. IEEE Trans. Softw. Eng. SE-7, 2 (Mar. 1981), 229-240.Google Scholar
- 24 SCHULERT, A. J., ROGERS, G. T., AND HAMILTON, J.A. ADM--A dialog manager. In Proceedings of the A CM SIGCHI Human Factors in Computer Systems Conference (San Francisco, Calif., Apr. 14-18). ACM, New York, 1985, pp. 177-183. Google Scholar
- 25 SHNEIDERMAN, B. Multi-party grammars and related.features for defining interactive systems. IEEE Trans. Syst. Man Cybern. SMC-12, 2 (Mar. 1982), 148-154.Google Scholar
- 26 SHNEIDERMAN, B. Direct manipulation: A step beyond programming languages. Computer 16, 8 (Aug. 1983), 57-69.Google Scholar
- 27 SIBERT, J. L., AND HURLEY, W.D. A prototype for a general user interface management system. Tech. Rep. GWU-IIST-84-47, Institute for Information Science and Technology, George Washington Univ., Washington, D.C., 1984.Google Scholar
- 28 SIBERT, J. L., HURLEY, W. D., AND BLESER, T.W. An object-oriented user interface management system. Comput. Graph. 20, 4 (Aug. 1986), 259-268. Google Scholar
- 29 SINGER, A. Formal methods and human factors in the design of interactive languages. Ph.D. dissertation, Computer and Information Science Dept., Univ. of Massachusetts, 1979. Google Scholar
- 30 SMITH, D. C., IRBY, C., KIMBALL, R., AND VERPLANK, B. Designing the Star user interface. Byte 7, 4 (Apr. 1982), 242-282.Google Scholar
- 31 WASSERMAN, A.I. Extending state transition diagrams for the specification of human-computer interactions. IEEE Trans. Softw. Eng. SE-11, 8 (Aug. 1985), 699-713. Google Scholar
- 32 WASSERMAN, A. I., AND SHEWMAKE, D. T. The role of prototypes in the user software engineering (USE) methodology. In Advances in Human-Computer Interaction, H. R. Hartson, Ed. Ablex, Norwood, N.J., 1985, pp. 191-209.Google Scholar
- 33 WASSERMAN, A. I., AND STINSON, S.K. A specification method for interactive information systems. In Proceedings of the Specifications of Reliable Software Conference. IEEE Press, New York, 1979, pp. 68-79. IEEE Catalog no. 79CH1401-9C.Google Scholar
- 34 WEINREB, D., AND MOON, D. Lisp Machine Manual. Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, Mass., 1981. Google Scholar
- 35 YUNTEN, T., AND HARTSON, H.R. A supervisory methodology and notation (SUPERMAN) for human-computer system development. In Advances in Human-Computer Interaction, H. R. Hartson, Ed. Ablex, Norwood, N.J., 1985, pp. 243-281.Google Scholar
Recommendations
Direct manipulation for comprehensible, predictable and controllable user interfaces
IUI '97: Proceedings of the 2nd international conference on Intelligent user interfacesCombining gestures and direct manipulation
CHI '92: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Reviews
Warren S. Hoffman
Many of us use and benefit daily from computers equipped with direct-manipulation interfaces (such as an Apple Macintosh or an MS-DOS machine equipped with Microsoft Windows). I, for one, have never stopped to think how I would specify the flow of logic of an application being developed for such an environment. The problem, of course, is that with a mouse or the equivalent and a number of windows and pull-down menus, the user can cause a wide variety of actions to take place, far more than the relatively few that exist in a traditional command-oriented application. This research is important and should be of interest to those who select a documentation method for what users refer to as window applications. A system for specifying the user interface is presented. It focuses upon the flow of logic. The language defined is based upon characteristics of direct-manipulation interfaces: :9BThey are usually made up of a number of simple dialogues. The dialogues are related to each other as a set of coroutines. The layout details can be dealt with separately. They seem to have nodes and states (despite appearances). The specification language has been implemented and tested, with some success, for some relatively simple applications. The development of these concepts to the point of generating an applications development environment, perhaps even a standard environment, would be useful.
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments