XML is frequently used or proposed for use in connection with device-independent user interface design. This document references several related activities.
- Alternate Abstract Interface Markup Language (AAIML)
- Abstract User Interface Markup Language (AUIML)
- Extensible Interface Markup Language (XIML)
- Extensible User Interface Language (XUL)
- Microsoft Extensible Application Markup Language (XAML)
- OASIS User Interface Markup Language (UIML) Specification Technical Committee
- SAP Web Dynpro Protocol
- SEESCOA Project [Software Engineering for Embedded Systems using a Component-Oriented Approach]
- USer Interface eXtensible Markup Language (UsiXML)
- User Interface Markup Language (UIML)
- W3C Device Independence Working Group (DIWG)
- W3C XForms Working Group [XForms]
- Other Projects
- Articles and Papers
- Conferences and Workshops
The V2 technical committee of the InterNational Committee for Information Technology Standards (INCITS) is developing an Alternate User Interface Access standard. The design includes an "XML-based language that would be used to communicate an abstract user interface definition for a service or device to a user's personal device which could act as a Universal Remote Console (URC)... The concept of a Universal Remote Console allows people with and without disabilities to remotely control any electronic and information technology device (target device/service), such as VCRs, copy machines, or elevators, from their personal remote control device. A standard for Universal Remote Consoles is required in order to facilitate interaction between the URC and target devices across different manufacturers."
"A key part in the Universal Remote Console specification is the definition of an XML-based language to convey a UI description from a service or device (target) to the URC. This 'Alternate Abstract Interface Markup Language' (AAIML) must be sufficiently abstract (in terms of modality independence), so that a particular URC device can render the provided UI in its own way, taking advantage of the specific interaction techniques the URC device is capable of. For example, a PDA could render the UI description by using GUI elements (visual) for output, and pointing with a stylus, as well as hand writing recognition for input; a car radio would render the same UI description auditorially with sound and synthetic speech for output, and speech recognition for input; and a braille note-taker would use its braille output and input capabilities in order to render the very same UI description tactily. Each URC device would allow access to all functions of the target, but in its own way..." [from the WWW 11 presentation.]
- "Universal Remote Console Prototyping of an Emerging XML Based Alternate User Interface Access Standard." By Gottfried Zimmermann, Gregg Vanderheiden, and Al Gilman. Presented at the The Eleventh International World Wide Web Conference, 7-11 May 2002, Honolulu, Hawaii.
- "Universal Remote Console: Prototyping The Alternate Interface Access Standard." By Gottfried Zimmermann, Gregg Vanderheiden, and Al Gilman. Presented at the "Center On Disabilities Technology and Persons With Disabilities Conference 2002."
- INCITS V2. InterNational Committee for Information Technology Standards, Technical Committee V2. - Information Technology Access Interfaces.
- V2 Support Project at Trace: Toward an Alternate User Interface Standard
"AUIML (Abstract User Interface Markup Language) is 'an XML vocabulary which has been designed to allow the intent of an interaction with a user to be defined.' This clearly contrasts with the conventional approach to user interface design, which focuses on appearance. With an intent based approach, designers are able to 'concentrate on the semantics of the interactions without having to concern themselves with which particular device type(s) need to be supported.' Being an XML vocabulary, AUIML allows device independent encoding of information. All the interaction information can be encoded once and subsequently rendered using 'device dependent rendering' so that users can actually interact with the system. AUIML is therefore 'intended to be independent of the client platform on which the user interface is rendered, the implementation language and the user interface implementation technology." [from the TUPIS 2000 paper]
- "AUIML: An XML Vocabulary for Describing User Interfaces. [Device Independent User Interfaces in XML.]" By Roland A. Merrick (IBM). 2001-05-09. "As pervasive computing evolves, and Web based e-commerce matures, application developers are faced with the task of providing solutions for simultaneous deployment on a growing number of platforms. Maintaining bespoke implementations for each target client platform will become a significant development and maintenance overhead, unless a better solution is found. IBM has been tackling this problem and in the process created Abstract User Interface Markup Language (AUIML), an XML vocabulary designed to describe the semantics of a user interaction, without making any assumptions about the presentation medium for the interaction... In 1998 IBM undertook an Advanced Technology project to develop a Device Independent Markup Language in XML. This project went through a number of phases but ended up with an XML Vocabulary called Abstract User Interface Markup Language (AUIML)..." [cache]
- "OVID to AUIML - User-Oriented Interface Modelling." By Pedro Azevedo (Software Engineer), Roland Merrick (AUIML point of contact), and Dave Roberts (OVID point of contact). IBM UK Ltd, PO Box 31, Birmingham Road, Warwick CV34 5JL, England. Paper presented at TUPIS 2000. [cache]
- "IBM's UI Unity: A Role Based, Device Independent, Administrative Console." [cache]
- "Object Modeling for User Centered Development and User Interface Design: The Wisdom Approach." By Duarte Nuno Jardim Nunes (Departamento de Matemática, Universidade da Madeira). April 2001. Dissertation. 285 pages. See the discussion of IBM's AUIML (Abstract User Interface Markup Language) on pages 203-212. Nuno is working on a set of extensions UML to support the Wisdom method and user interface design in general (Wisdom profile). [cache]
- See also: "OVID: an overview."
- Contact: Roland A. Merrick (IBM)
XIML is an XML-based "interface representation language for universal support of functionality across the entire lifecycle of a user interface: design, development, operation, management, organization, and evaluation."
"The XIML Forum is an industry organization dedicated to the research, dissemination, adoption, and standardization of the eXtensible Interface Markup Language (XIML), a comprehensive specification language for user interfaces. The Forum aims to provide a common specification and development infrastructure for user interface professionals of all types from interaction designers, to software engineers, to usability experts. It also seeks to afford human-computer interaction researchers in academia, industry, and the military a representation mechanism for the study of advanced technologies for user interfaces... XIML is available via a non-commercial, research license. Forum members use XIML for their own research purposes and make recommendations to the Forum towards the expansion and refinement of XIML. At their discretion, the members may also contribute tools and infrastructure support back to the XIML community."
- XIML Forum
- XIML Forum Members
- XIML Roadmap
- Technical papers
- "XIML: A Universal Language for User Interfaces." White paper. "This paper proposes XIML as a solution to the problem of standardizing the representation of interaction data. The paper describes key challenges facing the software industry in introducing interoperability to the field of user interfaces, and describes how XIML was built to meet those challenges. It also introduces validation samples of the language." [cache]
- "XIML: A Common Representation for Interaction Data.". By Angel Puerta and Jacob Eisenstein. Presented at the Sixth Intelligent User Interfaces Conference (IUI 2002), January 13-16, 2002, San Francisco, California, USA. "We introduce XIML (Extensible Interface Markup Language), a proposed common representation for interaction data. We claim that XIML fulfills the requirements that we have found essential for a language of its type: (1) it supports design, operation, organization, and evaluation functions, (2) it is able to relate the abstract and concrete data elements of an interface, and (3) it enables knowledge-based systems to exploit the captured data." [cache]
- "Applying Model-Based Techniques to the Development of UIs for Mobile Computers." By Jacob Eisenstein, Jean Vanderdonckt, and Angel Puerta. Presented at IUI 2001. 8 pages, 26 references.
"XUL" is glossed alternately as "XML-based User Interface Language," "XML User Interface Language," and "Extensible User Interface Language." The version 1.0 draft attests "XML User Interface Language." XUL is a "standards-based interface definition language" associated with the Mozilla XPToolkit Project.
- XML User Interface Language (XUL). Home page for Mozilla's XUL documentation,
- The Joy of XUL. Overview.
- XML User Interface Language (XUL) 1.0
- XRE (XUL Runtime Environment)
- XUL Tutorial. By Neil Deakin.
- XUL Developer Central
- "Getting Started." Sample Chapter 2 online. Introduction to XUL. In Creating Applications with Mozilla. By David Boswell, Brian King, Ian Oeschger, Pete Collins, and Eric Murphy. O'Reilly, September 2002.
- XULUX project (Apache) will "create an Object Model for the XUL language, so that we can process it easily with tools such as Betwixt or others; will create a XUL Jelly Tag Library, to transform a native XUL file into whatever..."
- Luxor - XML User Interface Language (XUL) Toolkit
- The jXUL project is an open source project to integrate the XUL language into the Java Platform.
- "Fooling with XUL." By Edd Dumbill. From XML.com. March 15, 2000.
- "Extensible User Interface Language (XUL)" - Local reference page.
XAML (Extensible Application Markup Language) is a code-name for the Microsoft 'Longhorn' Markup Language. "'Longhorn' application development raises abstraction to the level of a declarative programming model. To facilitate programming for this model a new XML-based declarative programming language (XAML) is being developed. XAML is based on Extensible Markup Language (XML) and enables developers to specify a hierarchy of objects with a set of properties and logic. The primary use of XAML is for 'Longhorn' client applications but you can use it to specify a hierarchy of any common language runtime objects. XAML enables you to create a UI without using code. You can create quite elaborate documents entirely in markup using controls, text, images, shapes and so forth... [excerpted from Microsoft's XAML Overview]"
- Create Real Apps Using New Code and Markup Model
- "Microsoft Extensible Application Markup Language (XAML)."
The OASIS TC will "develop a specification for an abstract meta-language that can provide a canonical XML representation of any user interface (UI). The language should be capable of specifying the requirements, design, and implementation of any UI. The committee will use as a starting point the UIML v3.0 specification."
- "OASIS Members Propose TC for User Interface Markup Language (UIML)."
- Announcement for UIML Discussion List . July 08, 2002.
- TC website
- General mailing list archive
- Comment mailing list archive
- 'UIML-Discuss' list [ephemeral]
- "The Relationship of the UIML 3.1 Spec. to Other Standards/Working Groups." Version 01.04 (Version 01, Draft 04, update based on Italy Workshop). July 26, 2004. 28 pages. PDF generated from the .DOC (Word) format posted by James Helms. "UIML is an answer to the question of what a declarative language would look like that could provide a canonical representation of any user interface (UI) suitable for multi-platform, multi-lingual, and multi-modal UIs. This document describes the influences from W3C and other complimentary efforts on UIML, and comments on how UIML fits into these various technologies." Description from the OASIS TC minutes: "...an update to the comparison of UIML to other existing standards and UI technologies. The update consisted of the addition of several technologies including UsiXML, AUI, and a catalog of the languages presented at the Workshop on XML user interface definition languages at the AVI 2004 conference..." Note that the diagram from Dave Raggett's document (cited in document section 1.5, "How UIML Fits W3C Architecture Today") was inaccessible to the conversion process creating this PDF format. See this diagram in slide #3 in "Device-Independence with UIML(User Interface Markup Language)," [.ppt] by Marc Abrams, presented at the W3C Workshop on Web Device Independent Authoring, October 2000.
From the SDN web site: "Developed to enable rich clients for sophisticated business applications, Web Dynpro is the next generation SAP GUI technology. The Web Dynpro Protocol provides efficient and adaptable HTTP-based exchange of UI (user interface) instances between client and server. It can be used to access Web Dynpro applications from different clients, such as the recently issued beta release of the SAP Web Dynpro client for Windows."
Model View Controller Programming Model: "Using Web Dynpro technology enables a clear separation of processing logic and presentation logic. A Web Dynpro application runs on the front end and has local or remote access to the back end system using a service. This means that the display logic is contained in the Web Dynpro application, while the business logic and the persistence of the business objects run in the back end system. Every Web Dynpro application is structured according to the Model View Controller programming model: (1) The Web Dynpro model forms the interface to the back-end system and thus enables the Web Dynpro application to access the data. (2) The view is the central logical element of the Web Dynpro application for the layout and processing of the presentation logic. (3) The controller is responsible for the data and control flow in the Web Dynpro application. There are different controller types. Since the controller contexts have the same structure, you can process every controller type in the same way using the same tool..." [from the SAP Library]
- An Introduction to the Web Dynpro Protocol. By Dipak Chopra (Standards Architect, SAP NetWeaver Industry Standards). SAP Developer Network (SDN). Copyright (c) 2005 SAP AG. 11 pages.
- Web Dynpro for SAP NetWeaver technology. From the SAP Library.
- "Highway 1: From Dynpro To Web Dynpro. SAP's New Model-Driven Programming Model Revolutionizes the Development of User Interfaces." By Karin Schattka. From JAX Magazine (June 14, 2004). "The Eclipse-based SAP NetWeaver Developer Studio offers intuitive, graphic tools for developing client applications and UIs. These tools visually support model-based development. This model-driven approach minimizes the number of lines of code that need to be written, and enables easy reuse of the UI components. Web Dynpro is SAP's programming model for developing user interfaces (UIs) for professional business applications. It is based on the flexible Model View Controller, which ensures that the UI (the presentation layer, that is) is separated from the business logic of the backend system. This architecture is already evident in the SAP NetWeaver Developer Studio — an integrated development environment."
The goal of SEESCOA "is to adapt the CBD (Component Based Development) technology that was developed for mainstream software to the needs of embedded systems. The research consortium consists of four partners from different Flemish universities, each bringing their own expertise in this project: (1) Coordinator, Research group DistriNet, Department of Computer Science, KULeuven; (2) Research groups PROG/SSEL, Department of Computer Science, VUB; (3) Research group Expertisecentrum Digitale Media (EDM), LUC; (4) Research group PARIS, Department ELIS, Universiteit Gent. The consortium is backed up by a user group consisting of the following members: Agfa-Gevaert, Alcatel, Barco, Imec, Philips, Siemens Atea, All participants are part of the Vlaams Software Platform (VSP), a non-profit organization grouping companies and research centers from universities and private institutions in the ICT sector."
- Project website
- Project description
- Project publications. Note that several papers report on the use of XML-based UI description languages.
"UsiXML (USer Interface eXtensible Markup Language) is a XML-compliant markup language that describes the UI for multiple contexts of use such as Character User Interfaces (CUIs), Graphical User Interfaces (GUIs), Auditory User Interfaces, and Multimodal User Interfaces. In other words, interactive applications with different types of interaction techniques, modalities of use, and computing platforms can be described in a way that preserves the design independently from peculiar characteristics of physical computing platform... UsiXML is defined in a set of XML schemas. Each schema corresponds to one of the models in the scope of the language...
UsiXML consists of a User Interface Description Language (UIDL), that is a declarative language capturing the essence of what a UI is or should be independently of physical characteristics. It describes at a high level of abstraction the constituting elements of the UI of an application: widgets, controls, containers, modalities, interaction techniques, etc. UsiXML allows cross-toolkit development of interactive application. A UI of any UsiXML-compliant application runs in all toolkits that implement it: compilers and interpreters...
UsiXML supports device independance: a UI can be described in a way that remains autonomous with respect to the devices used in the interactions such as mouse, screen, keyboard, voice recognition system, etc. In case of need, a reference to a particular device can be incorporated. UsiXML supports platform independance: a UI can be described in a way that remains autonomous with respect to the various computing platforms, such as mobile phone, Pocket PC, Tablet PC, laptop, desktop, etc. In case of need, a reference to a particular computing platform can be incorporated...
UsiXML supports modality independance: a UI can be described in a way that remains independent of any interaction modality such as graphical interaction, vocal interaction, 3D interaction, or haptics. It allows reuse of elements previously described in anterior UIs to compose a UI in new applications..." [from the UsiXML Overview page]
GrafiXML "is a graphical tool to draw user interfaces. You can save those UI in several formats like java or xhtml but the principal way is to save it in UsiXML, a xml user interface description. GrafiXML is released under the terms of the Apache License, Version 2.0.
- USer Interface eXtensible Markup Language web site
- About UsiXML
- USIXML draft documentation
- "USIXML: a Language Supporting Multi-Path Development of User Interfaces." By Quentin Limbourg, Jean Vanderdonckt, Benjamin Michotte, Laurent Bouillon, Víctor M. López Jaquero. In Proceedings of 9th IFIP Working Conference on Engineering for Human-Computer Interaction [jointly with 11th Int. Workshop on Design, Specification, and Verification of Interactive Systems, EHCI-DSVIS 2004, Hamburg, July 11-13, 2004). 30 slides.
- "USIXML: A User Interface Description Language for Context-Sensitive User Interfaces." By Quentin Limbourg, Jean Vanderdonckt, Benjamin Michotte, Laurent Bouillon, Murielle Florins, and Daniela Trevisan. With 25 references. 25 May 2004. Paper presented at "Developing User Interfaces with XML: Advances on User Interface Description Languages," a Workshop Organised at Advanced Visual Interfaces 2004. See Workshop Accepted Papers. "This paper presents USIXML (USer Interface eXtensible Markup Language), a User Interface Description Language aimed at describing user interfaces with various levels of details and abstractions, depending on the context of use. USIXML supports a family of user interfaces such as, but not limited to: device-independent, platform-independent, modality independent, and ultimately context-independent. This paper consequently details how context-sensitive user interfaces may be specified and produced from the USIXML specifications. USIXML allows specifying multiple models involved in user interface design such as: task, domain, presentation, dialog, and context of use, which is in turn decomposed into user, platform, and environment. These models are structured according to the four layers of the Cameleon framework: task and concepts, abstract user interface, concrete user interface, and final user interface. To support relationships between these models, a model for inter-model mapping is also introduced that cover forward and reverse engineering as well as translation from one context of use to another..." [cache]
- See also: GrafiXML, including the development version
"UIML is an XML-compliant language, so it looks a lot like HTML. UIML is designed to serve as a single language which permits creation of user interfaces for any device, any target language (e.g., Java, C, WML), and any operating system on the device. However, UIML is not a silver bullet: The UI designer must still design separate UIs for each device, and then represent those designs in UIML. UIML does not magically create multiple UIs from a single description; instead it is a language in which those multiple UIs can be recorded... UIML is designed to be a relatively simple markup language. In contrast to Dynamic HTML (HTML 4) or XSL, which have so many tags that few people in the work know all the tags, UIML has a little over two dozen tags. UIML is also independent of any user interface metaphor, such as graphical user interfaces or voice-response. UIML is designed to permit efficient compilation from UIML to any target language (e.g., Java, C, HTML, WML, VoiceXML)." UIML describes the appearance of a UI, the user interaction with the UI, and how the UI is connected to the application logic.
- See the OASIS User Interface Markup Language (UIML) Specification Technical Committee
- UIML homepage
- Introducing UIML
- UIML Tutorials
- UIML 3.0 specification. February 12, 2002. "A modest refinement of the UIML 2.0 specification."
- UIML 3.0 PDF [cache]
- "UIML: A Device-Independent User Interface Markup Language." By Constantinos Phanouriou. Dissertation.
- First European UIML Conference
- "Comparing Transcoding Tools for Use with a Generic User Interface Format." By Johan Plomp, Robbie Schaefer, and Wolfgang Mueller. Presented at Extreme Markup Languages 2002. "Small office printers, routers, etc. already present HTML-driven user interfaces via remote generic user-interaction devices connected through digital networks. Entertainment, air conditioning, and other everyday systems are expected to follow suit. HTML is suboptimal for use with small displays (such as wireless telephones) and nondisplay environments (such as speech-only). In an ITEA-VHE (Middleware for Virtual Home Environments) project, three transcoding tools, UIML's 'peers section' (extended), RDL/TT, and XSLT, were each applied to the problem of transforming a generic user interface format (UIML) to two target formats, HTML and VoiceXML. All three approaches worked; each appears to offer advantages in specific circumstances." Also in PDF format.
- "UIML and XForms." By Marc Abrams. Posted to 'firstname.lastname@example.org' by James Helms on August 26, 2002.
- "UIML: An XML Language for Building Device-Independent User Interfaces." By Marc Abrams. XML 1999, Philadelphia.
- "UIML: An Appliance-Independent XML User Interface Language." By Marc Abrams, Constantinos Phanouriou, Alan L. Batongbacal, Stephen M. Williams, and Jonathan E. Shuster (Harmonia). Presented at The Eighth International World Wide Web Conference, Toronto Convention Centre, Toronto, Canada. May 11-14, 1999.
- "User Interface Markup Language (UIML)" - Local reference page.
The W3C Device Independence Working Group (DIWG) is part of the Interaction Domain. "There are significant efforts to integrate Web technologies into various devices (e.g., mobile, TV sets) other than the traditional Web access equipment such as PCs. However, we are faced with the possibility that services for those devices may not interoperate with each other or with the existing Web. That would not only cause fragmentation of the Web space, but also make Web device independent authoring impossible... The Device Independence Activity is newly created and merges the "Mobile Access Activity" and "TV and the Web Activity" to facilitate interchange in the interest of device independent Web access and authoring..."
- Device Independence Working Group Activity
- Device Independence Working Group Charter
- "Authoring Challenges for Device Independence." W3C Working Draft 18-October-2002.
- Delivery Context Overview for Device Independence
- Device Independence Principles. W3C Working Draft 18-September-2001,
- DI mailing list archives
- W3C Workshop on Device Independent Authoring Techniques. 25-26 September 2002. SAP University, St. Leon-Rot, Germany. See the list of submissions, the agenda, and notes.
- W3C Delivery Context Workshop. 4-5 March 2002, INRIA Sophia-Antipolis, France.
"Key Goals of XForms: (1) Support for handheld, television, and desktop browsers, plus printers and scanners; (2) Richer user interface to meet the needs of business, consumer and device control applications; (3) Decoupled data, logic and presentation; (4) Improved internationalization; (5) Support for structured form data; (6) Advanced forms logic; (7) Multiple forms per page, and pages per form; (8) Suspend and Resume support; (9) Seamless integration with other XML tag sets."
- XForms website
- XForms WG Charter
- XForms 1.0 W3C Working Draft 21-August-2002. Note also XML Events in relation to the XForms Processing Model.
- XForms Requirements
- XForms mailing list archives
- XMLForm Framework. Inspired by Apache Jakarta Struts and W3C XForms. Supports reuse of Cocoon Actions for UI centric web applications (HTML, WML, UXL, VoiceML).
Provisionally noted as relevant:
- Sweet Project 'XSWT'. Approximately XSWT is "an XML-based markup language for describing SWT layouts... The Standard Widget Toolkit (SWT) is a cross-platform, portable, open source graphical user interface toolkit for the Java programming language. It is the toolkit that was used to create the Eclipse Java integrated development environment (IDE)."
- Web Services User Interface (WSUI) Initiative. See Web Service User Interface (WSUI) 1.0. Working Draft. July 26, 2002. XML Schema in text view and graphical documentation. Local references in "Web Services User Interface (WSUI) Initiative."
- Personal Universal Controller: Appliance Specification Language Documentation. From the Pebbles project. Documention by Jeffrey Nichols and Mathilde Pignol.
- Universal Speech Interface (Speech Graffiti) Project. CMU. See "Towards Every-Citizen's Speech Interface: An Application Generator for Speech Interfaces to Databases." 2002.
- Glade. Free (GPL) user interface builder for GTK+ and GNOME; uses XML interface description files
- Thinlet is a GUI toolkit; it separates the graphic presentation (described in an XML file) and the application methods (written as Java code).
- Java Gui Builder - "The Java Gui Builder program is designed to decouple the GUI building code from the rest of the application code, without hand-writing code. It allows one to describe the layout of windows and controls using an XML file. A full DTD was written to allow on-the-fly validation. Using an XML file to describe GUI components allows users the flexibility to rewrite their windows so that they suit their needs, without opening up the innards of the program to the users..." [pp]
- MotLib.NET is a library of C# classes and debugging tools that bring many MFC goodies to Windows.Forms. MotLib.NET implements the amazing MGL ('miggle', mot's gui language), a baby XUL-like GUI definition langauge that lets you code your UI in XML..." See "Streamline Your Code and Simplify Localization Using an XML-Based GUI Language Parser."
- XUP - Extensible User Interface Protocol W3C Note 28-May-2002.
- XML Events: An Events Syntax for XML W3C Working Draft 12-August-2002
- [July 14, 2001] XGL DTD. From Tom Van Laerhoven. 2001-07-14 or later. This DTD is based upon the official XGL specifications. The XGL specs don't force an order on the tags. In order to keep the DTD of reasonable size (no order in the content model means permutating the tags), we used the same order as used by the 3DStoXGL converter. This slightly differs from the order used by whatever program produced the 'greencubes.xgl' example, in the following way: (1) The converter puts the BACKGROUND tag first within the WORLD tag, in the greencubes example this tag is placed at the end. (2) In a MAT tag, the converter starts with the optional ALPHA and EMISS tags. Extension Ref Types are not included in this DTD for obvious reasons..." See the XGL File Format Specification. [cache]
[May 25, 2004] Workshop on Developing User Interfaces with XML: Advances on User Interface Description Languages." Part of Advanced Visual Interfaces 2004, Gallipoli (Lecce), Italy. From the Call for papers: "This workshop aims at bringing together people working on/with XML-based UIDLs and any person interested in describing and/or developing UIs with XML. UIDL developers, designers and users are all among the potential participants. It is expected that from this exchange of discussion, a better mutual understanding will be raised to enable people to work more closely on some UIDLs rather than reproducing the same efforts in multiple locations. This aim will be supported by open questions selected by the organizers... For many years, Human-Computer Interaction (HCI) has witnessed a perennial race for the ultimate User Interface Description Language (UIDL) that would ideally capture the essence of what a UI could be or should be. Many people and teams from both the industry and the academic worlds have introduced UIDLs during history. Early UIDLs appeared when people firstly attempted to specify the look and feel of a UI. This attempt was particularly vivid when software appeared that produces UIs for multiple computing platforms, thus ensuring some form of portability of UIs. Then, the activity almost vanished until XML was considered widely enough to initiate a new family of UIDLs. Such UIDLs may pursue various goals: (1) nsuring portability of UIs from one computing platform to another while preserving consistency; (2) Capturing UI requirements for an abstract definition that remains stable over time; (3) Improving the reusability of UI design; (4) Making one UI design for multiple devices, platforms, appliances; this goal is often referred to as the device-independence rendering; (5) Supporting extensibility and adaptability of UI; (6) Using a UI description to enable automated generation of UI code..." See the links to the Workshop Accepted Papers.
[October 27, 2003] "Code Name Avalon: Create Real Apps Using New Code and Markup Model." By Charles Petzold. In Microsoft MSDN Magazine Volume 19 Number 1 (January 2004). "The new presentation subsystem in the next version of Windows, code-named 'Longhorn,' offers powerful new capabilities to developers. This new subsystem, code-named 'Avalon,' allows developers to take advantage of its capabilities through a language -- Extensible Application Markup Language (code-named 'XAML'), as well as through modern object-oriented programming languages such as C#. Because most applications written to Avalon will probably be a mix of XAML and programming code, this article discusses XAML tags used to control page layout along with the procedural code written to respond to events. Avalon consists mostly of a new collection of classes added to the .NET Framework... In addition, Avalon also defines a new markup language you can use in Longhorn that's code-named 'XAML.' You use XAML much like HTML to define a layout of text, images, and controls. Being based on XML, XAML has stricter and much less ambiguous syntax than HTML. It is expected that most XAML will be machine-generated by visual design programs, but it may be a good learning experience to hand-write your own XAML (initially). Most applications written to Avalon will probably contain both program code and XAML. You'll use the XAML for defining the initial visual interface of your application, and write code for doing everything else. You can embed the program code directly in XAML or keep it in a separate file. Everything you can do in XAML you can also do in program code, so it's possible to write a program without using any XAML at all. The reverse is not true, however; there are many tasks that can only be done in program code, so only the simplest applications will consist entirely of XAML. Here's a little snippet of some XAML: <Button Background='LightSeaGreen' FontSize='24pt'>Calculate</Button>... XAML has very intimate ties with the Avalon class library: every element type you can use in XAML is actually a class and, specifically, a descendent of the UIElement or ContentElement classes declared in the MSAvalon.Windows namespace. Among the descendents of UIElement is Control, from which are descended all the common user-interface controls such as buttons, scroll bars, list boxes, edit fields, and so forth. Classes derived from ContentElement include Bold and Italic. Of course, the modern programmer wants to know: can I use my own classes as XAML elements? And the answer is: of course you can. XAML is called the Extensible Application Markup Language for a reason. Any class that has a public parameterless constructor and settable properties can be used in XAML... The hierarchical nature of XAML is one reason why a markup language makes more sense than programming code for defining a visual interface. The markup can mimic the hierarchy with nesting and indentation. In fact, when Windows 1.0 was first introduced in 1985, programmers used a text resource script to define the hierarchical structure of their menus and dialog boxes. At the time, the hierarchy only went one level deep, but it was a start. An XAML file is just the resource script... Avalon and XAML represent a departure from Windows-based application programming of the past. In many ways, designing your application's UI will be easier than it used to be and deploying it will be a snap. With a lightweight XAML markup for UI definition, Longhorn-based applications are the obvious next step in the convergence of the Web and desktop programming models, combining the best of both approaches..."
[June 2003] "A Review of XML-Compliant User Interface Description Languages." By Nathalie Souchon and Jean Vanderdonckt. In Proceedings of 10th International Conference on Design, Specification, and Verification of Interactive Systems DSV-IS 2003 (Madeira, 4-6 June 2003) pages 377-391. "A review of XML-compliant user interface description language is produced that compares a significant selection of various languages addressing different goals, such as multiplatform-user interfaces, device-independence, context delivery, and user interfaces virtually defined. There has been a long history and tradition to attempt to capture the essence of user interfaces at various levels of abstraction for different purposes. The return of this question today gained more attraction, along with the dissimination of XML markup languages, and gives birth to many proposal for a new user interface description language. Consequently, there is a need to conduct an in-depth analysis of features that make all these proposal discriminant and appropriate for significant subset of such languages based on an analysis grid and user interfaces that we tried to implement across these languages..." [alt URL, cache]
[October 31, 2002] "SWT, XML Put True Cross-platform GUIs Within Reach." By A. Russell Jones (DevX Executive Editor). From DevX XML Zone. ['The reality of a single cross-language, cross-platform GUI interface programming model is in sight, based on an XML description language supported by fast native runtimes. Will Mozilla, Eclipse, or someone else step in and complete the last mile that gives all developers a common way to design and program fast cross-platform user interfaces?'] "... if Mozilla were to take on the project of connecting XUL to SWT, developers could design cross-platform GUIs in a completely language-independent fashion. For example, by designing an application interface in XUL and using the appropriate platform-specific SWT implementation to activate it in the language of your choice, the only thing you'd need to change from one platform to another would be the SWT runtime. It's a powerful combination and opens a window of opportunity for any organization with the vision to implement it... Developers don't need multiple programming models for common widgets, such as windows, list controls, outline controls, drop-down menus, textboxes, and buttons. Widgets act much the same between platforms and the differences that do exist lie almost exclusively in the look and feel (how they're rendered and the events they raise). In short, the enormous efforts expended by developers who learn more than one GUI programming model interface are largely wasted, because the results are visually and functionally nearly identical yet remain quite different from a development point of view. I fully expect that an XML-based forms description language, backed by platform-specific, high-performance runtimes will eventually capture both the market and developers' mindshare..."
[June 2002] "Migratable User Interface Descriptions in Component-Based Development." By Kris Luyten, Chris Vandervelpen, and Karin Coninx (Expertise Centre for Digital Media, Limburgs Universitair Centrum, Belgium). Presented at the Ninth International Workshop on Design, Specification, and Verification of Interactive Systems (DSV-IS 2002), Rostock, Germany, June 12-14, 2002. 15 pages, with 14 references. "In this paper we describe how a component-based approach can be combined with a user interface (UI) description language to get more exible and adaptable UIs for embedded systems and mobile computing devices. We envision a new approach for building adaptable user interfaces for embedded systems, which can migrate from one device to another. Adaptability to the device constraints is especially important for adding reusability and extensibility to UIs for embedded systems: this way they are ready to keep pace with new technologies... To describe a UI on a suffciently abstract level the Extensible Markup Language (XML) is used. Listing 1.1 provides an example of how a UI can be described in XML. A list of advantages is given in ["An XML-based runtime user interface description language for mobile computing devices," Proceedings of the Eight Workshop of Design, Specification and Verification of Interactive Systems, June 2001]. One of the major advantages is that XML does not force any level of abstraction, so this level can be adapted to the requirements of the situation. Note that an XML document can be presented as a tree which turns out to be a great advantage in our approach. There are other approaches for describing User Interfaces, but we believe that an XML-based description offers the best solution in our component-based approach because of it heavily relies on hierarchical structures. [XML syntax goals: platform independent, declarative, consistency, constraints, extensible, reusable, transformations]... the component-oriented approach suggested in this paper has several advantages for developers of embedded systems and mobile computing devices in particular. It is (1) Flexible: changing the UI can be done by another renderer component or letting components provide another UI description; (2) Reusable: providing a high level description of the UI related to the functionality a component offers, allows easier reusability of previously designed UIs in contrast with hard-coded UIs; (3) Adaptable: by abstracting the UI, device constraints can be taken into account when rendering the concrete UI..." See also the presentation slides. [cache]
[May 17, 2002] "Specifying User Interfaces for Runtime Modal Independent Migration." By Kris Luyten, Tom Van Laerhoven, Karin ConinxM, and Frank Van Reeth. Paper presented at CADUI 2002, Fourth International Conference on Computer-Aided Design of User Interfaces, Université de Valenciennes, France, May 15-17, 2002. 12 pages, with 18 references. "In this paper an approach will be presented which provides a uniform interface to such services, without any dependence on modality, platform or programming language. Through the usage of general user interface descriptions, presented in XML, and converted using XSLT, a uniform framework is presented for runtime migration of user interfaces. As a consequence, future services will become easily extensible for all kinds of devices and modalities. An implementation serving as a proof of concept, a runtime conversion of a joystick in a 3D virtual environment into a 2D dialog-based user interface, is developed... At runtime a user interface description is generated by means of XML as a description language. The XML description provides an abstraction of the user interface using Abstract Interaction Objects (AIO) which can be mapped onto Concrete Interaction Objects (CIO)... Our work does not aim to extract a Task-based description of the User Interface but merely tries to abstract the 'contents' of the user interface, not the presentation. Although we acknowledge the idea of splitting the interface model into a User-task model, a domain model, a user model, a presentation model and a dialog model for getting a better mapping of AIOs on CIOs, we do not consider it to be useful when focusing on runtime migration of existing user interfaces. A working user interface is 'serialized' into an XML description at runtime and this description can be transported to another system using for example the Internet or an infrared communication protocol. Once arrived at the target system the XML document can be parsed and converted in a working user interface ('deserialized'). An advantage of this approach can be found in the abstraction of the original user interface that the XML document provides. While parsing the XML document that contains an abstraction of the user interface, the renderer of the target platform is free to choose other ways to present the same functionality in the user interface... As an abstraction of the user interface we have defined a DTD. This DTD restricts the possible elements and ordering used in the XML document. Our current DTD is inspired by [Müller/Forbrig/Cap 'Model-Based User Interface Design Using Markup Concepts'] and enriched by a subset of interactors... We are planning to replace it with an equivalent XML Schema as soon as there are more mature tools available to use this... The transformation of the original user interface description into a higher level description is realized using the XSLT. Every system or user interface toolkit has to define a conversion providing the appropriate XSLT. Once the higher level description is produced a system specific XML document for the target system has to be produced. For every system a XSLT is defined which maps the AIOs defined in the abstract user interface description to CIOs for that particular system..." See also the presentation slides. [cache]
[May 2002] "Building Multi-Platform User Interfaces with UIML." By Mir Farooq Ali, Manuel A. Pérez-Quiñones, Marc Abrams, and Eric Shell. Published in Proceedings of CADUI 2002, May 2002. 12 pages, with 21 references. "There has been a widespread emergence of computing devices in the past few years that go beyond the capabilities of traditional desktop computers. However, users want to use the same kinds of applications and access the same data and information on these appliances that they can access on their desktop computers. The user interfaces for these platforms go beyond the traditional interaction metaphors. It is a challenge to build User Interfaces (UIs) for these devices of differing capabilities that allow the end users to perform the same kinds of tasks. The User Interface Markup Language (UIML) is an XML-based language that allows the canonical description of UIs for different platforms. We describe the key aspects of our approach that makes UIML successful in building multi-platform UIs, namely the division in the representation of a UI, the use of a generic vocabulary, a process of transformations and an integrated development environment specifically designed for transformation-based UI development. Finally we describe the initial details of a multi-step usability engineering process for building multiplatform UI using UIML... [We show] some of our research in extending and utilizing UIML to generate multi-platform user interfaces. We are using a single language, UIML, to provide the multi-platform development support needed. This language is extended via the use of a logical model, alternate vocabularies and transformation algorithms. Our approach allows the developer to build a single specification for a family of devices. UIML and its associated tools transform this single representation to multiple platformspecific representations that can then be rendered in each device. We have presented our current research in extending UIML to allow building of interfaces for very-different platforms, such as VoiceXML, WML and desktop computers." [cache]
[November 21, 2001] "XML en User Interfaces." By Kris Luyten.
[June 2001] "Model-Based User Interface Design Using Markup Concepts." By Andreas Müller [WWW], Peter Forbrig, and Clemens Cap (University of Rostock Department of Computer Science, Rostock, Germany). In Proceedings of the Eight Workshop of Design, Specification and Verification of Interactive Systems, June 13-15, 2001, University of Glasgow. "In the field of model-based development of interactive systems, several approaches have been proposed to integrate task and object knowledge into the development process and its underlying representations. This paper follows such an approach with a special focus on mobile devices. It presents a concept of device independent user interface design based on the XML-technology. The concept is applied to an e-commerce example... There are already some approaches to specify user interfaces by markup languages. There is for instance UIML, a XML-based language for describing user interfaces. It comes with different rendering programs allowing the generation of user interfaces on different platforms. Up to now the generation for WML, Java AWT and swing are available. A special renderer for each target-device is needed. This is an important disadvantage The flexibility for selecting target-device features is limited because of a lack of a mapping concept. XUL has its focus on window-based graphical user interfaces by XML. Its disadvantage is the limitation to such graphical user interfaces. It is not applicable to interfaces of small mobile devices. There is no abstraction of interaction functionality available. The device-independent specification of user interfaces was not planned... The following requirements should be fulfilled by a markup language specifying user interfaces: device independence, flexibility in representing user interfaces depending on the contexts, ability to specify virtual and physical user interfaces, localisation, extensibility of abstractions (abstract UIOs describing interaction behaviour), and extensibility of concrete UIOs (UIOs of specific representations). The following paragraphs will show a specific way of using XML in the proposed manner... The current work includes the specification of XML-based languages describing different abstraction of a user interface for mobile devices. A concept for the mapping process from abstract UIOs to concrete UIOs was developed. A XML language was developed which allows the specification of the features of the user interface of mobile devices. The transformation process of the user interface is based on specifications written in this language. In an interactive design process based on an abstract device dependent abstract interaction model and some design rules a more refined xsl-description of the user interfaces is developed, which allows the generation of specific user interfaces. This concept is already evaluated by several examples. A tool supporting the mapping and design process is under development. A first prototype is already available, which delivers already promising results. Up to now this tool is not integrated in previous phases of the design process. The abstract interaction model has to be edited manually. This will be omitted in the near future. Our experiments show that XML is a promising technology for the platform independent generation of user interfaces..." [cache; other papers]
[June 2001] "An XML-based Runtime User Interface Description Language for Mobile Computing Devices." By Kris Luyten and Karin Coninx (Expertise Centre for Digital Media, Limburgs Universitair Centrum, Belgium). Pages 17-29 (with 28 references) in Proceedings of the Eight Workshop of Design, Specification and Verification of Interactive Systems, June 13-15, 2001, University of Glasgow. "In a time where mobile computing devices and embedded systems gain importance, too much time is spent to reinventing user interfaces for each new device. To enhance future extensibility and reusability of systems and their user interfaces we propose a runtime user interface description language, which can cope with constraints found in embedded systems and mobile computing devices. XML seems to be a suitable tool to do this, when combined with Java. Following the evolution of Java towards XML, it is logical to introduce the concept applied to mobile computing devices and embedded systems... Because of the evolving market towards mobile computing devices and embedded systems, a more general approach for describing a user interface for an embedded system or mobile computing device is necessary. In search of a notation for describing such a user interface it should satisfy the following requirements: (1) Platform independent: because of the heterogeneity of embedded systems, a user interface designer should be allowed to design without having to worry about the system on which the interface will be used; (2) Declarative: describing a user interface asks for a certain level of expressiveness for describing human-computer interaction; (3) Consistent: the notation should offer consistency of the user interface among different environments and systems; (4) Unconventional I/O : embedded and mobile computing devices are less conservative in input and output devices. For example: while 'normal' desktop computers have a mouse and a keyboard, this is not a requirement for a mobile device, which could very well have a touch-screen and speech recognition; (5) Rapid prototyping: in a highly competitive market, such as mobile devices, developers and designers want to tailor the software towards the users or user groups. A user interface notation should allow rapid prototyping to get the users involved in the development process sooner; (6) Constraint sensitive: because of the constraints embedded systems are coping with, the designers must be able to specify the constraints, or have the system automatically generate them; (7) Easily extensible: we want to extend our user interface with extra functionality, without starting from scratch; (6) Reusability: when a family of products is evolving, we want to reuse the design for the old devices in an optimal way... [cache, Postscript]
[June 2001] "Task Modelling for Context-Sensitive User Interfaces." By Costin Pribeanu, Quentin Limbourg, and Jean Vanderdonckt. In Proceedings of the Eight Workshop of Design, Specification and Verification of Interactive Systems, June 13-15, 2001, University of Glasgow. "With the explosion of devices, computing platforms, contextual conditions, user interfaces become more confronted to a need to be adapted to multiple configurations of the context of use... The paper presents a method for modelling context-sensitive tasks with the aid of the ConcurTaskTree notation. Several approaches were considered and analysed to come up with a final approach combining clear separation of context-sensitive part, non-context-sensitive part, and context decision tree, while preserving the hierarchical structure of the model, as expressed in CTT. In order to make a distinction between the normal choice operator and the ones used for the decision tree, we decided to introduce a marked structured annotation in the tasks representing the nodes of the decision tree. Each such task is then augmented with the current value (e.g., low/moderate/high) of the contextual condition considered at that time (e.g., what is the available bandwidth). We then choose to export the resulting CTT model into an XML-compliant file thanks to the facility provided by CTTE. This XML file is then transformed into an XIML-compliant file using XSLT transformations. In particular, the arcs and nodes used for the decision tree are isolated in dedicated tags with the following structure [XML document with root <decision-tree>] This XIML file can then be exploited by UI derivation algorithms to progressively derive a generic UI, then a specific UI, and a code generation/interpretation... The study of this derivation process is the subjects of another submission, where the context-sensitive task model, along with its domain model are separately or simultaneously considered to derive presentation and dialog models, that are in turn refined into a specific one after considering the user model... Many [other] efforts have already been deployed to cover the problems raised by context-sensitivity, often starting with change of computing-platform or cross-computing platforms changes. This is a crucial approach because some UIs are particularly tailored to specific computing platforms, sometimes with Web-based add-ins. Equally important are the UIs which are specialised and adapted to a series of computing platforms, for instance different Web-appliances... The Phone.com paradigm is intended to describe a UI dialog that will be rendered afterwards for fixed or mobile phone. It is therefore aimed at supporting phone-based applications, which is not the case of the GeoWorks solution: it supports the same idea for multiple platforms. Sisl uses an event-based model of services for platform independence. The User Interface Markup Language (UIML) is an XML-compliant language that allows a declarative description of a user interface in a device-independent manner. To create a UI, one writes a UIML document, which includes presentation style appropriate for devices on which the UI will be deployed. UIML is then automatically mapped to a language used by the target device of a particular computing platform, such as HTML, Java/JFC, WML, HTML, PalmOS and VoiceXML. The Abstract User Interface Markup Language (AUIML) is an XML vocabulary designed to allow the intent of an interaction with a user to be defined. It allows device independent encoding of information. All the interaction information can be encoded once and subsequently rendered using device dependent rendering. The Extensible User-interface Language (XUL) is an XML-based UI specification language specifically designed for network applications like browsers, mailers, etc. Thanks to the portability of its rendering engine Gecko and the use of its XPCOM Technology, a XUL definition can be rendered on several computing platforms. The eXtensible Interface Markup Language (XIML) is an extensible XML specification language for multiple facets of multiple models in a model-based approach. An XIML specification can lead to both an interpretation at runtime and a code generation at design time. The Vizzavi paradigm adopts another approach: rather than starting from initial models to produce a UI either at design time or at run-time, it reads a family of web pages and converts them on the fly to support other languages such as WML. Table 1 ['Comparison of languages for context-sensitive user interfaces'] reports on some major differences noted between some of these solutions..."
[May 09, 2001] "AUIML: An XML Vocabulary for Describing User Interfaces. [Device Independent User Interfaces in XML.]" By Roland A. Merrick (IBM). 2001-05-09. "As pervasive computing evolves, and Web based e-commerce matures, application developers are faced with the task of providing solutions for simultaneous deployment on a growing number of platforms. Maintaining bespoke implementations for each target client platform will become a significant development and maintenance overhead, unless a better solution is found. IBM has been tackling this problem and in the process created Abstract User Interface Markup Language (AUIML), an XML vocabulary designed to describe the semantics of a user interaction, without making any assumptions about the presentation medium for the interaction... In 1998 IBM undertook an Advanced Technology project to develop a Device Independent Markup Language in XML. This project went through a number of phases but ended up with an XML Vocabulary called Abstract User Interface Markup Language (AUIML)..." See also: (1) "IBM's UI Unity: A Role Based, Device Independent, Administrative Console."; (2) "OVID to AUIML - User-Oriented Interface Modelling." [cache]
[April 2001] "Object Modeling for User Centered Development and User Interface Design: The Wisdom Approach." By Duarte Nuno Jardim Nunes (Departamento de Matemática, Universidade da Madeira). April 2001. Dissertation. 285 pages. Nuno is working on a set of extensions UML to support the Wisdom method and user interface design in general (Wisdom profile). Note in the dissertation: the discussion of IBM's AUIML (Abstract User Interface Markup Language) on pages 203-212 and the discussion of "XForms and Flexible Automatic Generation" on pages 213-218. [cache]
- Workshop on Developing User Interfaces with XML: Advances on User Interface Description Languages." Part of Advanced Visual Interfaces 2004. May 25, 2004. Gallipoli (Lecce), Italy.
- DSV-IS 2003. Tenth International Workshop on Design, Specification and Verification of Interactive Systems. June 11-13, 2003. Funchal, Madeira Island, Portugal.
- UIML 2003 Conference. April, 2003. Paris, France. See Association Aristote: "UIML 2003 - La description et la génération automatique d'interfaces graphiques utilisateurs: où en est le langage UIML? 2 jours en Avril 2003, Paris."
- CADUI 2002
- Fourth International Conference on Computer-Aided Design of User Interfaces (CADUI 2002). "Towards a Uniform User Interface Modeling Language." Laboratoire d'Automatique, de Mécanique et d'Informatique, industrielles et Humaines (LAMIH), Université de Valenciennes, France, May 15-17, 2002.
- CADUI 2002 Program Listing [cache]
- Sixth Intelligent User Interfaces Conference (IUI 2002), January 13-16, 2002, San Francisco, California, USA.
- TUPIS 2000. Towards a UML Profile for Interactive Systems Development. See the program listing.