The Cover PagesThe OASIS Cover Pages: The Online Resource for Markup Language Technologies
SEARCH | ABOUT | INDEX | NEWS | CORE STANDARDS | TECHNOLOGY REPORTS | EVENTS | LIBRARY
SEARCH
Advanced Search
ABOUT
Site Map
CP RSS Channel
Contact Us
Sponsoring CP
About Our Sponsors
NEWS
Cover Stories
Articles & Papers
Press Releases
CORE STANDARDS
XML
SGML
Schemas
XSL/XSLT/XPath
XLink
XML Query
CSS
SVG
TECHNOLOGY REPORTS
XML Applications
General Apps
Government Apps
Academic Apps
EVENTS
LIBRARY
Introductions
FAQs
Bibliography
Technology and Society
Semantics
Tech Topics
Software
Related Standards
Historic
Last modified: October 03, 2005
XML Markup Languages for User Interface Definition

Overview

XML is frequently used or proposed for use in connection with device-independent user interface design. This document references several related activities.

Alternate Abstract Interface Markup Language (AAIML)

The V2 technical committee of the InterNational Committee for Information Technology Standards (INCITS) is developing an Alternate User Interface Access standard. The design includes an "XML-based language that would be used to communicate an abstract user interface definition for a service or device to a user's personal device which could act as a Universal Remote Console (URC)... The concept of a Universal Remote Console allows people with and without disabilities to remotely control any electronic and information technology device (target device/service), such as VCRs, copy machines, or elevators, from their personal remote control device. A standard for Universal Remote Consoles is required in order to facilitate interaction between the URC and target devices across different manufacturers."

"A key part in the Universal Remote Console specification is the definition of an XML-based language to convey a UI description from a service or device (target) to the URC. This 'Alternate Abstract Interface Markup Language' (AAIML) must be sufficiently abstract (in terms of modality independence), so that a particular URC device can render the provided UI in its own way, taking advantage of the specific interaction techniques the URC device is capable of. For example, a PDA could render the UI description by using GUI elements (visual) for output, and pointing with a stylus, as well as hand writing recognition for input; a car radio would render the same UI description auditorially with sound and synthetic speech for output, and speech recognition for input; and a braille note-taker would use its braille output and input capabilities in order to render the very same UI description tactily. Each URC device would allow access to all functions of the target, but in its own way..." [from the WWW 11 presentation.]

Abstract User Interface Markup Language (AUIML)

"AUIML (Abstract User Interface Markup Language) is 'an XML vocabulary which has been designed to allow the intent of an interaction with a user to be defined.' This clearly contrasts with the conventional approach to user interface design, which focuses on appearance. With an intent based approach, designers are able to 'concentrate on the semantics of the interactions without having to concern themselves with which particular device type(s) need to be supported.' Being an XML vocabulary, AUIML allows device independent encoding of information. All the interaction information can be encoded once and subsequently rendered using 'device dependent rendering' so that users can actually interact with the system. AUIML is therefore 'intended to be independent of the client platform on which the user interface is rendered, the implementation language and the user interface implementation technology." [from the TUPIS 2000 paper]

Extensible Interface Markup Language (XIML)

XIML is an XML-based "interface representation language for universal support of functionality across the entire lifecycle of a user interface: design, development, operation, management, organization, and evaluation."

"The XIML Forum is an industry organization dedicated to the research, dissemination, adoption, and standardization of the eXtensible Interface Markup Language (XIML), a comprehensive specification language for user interfaces. The Forum aims to provide a common specification and development infrastructure for user interface professionals of all types from interaction designers, to software engineers, to usability experts. It also seeks to afford human-computer interaction researchers in academia, industry, and the military a representation mechanism for the study of advanced technologies for user interfaces... XIML is available via a non-commercial, research license. Forum members use XIML for their own research purposes and make recommendations to the Forum towards the expansion and refinement of XIML. At their discretion, the members may also contribute tools and infrastructure support back to the XIML community."

References:

Extensible User Interface Language (XUL)

"XUL" is glossed alternately as "XML-based User Interface Language," "XML User Interface Language," and "Extensible User Interface Language." The version 1.0 draft attests "XML User Interface Language." XUL is a "standards-based interface definition language" associated with the Mozilla XPToolkit Project.

References:

Microsoft Extensible Application Markup Language (XAML)

XAML (Extensible Application Markup Language) is a code-name for the Microsoft 'Longhorn' Markup Language. "'Longhorn' application development raises abstraction to the level of a declarative programming model. To facilitate programming for this model a new XML-based declarative programming language (XAML) is being developed. XAML is based on Extensible Markup Language (XML) and enables developers to specify a hierarchy of objects with a set of properties and logic. The primary use of XAML is for 'Longhorn' client applications but you can use it to specify a hierarchy of any common language runtime objects. XAML enables you to create a UI without using code. You can create quite elaborate documents entirely in markup using controls, text, images, shapes and so forth... [excerpted from Microsoft's XAML Overview]"

References:

OASIS User Interface Markup Language (UIML) Specification Technical Committee

The OASIS TC will "develop a specification for an abstract meta-language that can provide a canonical XML representation of any user interface (UI). The language should be capable of specifying the requirements, design, and implementation of any UI. The committee will use as a starting point the UIML v3.0 specification."

References:

SAP Web Dynpro Protocol

In October 2005, SAP 'Inside SDN' published a paper An Introduction to the Web Dynpro Protocol. Article excerpt: "In last couple of years, the industry has been working on new initiatives to enable rich web-enabled clients; (e.g., W3C XForms 1.0; Mozilla XUL — XML User Interface Language; AJAX — Asynchronous JavaScript and XML). The SAP Web Dynpro Protocol is an enabling technology that provides efficient and adaptable HTTP-based exchange of UI (User Interface) instances between client and server. Enabling such exchange between different types of clients and servers poses a number of challenges. The Web Dynpro Protocol is an HTTP based protocol, which sends XML as its body. The main purpose of the protocol is to synchronize a server instance of a UI (User Interface) object tree with the client instance. The client request and server response refer to the same UI object model. The UI object model uses an XML representation to specify data, view and eventing objects for HTTP request and responses. There is a clear separation between data, data types, view and eventing. The object model is defined in different XML schemas. SCXML consists of XML fragments for specifying client UI and interaction: Header, Windows, Views, Data, ContainerTypes, SimpleTypes, Messages, Actions, Automation..."

From the SDN web site: "Developed to enable rich clients for sophisticated business applications, Web Dynpro is the next generation SAP GUI technology. The Web Dynpro Protocol provides efficient and adaptable HTTP-based exchange of UI (user interface) instances between client and server. It can be used to access Web Dynpro applications from different clients, such as the recently issued beta release of the SAP Web Dynpro client for Windows."

Model View Controller Programming Model: "Using Web Dynpro technology enables a clear separation of processing logic and presentation logic. A Web Dynpro application runs on the front end and has local or remote access to the back end system using a service. This means that the display logic is contained in the Web Dynpro application, while the business logic and the persistence of the business objects run in the back end system. Every Web Dynpro application is structured according to the Model View Controller programming model: (1) The Web Dynpro model forms the interface to the back-end system and thus enables the Web Dynpro application to access the data. (2) The view is the central logical element of the Web Dynpro application for the layout and processing of the presentation logic. (3) The controller is responsible for the data and control flow in the Web Dynpro application. There are different controller types. Since the controller contexts have the same structure, you can process every controller type in the same way using the same tool..." [from the SAP Library]

References:

SEESCOA Project [Software Engineering for Embedded Systems using a Component-Oriented Approach]

The goal of SEESCOA "is to adapt the CBD (Component Based Development) technology that was developed for mainstream software to the needs of embedded systems. The research consortium consists of four partners from different Flemish universities, each bringing their own expertise in this project: (1) Coordinator, Research group DistriNet, Department of Computer Science, KULeuven; (2) Research groups PROG/SSEL, Department of Computer Science, VUB; (3) Research group Expertisecentrum Digitale Media (EDM), LUC; (4) Research group PARIS, Department ELIS, Universiteit Gent. The consortium is backed up by a user group consisting of the following members: Agfa-Gevaert, Alcatel, Barco, Imec, Philips, Siemens Atea, All participants are part of the Vlaams Software Platform (VSP), a non-profit organization grouping companies and research centers from universities and private institutions in the ICT sector."

USer Interface eXtensible Markup Language (UsiXML)

"UsiXML (USer Interface eXtensible Markup Language) is a XML-compliant markup language that describes the UI for multiple contexts of use such as Character User Interfaces (CUIs), Graphical User Interfaces (GUIs), Auditory User Interfaces, and Multimodal User Interfaces. In other words, interactive applications with different types of interaction techniques, modalities of use, and computing platforms can be described in a way that preserves the design independently from peculiar characteristics of physical computing platform... UsiXML is defined in a set of XML schemas. Each schema corresponds to one of the models in the scope of the language...

UsiXML consists of a User Interface Description Language (UIDL), that is a declarative language capturing the essence of what a UI is or should be independently of physical characteristics. It describes at a high level of abstraction the constituting elements of the UI of an application: widgets, controls, containers, modalities, interaction techniques, etc. UsiXML allows cross-toolkit development of interactive application. A UI of any UsiXML-compliant application runs in all toolkits that implement it: compilers and interpreters...

UsiXML supports device independance: a UI can be described in a way that remains autonomous with respect to the devices used in the interactions such as mouse, screen, keyboard, voice recognition system, etc. In case of need, a reference to a particular device can be incorporated. UsiXML supports platform independance: a UI can be described in a way that remains autonomous with respect to the various computing platforms, such as mobile phone, Pocket PC, Tablet PC, laptop, desktop, etc. In case of need, a reference to a particular computing platform can be incorporated...

UsiXML supports modality independance: a UI can be described in a way that remains independent of any interaction modality such as graphical interaction, vocal interaction, 3D interaction, or haptics. It allows reuse of elements previously described in anterior UIs to compose a UI in new applications..." [from the UsiXML Overview page]

GrafiXML "is a graphical tool to draw user interfaces. You can save those UI in several formats like java or xhtml but the principal way is to save it in UsiXML, a xml user interface description. GrafiXML is released under the terms of the Apache License, Version 2.0.

  • USer Interface eXtensible Markup Language web site
  • About UsiXML
  • Publications
  • USIXML draft documentation
  • "USIXML: a Language Supporting Multi-Path Development of User Interfaces." By Quentin Limbourg, Jean Vanderdonckt, Benjamin Michotte, Laurent Bouillon, Víctor M. López Jaquero. In Proceedings of 9th IFIP Working Conference on Engineering for Human-Computer Interaction [jointly with 11th Int. Workshop on Design, Specification, and Verification of Interactive Systems, EHCI-DSVIS 2004, Hamburg, July 11-13, 2004). 30 slides.
  • "USIXML: A User Interface Description Language for Context-Sensitive User Interfaces." By Quentin Limbourg, Jean Vanderdonckt, Benjamin Michotte, Laurent Bouillon, Murielle Florins, and Daniela Trevisan. With 25 references. 25 May 2004. Paper presented at "Developing User Interfaces with XML: Advances on User Interface Description Languages," a Workshop Organised at Advanced Visual Interfaces 2004. See Workshop Accepted Papers. "This paper presents USIXML (USer Interface eXtensible Markup Language), a User Interface Description Language aimed at describing user interfaces with various levels of details and abstractions, depending on the context of use. USIXML supports a family of user interfaces such as, but not limited to: device-independent, platform-independent, modality independent, and ultimately context-independent. This paper consequently details how context-sensitive user interfaces may be specified and produced from the USIXML specifications. USIXML allows specifying multiple models involved in user interface design such as: task, domain, presentation, dialog, and context of use, which is in turn decomposed into user, platform, and environment. These models are structured according to the four layers of the Cameleon framework: task and concepts, abstract user interface, concrete user interface, and final user interface. To support relationships between these models, a model for inter-model mapping is also introduced that cover forward and reverse engineering as well as translation from one context of use to another..." [cache]
  • See also: GrafiXML, including the development version

User Interface Markup Language (UIML)

"UIML is an XML-compliant language, so it looks a lot like HTML. UIML is designed to serve as a single language which permits creation of user interfaces for any device, any target language (e.g., Java, C, WML), and any operating system on the device. However, UIML is not a silver bullet: The UI designer must still design separate UIs for each device, and then represent those designs in UIML. UIML does not magically create multiple UIs from a single description; instead it is a language in which those multiple UIs can be recorded... UIML is designed to be a relatively simple markup language. In contrast to Dynamic HTML (HTML 4) or XSL, which have so many tags that few people in the work know all the tags, UIML has a little over two dozen tags. UIML is also independent of any user interface metaphor, such as graphical user interfaces or voice-response. UIML is designed to permit efficient compilation from UIML to any target language (e.g., Java, C, HTML, WML, VoiceXML)." UIML describes the appearance of a UI, the user interaction with the UI, and how the UI is connected to the application logic.

References:

W3C Device Independence Working Group (DIWG)

The W3C Device Independence Working Group (DIWG) is part of the Interaction Domain. "There are significant efforts to integrate Web technologies into various devices (e.g., mobile, TV sets) other than the traditional Web access equipment such as PCs. However, we are faced with the possibility that services for those devices may not interoperate with each other or with the existing Web. That would not only cause fragmentation of the Web space, but also make Web device independent authoring impossible... The Device Independence Activity is newly created and merges the "Mobile Access Activity" and "TV and the Web Activity" to facilitate interchange in the interest of device independent Web access and authoring..."

W3C XForms Working Group [XForms]

"Key Goals of XForms: (1) Support for handheld, television, and desktop browsers, plus printers and scanners; (2) Richer user interface to meet the needs of business, consumer and device control applications; (3) Decoupled data, logic and presentation; (4) Improved internationalization; (5) Support for structured form data; (6) Advanced forms logic; (7) Multiple forms per page, and pages per form; (8) Suspend and Resume support; (9) Seamless integration with other XML tag sets."

Other Projects

Provisionally noted as relevant:

Articles and Papers

  • [May 25, 2004] Workshop on Developing User Interfaces with XML: Advances on User Interface Description Languages." Part of Advanced Visual Interfaces 2004, Gallipoli (Lecce), Italy. From the Call for papers: "This workshop aims at bringing together people working on/with XML-based UIDLs and any person interested in describing and/or developing UIs with XML. UIDL developers, designers and users are all among the potential participants. It is expected that from this exchange of discussion, a better mutual understanding will be raised to enable people to work more closely on some UIDLs rather than reproducing the same efforts in multiple locations. This aim will be supported by open questions selected by the organizers... For many years, Human-Computer Interaction (HCI) has witnessed a perennial race for the ultimate User Interface Description Language (UIDL) that would ideally capture the essence of what a UI could be or should be. Many people and teams from both the industry and the academic worlds have introduced UIDLs during history. Early UIDLs appeared when people firstly attempted to specify the look and feel of a UI. This attempt was particularly vivid when software appeared that produces UIs for multiple computing platforms, thus ensuring some form of portability of UIs. Then, the activity almost vanished until XML was considered widely enough to initiate a new family of UIDLs. Such UIDLs may pursue various goals: (1) nsuring portability of UIs from one computing platform to another while preserving consistency; (2) Capturing UI requirements for an abstract definition that remains stable over time; (3) Improving the reusability of UI design; (4) Making one UI design for multiple devices, platforms, appliances; this goal is often referred to as the device-independence rendering; (5) Supporting extensibility and adaptability of UI; (6) Using a UI description to enable automated generation of UI code..." See the links to the Workshop Accepted Papers.

  • [October 27, 2003] "Code Name Avalon: Create Real Apps Using New Code and Markup Model." By Charles Petzold. In Microsoft MSDN Magazine Volume 19 Number 1 (January 2004). "The new presentation subsystem in the next version of Windows, code-named 'Longhorn,' offers powerful new capabilities to developers. This new subsystem, code-named 'Avalon,' allows developers to take advantage of its capabilities through a language -- Extensible Application Markup Language (code-named 'XAML'), as well as through modern object-oriented programming languages such as C#. Because most applications written to Avalon will probably be a mix of XAML and programming code, this article discusses XAML tags used to control page layout along with the procedural code written to respond to events. Avalon consists mostly of a new collection of classes added to the .NET Framework... In addition, Avalon also defines a new markup language you can use in Longhorn that's code-named 'XAML.' You use XAML much like HTML to define a layout of text, images, and controls. Being based on XML, XAML has stricter and much less ambiguous syntax than HTML. It is expected that most XAML will be machine-generated by visual design programs, but it may be a good learning experience to hand-write your own XAML (initially). Most applications written to Avalon will probably contain both program code and XAML. You'll use the XAML for defining the initial visual interface of your application, and write code for doing everything else. You can embed the program code directly in XAML or keep it in a separate file. Everything you can do in XAML you can also do in program code, so it's possible to write a program without using any XAML at all. The reverse is not true, however; there are many tasks that can only be done in program code, so only the simplest applications will consist entirely of XAML. Here's a little snippet of some XAML: <Button Background='LightSeaGreen' FontSize='24pt'>Calculate</Button>... XAML has very intimate ties with the Avalon class library: every element type you can use in XAML is actually a class and, specifically, a descendent of the UIElement or ContentElement classes declared in the MSAvalon.Windows namespace. Among the descendents of UIElement is Control, from which are descended all the common user-interface controls such as buttons, scroll bars, list boxes, edit fields, and so forth. Classes derived from ContentElement include Bold and Italic. Of course, the modern programmer wants to know: can I use my own classes as XAML elements? And the answer is: of course you can. XAML is called the Extensible Application Markup Language for a reason. Any class that has a public parameterless constructor and settable properties can be used in XAML... The hierarchical nature of XAML is one reason why a markup language makes more sense than programming code for defining a visual interface. The markup can mimic the hierarchy with nesting and indentation. In fact, when Windows 1.0 was first introduced in 1985, programmers used a text resource script to define the hierarchical structure of their menus and dialog boxes. At the time, the hierarchy only went one level deep, but it was a start. An XAML file is just the resource script... Avalon and XAML represent a departure from Windows-based application programming of the past. In many ways, designing your application's UI will be easier than it used to be and deploying it will be a snap. With a lightweight XAML markup for UI definition, Longhorn-based applications are the obvious next step in the convergence of the Web and desktop programming models, combining the best of both approaches..."

  • [August 05, 2003] "Build Rich, Thin Client Applications Automatically Using XML." By Laurence Moroney. In DevX Network (July 30, 2003). ['New products let you build highly graphical client-side applications without the performance headaches of applets or the security concerns of ActiveX. Find out what these XWT-based technologies can do.'] " "The decoupling of data and presentation in HTML -- by using XML for the data and XSLT for the presentation of data -- has led to much innovation and flexibility, not least of which is the ability to deliver a document as data in XML and deliver custom styling for that document with different XSLTs. The next big trend in decoupling appears to be separating the user interface implementation from the user interface definition. There are countless initiatives, both open source and commercial, that will have at their core this very principle. The next big revolution in the desktop Windows operating system from Microsoft, codenamed Longhorn, is perhaps the most notable of these. With this type of decoupling, when a user interface is defined in a document such as XML, users would not have to download and install their GUIs; they can simply browse them as easily as they browse Web pages. A runtime engine would be present on the desktop, and servers would be able to deliver the GUI to the browser with an XML document. This will be huge for the corporate environment where at present, rich desktops are difficult and expensive to produce and maintain. Corporations are necessarily paranoid about allowing users to download and install binary files, and in general block users from doing this. The only options are to build a rich interface in HTML, or to build Windows applications and install them using a CD... The world is becoming used to XML driving HTML, with XSLT stylesheets to brand the data differently for different users. So why should more complex user interfaces be any different? They shouldn't, and that's the philosophy behind these new initiatives, including Longhorn, which will have its user interfaces defined using a language called XAML (XML Application Markup Language). In short, if a user interface is defined using XML as opposed to programmed using something such as C# or Java, then a runtime engine can parse it and render it on screen. Hooks within the XML could link to server-side applications that process information... Some of the better known initiatives in this area are XUL (XML User interface Language) and XWT (XML Windowing Toolkit) in the open source community, and Bambookit -- a commercial offering... Bambookit is a particularly impressive implementation as it is very easy to use and gives some great functionality and the ability to build complex, visually appealing, and high-performing user interfaces. Another player in this space is XWT, an open source initiative; XWT gives you either a Java-based or ActiveX-based runtime, and has the peculiar implementation of having to run off a server. This means that the XWT runtime has to be on a server and the configuration that you want to launch is passed to that server as a parameter. The configuration is an XML file, but it has to be zipped up in a special .XWAR file along with all of its dependencies... XWT has a flexible scripting model, using Javascript to process actions instead of the property pattern that Bambookit utilizes. It can also communicate with middleware using RPC or SOAP. Other players in this space are XUL, Luxor, Thinlets, and JEasy... At present, hand-coding of the XML documents to define the interfaces -- or the middleware servers to generate the interfaces -- is still necessary, but as time unfolds, IDE packages will have UI designers that compile to an XML format..."

  • [June 2003] "A Review of XML-Compliant User Interface Description Languages." By Nathalie Souchon and Jean Vanderdonckt. In Proceedings of 10th International Conference on Design, Specification, and Verification of Interactive Systems DSV-IS 2003 (Madeira, 4-6 June 2003) pages 377-391. "A review of XML-compliant user interface description language is produced that compares a significant selection of various languages addressing different goals, such as multiplatform-user interfaces, device-independence, context delivery, and user interfaces virtually defined. There has been a long history and tradition to attempt to capture the essence of user interfaces at various levels of abstraction for different purposes. The return of this question today gained more attraction, along with the dissimination of XML markup languages, and gives birth to many proposal for a new user interface description language. Consequently, there is a need to conduct an in-depth analysis of features that make all these proposal discriminant and appropriate for significant subset of such languages based on an analysis grid and user interfaces that we tried to implement across these languages..." [alt URL, cache]

  • [October 31, 2002] "SWT, XML Put True Cross-platform GUIs Within Reach." By A. Russell Jones (DevX Executive Editor). From DevX XML Zone. ['The reality of a single cross-language, cross-platform GUI interface programming model is in sight, based on an XML description language supported by fast native runtimes. Will Mozilla, Eclipse, or someone else step in and complete the last mile that gives all developers a common way to design and program fast cross-platform user interfaces?'] "... if Mozilla were to take on the project of connecting XUL to SWT, developers could design cross-platform GUIs in a completely language-independent fashion. For example, by designing an application interface in XUL and using the appropriate platform-specific SWT implementation to activate it in the language of your choice, the only thing you'd need to change from one platform to another would be the SWT runtime. It's a powerful combination and opens a window of opportunity for any organization with the vision to implement it... Developers don't need multiple programming models for common widgets, such as windows, list controls, outline controls, drop-down menus, textboxes, and buttons. Widgets act much the same between platforms and the differences that do exist lie almost exclusively in the look and feel (how they're rendered and the events they raise). In short, the enormous efforts expended by developers who learn more than one GUI programming model interface are largely wasted, because the results are visually and functionally nearly identical yet remain quite different from a development point of view. I fully expect that an XML-based forms description language, backed by platform-specific, high-performance runtimes will eventually capture both the market and developers' mindshare..."

  • [June 2002] "Migratable User Interface Descriptions in Component-Based Development." By Kris Luyten, Chris Vandervelpen, and Karin Coninx (Expertise Centre for Digital Media, Limburgs Universitair Centrum, Belgium). Presented at the Ninth International Workshop on Design, Specification, and Verification of Interactive Systems (DSV-IS 2002), Rostock, Germany, June 12-14, 2002. 15 pages, with 14 references. "In this paper we describe how a component-based approach can be combined with a user interface (UI) description language to get more exible and adaptable UIs for embedded systems and mobile computing devices. We envision a new approach for building adaptable user interfaces for embedded systems, which can migrate from one device to another. Adaptability to the device constraints is especially important for adding reusability and extensibility to UIs for embedded systems: this way they are ready to keep pace with new technologies... To describe a UI on a suffciently abstract level the Extensible Markup Language (XML) is used. Listing 1.1 provides an example of how a UI can be described in XML. A list of advantages is given in ["An XML-based runtime user interface description language for mobile computing devices," Proceedings of the Eight Workshop of Design, Specification and Verification of Interactive Systems, June 2001]. One of the major advantages is that XML does not force any level of abstraction, so this level can be adapted to the requirements of the situation. Note that an XML document can be presented as a tree which turns out to be a great advantage in our approach. There are other approaches for describing User Interfaces, but we believe that an XML-based description offers the best solution in our component-based approach because of it heavily relies on hierarchical structures. [XML syntax goals: platform independent, declarative, consistency, constraints, extensible, reusable, transformations]... the component-oriented approach suggested in this paper has several advantages for developers of embedded systems and mobile computing devices in particular. It is (1) Flexible: changing the UI can be done by another renderer component or letting components provide another UI description; (2) Reusable: providing a high level description of the UI related to the functionality a component offers, allows easier reusability of previously designed UIs in contrast with hard-coded UIs; (3) Adaptable: by abstracting the UI, device constraints can be taken into account when rendering the concrete UI..." See also the presentation slides. [cache]

  • [May 17, 2002] "Specifying User Interfaces for Runtime Modal Independent Migration." By Kris Luyten, Tom Van Laerhoven, Karin ConinxM, and Frank Van Reeth. Paper presented at CADUI 2002, Fourth International Conference on Computer-Aided Design of User Interfaces, Université de Valenciennes, France, May 15-17, 2002. 12 pages, with 18 references. "In this paper an approach will be presented which provides a uniform interface to such services, without any dependence on modality, platform or programming language. Through the usage of general user interface descriptions, presented in XML, and converted using XSLT, a uniform framework is presented for runtime migration of user interfaces. As a consequence, future services will become easily extensible for all kinds of devices and modalities. An implementation serving as a proof of concept, a runtime conversion of a joystick in a 3D virtual environment into a 2D dialog-based user interface, is developed... At runtime a user interface description is generated by means of XML as a description language. The XML description provides an abstraction of the user interface using Abstract Interaction Objects (AIO) which can be mapped onto Concrete Interaction Objects (CIO)... Our work does not aim to extract a Task-based description of the User Interface but merely tries to abstract the 'contents' of the user interface, not the presentation. Although we acknowledge the idea of splitting the interface model into a User-task model, a domain model, a user model, a presentation model and a dialog model for getting a better mapping of AIOs on CIOs, we do not consider it to be useful when focusing on runtime migration of existing user interfaces. A working user interface is 'serialized' into an XML description at runtime and this description can be transported to another system using for example the Internet or an infrared communication protocol. Once arrived at the target system the XML document can be parsed and converted in a working user interface ('deserialized'). An advantage of this approach can be found in the abstraction of the original user interface that the XML document provides. While parsing the XML document that contains an abstraction of the user interface, the renderer of the target platform is free to choose other ways to present the same functionality in the user interface... As an abstraction of the user interface we have defined a DTD. This DTD restricts the possible elements and ordering used in the XML document. Our current DTD is inspired by [Müller/Forbrig/Cap 'Model-Based User Interface Design Using Markup Concepts'] and enriched by a subset of interactors... We are planning to replace it with an equivalent XML Schema as soon as there are more mature tools available to use this... The transformation of the original user interface description into a higher level description is realized using the XSLT. Every system or user interface toolkit has to define a conversion providing the appropriate XSLT. Once the higher level description is produced a system specific XML document for the target system has to be produced. For every system a XSLT is defined which maps the AIOs defined in the abstract user interface description to CIOs for that particular system..." See also the presentation slides. [cache]

  • [May 2002] "Building Multi-Platform User Interfaces with UIML." By Mir Farooq Ali, Manuel A. Pérez-Quiñones, Marc Abrams, and Eric Shell. Published in Proceedings of CADUI 2002, May 2002. 12 pages, with 21 references. "There has been a widespread emergence of computing devices in the past few years that go beyond the capabilities of traditional desktop computers. However, users want to use the same kinds of applications and access the same data and information on these appliances that they can access on their desktop computers. The user interfaces for these platforms go beyond the traditional interaction metaphors. It is a challenge to build User Interfaces (UIs) for these devices of differing capabilities that allow the end users to perform the same kinds of tasks. The User Interface Markup Language (UIML) is an XML-based language that allows the canonical description of UIs for different platforms. We describe the key aspects of our approach that makes UIML successful in building multi-platform UIs, namely the division in the representation of a UI, the use of a generic vocabulary, a process of transformations and an integrated development environment specifically designed for transformation-based UI development. Finally we describe the initial details of a multi-step usability engineering process for building multiplatform UI using UIML... [We show] some of our research in extending and utilizing UIML to generate multi-platform user interfaces. We are using a single language, UIML, to provide the multi-platform development support needed. This language is extended via the use of a logical model, alternate vocabularies and transformation algorithms. Our approach allows the developer to build a single specification for a family of devices. UIML and its associated tools transform this single representation to multiple platformspecific representations that can then be rendered in each device. We have presented our current research in extending UIML to allow building of interfaces for very-different platforms, such as VoiceXML, WML and desktop computers." [cache]

  • [November 21, 2001] "XML en User Interfaces." By Kris Luyten.

  • [June 2001] "Model-Based User Interface Design Using Markup Concepts." By Andreas Müller [WWW], Peter Forbrig, and Clemens Cap (University of Rostock Department of Computer Science, Rostock, Germany). In Proceedings of the Eight Workshop of Design, Specification and Verification of Interactive Systems, June 13-15, 2001, University of Glasgow. "In the field of model-based development of interactive systems, several approaches have been proposed to integrate task and object knowledge into the development process and its underlying representations. This paper follows such an approach with a special focus on mobile devices. It presents a concept of device independent user interface design based on the XML-technology. The concept is applied to an e-commerce example... There are already some approaches to specify user interfaces by markup languages. There is for instance UIML, a XML-based language for describing user interfaces. It comes with different rendering programs allowing the generation of user interfaces on different platforms. Up to now the generation for WML, Java AWT and swing are available. A special renderer for each target-device is needed. This is an important disadvantage The flexibility for selecting target-device features is limited because of a lack of a mapping concept. XUL has its focus on window-based graphical user interfaces by XML. Its disadvantage is the limitation to such graphical user interfaces. It is not applicable to interfaces of small mobile devices. There is no abstraction of interaction functionality available. The device-independent specification of user interfaces was not planned... The following requirements should be fulfilled by a markup language specifying user interfaces: device independence, flexibility in representing user interfaces depending on the contexts, ability to specify virtual and physical user interfaces, localisation, extensibility of abstractions (abstract UIOs describing interaction behaviour), and extensibility of concrete UIOs (UIOs of specific representations). The following paragraphs will show a specific way of using XML in the proposed manner... The current work includes the specification of XML-based languages describing different abstraction of a user interface for mobile devices. A concept for the mapping process from abstract UIOs to concrete UIOs was developed. A XML language was developed which allows the specification of the features of the user interface of mobile devices. The transformation process of the user interface is based on specifications written in this language. In an interactive design process based on an abstract device dependent abstract interaction model and some design rules a more refined xsl-description of the user interfaces is developed, which allows the generation of specific user interfaces. This concept is already evaluated by several examples. A tool supporting the mapping and design process is under development. A first prototype is already available, which delivers already promising results. Up to now this tool is not integrated in previous phases of the design process. The abstract interaction model has to be edited manually. This will be omitted in the near future. Our experiments show that XML is a promising technology for the platform independent generation of user interfaces..." [cache; other papers]

  • [June 2001] "An XML-based Runtime User Interface Description Language for Mobile Computing Devices." By Kris Luyten and Karin Coninx (Expertise Centre for Digital Media, Limburgs Universitair Centrum, Belgium). Pages 17-29 (with 28 references) in Proceedings of the Eight Workshop of Design, Specification and Verification of Interactive Systems, June 13-15, 2001, University of Glasgow. "In a time where mobile computing devices and embedded systems gain importance, too much time is spent to reinventing user interfaces for each new device. To enhance future extensibility and reusability of systems and their user interfaces we propose a runtime user interface description language, which can cope with constraints found in embedded systems and mobile computing devices. XML seems to be a suitable tool to do this, when combined with Java. Following the evolution of Java towards XML, it is logical to introduce the concept applied to mobile computing devices and embedded systems... Because of the evolving market towards mobile computing devices and embedded systems, a more general approach for describing a user interface for an embedded system or mobile computing device is necessary. In search of a notation for describing such a user interface it should satisfy the following requirements: (1) Platform independent: because of the heterogeneity of embedded systems, a user interface designer should be allowed to design without having to worry about the system on which the interface will be used; (2) Declarative: describing a user interface asks for a certain level of expressiveness for describing human-computer interaction; (3) Consistent: the notation should offer consistency of the user interface among different environments and systems; (4) Unconventional I/O : embedded and mobile computing devices are less conservative in input and output devices. For example: while 'normal' desktop computers have a mouse and a keyboard, this is not a requirement for a mobile device, which could very well have a touch-screen and speech recognition; (5) Rapid prototyping: in a highly competitive market, such as mobile devices, developers and designers want to tailor the software towards the users or user groups. A user interface notation should allow rapid prototyping to get the users involved in the development process sooner; (6) Constraint sensitive: because of the constraints embedded systems are coping with, the designers must be able to specify the constraints, or have the system automatically generate them; (7) Easily extensible: we want to extend our user interface with extra functionality, without starting from scratch; (6) Reusability: when a family of products is evolving, we want to reuse the design for the old devices in an optimal way... [cache, Postscript]

  • [June 2001] "Task Modelling for Context-Sensitive User Interfaces." By Costin Pribeanu, Quentin Limbourg, and Jean Vanderdonckt. In Proceedings of the Eight Workshop of Design, Specification and Verification of Interactive Systems, June 13-15, 2001, University of Glasgow. "With the explosion of devices, computing platforms, contextual conditions, user interfaces become more confronted to a need to be adapted to multiple configurations of the context of use... The paper presents a method for modelling context-sensitive tasks with the aid of the ConcurTaskTree notation. Several approaches were considered and analysed to come up with a final approach combining clear separation of context-sensitive part, non-context-sensitive part, and context decision tree, while preserving the hierarchical structure of the model, as expressed in CTT. In order to make a distinction between the normal choice operator and the ones used for the decision tree, we decided to introduce a marked structured annotation in the tasks representing the nodes of the decision tree. Each such task is then augmented with the current value (e.g., low/moderate/high) of the contextual condition considered at that time (e.g., what is the available bandwidth). We then choose to export the resulting CTT model into an XML-compliant file thanks to the facility provided by CTTE. This XML file is then transformed into an XIML-compliant file using XSLT transformations. In particular, the arcs and nodes used for the decision tree are isolated in dedicated tags with the following structure [XML document with root <decision-tree>] This XIML file can then be exploited by UI derivation algorithms to progressively derive a generic UI, then a specific UI, and a code generation/interpretation... The study of this derivation process is the subjects of another submission, where the context-sensitive task model, along with its domain model are separately or simultaneously considered to derive presentation and dialog models, that are in turn refined into a specific one after considering the user model... Many [other] efforts have already been deployed to cover the problems raised by context-sensitivity, often starting with change of computing-platform or cross-computing platforms changes. This is a crucial approach because some UIs are particularly tailored to specific computing platforms, sometimes with Web-based add-ins. Equally important are the UIs which are specialised and adapted to a series of computing platforms, for instance different Web-appliances... The Phone.com paradigm is intended to describe a UI dialog that will be rendered afterwards for fixed or mobile phone. It is therefore aimed at supporting phone-based applications, which is not the case of the GeoWorks solution: it supports the same idea for multiple platforms. Sisl uses an event-based model of services for platform independence. The User Interface Markup Language (UIML) is an XML-compliant language that allows a declarative description of a user interface in a device-independent manner. To create a UI, one writes a UIML document, which includes presentation style appropriate for devices on which the UI will be deployed. UIML is then automatically mapped to a language used by the target device of a particular computing platform, such as HTML, Java/JFC, WML, HTML, PalmOS and VoiceXML. The Abstract User Interface Markup Language (AUIML) is an XML vocabulary designed to allow the intent of an interaction with a user to be defined. It allows device independent encoding of information. All the interaction information can be encoded once and subsequently rendered using device dependent rendering. The Extensible User-interface Language (XUL) is an XML-based UI specification language specifically designed for network applications like browsers, mailers, etc. Thanks to the portability of its rendering engine Gecko and the use of its XPCOM Technology, a XUL definition can be rendered on several computing platforms. The eXtensible Interface Markup Language (XIML) is an extensible XML specification language for multiple facets of multiple models in a model-based approach. An XIML specification can lead to both an interpretation at runtime and a code generation at design time. The Vizzavi paradigm adopts another approach: rather than starting from initial models to produce a UI either at design time or at run-time, it reads a family of web pages and converts them on the fly to support other languages such as WML. Table 1 ['Comparison of languages for context-sensitive user interfaces'] reports on some major differences noted between some of these solutions..."

  • [May 09, 2001] "AUIML: An XML Vocabulary for Describing User Interfaces. [Device Independent User Interfaces in XML.]" By Roland A. Merrick (IBM). 2001-05-09. "As pervasive computing evolves, and Web based e-commerce matures, application developers are faced with the task of providing solutions for simultaneous deployment on a growing number of platforms. Maintaining bespoke implementations for each target client platform will become a significant development and maintenance overhead, unless a better solution is found. IBM has been tackling this problem and in the process created Abstract User Interface Markup Language (AUIML), an XML vocabulary designed to describe the semantics of a user interaction, without making any assumptions about the presentation medium for the interaction... In 1998 IBM undertook an Advanced Technology project to develop a Device Independent Markup Language in XML. This project went through a number of phases but ended up with an XML Vocabulary called Abstract User Interface Markup Language (AUIML)..." See also: (1) "IBM's UI Unity: A Role Based, Device Independent, Administrative Console."; (2) "OVID to AUIML - User-Oriented Interface Modelling." [cache]

  • [April 2001] "Object Modeling for User Centered Development and User Interface Design: The Wisdom Approach." By Duarte Nuno Jardim Nunes (Departamento de Matemática, Universidade da Madeira). April 2001. Dissertation. 285 pages. Nuno is working on a set of extensions UML to support the Wisdom method and user interface design in general (Wisdom profile). Note in the dissertation: the discussion of IBM's AUIML (Abstract User Interface Markup Language) on pages 203-212 and the discussion of "XForms and Flexible Automatic Generation" on pages 213-218. [cache]

  • [September 26, 2000] "UIML: A Device-Independent User Interface Markup Language." By Constantinos Phanouriou. Blacksburg, Virginia. Dissertation submitted to the faculty of the Virginia Polytechnic Institute and State University in partial fulfillment of the requirements for the degree of 'Doctor of Philosophy' in Computer Science and Applications. September 26, 2000. 172 pages. This dissertation proposes a comprehensive solution to the problem of building device-independent (or multichannel) user interfaces promoting the separation of the interface from the application logic. It introduces an interface model (Meta-Interface Model, or MIM) for separating the user interface from the application logic and the presentation device. MIM divides the interface into three components, presentation, interface, and logic, that are connected with abstract vocabularies designed in terms of user chosen abstraction. The logic component provides a canonical way for the user interface to communicate with an application. The presentation component provides a canonical way for the user interface to render itself independently of the platform. The interface component describes the interaction between the user and the application using a set of abstract parts, events, and method calls that are device and application independent. MIM goes one step further than earlier models and subdivides the interface into four additional subcomponents: structure, style, content, and behavior. The structure describes the organization of the parts in the interface, the style describes the presentation specific properties of each part, the content describes the information that is presented to the user, and the behavior describes user interaction with the interface in a platform-independent manner. This dissertation also presents the second version of the User Interface Markup Language (UIML2), a declarative language that derives its syntax from XML and realizes the MIM model. It also gives the design rationale behind the language and discusses the implementation issues for mapping UIML2 to various devices (Java/JFC, PalmOS, WML, HTML, and VoiceXML). Finally, this dissertation evaluates UIML2 in terms of its goals, and among the major ones are to provide a canonical format for describing interfaces that map to multiple devices and to generate one description of a user interface connection to the application logic independent of target device." [cache]

Conferences and Workshops

[Incomplete]
Hosted By
OASIS - Organization for the Advancement of Structured Information Standards

Sponsored By

IBM Corporation
ISIS Papyrus
Microsoft Corporation
Oracle Corporation

Primeton

XML Daily Newslink
Receive daily news updates from Managing Editor, Robin Cover.

 Newsletter Subscription
 Newsletter Archives
Globe Image

Document URI: http://xml.coverpages.org/userInterfaceXML.html  —  Legal stuff
Robin Cover, Editor: robin@oasis-open.org