MorphGUI: Real-time GUIs Customization with Large Language Models
Web app interface - Users can use the web app like a standard application.
Enhancing with AI - Users describe changes, select components to redesign, separate “What it should do” (functional) from “How it should appear” (visual).
New web app interface - The new interface is generated and shown to the user that can use the new UI or further modify it.
MorphGUI is a framework that leverages Large Language Models to enable users to customize interfaces using natural language without altering the application’s core functionalities. Its structured dual-input design separates functional goals (“what it should do”) from visual specifications (“how it should appear”), allowing users to modify interface elements while maintaining the integrity of the underlying system logic. Powered by a dynamic component architecture, MorphGUI applies updates at runtime, preserving both the application state and baseline functionality.
Abstract
Graphical user interface (GUI) customization relies on predefined configuration options and settings, constraining diverse individual needs and preferences within predetermined boundaries and often requiring technical expertise. To address these limitations, this work introduces MorphGUI, a framework leveraging Large Language Models (LLMs) to enable interface customization through natural language. By allowing users to express desired changes using their own words and harnessing the generative capabilities of LLMs, MorphGUI mitigates the limitations of predefined options and reduces the need for technical expertise. The framework translates functional and stylistic requests into either modifications of existing application components or generation of new ones. Through a use case implementation with a calendar application and a user study (n=18), where participants were tasked with modifying interfaces towards a target goal, we investigate if MorphGUI can enable effective natural language-driven interface customization for non-expert users through both functional and visual modifications. Results show that participants successfully customized interfaces using natural language. Users found the system intuitive and achieved good performance regardless of technical background, we report analysis of optimal prompt length, challenges in separating functional and visual instructions in structured templates, correlation between LLM experience and success, and learning effects. The study revealed opportunities for enhanced guidance, examples, and scaffolding to help users structure their customization requests more effectively.
Customization Interface
The "Enhance With AI" panel in MorphGUI enables users to personalize and modify their interface through AI-assisted customization. It features two main sections, Static Settings and Dynamic Settings, allowing configuration of both fixed and adaptive interface components. Within the Dynamic Settings tab, the Personalize section introduces component-level targeting for modifying or adding UI elements. Users specify What it should do to define the element’s functional behavior and How it should appear to control its styling and layout. The panel also provides three main actions: Generate to apply the defined changes, Previous to roll back to a previous configuration, and Restore to return the interface to its last stable state.
Workflow & Real-time Updates
MorphGUI combines selected component context, user inputs, and constraints into structured prompts for the LLM, then validates specifications and transforms them into executable UI code. The dynamic component bridges generated code and the app’s runtime, handling dependency resolution and compilation in real-time effectively rendering the new UI.
Architecture
The system architecture implements a modular client-server model that enables real-time GUI customization through natural language interaction. The Front-end is a web-based interface built on a component-driven framework that communicates bidirectionally with Dynamic Components for runtime rendering and update of interface elements. These components interface with the Server layer via RESTful APIs to support synchronous data exchange and asynchronous updates. The Server, developed using Node.js and the Express.js framework, orchestrates communication with the AI module,a large language model that parses natural language inputs and translates them into executable interface specifications. The Database layer, implemented with SQLite, manages persistent storage of user preferences, interface states, and component configurations, enabling rollback and version control functionalities. The User Preferences subsystem provides contextual adaptation by maintaining individualized configuration data that influence interface behavior and visual presentation across sessions.
User Study Results
The user study evaluated MorphGUI in terms of usability, learning curve, and interaction effectiveness when customizing graphical interfaces through natural language. Participants performed a series of interface modification tasks using a calendar application. Each participant started with a base calendar layout and used MorphGUI to reproduce one or more target interfaces of increasing complexity. The easy task involved straightforward visual or structural edits (e.g., changing colors, repositioning elements), while the complex task required functional and aesthetic modifications, such as adding components, reorganizing layouts, or combining multiple requests into a single prompt.
Results from the System Usability Scale (SUS) and custom Likert-scale items indicate that participants found MorphGUI intuitive, well-integrated, and easy to use, with low perceived complexity. Most participants disagreed with statements suggesting high cognitive demand or inconsistency, confirming a low learning overhead and high user confidence.
Correlation analyses revealed no significant relationship between users’ familiarity with large language models (LLMs) and overall task performance (Pearson r = -0.03, p = 0.883), suggesting that even participants unfamiliar with LLMs were able to effectively complete customization tasks. However, a mild positive trend (r = 0.24, p = 0.165) was observed between prompt descriptiveness and task success, implying that clearer and more detailed natural language prompts tended to produce better customization outcomes. Additionally, analysis of template usage patterns showed that participants primarily relied on the “what” field for functional modifications and the “how” field for styling and layout changes, supporting MorphGUI’s dual-input design principle.
