US20050125787A1 - Convertible runtime graphical user interface - Google Patents

Convertible runtime graphical user interface Download PDF

Info

Publication number
US20050125787A1
US20050125787A1 US10/707,343 US70734303A US2005125787A1 US 20050125787 A1 US20050125787 A1 US 20050125787A1 US 70734303 A US70734303 A US 70734303A US 2005125787 A1 US2005125787 A1 US 2005125787A1
Authority
US
United States
Prior art keywords
cui
widget
application
gui
widgets
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US10/707,343
Inventor
Leonid Tertitski
Andrei Ivanov
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to US10/707,343 priority Critical patent/US20050125787A1/en
Publication of US20050125787A1 publication Critical patent/US20050125787A1/en
Priority to US11/948,693 priority patent/US7904821B1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/451Execution arrangements for user interfaces

Definitions

  • GUI Graphical User Interface
  • Modern, sometimes called “configurable”, GUIs allow the end-user to modify some of the GUI widget's attributes: geometrical sizes, position, background color, and text font.
  • GUIs provide no possibility to change the behavior of the widget associated with some program variable at runtime, or create new link(s) between this variable and new widget(s).
  • a UIMS that would allow the runtime creation of new widget shapes and/or conversion of a widget or a group of widgets to another widget or group of widgets without recompilation and/or restart of the application executable would be desirable.
  • FIG. 1 shows contents of the example program HelloWorld.cpp
  • FIG. 2 shows window with parameter “Param” when program HelloWorld.cpp runs for the first time
  • FIG. 3 shows window with new (converted) representation of parameter “Param”
  • FIG. 4 shows contents of the example program SumOfTwoNumbers.cpp;
  • FIG. 5 shows window with parameters of the program from FIG. 4 when it runs for the first time
  • FIG. 6 shows window from FIG. 5 after the values of parameters have been changed
  • FIG. 7 shows window with new (converted) representation of variables of the program presented in FIG. 4 ;
  • FIG. 8 shows window presented by FIG. 7 with duplicated parameters
  • FIG. 9 shows window presented by FIG. 8 with parameters converted to sliders
  • FIG. 10 provides a list of members of the class CUIParamEntry
  • FIG. 11 provides a list of members of the class CUIObject
  • FIG. 12 provides a list of CUI Viewer commands currently supported in “Edit” mode
  • the current embodiment of this invention is a set of methods, procedures and functions implemented either in specialized Dynamic Link Library (CUI DLL) or standalone executable (CUI Viewer), and data representing Convertible User Interface Description (CUI Description).
  • CUI DLL Dynamic Link Library
  • CUI Viewer standalone executable
  • CUI Description Convertible User Interface Description
  • CUI variables The registration of CUI variables is done automatically when the application uses special constructors.
  • For manipulation of CUI variables inside of the application set/get/compare value(s) we implemented special methods and operators, which override analogous regular methods and operators used in C++ or Java.
  • CUI DLL provides these special constructors, methods and operators.
  • For manipulation of CUI variables outside of the application registration, visualization in GUI, modification of their values according the user's input) we use the set of methods, procedures and functions implemented in CUI Viewer.
  • CUI Description is a text file used to define the application user interface as a set of descriptions of widgets with their respective sizes, locations, graphical and other attributes.
  • CUI Viewer plays the role of a middleman between the application and its GUI, defined by GUI Description. It is used for the registration of CUI parameters, visualization and modification of the GUI(s) of one or several applications.
  • an application sends a CUI Description file name and its own name to CUI Viewer. If CUI Viewer is able to read CUI Description file, it converts its contents into a tree of widgets and displays them according their attributes.
  • the application generates each CUI variable by using a corresponding constructor from CUI DLL, and the constructor registers each variable with CUI Viewer. At this moment CUI Viewer attempts to create a link between each registered CUI variable and its corresponding widget. If it is successful, then the widget is updated. If not, CUI Viewer creates some default widget for the new CUI variable. This new widget is added to the chain of widgets and is displayed by CUI Viewer in the window “New Parameters”.
  • CUI Viewer uses the application name to construct some default file name for CUI Description.
  • CUI Memory Manager uses Application Shared Memory Buffer (ASMB). This design not only simplifies the treatment of CUI variables but make possible the use of common variables in different applications working with the same CUI Viewer.
  • ASMB Application Shared Memory Buffer
  • CUI Viewer and CUI DLL do not depend on the purpose of the application. They could be written and/or compiled for any platform once or even become part of operating systems or Internet browsers.
  • FIG. 1 An example of a simple “Hello World” application is presented by FIG. 1 .
  • CUI Viewer displays an editable variable named “Test” having the value “Hello World” by using the default widget designed for CUI Variables of character string type in the default window “New Parameters” ( FIG. 2 ) and waits for user input.
  • the text “Hello World” is the value of CUI variable Param, which can be modified by user.
  • the application will end its execution.
  • CUI Viewer asks user to save new (converted) CUI Description. If the user agrees, CUI Viewer saves the new CUI Description for “HelloWorld.exe” in a text file with the name “HelloWord.CUI”. CUI Viewer will use this file during the next launch of application “HelloWorld.exe” and this application GUI will show up as in FIG. 3 .
  • CUI variable Param being editable is defined by the default value of one parameter of constructor CUIChar(char *). The full set of parameters for this constructor will be described later.
  • Any CUI variable has two mandatory and unchangeable attributes “Name” and “Type”, which distinguish it among the other CUI variables.
  • CUI Viewer will create only one CUI variable for any given pair (Name, Type) and it will provide read/write access to this variable attributes for any application working under its supervision.
  • the attribute “Name” is a non-empty character string.
  • CUI DLL provides read/write access to the value of any CUI Variable and other manipulation of CUI variables by using a set of overloaded operators defined separately for each type in the corresponding C++ class.
  • FIG. 4 shows an example of a program for the calculation of the sum of two numbers.
  • This program defines two user editable CUI variables FirstNumber and SecondNumber and CUI variable SumOfTwoNumbers that can be modified only by the application program. These three variables have the names “A”, “B” and “Sum of A and B”, respectively. Notice that the application program does not define how and where these variables must be displayed.
  • the application program sets the initial values and then stays in a while loop and calculates the sum of these two numbers.
  • the function CUISleep( ) belongs to CUI DLL. It waits for the requested number (100) of milliseconds and then checks the status of the application. If application program state is “End Application”, then CUISleep( ) returns 1.
  • the CUI DLL function supporting “set value” function sends the message “Update Value” to the CUI Viewer only if the value of SumOfTwoNumbers was actually changed.
  • CUI DLL will modify FirstNumber and/or SecondNumber
  • User can modify size and display style of the first widget representing A (B) and convert the second widget representing the same CUI variable into a slider ( FIG. 9 ).
  • FIG. 10 shows the information stored by any instance of CUIParamEntry class.
  • “Actual Value” ( FIG. 10 line 3 ) can be changed either by CUI Application.
  • CUI Viewer can change “Actual Value” only if an application defined the associated CUI variable as CUI_EDITABLE and the user changes the corresponding field of the widget linked with the variable.
  • Each CUI variable is an instance of a type specific class (CUIint, CUIdouble or CUIchar) derived from class CUIParam. Any instance of CUIParam contains only one member “Index of Parameter”, which is the index of an instance of class CUIParamEntry previously allocated by CUI MemoryManager.
  • the constructor of class CUIParam uses one of its input parameters, Name, passed to it by the constructor of one of the derived classes (CUIint, CUIdouble or CUIchar), and corresponding Type and attempts to find an instance of CUIParamEntry with the same Name and Type in the memory allocated by CUI Memory Manager. In case of a match it makes Index of Parameter equal to the index of found instance. Otherwise the new instance of CUIParamEntry with given (Name, Type) is constructed and Index of Parameter stores its index.
  • FIG. 11 shows the information stored by any instance of CUIObject class.
  • CUI_Var Every time the value of any CUI Variable (CUI_Var) is changed, a special CUI DLL function sends a “SET_VALUE” message with the address of corresponding instance of CUIParamEntry class to CUI Viewer. Upon receiving this message CUI Viewer performs the following sequence of operations:
  • CUI Viewer It checks the value of the field “CUIObject Address” ( FIG. 10 line 11 ). If it is NULL, CUI Viewer searches for an instance of CUIObject class with the same Name ( FIG. 10 line 1 ) and Type ( FIG. 10 line 2 ) in the tree of CUIObjects. If no such instance is found, the CUI Viewer creates a new instance of CUIObject class with the default parameters for the visualization of CUI_Var and address of the CUIParamEntry instance stored in “Reference to parameter” field ( FIG. 11 line 19 ). The address of the newly created or found instance is stored in the field “CUIObject Address” ( FIG. 10 line 11 ).
  • Any CUI Variable can have more than one representation in GUI described by different CUIObject instances. All these instances comprise a doubly linked list ( FIG. 11 lines 34 , 35 ) used by CUI Viewer when it processes the “SET_VALUE” message.
  • CUI Viewer When the application calls any CUI function for the first time, CUI Viewer starts a special thread CUIAppThread used for waiting and processing of “RUN_CALLBACK” messages. CUI Viewer sends such a message every time the value of CUI Variable CUI_Var is modified by user's interaction with the GUI (or by another program) and the field “Callback address” ( FIG. 10 line 5 ) of the CUIParamEntry instance associated with CUI_Var contains a non-NULL address of some callback routine of the application.
  • FIG. 12 shows the full list of CUI Viewer commands currently supported in “Edit” mode.
  • CUI Viewer itself is written as CUI application; therefore the user can add or modify the list of widgets in the “Convert To” menu.
  • CUI Viewer does not send any messages regarding the widget conversion to the application program, and the application continues its execution as usual.
  • CUI Viewer stores only information about CUI Variables changes. Each record contains following information:
  • the CUI Viewer “Play” command can be used even if the positions, sizes or types of any Widgets have been changed (claim 12 ).
  • Play CUI Viewer reads these records and simulates execution (moves mouse pointer to widget with the same Variable Name as in the record and changes Variable Value) according to recorded time.
  • CUI Viewer calculates the “Time To Set Value” for each such widget.
  • the “Time To Set Value” is calculated as the time needed to move the mouse cursor at “Mouse Simulation Speed” to the closest point of the widget plus the time for “Value Simulation Change”. If the widget is not visible (for example, it is located in a closed window), the “Time To Set Value” is equal to the time needed to move mouse cursor and make this widget visible plus time needed for “Value Simulation Change”. The widget with minimum “Time To Set Value” is used.
  • “Mouse Simulation Speed” is equal to 5 centimeter per second
  • “Time To Set Value” is equal to number of the Mouse/Keyboard button clicks multiplied by 0.3 seconds per click.
  • Some methods of current invention are based on or intensively use the external and internal representation of different CUI data in XML format. All these XML-methods can be described as “static” or “dynamic” according to their usage of transformation of XML representations involved in the corresponding processes.
  • the static XML-methods use XML representation for storage and reading/writing procedures of CUI Description (claim 7 ) and widget library (claims 13 , 17 ). They use no transformations of XML data.
  • Every dynamic method uses some XSL transformation to convert the initial XML representation of some part of CUI data into its new XML representation.
  • the CUI Viewer uses this new XML representation for visualization of the same CUI data in some other (converted) way.
  • This approach is implemented for automatic recalculation of coordinates and sizes of widgets (claim 14 ) and for automatic rule based conversion of attributes of replaced widgets (claim 15 ).
  • the XSL files used for our dynamic XML-methods are the external text files that form the special Transformation Library. Any of these files can be modified or replaced by a new XSL files, which can provide new rules for some transformation (claim 16 ).

Abstract

Method to change Graphical User Interface that support runtime modification of the application's GUI without access to the application source code and/or needs no recompilation/restart of the application executable. This method allows the end-user to make runtime modifications of a widget's attributes and/or runtime conversion of widgets into different types of widgets

Description

    CROSS REFERENCE TO RELATED APPLICATIONS
  • U.S. Pat. Nos. 5,600,780/5,603,034/5,652,884/5,786,815/6,208,336
  • BACKGROUND OF INVENTION
  • The present invention field of endeavor is user interface management systems (UIMS). Graphical User Interface (GUI) is “the Front End”, “the Face” of any application. Modern, sometimes called “configurable”, GUIs allow the end-user to modify some of the GUI widget's attributes: geometrical sizes, position, background color, and text font. Unfortunately, they provide no possibility to change the behavior of the widget associated with some program variable at runtime, or create new link(s) between this variable and new widget(s). For example, it is impossible to convert the widget “editable integer” to the widget “slider” without changing the application source code and/or without compiling a new application executable. A UIMS that would allow the runtime creation of new widget shapes and/or conversion of a widget or a group of widgets to another widget or group of widgets without recompilation and/or restart of the application executable would be desirable.
  • BRIEF DESCRIPTION OF DRAWINGS
  • FIG. 1 shows contents of the example program HelloWorld.cpp;
  • FIG. 2 shows window with parameter “Param” when program HelloWorld.cpp runs for the first time;
  • FIG. 3 shows window with new (converted) representation of parameter “Param”;
  • FIG. 4 shows contents of the example program SumOfTwoNumbers.cpp;
  • FIG. 5 shows window with parameters of the program from FIG. 4 when it runs for the first time;
  • FIG. 6 shows window from FIG. 5 after the values of parameters have been changed;
  • FIG. 7 shows window with new (converted) representation of variables of the program presented in FIG. 4;
  • FIG. 8 shows window presented by FIG. 7 with duplicated parameters;
  • FIG. 9 shows window presented by FIG. 8 with parameters converted to sliders;
  • FIG. 10 provides a list of members of the class CUIParamEntry;
  • FIG. 11 provides a list of members of the class CUIObject;
  • FIG. 12 provides a list of CUI Viewer commands currently supported in “Edit” mode;
  • DETAILED DESCRIPTION
  • The current embodiment of this invention is a set of methods, procedures and functions implemented either in specialized Dynamic Link Library (CUI DLL) or standalone executable (CUI Viewer), and data representing Convertible User Interface Description (CUI Description).
  • All variables of any application can be divided into two groups with respect to their relationship with the GUI. Some are associated with some widgets, and thus they are “visible” or perhaps even “changeable” in the GUI, and the others are not. We refer to GUI-“visible” variables as CUI variables and we treat them in a special way. First of all, we implemented a special procedure for their registration, which uses their names and types. This procedure uses a special memory manager (CUI Memory Manager) for their memory allocation. Second, we provide a set of functions for the manipulation of CUI variables.
  • The registration of CUI variables is done automatically when the application uses special constructors. For manipulation of CUI variables inside of the application (set/get/compare value(s)) we implemented special methods and operators, which override analogous regular methods and operators used in C++ or Java. CUI DLL provides these special constructors, methods and operators. For manipulation of CUI variables outside of the application (registration, visualization in GUI, modification of their values according the user's input) we use the set of methods, procedures and functions implemented in CUI Viewer.
  • CUI Description is a text file used to define the application user interface as a set of descriptions of widgets with their respective sizes, locations, graphical and other attributes.
  • In our design CUI Viewer plays the role of a middleman between the application and its GUI, defined by GUI Description. It is used for the registration of CUI parameters, visualization and modification of the GUI(s) of one or several applications.
  • During its start an application sends a CUI Description file name and its own name to CUI Viewer. If CUI Viewer is able to read CUI Description file, it converts its contents into a tree of widgets and displays them according their attributes. During its work the application generates each CUI variable by using a corresponding constructor from CUI DLL, and the constructor registers each variable with CUI Viewer. At this moment CUI Viewer attempts to create a link between each registered CUI variable and its corresponding widget. If it is successful, then the widget is updated. If not, CUI Viewer creates some default widget for the new CUI variable. This new widget is added to the chain of widgets and is displayed by CUI Viewer in the window “New Parameters”.
  • Notes:
  • 1. An application never needs the information about the current contents or even existence of CUI Description. Generally speaking, it should work fine even when this file does not exist.
  • 2. If the CUI Description file name provided by application is empty, CUI Viewer uses the application name to construct some default file name for CUI Description.
  • 3. All CUI variables of current application are linked to new widgets, if CUI Viewer cannot find or open CUI Description file.
  • 4. The storage of all CUI variables is managed by CUI Memory Manager. In some implementations CUI Viewer uses Application Shared Memory Buffer (ASMB). This design not only simplifies the treatment of CUI variables but make possible the use of common variables in different applications working with the same CUI Viewer.
  • 5. CUI Viewer and CUI DLL do not depend on the purpose of the application. They could be written and/or compiled for any platform once or even become part of operating systems or Internet browsers.
  • 6. All functions and procedures of CUI Viewer could be easily moved into CUI DLL. In this case this DLL could be statically linked to any application program.
  • 7. It is very natural to use XML representation for CUI Description. However, since today XML is not very well supported on different platforms our program is capable of reading and writing CUI Description either in XML or in our own internal text format.
  • When the application is running the user can:
  • a) change different attributes of any widget: size, position, color etc.
  • b) move widgets from one window to another;
  • c) convert any widget to another widget type;
  • d) construct new and delete old links between GUI variables and widgets;
  • e) add new widgets not connected to any application variables and which can be used, for example, as a new window or button to open some window;
  • f) save current widget configuration as the GUI description in the text file that will be used by CUI Viewer during the next start of application.
  • An example of a simple “Hello World” application is presented by FIG. 1.
  • For simplicity we suppose that file “HelloWorld.cpp” was compiled into “HelloWorld.exe” application. When this application starts running it calls constructor CUIChar(char *) to initiate the editable CUI variable Param of character string type. This variable is registered with CUI Viewer under the name “Test”. Then the application sets the value of Param to string “Hello World”. Finally, it waits in a loop until the value of Param becomes equal to the string “End”.
  • When this application starts for the first time (or if Application CUI Description file is not presented), CUI Viewer displays an editable variable named “Test” having the value “Hello World” by using the default widget designed for CUI Variables of character string type in the default window “New Parameters” (FIG. 2) and waits for user input.
  • In this example the text “Hello World” is the value of CUI variable Param, which can be modified by user. When user inputs the text “End”, the application will end its execution.
  • Without exiting the application by using the capability provided by CUI Viewer, user can create a new window “Test Window” and convert the initial default widget previously used for representation of CUI variable Param into a pull-down menu “Hello World” with two options: “Hello World” and “End” (FIG. 3).
  • If this is done, the application stops its execution when the user selects the option “End” from the described pull-down menu. At this point CUI Viewer asks user to save new (converted) CUI Description. If the user agrees, CUI Viewer saves the new CUI Description for “HelloWorld.exe” in a text file with the name “HelloWord.CUI”. CUI Viewer will use this file during the next launch of application “HelloWorld.exe” and this application GUI will show up as in FIG. 3.
  • In this example (FIG. 1) the property of CUI variable Param being editable is defined by the default value of one parameter of constructor CUIChar(char *). The full set of parameters for this constructor will be described later.
  • Any CUI variable has two mandatory and unchangeable attributes “Name” and “Type”, which distinguish it among the other CUI variables. In other words, CUI Viewer will create only one CUI variable for any given pair (Name, Type) and it will provide read/write access to this variable attributes for any application working under its supervision.
  • The attribute “Name” is a non-empty character string. Currently CUI DLL and CUI Viewer support the following CUI variable types:
  • a) CUIint for integers;
  • b) CUIdouble for doubles;
  • c) CUIchar for character strings;
  • d) CUIintArray for arrays of integers;
  • e) CUIdoubleArray for arrays of doubles;
  • f) CUIcharArray for arrays of strings.
  • CUI DLL provides read/write access to the value of any CUI Variable and other manipulation of CUI variables by using a set of overloaded operators defined separately for each type in the corresponding C++ class.
  • Two CUI Variables defined by one or several applications with the same pair of attributes (Name, Type) always have references to the same address provided by CUI Memory-Manager.
  • FIG. 4 shows an example of a program for the calculation of the sum of two numbers.
  • This program defines two user editable CUI variables FirstNumber and SecondNumber and CUI variable SumOfTwoNumbers that can be modified only by the application program. These three variables have the names “A”, “B” and “Sum of A and B”, respectively. Notice that the application program does not define how and where these variables must be displayed.
  • The application program sets the initial values and then stays in a while loop and calculates the sum of these two numbers. The function CUISleep( ) belongs to CUI DLL. It waits for the requested number (100) of milliseconds and then checks the status of the application. If application program state is “End Application”, then CUISleep( ) returns 1. The CUI DLL function supporting “set value” function sends the message “Update Value” to the CUI Viewer only if the value of SumOfTwoNumbers was actually changed.
  • When the application starts for the first time (or if file “SumOfTwoNumbers.CUI” is not found) the user is presented with Initial state of window “New Parameters” (FIG. 5). User can modify the values of A and/or B and
  • a) CUI DLL will modify FirstNumber and/or SecondNumber;
  • b) application will calculate the result A+B;
  • c) CUI Viewer will automatically display new result (FIG. 6)
  • Let's describe the hypothetical sequence of user's actions modifying the GUI of this application when it is running.
  • 1. User can create new window “Sum of A and B” and move all widgets representing CUI variables into this window (FIG. 7).
  • 2. User can duplicate widgets representing the values of A and B (FIG. 8).
  • 3. User can modify size and display style of the first widget representing A (B) and convert the second widget representing the same CUI variable into a slider (FIG. 9).
  • 4. User can test the newly created interface by using sliders to modify the values of editable CUI variables A and/or B.
  • 5. Finally, user can save the new CUI Description of the newly constructed interface and see it in the next launch of the application.
  • Let's describe the data structures we are using to make CUI work. For this we will rather use “functional” descriptions of class members and not their formal names.
  • We are using three main base classes: CUIParamEntry, CUIParam and CUIObject. The first two of these contain information about CUI variables and methods for their manipulation; the last contains information about widgets.
  • FIG. 10 shows the information stored by any instance of CUIParamEntry class.
  • “Actual Value” (FIG. 10 line 3) can be changed either by CUI Application. Note that CUI Viewer can change “Actual Value” only if an application defined the associated CUI variable as CUI_EDITABLE and the user changes the corresponding field of the widget linked with the variable. Each CUI variable is an instance of a type specific class (CUIint, CUIdouble or CUIchar) derived from class CUIParam. Any instance of CUIParam contains only one member “Index of Parameter”, which is the index of an instance of class CUIParamEntry previously allocated by CUI MemoryManager.
  • The constructor of class CUIParam uses one of its input parameters, Name, passed to it by the constructor of one of the derived classes (CUIint, CUIdouble or CUIchar), and corresponding Type and attempts to find an instance of CUIParamEntry with the same Name and Type in the memory allocated by CUI Memory Manager. In case of a match it makes Index of Parameter equal to the index of found instance. Otherwise the new instance of CUIParamEntry with given (Name, Type) is constructed and Index of Parameter stores its index.
  • All constructors of CUIint, CUIdouble, and CUIchar classes pass additional parameters to the constructor of CUIParam class, which define the initial value of constructed CUI variable and some attributes.
  • FIG. 11 shows the information stored by any instance of CUIObject class.
  • We use the following three classes derived from CUIObject to represent different types of widgets: CUIEdit, CUIButton and CUIPicture. Each of these classes contains the full description of graphical representation and location of corresponding widget.
  • The operation of any application working in CUI environment is based on the message exchange between some functions provided by CUI DLL, CUI Viewer and the special thread automatically launched for the application. Among these messages the most important are the “SET_VALUE” and “RUN_CALLBACK” messages. The first of these initializes the changes in the visualization of CUI Variables on the screen, and the other initializes some reactions of the application related to the user's input.
  • Every time the value of any CUI Variable (CUI_Var) is changed, a special CUI DLL function sends a “SET_VALUE” message with the address of corresponding instance of CUIParamEntry class to CUI Viewer. Upon receiving this message CUI Viewer performs the following sequence of operations:
  • a) It checks the value of the field “CUIObject Address” (FIG. 10 line 11). If it is NULL, CUI Viewer searches for an instance of CUIObject class with the same Name (FIG. 10 line 1) and Type (FIG. 10 line 2) in the tree of CUIObjects. If no such instance is found, the CUI Viewer creates a new instance of CUIObject class with the default parameters for the visualization of CUI_Var and address of the CUIParamEntry instance stored in “Reference to parameter” field (FIG. 11 line 19). The address of the newly created or found instance is stored in the field “CUIObject Address” (FIG. 10 line 11).
  • b) It displays the new value of CUI_Var by using the virtual function Draw( ) for the instance of CUIObject linked with CUI_Var.
  • Notes:
  • 1) When the search of CUIObject instance for current Name (FIG. 10 line 1) and Type (FIG. 10 line 2) is unsuccessful, the default parameters for visualization of CUI_Var by newly constructed CUIObject instance make possible the visualization of CUI_Var in the special window “New Parameters”.
  • 2) Any CUI Variable can have more than one representation in GUI described by different CUIObject instances. All these instances comprise a doubly linked list (FIG. 11 lines 34, 35) used by CUI Viewer when it processes the “SET_VALUE” message.
  • When the application calls any CUI function for the first time, CUI Viewer starts a special thread CUIAppThread used for waiting and processing of “RUN_CALLBACK” messages. CUI Viewer sends such a message every time the value of CUI Variable CUI_Var is modified by user's interaction with the GUI (or by another program) and the field “Callback address” (FIG. 10 line 5) of the CUIParamEntry instance associated with CUI_Var contains a non-NULL address of some callback routine of the application.
  • In the special “Edit” mode CUI Viewer allows the user to convert any widget into a widget of another type or change its attributes. “Edit” mode can be selected by pressing “Ctrl-E” (“Ctrl”+“E” buttons simultaneously). When this is done the user should:
  • a) select the widget he wants to convert or modify by making a right mouse button click inside this widget;
  • b) open the CUI Viewer pull down menu “Convert To . . . ” and select the new widget type by pressing a button inside desired widget type image.
  • FIG. 12 shows the full list of CUI Viewer commands currently supported in “Edit” mode.
  • Notes:
  • 1) CUI Viewer itself is written as CUI application; therefore the user can add or modify the list of widgets in the “Convert To” menu.
  • 2) CUI Viewer does not send any messages regarding the widget conversion to the application program, and the application continues its execution as usual.
  • The user can record input information to file and then play this file by using CUI Viewer commands “Record” and “Play” (FIG. 12, lines 11 and 12). In “Record” mode CUI Viewer stores only information about CUI Variables changes. Each record contains following information:
  • 1. Variable Name;
  • 2. Variable Value;
  • 3. Time when user changed Variable Value.
  • The CUI Viewer “Play” command can be used even if the positions, sizes or types of any Widgets have been changed (claim 12). During Play CUI Viewer reads these records and simulates execution (moves mouse pointer to widget with the same Variable Name as in the record and changes Variable Value) according to recorded time.
  • If there is more than one widget with the same name, CUI Viewer calculates the “Time To Set Value” for each such widget. The “Time To Set Value” is calculated as the time needed to move the mouse cursor at “Mouse Simulation Speed” to the closest point of the widget plus the time for “Value Simulation Change”. If the widget is not visible (for example, it is located in a closed window), the “Time To Set Value” is equal to the time needed to move mouse cursor and make this widget visible plus time needed for “Value Simulation Change”. The widget with minimum “Time To Set Value” is used.
  • In the current implementation the default value of “Mouse Simulation Speed” is equal to 5 centimeter per second, “Time To Set Value” is equal to number of the Mouse/Keyboard button clicks multiplied by 0.3 seconds per click.
  • Some methods of current invention are based on or intensively use the external and internal representation of different CUI data in XML format. All these XML-methods can be described as “static” or “dynamic” according to their usage of transformation of XML representations involved in the corresponding processes.
  • The static XML-methods use XML representation for storage and reading/writing procedures of CUI Description (claim 7) and widget library (claims 13, 17). They use no transformations of XML data.
  • Every dynamic method uses some XSL transformation to convert the initial XML representation of some part of CUI data into its new XML representation. When the transformation is complete the CUI Viewer uses this new XML representation for visualization of the same CUI data in some other (converted) way. This approach is implemented for automatic recalculation of coordinates and sizes of widgets (claim 14) and for automatic rule based conversion of attributes of replaced widgets (claim 15).
  • Notes:
  • 1) The XSL files used for our dynamic XML-methods are the external text files that form the special Transformation Library. Any of these files can be modified or replaced by a new XSL files, which can provide new rules for some transformation (claim 16).
  • 2) The use of a special naming convention for XSL files of Transformation Library makes it possible to support predefined lists for widget conversion (claims 8, 9)
  • 3) The use of this naming convention makes the Transformation Library expandable. There is a possibility not only to replace some or all XSL files, but also to add new XSL files providing conversion rules for a new customer's widgets.
  • 4) The names of CUI Variables can be used in XSL transformation files names.
  • 5) While processing an XSL file, a special function from CUI DLL has access to the current value of any CUI Variable registered with CUI Viewer. This makes it possible to use not only the names of CUI Variables but also their values during XSL transformation.

Claims (17)

1. A method of managing graphical user interfaces associated with application programs operable on a computing device having a visual display device and a user input device for controlling a pointer on said display device, said graphical user interfaces having a plurality of windows containing widgets for visualization and manipulation of application program data (variables and/or callback functions).
2. A method of claim 1, comprising the usage of dynamic links (connections) between said data and their corresponding representations (widgets) in the graphical user interface.
3. A method of claim 2, comprising the possibilities to create new, remove or replace existing said dynamic links.
4. A method of claim 1, comprising the possibilities for GUI modification during an execution of said application programs.
5. A method of claim 4, comprising the default said dynamic links for newly implemented said data.
6. A method of claim 4, comprising the change of default attributes for widgets used by said dynamic links.
7. A method of claim 4, comprising read/write procedures for previously established said dynamic links and/or the widget attributes.
8. A method of claim 4, comprising the replacement of a widget in said dynamic link by another widget from a predefined list.
9. A method of claim 8, wherein said predefined list is constructed according to the type of data associated with the said dynamic link.
10. A method of claim 4, wherein the GUI's element (widget) representing an integer editable limited value can be converted into the slider type element (widget).
11. A method of claim 1, comprising the recording/replaying functionality for user's interaction with convertible GUI.
12. A method of claim 11, wherein a prerecorded sequence of said user's interaction for any initial state of GUI can be replayed for any new (converted) state of the same GUI.
13. A method of claim 1, comprising the support of widget library, designed to work with the said dynamic links.
14. A method of claim 6, comprising the automatic recalculation of coordinates and sizes for widget during resizing of its parent widget.
15. A method of claim 8, comprising the automatic rule based conversion of attributes of replaced widget.
16. A method of claim 15, comprising the customization of rules for said rule based conversion of attributes of replaced widget.
17. A method of claim 13, comprising the run time creation of new and modification of existing widgets including their associations with the callback functions.
US10/707,343 2003-12-05 2003-12-05 Convertible runtime graphical user interface Abandoned US20050125787A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US10/707,343 US20050125787A1 (en) 2003-12-05 2003-12-05 Convertible runtime graphical user interface
US11/948,693 US7904821B1 (en) 2003-12-05 2007-11-30 Graphical user interface that is convertible at runtime

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US10/707,343 US20050125787A1 (en) 2003-12-05 2003-12-05 Convertible runtime graphical user interface

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US11/948,693 Continuation US7904821B1 (en) 2003-12-05 2007-11-30 Graphical user interface that is convertible at runtime

Publications (1)

Publication Number Publication Date
US20050125787A1 true US20050125787A1 (en) 2005-06-09

Family

ID=34633149

Family Applications (2)

Application Number Title Priority Date Filing Date
US10/707,343 Abandoned US20050125787A1 (en) 2003-12-05 2003-12-05 Convertible runtime graphical user interface
US11/948,693 Expired - Fee Related US7904821B1 (en) 2003-12-05 2007-11-30 Graphical user interface that is convertible at runtime

Family Applications After (1)

Application Number Title Priority Date Filing Date
US11/948,693 Expired - Fee Related US7904821B1 (en) 2003-12-05 2007-11-30 Graphical user interface that is convertible at runtime

Country Status (1)

Country Link
US (2) US20050125787A1 (en)

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070038934A1 (en) * 2005-08-12 2007-02-15 Barry Fellman Service for generation of customizable display widgets
US20070101291A1 (en) * 2005-10-27 2007-05-03 Scott Forstall Linked widgets
US20080295004A1 (en) * 2007-05-22 2008-11-27 Coca Dinesh C Apparatus, system, and method for customizing a graphical user interface
US20090055757A1 (en) * 2007-08-20 2009-02-26 International Business Machines Corporation Solution for automatically generating software user interface code for multiple run-time environments from a single description document
US20090158180A1 (en) * 2007-12-17 2009-06-18 Yahoo! Inc. System and method of styling an embedded widget
US20090248996A1 (en) * 2008-03-25 2009-10-01 Mandyam Giridhar D Apparatus and methods for widget-related memory management
US20090249359A1 (en) * 2008-03-25 2009-10-01 Caunter Mark Leslie Apparatus and methods for widget intercommunication in a wireless communication environment
US20090248883A1 (en) * 2008-03-25 2009-10-01 Lalitha Suryanarayana Apparatus and methods for managing widgets in a wireless communication environment
US20090249321A1 (en) * 2008-03-25 2009-10-01 Mandyam Giridhar D Apparatus and methods for widget update scheduling
US20090271778A1 (en) * 2008-03-25 2009-10-29 Mandyam Giridhar D Apparatus and methods for transport optimization for widget content delivery
US20110029949A1 (en) * 2009-07-28 2011-02-03 International Business Machines Corporation Widget combos: a widget programming model
US7904821B1 (en) * 2003-12-05 2011-03-08 Leonid M. Tertitski Graphical user interface that is convertible at runtime
US20120089939A1 (en) * 2010-10-06 2012-04-12 International Business Machines Corporation Combined properties dialog from different sources
CN101763255B (en) * 2008-12-23 2012-10-31 中国移动通信集团公司 Format conversion method and device of special interface tool
US20140108985A1 (en) * 2012-10-08 2014-04-17 Fisher-Rosemount Systems, Inc. Configurable User Displays in a Process Control System
US8826158B1 (en) * 2011-12-14 2014-09-02 The United States Of America As Represented By The Director, National Security Agency Device for and method of determining changes to GUI
US9047106B1 (en) * 2010-06-29 2015-06-02 Emc Corporation Managing application features
US9417888B2 (en) 2005-11-18 2016-08-16 Apple Inc. Management of user interface elements in a display environment
US9483164B2 (en) 2007-07-18 2016-11-01 Apple Inc. User-centric widgets and dashboards
US9507503B2 (en) 2004-06-25 2016-11-29 Apple Inc. Remote access to layer and user interface elements
US9513930B2 (en) 2005-10-27 2016-12-06 Apple Inc. Workflow widgets
US11216159B2 (en) 2012-10-08 2022-01-04 Fisher-Rosemount Systems, Inc. Configuration element for graphic elements
US11774927B2 (en) 2012-10-08 2023-10-03 Fisher-Rosemount Systems, Inc. Methods and apparatus to provide a role-based user interface

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8392878B2 (en) * 2007-10-31 2013-03-05 National Instruments Corporation In-place structure in a graphical program
US8656314B2 (en) 2009-07-30 2014-02-18 Lenovo (Singapore) Pte. Ltd. Finger touch gesture for joining and unjoining discrete touch objects
US8762886B2 (en) 2009-07-30 2014-06-24 Lenovo (Singapore) Pte. Ltd. Emulating fundamental forces of physics on a virtual, touchable object
US20110029864A1 (en) * 2009-07-30 2011-02-03 Aaron Michael Stewart Touch-Optimized Approach for Controlling Computer Function Using Touch Sensitive Tiles
US20110029904A1 (en) * 2009-07-30 2011-02-03 Adam Miles Smith Behavior and Appearance of Touch-Optimized User Interface Elements for Controlling Computer Function
US8954880B1 (en) * 2012-09-28 2015-02-10 Emc Corporation User interface having user-customized default values
US20140129077A1 (en) * 2012-11-05 2014-05-08 Service Solutions U.S. Llc Scan tool with configurable shortcuts

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5600780A (en) * 1990-03-30 1997-02-04 International Business Machines Corporation Programming environment system for customizing a program application based upon user input
US5603034A (en) * 1992-09-08 1997-02-11 International Business Machines Corporation Graphical resource editor for software customization
US5652884A (en) * 1994-11-14 1997-07-29 Object Technology Licensing Corp. Method and apparatus for dynamic update of an existing object in an object editor
US5786815A (en) * 1996-05-31 1998-07-28 Sun Microsystems, Inc. Configurable runtime graphical user interface widget management
US6208336B1 (en) * 1998-03-20 2001-03-27 Sun Microsystems, Inc. Dynamic graphical user interface feature-set configuration
US20020099456A1 (en) * 2000-11-13 2002-07-25 Mclean Alistair William User interfaces
US6577319B1 (en) * 1998-09-18 2003-06-10 Sharp Kabushiki Kaisha Method for controlling data display
US20030159129A1 (en) * 2002-02-15 2003-08-21 Tridium, Inc. Component model for real time system control
US20040236860A1 (en) * 2000-05-30 2004-11-25 Gary Logston Method and apparatus for balancing distributed applications

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1999004349A1 (en) * 1997-07-15 1999-01-28 Koninklijke Philips Electronics N.V. A method and system for designing a graphical user interface for an electronic consumer product
US6237004B1 (en) * 1998-02-24 2001-05-22 International Business Machines Corporation System and method for displaying data using graphical user interface control elements
US20030061482A1 (en) * 2001-08-23 2003-03-27 Efunds Corporation Software security control system and method
US7895522B2 (en) * 2001-09-28 2011-02-22 Ntt Docomo, Inc. Layout of platform specific graphical user interface widgets migrated between heterogeneous device platforms
CA2388101A1 (en) 2002-02-01 2003-08-01 Concepts Egeria Inc. Conceptual user interface
US7415484B1 (en) * 2003-05-09 2008-08-19 Vignette Corporation Method and system for modeling of system content for businesses
US8479101B2 (en) 2003-11-13 2013-07-02 Koninklijke Philips Electronics N.V. Consistent user interface front end for remote user interfaces
US20050125787A1 (en) * 2003-12-05 2005-06-09 Leonid Tertitski Convertible runtime graphical user interface

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5600780A (en) * 1990-03-30 1997-02-04 International Business Machines Corporation Programming environment system for customizing a program application based upon user input
US5603034A (en) * 1992-09-08 1997-02-11 International Business Machines Corporation Graphical resource editor for software customization
US5652884A (en) * 1994-11-14 1997-07-29 Object Technology Licensing Corp. Method and apparatus for dynamic update of an existing object in an object editor
US5786815A (en) * 1996-05-31 1998-07-28 Sun Microsystems, Inc. Configurable runtime graphical user interface widget management
US6208336B1 (en) * 1998-03-20 2001-03-27 Sun Microsystems, Inc. Dynamic graphical user interface feature-set configuration
US6577319B1 (en) * 1998-09-18 2003-06-10 Sharp Kabushiki Kaisha Method for controlling data display
US20040236860A1 (en) * 2000-05-30 2004-11-25 Gary Logston Method and apparatus for balancing distributed applications
US20020099456A1 (en) * 2000-11-13 2002-07-25 Mclean Alistair William User interfaces
US20030159129A1 (en) * 2002-02-15 2003-08-21 Tridium, Inc. Component model for real time system control

Cited By (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7904821B1 (en) * 2003-12-05 2011-03-08 Leonid M. Tertitski Graphical user interface that is convertible at runtime
US10489040B2 (en) 2004-06-25 2019-11-26 Apple Inc. Visual characteristics of user interface elements in a unified interest layer
US9753627B2 (en) 2004-06-25 2017-09-05 Apple Inc. Visual characteristics of user interface elements in a unified interest layer
US9507503B2 (en) 2004-06-25 2016-11-29 Apple Inc. Remote access to layer and user interface elements
US20070038934A1 (en) * 2005-08-12 2007-02-15 Barry Fellman Service for generation of customizable display widgets
US8924869B2 (en) 2005-08-12 2014-12-30 Barry Fellman Service for generation of customizable display widgets
US9104294B2 (en) * 2005-10-27 2015-08-11 Apple Inc. Linked widgets
US20070101291A1 (en) * 2005-10-27 2007-05-03 Scott Forstall Linked widgets
US11150781B2 (en) 2005-10-27 2021-10-19 Apple Inc. Workflow widgets
US9513930B2 (en) 2005-10-27 2016-12-06 Apple Inc. Workflow widgets
US9417888B2 (en) 2005-11-18 2016-08-16 Apple Inc. Management of user interface elements in a display environment
US20080295004A1 (en) * 2007-05-22 2008-11-27 Coca Dinesh C Apparatus, system, and method for customizing a graphical user interface
US7987430B2 (en) 2007-05-22 2011-07-26 International Business Machines Corporation Apparatus, system, and method for customizing a graphical user interface
US9483164B2 (en) 2007-07-18 2016-11-01 Apple Inc. User-centric widgets and dashboards
US20090055757A1 (en) * 2007-08-20 2009-02-26 International Business Machines Corporation Solution for automatically generating software user interface code for multiple run-time environments from a single description document
US20090158180A1 (en) * 2007-12-17 2009-06-18 Yahoo! Inc. System and method of styling an embedded widget
US9069575B2 (en) 2008-03-25 2015-06-30 Qualcomm Incorporated Apparatus and methods for widget-related memory management
US9600261B2 (en) 2008-03-25 2017-03-21 Qualcomm Incorporated Apparatus and methods for widget update scheduling
US10481927B2 (en) 2008-03-25 2019-11-19 Qualcomm Incorporated Apparatus and methods for managing widgets in a wireless communication environment
US20090248996A1 (en) * 2008-03-25 2009-10-01 Mandyam Giridhar D Apparatus and methods for widget-related memory management
US10558475B2 (en) 2008-03-25 2020-02-11 Qualcomm Incorporated Apparatus and methods for widget intercommunication in a wireless communication environment
US20090248883A1 (en) * 2008-03-25 2009-10-01 Lalitha Suryanarayana Apparatus and methods for managing widgets in a wireless communication environment
US10061500B2 (en) 2008-03-25 2018-08-28 Qualcomm Incorporated Apparatus and methods for widget-related memory management
US9110685B2 (en) * 2008-03-25 2015-08-18 Qualcomm, Incorporated Apparatus and methods for managing widgets in a wireless communication environment
US9269059B2 (en) 2008-03-25 2016-02-23 Qualcomm Incorporated Apparatus and methods for transport optimization for widget content delivery
US20090249321A1 (en) * 2008-03-25 2009-10-01 Mandyam Giridhar D Apparatus and methods for widget update scheduling
US20090271778A1 (en) * 2008-03-25 2009-10-29 Mandyam Giridhar D Apparatus and methods for transport optimization for widget content delivery
US20090249359A1 (en) * 2008-03-25 2009-10-01 Caunter Mark Leslie Apparatus and methods for widget intercommunication in a wireless communication environment
US9747141B2 (en) 2008-03-25 2017-08-29 Qualcomm Incorporated Apparatus and methods for widget intercommunication in a wireless communication environment
CN101763255B (en) * 2008-12-23 2012-10-31 中国移动通信集团公司 Format conversion method and device of special interface tool
US20110029949A1 (en) * 2009-07-28 2011-02-03 International Business Machines Corporation Widget combos: a widget programming model
US8495566B2 (en) 2009-07-28 2013-07-23 International Business Machines Corporation Widget combos: a widget programming model
US9047106B1 (en) * 2010-06-29 2015-06-02 Emc Corporation Managing application features
US8812974B2 (en) * 2010-10-06 2014-08-19 International Business Machines Corporation Combined properties dialog from different sources
US20120089939A1 (en) * 2010-10-06 2012-04-12 International Business Machines Corporation Combined properties dialog from different sources
US8826158B1 (en) * 2011-12-14 2014-09-02 The United States Of America As Represented By The Director, National Security Agency Device for and method of determining changes to GUI
US20140108985A1 (en) * 2012-10-08 2014-04-17 Fisher-Rosemount Systems, Inc. Configurable User Displays in a Process Control System
US10444949B2 (en) * 2012-10-08 2019-10-15 Fisher-Rosemount Systems, Inc. Configurable user displays in a process control system
US10691311B2 (en) 2012-10-08 2020-06-23 Fisher-Rosemount Systems, Inc. Derived and linked definitions with override
US9792004B2 (en) 2012-10-08 2017-10-17 Fisher-Rosemount Systems, Inc. Derived and linked definitions with override
US11216159B2 (en) 2012-10-08 2022-01-04 Fisher-Rosemount Systems, Inc. Configuration element for graphic elements
US11599251B2 (en) 2012-10-08 2023-03-07 Fisher-Rosemount Systems, Inc Derived and linked definitions with override
US11650718B2 (en) 2012-10-08 2023-05-16 Fisher-Rosemount Systems, Inc. Derived and linked definitions with override
US11774927B2 (en) 2012-10-08 2023-10-03 Fisher-Rosemount Systems, Inc. Methods and apparatus to provide a role-based user interface

Also Published As

Publication number Publication date
US7904821B1 (en) 2011-03-08

Similar Documents

Publication Publication Date Title
US7904821B1 (en) Graphical user interface that is convertible at runtime
CN108021363B (en) Visual game logic editing method and system
US6792595B1 (en) Source editing in a graphical hierarchical environment
US6038395A (en) System and method for implementing proxy objects in a visual application builder framework
US7190369B2 (en) Application programming interface for changing the visual style
US5551055A (en) System for providing locale dependent user interface for presenting control graphic which has different contents or same contents displayed in a predetermined order
US5715416A (en) User definable pictorial interface for a accessing information in an electronic file system
US5530864A (en) Command object system for an object-oriented software platform
US8386919B2 (en) System for displaying an annotated programming file
Petzold Creating Mobile Apps with Xamarin. Forms Preview Edition 2
US20090128581A1 (en) Custom transition framework for application state transitions
Mark et al. Beginning iphone 4 development
US20220229545A1 (en) Intelligent manipulation of dynamic declarative interfaces
US20090288069A1 (en) Dynamic Declarative Application Description
KR100562905B1 (en) Method for dynamic layout in application program module
Mark Beginning iOS 5 Development: Exploring the iOS SDK
Craig et al. Learn android studio: build android apps quickly and effectively
US7574336B1 (en) Model workspaces in modeling environments
US7882486B2 (en) Adding interactivity to artwork
Vullinghs et al. Lightweight GUIs for functional programming
Lewis et al. Native mobile development: a cross-reference for iOS and Android
Chin et al. Javafx fundamentals
Mark et al. Beginning iPhone development
Fitzpatrick Create Simple GUI Applications with Python & Qt5
Fields et al. IntelliJ IDEA in Action

Legal Events

Date Code Title Description
STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION