Revolutionary User Interface Design: Tackling Cognitive Overload in Graphics Software

Content Overview

• An Example of Cognitive Overload
• A Mad Proposal
• A Mad Comparison to Digital Compositing Node Graph
• What Changed? Nothing, Really
• An Insane User Interface
• Then, things Start to get Crazier
• Digital Compositing Paradigm in a Pipeline
• Omg! Advantages. Lol!
• You must be Mad too

An Example of Cognitive Overload

Imagine a graphic artist performing a series of photo edits:

1. Apply Disc Blur to an image.
2. Apply Sepia Tone to Disc Blur.
3. Blend the Sepia Tone with the original.

The intent of the artist usually consists of a sequence of actions to be translated into operations carried out with the graphics software.

For example, the process of the photo edits above may involve creating a new document, selecting the optimal image options in the application menu, creating Layers to apply a Disc Blur, changing properties in a side panel, bringing up an object dialog box, and repeating the process to add additional layers for the Sepia Tone/Blend to achieve the desired effect.

This NON-STREAMLINED approach of user interface disrupts the flow of thoughts of the artist and when exacerbated with an increasing number of steps results in Cognitive Overload and loss of creative productivity.

We are OK with the above. What's WRONG?

A Mad Proposal

We streamline the user interface into a linear sequence of actions that can be compounded or stacked like a digital compositing node graph but with simplification to express how to achieve the desired outcome succinctly, like below:

Pipeline

1. Read Image
2. Disc Blur
3. Sepia Tone
4. Blend (2,3)

A Mad Comparison to Digital Compositing Node Graph

To someone familiar with Digital Compositing, they will immediately recognize that the list of steps, the Pipeline, above is "almost" identical to a digital compositing node graph. A digital compositing node graph is shown below:

Read Image
|         |
|         |
|         V
|         Disc Blur
|         |
|         |
|         V
|         Sepia Tone
|         |
|         |
V         V
   Blend

What Changed? Nothing, Really

A Pipeline is indeed similar to a digital compositing node graph with a subtle yet significant change. Instead of displaying a full node graph graphically, with two lines from the Read Image node and Sepia Tone node connecting to the Blend node. The Pipeline simplifies the node graph by just showing a list of steps and a spreadsheet-like function for the final Blend step.

4. Blend (2,3)

This creativity of arranging a digital compositing node graph as a Pipeline enables the intent of the artist to be conveyed as a succinct list of steps, following the artist's flow of thoughts from start to end, without a graphical map. The Pipeline also guides the artist into thinking of what to do next in a streamlined manner. Without using a traditional node graph, more information can be conveyed in a smaller amount of space linearly, providing a larger contextual overview to the creative mind.

An Insane User Interface

A Pipeline, in software, maps directly to a list and is easy to implement. The list removes the need for a complicated graphical map that may require panning (scrolling) to identify the right node to work on. The properties screen of a node can also be brought up intuitively and naturally by drilling into the node.

Pipeline

1. Read Image
2. Disc Blur  ----> Drill into the Properties screen
3. Sepia Tone
4. Blend (2,3)

PROTOTYPE - Fits into a mobile phone

Then, Things Start to get Crazier

No loss of information

The important thing to note is the above creativity does not lose any information. The simplified node graph above, the Pipeline, in a mathematical sense, can be fully represented as a node graph (Directed Acyclic Graph) in any manner or form. For example, if we have the following node graph for video processing.

Read media Read media 
|          |
V          V
Premult    Color Correction 
|          |
|          V 
|          Transform 
|          |
|          V 
|          Gaussian Blur 
|          | 
V          V 
Merge 
|
V 
Viewer

It can be flattened, streamlined, and represented as follows:

Read media
|
V
Premult

Read media
|
V
Color Correction
|
V
Transform
|
V
Gaussian Blur

Merge
|
V
Viewer

Note the break from the first series after the 'Premult' node and the second break after the 'Gaussian Blur' node.

Pipeline

1. Read media         >
2. Premult            >

3. Read media         >
4. Color Correction   >
5. Transform          >
6. Blur               >

7. Merge (2,6)        >
8. Viewer             >

The space between node 2 and node 3 represents the break, while the '>' is optional and used to indicate the availability of a properties screen.

Digital Compositing Paradigm in a Pipeline

The paradigm of node-based digital compositing where we can lay out steps in a progression and make modifications to an earlier step to change the outcome is completely kept. Each node can be easily reused by referencing it with a node number as in Step 7, and if required, used further in a later step, saving time and effort. In the same way that a digital compositing node graph is broken into a series of nodes in a graph, a Pipeline breaks it into a series of actions or steps. A Pipeline, from another angle, is thus like a single-column spreadsheet, a metaphor that many computer users are quite familiar with.

Viewing the output of each step: Viewer Cycling

The arrangement in a series of steps enables the output of each progression step to be viewed easily, for example, by long pressing on a node on the Pipeline. This enables an artist to step through the progression to reinforce his or her cognitive working memory of what has been performed.

Connecting the steps: Auto-Chaining

Each node on the Pipeline that requires an input can be assumed to be taking the output of the previous node automatically. For example, the 'Premult' node in Step 2 is assumed to automatically take the output from 'Read Media' in Step 1.

Connecting the steps like a node graph: Reverse Compositing

The auto-chaining or automatic preceding node referencing behavior can also be altered by enabling the artist to make changes in the properties of a node. For example, in Step 7, the artist can choose from its properties screen any of the previous nodes, such as nodes 2 and 6, for blending.

Omg! Advantages. Lol!

Modular Extensibility

A Pipeline, being almost identical to a digital compositing node graph, enables it to achieve the same modular extensibility in software development. Software developers can think of developing graphical effects, filters, animations, and 3D operations modularly as a node. Each node is accompanied by a properties user interface that does not interfere with other parts of the software. A node and its user interface can thus be developed and tested independently while knowing ultimately that it can be easily integrated into a full node graph, aka the Pipeline, to achieve any complex graphics operation. Such a design will benefit the open-source community greatly where many different developers can build an open-source Pipeline, the Open Pipeline, together in a distributed and collaborative manner.

Reduce Information Overload

To start using a graphics application, artists usually need to learn the way of achieving it through documentation. Often, the documentation requires an artist to follow a series of screenshots, explaining where and how to click or tap on different parts of the screen using a non-streamlined user interface offered by the graphics application. The non-streamlined user interface makes the information hard for an artist to remember and may require the artist to constantly refer to the documentation. Over time, this introduces an additional cost that, when accumulated, can lead to Information Overload.

Succinct Self-Describing Pipeline

The Open Pipeline reduces Information Overload as it succinctly describes what needs to be done and allows all necessary details to be shown in context nearby. This significantly benefits documentation: tutorials, articles, or books.

Open Pipeline

1. Disc Blur: radius = 8
2. Sepia Tone: intensity = 1
3. Blend (1,2)

The details of the radius for Disc Blur are shown as part of the node in the Pipeline. Moreover, besides being succinct, it is common for the above description to NOT REQUIRE supporting screenshots as it is self-explanatory.

You must be Mad too

If you have followed this article till here, find out more about the author's banishment due to his 'NOT SO' Open Pipeline, and where he is living in exile. His ‘BLUFF’ of implementing a Photo Curve node that ‘CAN BE INTEGRATED EASILY’ into the Pipeline. And him performing a Photo Fire Effect MAGIC TRICK. REALLY. He compares himself with the Roman poet Ovid by quoting on ‘how shapes changed into new bodies’ in Metamorphoses:

In nova fert animus mutatas dicere formas corpora; di, coeptis (nam vos mutastis et illas) adspirate meis primaque ab origine mundi ad mea perpetuum deducite tempora carmen. – OVID

Sigh, I give up

He further mumbles about animating a graphics object with a METAMORPHOSIS node.