It was a tremendous honor to give a GDC talk this year and the positive reception I’ve received took me by surprise. The opportunity has connected me with UI folks from all different backgrounds across the world and has resulted in a steady stream of people reaching out for advice, guidance, and mentorship.
So I think it’s worth putting together some of the tips/tricks/best practices & terminology to help give everyone a solid foundation for approaching UI.
But please take this all with a grain of salt. My viewpoint stems from a particular lens of providing UI for traditional games & there are a lot of other mediums out there. So with that said, let’s get started.
A Note on UI Roles
Unlike film or television, roles & titles in the games industry are fairly loose. What a UI Designer might do at one studio could be vastly different from another. However, I do want to provide a baseline for the kinds of responsibilities that are common for UI Roles.
- UI Designer – Traditionally a UI Designer is NOT a technical role. UI Designers are a bridge role that connects the overall design team to the UI group. They are responsible for creating, pitching & iterating on Wireframes. They often cross over into the UX space to oversee internal/external testing & can often be a point person for disseminating learnings from these tests.
- UI Artist – A UI Artist is traditionally a 2D artist with some graphic design background. They often author assets in Photoshop or Illustrator. (At Santa Monica, we have both 2D & 3D UI Artists due to our pipeline) may also be responsible for implementing art within an engine or working in conjunction with a UI Programmer or scripter to iterate on the look of an in-game asset.
- UI Programmer – UI Programmers are generally Gameplay Programmers, with a background in C++. Depending on the engine used, they may be the primary feature implementor for the UI. I.E. the one who builds the functionality for a given UI element, or they play more of a support role in owning specific UI architecture while exposing gameplay data to UI Scripters/Technical Designers.
- UI Scripter/Technical Designer – UI Scripters/Technical designers exist when the UI framework is either a custom engine or a 3rd party UI middleware, like Scaleform (Now defunct) or Coherent. They are generally the primary implementor for the UI Team & are skilled in script-like languages such as Lua or Javascript and may also end up working in a visual scripting framework like Unreal’s Blueprint.
The UI Pipeline
Lesson 1 – Design Intent
UI Development starts the moment that design starts. This is called a Design Intent, or an Ask. Sometimes it’s framed directly as a visual representation of a system, I.E. We need to represent a player’s health. Other times it’s in response to a User Experience (UX) problem such as, Players don’t know when they’ve picked something up and the answer is a UI solution.
So for example to solve the above problem that, Players don’t know when they’ve picked something up, we’re going to create an Item Log. A small message system that will temporarily show an indicator of the item the player picked up.
But when building UI it’s important to iron out the details such as:
- Can players pick up multiple items at once? If so, what is the max number we want to allow onscreen?
- What happens if the player picks up something & then steps into a cinematic?
- What kind of information should we display? An Icon? The name of the Item? How about its rarity?
Lesson 2 – Wireframing
Once we know what we’re building, the next step is a wireframe which generally is a simple 2D Mockup of your UI. It can be used for whole menus, player HUDs, or even individual UI elements. The concept behind a wireframe is that it’s quick, you can make several in succession to explore a variety of ideas & it gives an overall idea of what a UI element consists of & its general shape.
In the AAA world, we wireframe EVERY element. We do this to explore the problem space, to align everyone on the initial direction of a prototype element, to agree on exactly what data that element needs/should display, and to get buy-in from the team we’re usually partnering to work with on this feature.
Doing a wireframe is super cheap in terms of development time & provides a visual reference for whoever is implementing to constantly check against.
A wireframe can be as simple as a napkin/whiteboard drawing, or as complex as a fully art’ed clickable prototype. The most common tool for wireframing is Figma, however, you should use whatever the team feels gets the job done. I’ve been using Adobe Illustrator for 10+ years for wireframing.
Lesson 3A – Initial Prototype
Just like in Level Design, the first pass implementation for a UI element is often made of blocks or simplified shapes. Your goal should be to get the raw functionality in without worrying too much about the visual appearance of the UI element. There are a few reasons to do this.
- Rapid Prototype Mentality – This stands up the feature as quickly as possible to unblock other groups & to begin testing/feedback.
- Solves Data Workflow – It forces your team to figure out the data-side workflow to support this element. I.E. How is the UI receiving the specific information needed for this element? In some instances it may uncover that additional API is needed or flaws in the assumptions made in what kinds of data you needed.
- Scope Bet – It’s a hedge bet against scope, design changes rapidly & the initial concept everyone agreed on during the Design Intent might already be out of date so standing up the bare-bones element may help expose changes or immediate iterations of the element. Changing an element after it’s received a full art-support pass is far more time-consuming/risk-prone than changing a skeleton version.
Lesson 3B – Art Mockup & Iteration
If your studio has the resources, this is where specialized UI roles come into play as this work can run in parallel with Lesson 3A. A UI Artist can then take the approved wireframe & continue to iterate visually creating a High-Fidelity Mockup.
Creating a high-fidelity mockup early allows the UI team to begin to understand the overall art scope related to the UI element, provides time for critique, and an opportunity to buy consensus from other teams or the project’s art director.
What’s valuable about this early mockup is that it allows the team to see if there’s an opportunity for Asset Reuse (Utilizing previously made UI elements/assets). Reusing assets is key to a low Memory Footprint & overall UI optimization. This is often called Object Pooling, which I’ll dig into more in a future post.
An early mockup also helps ensure Consistency throughout the UI, which is one of the important tenets of UI Design/Graphic Design. The UI should be consistent throughout the game’s experience in order to build Player Confidence & mastery in how they navigate & use the UI. When a player’s confidence reaches a certain level, navigating the UI becomes nearly automatic or instinctual resulting in a term called ‘Invisible UI‘. The gold standard for most UI systems is to have it become invisible so that players don’t even notice it anymore & instead are deeply engaging with the system it represents.
Lesson 4 – Playtest, Playtest, Playtest
But you can only achieve great UI through iteration & putting it in front of players. And it’s so fundamental to the process of creating great UI that we’re often called User Interface & User Experience (UI/UX) designers or belong to the UI/UX Team. We’re a part of every playtest simply because of the immediacy of the UI, it’s literally in front of the player at all times.
Depending on your role & the resources at your studio, your direct involvement with setting up & running a testing session may vary (At SMS we have a UX Director who manages the testing process).
Typically the UI Teams I’ve been a part of all sit together once a week to have a group testing/critique session. We pick some aspect of the UI, maybe something that has changed recently, or maybe something we haven’t looked at for a while, and as a group play through the content. As we discuss, we keep a running list of ideas/changes that get turned into Jira tickets at the end of the meeting and assigned out.
For larger playtest sessions, we’re typically watching multiple users simultaneously to get an overall feeling about what aspects are working, not working, or being ignored. At specific stages of the test, playtesters are asked to stop & fill out a survey. We often work with the UX Director beforehand to generate any UI-specific questions.
Lesson 5 – Ship it
The last lesson is to step away.
Because no game is ever truly done, they just reach a point where it’s good enough to ship. Game development is a continuous lesson in Triage. You only ever have so much time & and are constantly confronted with more than you can possibly do. The only way forward is one step at a time, prioritizing the elements/systems that have the greatest impact while balancing scope and making the hard decisions about what to leave on the cutting room floor.
Stepping away closes a chapter, but it also opens one with the start of a new project where you can apply all those lessons you’ve learned to make something even greater.