(under construction)

Presence Event entry system

Date: Apr. 2018 - Ongoing
Roles: UX Researcher
Methods: Remote Moderated Interviews & Usability Testing, Concept Ideation, Concept Testing, Contextual Inquiry, In-Person Observation & Interviews, Moderated Card-Sorting, Persona Development, Survey, Affinity Diagramming

Background: The Presence event entry system is one of the most cutting edge technologies in entry systems today. Not just a platform to help live event venues and promoters check real-time attendance data, the Presence system allows for better fan identity and the potential for in-app messaging to fans while they are at events in the future. The Presence platform has multiple components: a frontend UI and an Android scanner app under the nomenclature TM1 Entry are the main components in which clients interact. On the frontend UI, a platform where clients in the live event space can set their rules for ensuring only valid tickets scan into their event, and also which ticket types can enter which areas of their venue. For example, they set rules in the frontend UI that allow VIP ticket types to only be able to scan into VIP areas, and prevent people who don’t have VIP tickets from entering. The scanner hardware is an Android phone fitted with special protective hardware for scanning tickets. Presence promises to not just be an entry control system, but also a way to When I first started on this project, a new designer had taken over as the lead designer. We quickly became aware that this product had not had much research done prior to us taking it on and so, we needed to do a lot of research to understand any gaps in the current experience, current usability issues, and understand what opportunities we have in the future.

Goal: Begin to unpack and understand client pain points with the current Presence system and start building in new features that will actually matter to them. Begin understanding any new opportunities we weren’t fully realizing and how to design them into Presence.

The suite of Presence products (from left-to-right): an Android scanner app retro-fitted with surrounding hardware, frontend UI called TM1 Entry with an updated dashboard displaying real-time attendance data, and messaging capabilities through the Ticketmaster app (coming soon)


Research questions

  • On a high-level, what is the typical event day like for our clients?

  • What our are clients’ mental models when opening up TM1 Entry?

  • What enhancements can we make to the TM1 Entry frontend UI that provide the most value to our clients? Why do they provide value?

  • What are the current pain points in the experience of using TM1 Entry and Presence scanner app for our clients? Can we ideate on how to help alleviate the pain points?

We had many more research questions, but in the in the interest of keeping this brief, I won’t elaborate on all of them

At an internal conference on scaling Presence to more clients, we touched on the fact that Presence began as an experiment and proof of concept (POC)… an experiment with no research done before building.

Discover

Approach: Client Interviews, Usability Testing, Contextual Inquiry, In-Field Observation

The need for intense research was apparent from the beginning of the project. Presence was originally an internal experiment in 2015 to see if we could “kill the barcode” in ticketing. However, what didn’t go into developing Presence was research, as we discovered. So, in spring of 2018, the new lead designer of Presence and I set out to really understand how our clients not only use Presence to track event entry data, but just how they operate on event days in general. What are their goals, challenges, etc. when thousands of fans are all coming into their venue at the same time? So, we initially set up 7 client interviews to really discover the landscape of how our clients were currently using Presence and potential pain points. Then, we set-up two in-person observation and interview sessions on site with two major sports league teams in their arenas. We really needed a foundational lay-of-the-land to understand what our clients were trying to accomplish on the day-of-an-event and if our product was meeting their needs; if it wasn’t, we had to dig into why not and begin ideating on opportunities to improve the user experience. To begin testing, we had a few mocks to test a some re-designs, with an emphasis on a new feature called Access Groups, where the client is able to set entry rules for fans based on a ticket type, not just a particular section. We also wanted to test an enhanced dashboard of aggregate event attendee data to really understand which data points were most important to our clients and why.

Initial Findings

When testing a new feature called Access Groups, every client requested the feature to function in a way they can search across their ticket types and add them, instead of manually typing them in. We wouldn’t have known to build the feature this way without research!

When testing a new feature called Access Groups, every client requested the feature to function in a way they can search across their ticket types and add them, instead of manually typing them in. We wouldn’t have known to build the feature this way without research!

One of the biggest findings from our initial client interviews and usability testing was a gap in real-time data feeding into TM1 Entry (frontend UI component of Presence). This is purely a systemic bug that could hopefully be resolved by the development team. A few other bugs were discovered during the on-site observations on event days, meaning going on-site and observing our clients encounter bugs during their workflow within TM1 Entry was key to getting them resolved. We were able to report back bugs and the development team had patches for them quickly. As far as new features, every single client gave us feedback that our concept of Access Groups would work better for them if the functionality mirrored another system we have called Arctics, one of the systems clients use to build events and sell tickets. When clients build events, they have to input event codes (alphanumeric characters) to help label the event, in addition to the title. In Archtics, clients can use a wildcard syntax to call out all tickets that have a similar prefix in their event codes and select multiple events at once. This functionality was not apparent to us in the original design, so the client interviews 100% shaped how Access Groups should function. We also discovered that clients had to set their entry rules for their event every single day they have an event. There was no “save” function within TM1 Entry. Clients had to go into the UI every single time they needed to set entry rules, meaning the process is extremely manual and means the clients are spending a lot of time doing repetitive tasks. They ask for a way in which they could set entry rules once and apply those across multiple events as a way to save them hours of time.

We also learned that a part of the UI called the Scan Log is an area where clients will trouble-shoot during the event to check is their scanners are all online and scanning properly. Enhanced filters to the Scan Log would help our clients trouble-shoot better in real-time. In addition, our clients mentioned they would like a few enhancements to our entry data dashboard, including a scan rate chart, which will show the amount of fans entering their venue in specific time intervals. For them, they universally wanted options: 5 mins, 10 mins, and 15 mins. We found out that their senior management want updated, accurate information in those increments, and knowing the scan rate helps them with logistical planning in real-time and for their next event. We gathered feedback on many other parts of the system and on the new prototype, including several major pain points, but these were some of the key areas where we discovered interesting takeaways and areas of opportunity to really enhance the experience.

The updated dashboard which will show clients to amount of fans inside their venue at any given time, how many fans have entered in specific time intervals, and which seats are populated with the seat map.

The updated dashboard which will show clients to amount of fans inside their venue at any given time, how many fans have entered in specific time intervals, and which seats are populated with the seat map.

Two photos from the initial on-site observations and interviews we did at two separate major league sports arenas. In-person observation and contextual inquiry are absolutely necessary in really understanding what our users are actually trying to accomplish and understanding their goals and current challenges. Contextual inquiry can really never be replaced with usability testing or any method with a more controlled environment. The real world is a must.


Define

Approach: Card-Sorting, Usability Testing, Personas

Results from one of the feature priority card-sorting activities for the frontend UI component. We also did a similar card-sort for features for the scanner app.

Results from one of the feature priority card-sorting activities for the frontend UI component. We also did a similar card-sort for features for the scanner app.

After discovering so many areas of opportunities on how to enhance the user experience of both the frontend UI and scanner app, I decided a card-sorting activity where our clients could actually rank new features by relative priority would be a worthwhile activity. This is not the typical use case for a card-sorting activity, but I knew this was important to help impact the UX roadmap and let the product team understand what could be the “small wins” that they could deliver on now, versus the larger lifts for the team that would take longer to complete. Letting the product team know directly from clients what they value most is hugely beneficial for them to make evidence-based decisions about what to prioritize. We decided to do moderated card-sorting with select clients to better understand why certain features were higher priority than others.


collaborative synthesis of the research

Bringing key stakeholders together to go through all the research findings and begin grouping the major takeaways and themes is my preferred method of synthesis. I find just ensuring everyone is communicating and level-setting is worth the effort. Sometimes in the fast-paced tech environment we get too busy to really deep-dive, but I try to do this collaborative sticky note synthesis with almost every major project. Otherwise, different stakeholders might come away with different interpretations of the data. For this round of synthesis, we wrote down findings on sticky notes, grouped them according to their theme, and then I placed sticker dots on sticky notes to represent different clients. If a sticky note had 4 dots for example, then that was a key point that was brought up with 4 different clients, meaning the idea or pain point held a lot of weight.

Getting the key stakeholders together and affinity diagramming is my preferred method of synthesizing data from research. I find open conversations and getting people involved and active is a great way to get them invested in the research versus just sharing out a report. It all depends on timing, of course. We do get busy but I try to do this when I can, especially for large projects like this one.

Getting the key stakeholders together and affinity diagramming is my preferred method of synthesizing data from research. I find open conversations and getting people involved and active is a great way to get them invested in the research versus just sharing out a report. It all depends on timing, of course. We do get busy but I try to do this when I can, especially for large projects like this one.


Persona Development

The three personas I developed as a result of this initial research.

The three personas I developed as a result of this initial research.

From our initial research, I noted that the people who are using Presence are typically ticketing operations, venue operations, and ticket scanners who use the scanning equipment. I realized that we did not have any personas to represent these people in our organization, which was a huge gap, in my opinion. So, I set out to create new personas that could humanize our end users since they were historically not represented. I went through all our qualitative interview findings for common patterns on our personas’ goals, tasks, and challenges. I also crafted and sent out a survey to our clients to get more well-rounded data to fill out these personas. I presented these personas in a company wide-presentation about how research was now completely shaping the product development of Presence going forward. received many thanks from the product team for helping them understand the humans who are using the product they build. The decisions the product team makes completely impact their lives as they must use our products to do their jobs. Building empathy is a huge initiative of mine.


Evangelizing the Research

Some images captured from our first company-wide presentation on the results of the initial round of research, entitled “How User Research is Informing the Presence Product Development.” We had over 100 people tune in. I also used the opportunity to evangelize what UX research is and how it will help us build better products.

It’s one thing to actually do the research; it’s another thing to ensure people know about the research! After all of our initial product stakeholder conversations about the research findings and how that will impact the product, the lead designer and I decided to do a company-wide presentation on our research in June 2018. During our “tech talk,” we touched upon some of the main points about user research. The main focal point was how we were developing and permeating empathy to the product and development teams. Clearly we found some major issues in the experience of using TM1 Entry and the Presence system, and I wanted to ensure everyone knew this was coming from a place of empathy. Empathy will help us make data-driven decisions.

Iterative Research Process

Coming out of the first round of research, we discovered so much about the people using our products and were really able to step into their worlds. In particular, we learned of some of the common major challenges they faced. At the state TM1 Entry was in, the product was actually making their jobs more difficult and labor-intensive because of some major pain points. So, the lead designer and I set out to design some potential solutions.

One potential solution was “Copying Configurations,” which was designed to alleviate the largest pain point and the top priority for everyone, which was the fact they had to manually set configuration rules before every single event, even if the rules didn’t change. We also wanted to provide an update event list.

We worked together to solve more problems, but these were some high-level

The lead designer and I worked over the next several months to conduct more rounds of usability testing and interviews.

Survey

During this time, I also designed and sent out a survey to all Presence clients to ask about their current satisfaction with the product, and how we could improve. I received about 50 responses from our roughly 180 clients, which is an amazing response rate. The survey allowed me to gather some quantitative feedback and rank the feature enhancements and requests by tallying up the comments.

Copy Configurations was one of the highest priority features for us implement, mainly because it would actually save our clients’ a lot of time by automating a tedious task.

A re-imagined event list UI that cleans up a lot of the messiness of the original UI.


…Even More Research LED TO MORE IDEAS!

I won’t go into much further detail, but through our rounds of interviewing and usability testing, a common pattern was started to emerge. One of the key takeaways was that Presence could not support crucial use cases such as setting entry rules active on events that span multiple days, or set multiple active rules within the same day. Think of a theater that hosts live shows; they usually have matinee and evening shows so they have multiple events in one day. With the current build of the Presence backend, the frontend UI of TM1 Entry could not support these use cases.

Talking to the product team, we discovered the way Presence was built on the backend was to activate entry rules based on a date, not by an event. However, the people using our product have a different mental model: they think about setting rules based on an event, not a date. Because of this mismatch in mental models, Presence was not designed and built in a way originally that really took our users’ mental models in place.

There were many more features we discussed, but these key use cases impacted hundreds of our clients and thus, we had to re-do the entire backend of Presence. There was no way around it. Back in summer of 2018, the development team started work on the Configurations Re-Factor.

Not only was the Configurations Re-Factor important for our current clients, but I started on a project in the same time to look at how to scale Presence to our remaining clients, which you can read all about here!

This feature gap completely impacted the clients who could install Presence, and the larger company saw how important building the product the right way the first time really is. Research really is at the heart of not just building a product, but crucial for scaling it as well.


Evangelizing the Research (again)

A natural follow-up to the presentation in June 2018, we did another company-wide presentation in April 2019 entitled “How Client Empathy is Building A Better Presence.” Yes, I am wearing the same blouse as I did in the original presentation.

A natural follow-up to the presentation in June 2018, we did another company-wide presentation in April 2019 entitled “How Client Empathy is Building A Better Presence.” Yes, I am wearing the same blouse as I did in the original presentation.

In April of 2019, we did an updated presentation on how client empathy is building a better Presence in the wake of the major re-factor. We briefly recapped the research we’ve already chronicled, but also outlined all the research and work that has been going on since that last presentation. We placed a large emphasis on the fact that if this product had generative research like requirements gathering and use case identification done from the get-go, then we might have been able to avoid the year-long configurations re-factor.

Of course, I took this time to weave together all the elements of UX research and how it was now positively impacting not just the product, but our clients’ lives.

Where are we and where are we going?

The configurations re-factor is still underway. The original timelines got pushed back quite a bit. The product team is hoping to be done within the next couple of months. The year-long re-factor was a huge wake-up call to the organization that doing some research upfront and outlining jobs-to-be-done and major use cases upfront will save so much time, money, and energy in the long run.

At the end of the day, everyone has the same goal of continuously enhancing the experience for the people who use our products. Not just the UX of the product, but trying to enhance their lives just a little bit.

Now we do either generative or evaluative research on absolutely everything Presence-related. Our small research department takes requests for research for all products and market research so we stay super busy these days now that we’ve evangelized how important research is in all aspects of product development.