Presence Event entry system
Date: Apr. 2018 - Ongoing
Roles: UX Researcher
Methods: Remote Moderated Interviews & Usability Testing, Concept Ideation, Concept Testing, Contextual Inquiry, In-Person Observation & Interviews, Moderated Card-Sorting, Persona Development, Survey, Affinity Diagramming
Background: The Presence event entry system is one of the most cutting edge technologies in entry systems today. Not just a platform to help live event venues and promoters check real-time attendance data, the Presence system allows for better fan identity and the potential for in-app messaging to fans while they are at events in the future. The Presence platform has multiple components: a frontend UI and an Android scanner app under the nomenclature TM1 Entry are the main components in which clients interact. On the frontend UI, a platform where clients in the live event space can set their rules for ensuring only valid tickets scan into their event, and also which ticket types can enter which areas of their venue. For example, they set rules in the frontend UI that allow VIP ticket types to only be able to scan into VIP areas, and prevent people who don’t have VIP tickets from entering. The scanner hardware is an Android phone fitted with special protective hardware for scanning tickets. Presence promises to not just be an entry control system, but also a way to When I first started on this project, a new designer had taken over as the lead designer. We quickly became aware that this product had not had much research done prior to us taking it on and so, we needed to do a lot of research to understand any gaps in the current experience, current usability issues, and understand what opportunities we have in the future.
Goal: Begin to unpack and understand client pain points with the current Presence system and start building in new features that will actually matter to them. Begin understanding any new opportunities we weren’t fully realizing and how to design them into Presence.
On a high-level, what is the typical event day like for our clients?
What our are clients’ mental models when opening up TM1 Entry?
What enhancements can we make to the TM1 Entry frontend UI that provide the most value to our clients? Why do they provide value?
What are the current pain points in the experience of using TM1 Entry and Presence scanner app for our clients? Can we ideate on how to help alleviate the pain points?
We had many more research questions, but in the in the interest of keeping this brief, I won’t elaborate on all of them
Approach: Client Interviews, Usability Testing, Contextual Inquiry, In-Field Observation
The need for intense research was apparent from the beginning of the project. Presence was originally an internal experiment in 2015 to see if we could “kill the barcode” in ticketing. However, what didn’t go into developing Presence was research, as we discovered. So, in spring of 2018, the new lead designer of Presence and I set out to really understand how our clients not only use Presence to track event entry data, but just how they operate on event days in general. What are their goals, challenges, etc. when thousands of fans are all coming into their venue at the same time? So, we initially set up 7 client interviews to really discover the landscape of how our clients were currently using Presence and potential pain points. Then, we set-up two in-person observation and interview sessions on site with two major sports league teams in their arenas. We really needed a foundational lay-of-the-land to understand what our clients were trying to accomplish on the day-of-an-event and if our product was meeting their needs; if it wasn’t, we had to dig into why not and begin ideating on opportunities to improve the user experience. To begin testing, we had a few mocks to test a some re-designs, with an emphasis on a new feature called Access Groups, where the client is able to set entry rules for fans based on a ticket type, not just a particular section. We also wanted to test an enhanced dashboard of aggregate event attendee data to really understand which data points were most important to our clients and why.
One of the biggest findings from our initial client interviews and usability testing was a gap in real-time data feeding into TM1 Entry (frontend UI component of Presence). This is purely a systemic bug that could hopefully be resolved by the development team. A few other bugs were discovered during the on-site observations on event days, meaning going on-site and observing our clients encounter bugs during their workflow within TM1 Entry was key to getting them resolved. We were able to report back bugs and the development team had patches for them quickly. As far as new features, every single client gave us feedback that our concept of Access Groups would work better for them if the functionality mirrored another system we have called Arctics, one of the systems clients use to build events and sell tickets. When clients build events, they have to input event codes (alphanumeric characters) to help label the event, in addition to the title. In Archtics, clients can use a wildcard syntax to call out all tickets that have a similar prefix in their event codes and select multiple events at once. This functionality was not apparent to us in the original design, so the client interviews 100% shaped how Access Groups should function. We also discovered that clients had to set their entry rules for their event every single day they have an event. There was no “save” function within TM1 Entry. Clients had to go into the UI every single time they needed to set entry rules, meaning the process is extremely manual and means the clients are spending a lot of time doing repetitive tasks. They ask for a way in which they could set entry rules once and apply those across multiple events as a way to save them hours of time.
We also learned that a part of the UI called the Scan Log is an area where clients will trouble-shoot during the event to check is their scanners are all online and scanning properly. Enhanced filters to the Scan Log would help our clients trouble-shoot better in real-time. In addition, our clients mentioned they would like a few enhancements to our entry data dashboard, including a scan rate chart, which will show the amount of fans entering their venue in specific time intervals. For them, they universally wanted options: 5 mins, 10 mins, and 15 mins. We found out that their senior management want updated, accurate information in those increments, and knowing the scan rate helps them with logistical planning in real-time and for their next event. We gathered feedback on many other parts of the system and on the new prototype, including several major pain points, but these were some of the key areas where we discovered interesting takeaways and areas of opportunity to really enhance the experience.
Approach: Card-Sorting, Usability Testing, Personas
After discovering so many areas of opportunities on how to enhance the user experience of both the frontend UI and scanner app, I decided a card-sorting activity where our clients could actually rank new features by relative priority would be a worthwhile activity. This is not the typical use case for a card-sorting activity, but I knew this was important to help impact the UX roadmap and let the product team understand what could be the “small wins” that they could deliver on now, versus the larger lifts for the team that would take longer to complete. Letting the product team know directly from clients what they value most is hugely beneficial for them to make evidence-based decisions about what to prioritize. We decided to do moderated card-sorting with select clients to better understand why certain features were higher priority than others.
collaborative synthesis of the research
Bringing key stakeholders together to go through all the research findings and begin grouping the major takeaways and themes is my preferred method of synthesis. I find just ensuring everyone is communicating and level-setting is worth the effort. Sometimes in the fast-paced tech environment we get too busy to really deep-dive, but I try to do this collaborative sticky note synthesis with almost every major project. Otherwise, different stakeholders might come away with different interpretations of the data. For this round of synthesis, we wrote down findings on sticky notes, grouped them according to their theme, and then I placed sticker dots on sticky notes to represent different clients. If a sticky note had 4 dots for example, then that was a key point that was brought up with 4 different clients, meaning the idea or pain point held a lot of weight.
From our initial research, I noted that the people who are using Presence are typically ticketing operations, venue operations, and ticket scanners who use the scanning equipment. I realized that we did not have any personas to represent these people in our organization, which was a huge gap, in my opinion. So, I set out to create new personas that could humanize our end users since they were historically not represented. I went through all our qualitative interview findings for common patterns on our personas’ goals, tasks, and challenges. I also crafted and sent out a survey to our clients to get more well-rounded data to fill out these personas. I presented these personas in a company wide-presentation about how research was now completely shaping the product development of Presence going forward. received many thanks from the product team for helping them understand the humans who are using the product they build. The decisions the product team makes completely impact their lives as they must use our products to do their jobs. Building empathy is a huge initiative of mine.
Evangelizing the Research
It’s one thing to actually do the research; it’s another thing to ensure people know about the research! After all of our initial product stakeholder conversations about the research findings and how that will impact the product, the lead designer and I decided to do a company-wide presentation on our research in June 2018. During our “tech talk,” we touched upon some of the main points about user research. The main focal point was how we were developing and permeating empathy to the product and development teams. Clearly we found some major issues in the experience of using TM1 Entry and the Presence system, and I wanted to ensure everyone knew this was coming from a place of empathy. Empathy will help us make data-driven decisions.
Iterative Research Process
Coming out of the first round of research, we discovered so much about the people using our products and were really able to step into their worlds. In particular, we learned of some of the common major challenges they faced. At the state TM1 Entry was in, the product was actually making their jobs more difficult and labor-intensive because of some major pain points. So, the lead designer and I set out to design some potential solutions.
One potential solution was “Copying Configurations,” which was designed to alleviate the largest pain point and the top priority for everyone, which was the fact they had to manually set configuration rules before every single event, even if the rules didn’t change. We also wanted to provide an update event list.
We worked together to solve more problems, but these were some high-level
The lead designer and I worked over the next several months to conduct more rounds of usability testing and interviews.
During this time, I also designed and sent out a survey to all Presence clients to ask about their current satisfaction with the product, and how we could improve. I received about 50 responses from our roughly 180 clients, which is an amazing response rate. The survey allowed me to gather some quantitative feedback and rank the feature enhancements and requests by tallying up the comments.
…Even More Research LED TO MORE IDEAS!
I won’t go into much further detail, but through our rounds of interviewing and usability testing, a common pattern was started to emerge. One of the key takeaways was that Presence could not support crucial use cases such as setting entry rules active on events that span multiple days, or set multiple active rules within the same day. Think of a theater that hosts live shows; they usually have matinee and evening shows so they have multiple events in one day. With the current build of the Presence backend, the frontend UI of TM1 Entry could not support these use cases.
Talking to the product team, we discovered the way Presence was built on the backend was to activate entry rules based on a date, not by an event. However, the people using our product have a different mental model: they think about setting rules based on an event, not a date. Because of this mismatch in mental models, Presence was not designed and built in a way originally that really took our users’ mental models in place.
There were many more features we discussed, but these key use cases impacted hundreds of our clients and thus, we had to re-do the entire backend of Presence. There was no way around it. Back in summer of 2018, the development team started work on the Configurations Re-Factor.
Not only was the Configurations Re-Factor important for our current clients, but I started on a project in the same time to look at how to scale Presence to our remaining clients, which you can read all about here!
This feature gap completely impacted the clients who could install Presence, and the larger company saw how important building the product the right way the first time really is. Research really is at the heart of not just building a product, but crucial for scaling it as well.
Evangelizing the Research (again)
In April of 2019, we did an updated presentation on how client empathy is building a better Presence in the wake of the major re-factor. We briefly recapped the research we’ve already chronicled, but also outlined all the research and work that has been going on since that last presentation. We placed a large emphasis on the fact that if this product had generative research like requirements gathering and use case identification done from the get-go, then we might have been able to avoid the year-long configurations re-factor.
Of course, I took this time to weave together all the elements of UX research and how it was now positively impacting not just the product, but our clients’ lives.
Where are we and where are we going?
The configurations re-factor is still underway. The original timelines got pushed back quite a bit. The product team is hoping to be done within the next couple of months. The year-long re-factor was a huge wake-up call to the organization that doing some research upfront and outlining jobs-to-be-done and major use cases upfront will save so much time, money, and energy in the long run.
At the end of the day, everyone has the same goal of continuously enhancing the experience for the people who use our products. Not just the UX of the product, but trying to enhance their lives just a little bit.
Now we do either generative or evaluative research on absolutely everything Presence-related. Our small research department takes requests for research for all products and market research so we stay super busy these days now that we’ve evangelized how important research is in all aspects of product development.