Emburse Mobile Beta Test

This project demonstrated my ability to uncover significant usability and engineering issues prior to the official release of emburse mobile app.

What is Emburse Mobile app?

What drove my research?

1. Disjoint user experience between two devices: Having worked closely with both the mobile and expense teams, I have developed a thorough understanding of the product flows, step by step. While observing from a user’s standpoint, I proposed a research hypothesis that the experience becomes fragmented when transitioning from one device to another.

2. Early usability testing & troubleshooting for newly-launched app: mobile is a newly launched product, my research intuition drove me to initiate this research. Otherwise, we would spend huge costs and efforts to fix usability AFTER the official release.

Project timeline

Understand business goals and tie my research to product vision

Understand business goals
Before diving into the research, I aimed to understand the significance of Emburse Mobile and the value it brings to our customers. Prior to Emburse Mobile, we had two other mobile apps, ChromeRiver Snap and ChromeRiver Mobile. Previous studies indicated that these existing apps were not delivering sufficient value to users, with persistent bugs causing frustration and potentially extending the time required for expense management—contrary to our overarching product strategy. Therefore, we also sought positive testimonials from clients, hoping the new mobile app would bridge these gaps and eliminate these frictions. Together with the mobile team and the VP of Product, we collaboratively finalized the business goals for this beta test

• Show that our mobile product is the best practices in the market

• Increase the adoption rate of Emburse mobile so that the automation process can upsell bookings to revenue for the company

Select target audiences

Finalizing research goals & Setting up communication protocol

With all the context information I gathered, I had several rounds of discussion with the PM, and we agreed upon these 2 overarching research goals:

1. Understand mobile-to-web experience with both admins & end users

2. Understand from admins’ perspective:

   • Value proposition

   • Best practice

   • Challenges in adoption

To better collaborate with the PM, I also set up our regular communication protocol. We put a weekly 1:1 on our calendar to have weekly check-ins over the progress of the project

Testing metrics

With the overarching goal in mind: Understand mobile-to-web experience with both admins & end users, I started looking into product features, analyze user flow step by step, and think aloud over the metrics I would like to gather. Here I gave mobile (receipt capture feature) as an example, I looked into comprehension, specifically knowledge & awareness.

Methods

• In-depth Interview

• Usability Testing

Beta Test Execution

Section A — Beta test installation

In section A, we required every participant to download the emburse mobile app through Test Flight, and we prepared instruction documents for both IOS & Android users.

Troubleshooting & My role as a communicator

This app installation step may seem easy. However, it took the longest time & efforts because our clients / admins encountered all kinds of errors when they tried to download & login.

For instance, Android users could not download successfully because they were using personal accounts for Google play store, but we activated their beta with company email. For login, they have restricted access or SSO security issues.

Thus, I played a major role as a communicator. On average, I sent out 12.8 emails to each client to understand their issues, get screenshots of the bugs, and have them re-try. And it took me around 12.6 days per client to get the problem resolved.

Within internal communication, I first shared all the bugs through the product manager, while later to decrease the communication costs, I created direct Slack channels that include all important stakeholders. I also managed to report issues and documented 4 pages of issues for quick reference.

Section B — In-depth Interview

I recruited 8 admins from our client repository and the through outreach with CSMs. Originally, we tried to ask admins to promote this study with their employees who fit into the persona criteria. Unfortunately, we heard 0 response. Thus, I pivoted and found another 2 end users who fit into our road warrior persona from UserTesting.com.

Actionable insights

Impacts

Engineering (beta phase):

• App download success rate from 70% to 100%

• Login success rate from 10% to 100%

• Receipt capture success rate 70% to 90%

Product:

3 design changes

• Change of home screen information display

• Change of receipt confirmation UI

• Change of UX writing between mobile & web

3 key features on Q3 product roadmap

• Feature to edit receipt notes

• Feature to resize receipt image

• Feature of confirming receipts synced to web