Case Study: DoorDash Rating Systems‍

Timeline: 1.5 months

Role: UI/UX Designer, UX Researcher

Overview

We redesigned two different rating systems, and introduced two new features in the Dasher app, all aimed at improving rating transparency for Dashers. Specifically, we wanted to improve the clarity behind the Acceptance Rating system for Dashers, and clear customer misconceptions about Dasher and restaurant issues.

‍

Dasher Interviews

We interviewed four Dashers, aged 20-23, with diverse locations and experiences. Locations included California, Oklahoma, and Indiana. All of them were Dashers in the past, and they had all done it part time. Some key findings are:

• Dashers felt that they would receive poor ratings because customers get “hangry” even if there were external factors.
• Dealing with restaurants was the “hardest part of the job”.
• Acceptance Rating lacked clarity and had a major impact on the orders they would receive from DoorDash.
• Customer service support was found to be lacking.

‍

Customer Surveys

In order to increase our understanding of customer reviews,we sent out a survey. Since we were mainly interested in what motivated them to rate Dashers and restaurants, we determined that a survey would be the best method because of how limited our field of questions would be. About 85% of customers actually used the system to rate their Dashers, and timeliness and order handling were the most important factors to the customers.

‍

Design Solution:

Our design solutions can be broadly broken down into four features:

1. Improved customer rating system (Customer Rating).
2. AR simulator for increased acceptance rating understanding (AR).
3. Reduced penalty to Dashers if they decline an order for legitimate reasons (Order Declining).
4. Meaningful manner of restaurant ratings by Dashers for Dashers (Restaurant Rating).

‍

Based on current DoorDash UI at the time of this project, we created a design system in order to remain consistent across our designs.

Below are some of our concept sketches, based on the four features we outlined above. To execute 1, were ordered the rating screen such that the restaurant gets rated first, and then the restaurant. For 2, we developed a simulator based on a rolling set of 20 concept. For 3 and 4, we drew from Dasher frustrations – ratings going down because of factors out of their control (restaurant taking too long, no parking area), and wanting to know which restaurants to avoid for the same reasons.

Version 1

Based on the sketches above, we developed the first version of our redesigns, which were then tested via cognitive walk-throughs with three non-Dashers. We found that while they liked our redesign of Customer Rating and Order Declining, they did not like just the smileys in Restaurant Rating. The AR redesign showed no improvement in understanding, as our testers remained confused.

Version 2

Based on testing feedback, we tweaked all our screens. For Customer Rating, we updated the copy in the text bubbles. For AR, we completely redid the system, switching from a table to a rolling 20 simulator, as shown in the concept sketches. For Restaurant Rating,we changed the color in the first screen to create a better association. For the second screen, we added options about what went wrong/well, as our testers did not want to fill out the text box and would rather just hit a checkbox. For Order Declining, the testers were satisfied with the flow, only pointing out that the system could be abused by Dashers not wanting their AR to drop. We decided to let the honor system remain, as all Dashers we interviewed and most online stuck to the honor system and prioritized the customer.

This version was used to conduct usability testing with the interviewees. Our testers we happy and could not find any faults with three of the four scenarios. However, AR still left them puzzled. On further questioning and further testing with two others, we learnt that the direction of new orders replacing old orders (R <- L) was confusing them, as they expected it to be (R -> L). This seemed to clear up some more uncertainty. While our redesign cleared up some ambiguity, we were never able to fully resolve this issue, as we determined that it would be impossible to simulate without getting access to the algorithm that powered it.

Final Scenarios

#1: Customer Ratings once order is delivered. View prototype here.

#2: Acceptance Rating (AR) visualization in the Ratings section. View prototype here.

#3: Restaurant Ratings after order is delivered. View prototype here.

#4: Order Declining reasons after a Dasher declines an order. View prototype here.

Limitations

• We were only able to interview part-time Dashers, for whom DoorDash was not their major revenue stream.
• Extremely limited Customer Survey pool.
• Lack of full access to Dasher App.
• Limited AR testing, no access to DoorDash algorithm for AR.

Future Considerations

To develop these ideas further, we would interview full time Dashers, for whom DoorDash is a major revenue stream. We would also like to create various versions of the AR simulator and thoroughly test them. As the UI of DoorDash keeps evolving, so would our designs for the above four scenarios.