Papa John’s: Usability Report

Papa John’s Pizza is an American restaurant company. It runs the third largest take-out and pizza delivery restaurant chain in the United States, with headquarters in Jeffersontown, Kentucky, a suburb of Louisville.

The following usability report was written about Papa John’s Pizza’s desktop website.  I performed usability testing along with analyzing tests performed by my colleagues. This report covers the user experience of first time online ordering.  Though this report was done as part of my masters, I had the opportunity to send this report to the company.

What We Did

This report provides an analysis and evaluation of a usability study performed on Papa John’s Pizza. These tests provide insight to the ordering process and ease of use on the site. We looked to see what the actual ordering experience entailed and what user expectations of the process would be. Ultimately this study will help improve the Papa John’s website.

What We Found

It was common to listen to a user explain what they want to do and scroll up and down the homepage without being able to locate what they were looking for. This was a common theme throughout the site that vital points in the user’s journey were not intuitive. Users were tempted to give up but were encouraged to focus on the tasks and find some kind of solution. This is not to say that each facet of the site included a negative experience. Through recorded observations we are able to find positives and negatives throughout the ordering process

Each participant that was recruited for this usability study on Papa John’s Pizza’s website (www.papajohns.com) did not have online ordering experience. Tests were conducted with professional testers to exclude any bias that could come about from leading questions or proctor opinions. Participants were asked to perform three tasks:

• Place three complex orders
• Request coupons without creating an account
• File a complaint with the corporate office

During each session participants were instructed to think-aloud as they completed the given tasks which aided in pinpointing where usability issues lie. User comments, errors, and behaviors were observed during each session which lasted anywhere from 10-30 minutes. Users were instructed to take their time and not to worry about offending the tester in an attempt to help them relax.

Task 1 Analysis – Placing Complex Orders

Prompt: We are going to be looking at papajohns.com web site. Even if you are not a Papa John’s fan, imagine that the people you are with are fans of papa johns and that is where you will be ordering the pizza. There are bunch of people at this party and you need to order 3 pizzas.

• Mushroom and Pepperoni
• Half onion and half sausage with light sauce
• A specialty pizza (you want some variety)

Participant #1

• She notices the difference between delivery and pick up, puts in info for delivery
  • Delivery error
• Scrolls up and down a lot on the front page to eventually arrive at “create your own”
  • Never ordered pizza
• Felt she had to search a lot to find coupons

Participant struggles with errors from inputting her address.

“I don’t know how you would do this without searching lots…”

Participant #2

• Had difficulty finding information on pizza companies
• She got an error for delivery and opts for carry out, does not try to find a solution to this error
• She wanted to know how much extra sauce and other toppings are price conscious
• Wanted to see dynamic pricing
• She customized a ready made pizza from the homepage rather than building her own

Participant struggles to remove the sausage and replace with pepperoni.

“I just want to know the phone number.”

Participant #3

• Insisted on calling in pizzas instead of ordering online
• Seemed like Papa John’s wanted him to make an account
  • Went down this path and got an error… decided to go to the phone
• Appeared totally lost, wanted to be able to call them easily
• Searched through pre-made pizzas first but doesn’t see a mushroom and pepperoni pizza • Stuck and he wants to call

Participant admits that he would like to give up before ordering any pizzas

“I have enough passwords in my life, just bring me a pizza.”

Participant #4

• Create Your Own is done within a minute for the first pizza
• He did not really have any expectations of what the online ordering process should be like
• Breezed through tasks, had little to no trouble with site

Participant breezes through pizza ordering with “Create Your Own” option

“Perfect.”

Task 2 Analysis – Finding Coupons

Prompt: You want to sign up for deals and coupons but you don’t want to register. You just want to give them your email. Can you do this and what do you think you will receive by email?

Participant #1

• Felt she had to search a lot to find coupons
• She knew you could get coupons through creating an account and searched that area for a different solution
• Did not know how to get coupons without signing up for an account

Participant searches account area for a way to receive coupons

Participant #2

• Initially searched Google for promo codes
• Struggled to find a way to get deals without giving personal information
• If Papa John’s had an advertisement for free food she would sign up or their rewards program
• Could not discover a simple path to provide email address in order to receive promotions

Participant uses Google to find promo codes for Papa John’s

“I’m not going to go out of my way to give you my email address if you’re not going to take it.”

Participant #3

• Wanted coupons to be available at the top of the page, when he clicked in the footer he found text deals
• Participant found that coupons weren’t difficult to find but they also weren’t incredibly easy

Participant #4

• Found email and text offers easily in the footer
  • The other three participants either took a lot of time or did not complete this task

Task 3 Analysis – Complaint

Prompt: Your pizza arrives and it is terrible! Your driver was rude and you are really upset about what just happened. You call the local store but get nowhere. You need to contact the corporate office, how would you do that?

Patricipant #1

• This task was the most simple for this participant
• Scrolled to footer quickly and found customer service, clicked and arrived at feedback form
• She correctly navigated and completed this task quickly

Participant #2 

• She appeared confused at first about how to find the corporate office
• This task was the most simple for this participant
• She navigated to the feedback form from the footer as Participant #1 also did

Participant #3

• Wouldn’t normally get ahold of corporate or report an issue
• Google searched on Papa John’s corporate in order to find where he wanted to go rather than navigating through the site
• Went to the footer to search for customer service and easily found the feedback form

Participant #4 

• Participant found the feedback form so quickly from the last task there was not much time to take notes. This process appeared to be very intuitive for him

Users would have liked a more intuitive experience as they were first time users. The ordering process was enjoyable as they were able to actually see the pizza they were building and take visual cues of mistakes they were making. Overall they did not have bad experiences and perhaps would have an easier time now that they are familiar with the Papa John’s website.

Area for Improvement: Find Your Store

Participants had difficulty in this area of the site as they wanted to see screens to order a pizza first, rather than set up delivery or carryout. Another issue with this screen was that participants interpreted the delivery address form as helping them find a nearby store for carryout. Papa John’s should consider simply asking for an address or zip code and letting the participants choose carryout or delivery at checkout since delivery is also a paid service.

Area for Improvement: Homepage Layout

The Homepage is overwhelmed with similar photos and graphics of similar colors. Finding the CTA for “Create Your Own Pizza” was difficult for most participants to find and often led to frustration where they would either decide to place an order over the phone or abandon Papa John’s altogether. This could be remedied by using a different graphic altogether or adjusting the size; something to make it stand out more to users.

Area of Success: Useful Footer

On Task 3, each participant found the feedback form quickly and easily. They intuitively navigated to the footer and completed this task in the shortest amount of time with the least amount of errors. Participants could be found referencing the footer for other information such as coupons for Task 2 as well. The footer contains appropriate information for users and can be considered a success of the Papa John’s website.

Follow up research questions will help us dig deeper into usability issues and help us find the best solutions. Below are questions that would be advised for future research…

Where are users becoming the most frustrated within tasks?

This qualitative question can be answered through judging task time, how a user is acting, and if they are not able to articulate exactly what they are trying to accomplish. Pinpointing frustration is imperative to discovering usability issues.

Although users had never ordered pizza online, what was their other experience?

Being familiar with other forms of ordering could factor into how easily a participant used the Papa John’s website. This could be seen for example with Participant #3 and #4, they have vastly different usability tests. This could be due to having varying experience with online ordering patterns.

How do user answers connect to each other?

This question will check for consistency in participants in their experience. It allows us to synthesize provided test information.