Ovo Logger ADC Sample Project

View by User:
View by Task:
View by Quicklog:
View by Observer:
Search Notes for: 
Page layout doesn't match participants preconceived priorities.
When participants learn what something is they also want to know what something costs.
Participants expected more details about square envelopes when square envelopes were mentioned.

This page presents summaries for all surveys that were used in this activity, including the following survey sections:

Post-Test Survey

This survey section had the following questions:

  1. This is instruction text. You can put instructions to participants here. (Instructions)
  2. 1.) This is a short answer question type. What is your job title? (Short Answer)
  3. 2.) This is an essay question type. Please describe your primary job duties. (Essay)
  4. 3.) This is a bulleted question type. List three toppings you would eat on pizza. (Bulleted Answer)
  5. 4.) This is a Likert question type which defaults to a 5-point scale. Please rate the following statement: I like pinapple on pizza. (Likert)
  6. 5.) This is a multiple choice question configured to only allow one choice. Who is your favorite Stooge? (Multiple Choice, Pick One)
  7. 6.) This is a multiple choice question configured to permit participants to select more than one item. What ice cream flavors do you like? (Pick as many as you want.) (Multiple Choice, Pick Many)
  8. 7.) This is a ranking question. Rank the following Star Wars creatures in order of preference from 1 (favorite) to 3 (least favorite.) (Ranking)

Instructions to User

This is instruction text. You can put instructions to participants here.

1.) This is a short answer question type. What is your job title? (Short Answer)

Word Frequency Analysis

Demographic Range/Level

Word

Frequency

Overall

Manager

1

 

Kitchen

1

 

Washer

1

 

Bottle

1

 

Cook

1

 

Chief

1

Individual User Responses

Participant
Response
User 01
Chief Cook
User 02
Bottle Washer
User 03
Kitchen Manager

2.) This is an essay question type. Please describe your primary job duties. (Essay)

Word Frequency Analysis

Demographic Range/Level

Word

Frequency

Overall

Washer

2

 

Bottle

2

 

cook

2

 

cats

1

 

herding

1

 

it

1

 

tell

1

 

Let

1

 

Chief

1

 

manage

1

 

call

1

 

why

1

 

idea

1

 

no

1

 

dishes

1

 

pans

1

 

pots

1

 

wash

1

 

duties

1

 

job

1

 

dinners

1

 

lunches

1

 

breakfasts

1

 

everything

1

Individual User Responses

Participant
Response
User 01
I cook everything: breakfasts, lunches, and dinners.
User 02
My job duties are to wash pots, pans, and dishes. I have no idea why they call me Bottle Washer.
User 03
I manage the Chief Cook and the Bottle Washer. Let me tell you, it is like herding cats.

3.) This is a bulleted question type. List three toppings you would eat on pizza. (Bulleted)

Word Frequency Analysis

Demographic Range/Level

Word

Frequency

Overall

Cheese

4

 

peppers

1

 

hot

1

 

mildly

1

 

Those

1

 

only

1

 

Bovril

1

 

Onion

1

 

Vegemite

1

 

Pepperoni

1

 

Marmite

1

Individual User Responses

Participant
Response
User 01
(1) Marmite
(2) Vegemite
(3) Bovril
User 02
(1) Cheese
(2) Cheese
(3) Cheese (I only like cheese)
User 03
(1) Pepperoni
(2) Onion
(3) Those mildly hot peppers

4.) This is a Likert question type which defaults to a 5-point scale. Please rate the following statement: I like pinapple on pizza. (Likert)

Legend

1

2

3

4

5

Strongly Disagree

Somewhat Disagree

Neutral or N/A

Somewhat Agree

Strongly Agree

Statistical Analysis: Overall

Demographic Range/Level

n

Sum

Min

Max

s

Mean

Median

Overall

3

9

1

5

2.0

3

1

Individual User Responses

Participant
Response
Stat Value
Additional Comments
User 01
Strongly Agree
5
I would get a pizza with just pineapple. No cheese no sauce, just pineapple.
User 02
Strongly Disagree
1
Like I said, I only like cheese.
User 03
Neutral or N/A
3
I can take it or leave it.

5.) This is a multiple choice question configured to only allow one choice. Who is your favorite Stooge? (Multiple Choice, Pick One)

Frequency Analysis

Demographic Range/Level

Choice

Frequency

Overall

Curly

1

 

Mo

2

 

Shemp

0

Individual User Responses

Participant
Response
Additional Comments
User 01
Curly
I like his hair.
User 02
Mo
I like strong authority figures, so Mo is my fave.
User 03
Mo
I like Mo because he's the boss.

6.) This is a multiple choice question configured to permit participants to select more than one item. What ice cream flavors do you like? (Pick as many as you want.) (Multiple Choice, Pick Many)

Frequency Analysis

Demographic Range/Level

Choice

Frequency

Overall

Vanilla

2

 

Chocolate

2

 

Strawberry

1

Individual User Responses

Participant
Response
Additional Comments
User 01
Vanilla, Chocolate, Strawberry
I like all ice cream flavors.
User 02
Vanilla
If there was a cheese flavored ice cream I would like that best.
User 03
Chocolate
I only like chocolate.

7.) This is a ranking question. Rank the following Star Wars creatures in order of preference from 1 (favorite) to 3 (least favorite.) (Ranking)

Ranking Analysis

Demographic Range/Level

Ranking Item:

n

Min

Max

s

Mean

Median

Overall

Jar Jar Binks

3

1

2

0.6

1.7

2

 

The slimy creature in the trash compactor

3

1

3

1.2

1.7

1

 

Chewbacca

3

2

3

0.6

2.7

3

Individual User Responses

Participant
Ranking
Additional Comments
User 01
1: The slimy creature in the trash compactor, 2: Jar Jar Binks, 3: Chewbacca
Can't stand Jar Jar.
User 02
1: Jar Jar Binks, 2: Chewbacca, 3: The slimy creature in the trash compactor
I think Jar Jar's comic stylings are overlooked.
User 03
1: The slimy creature in the trash compactor, 2: Jar Jar Binks, 3: Chewbacca
Can't stand Jar Jar.
Time
User
Task
Observer
QuickLog
Note
User 01
#10 Business
Main Logger
N/A
Recording Started
Computer
Participant
2-Source View
User 01
Introduction
Scott
Introduction
Quicklog: Introduction
Computer
Participant
2-Source View
User 01
Introduction
Scott
Introduction
Participant clicks Ovo Study Hub link to access web-based scenarios.
Computer
Participant
2-Source View
User 01
#10 Business
Scott
N/A
Reads scenario
Computer
Participant
2-Source View
User 01
#10 Business
Scott
Layout
Those are big icons. I can do 9 things.
Computer
Participant
2-Source View
User 01
#10 Business
Scott
Mental Model
If these are the 9 most important things that the post office does and the post office is all about letter, then boom. Click and Ship.
Computer
Participant
2-Source View
User 01
#10 Business
Scott
Mental Model
Click and Ship. OK that's got to be wrong because I don't have an account. I'm 50 and I've never had an account with the post office.
Computer
Participant
2-Source View
User 01
#10 Business
Scott
N/A
So maybe I want to buy stamps.
Computer
Participant
2-Source View
User 01
#10 Business
Scott
N/A
Cards and envelopes. I don't want to buy cards from the PO, I can buy cards from the Hallmark store.
Computer
Participant
2-Source View
User 01
#10 Business
Scott
N/A
We have the postage you need.... (Goes to Postal Store.)
Computer
Participant
2-Source View
User 01
#10 Business
Scott
N/A
I didn't find out cost to mail letter. Assumes his letter is OK for first class. Sees Forever Stamp and .55 cents.
Computer
Participant
2-Source View
User 01
#10 Business
Scott
Workflow
Quicklog: Workflow
Computer
Participant
2-Source View
User 01
#10 Business
Scott
N/A
Quicklog: Xray
Computer
Participant
2-Source View
User 01
Square Envelope
Scott
N/A
Reads scenario materials.
Computer
Participant
2-Source View
User 01
Square Envelope
Scott
Layout
Sees 3x3 layout on home page. Drags page wider.
Computer
Participant
2-Source View
User 01
Square Envelope
Scott
Layout
(See Stamps and Supplies) Isn't that nice. So in that tiny little icon they give me the answer, but in that big icon (in 3x3) they don't
Computer
Participant
2-Source View
User 01
Square Envelope
Scott
Search
Square envelope (searches)
Computer
Participant
2-Source View
User 01
Square Envelope
Scott
Search
Square corners. It's funny that they did not make the connection that the entire phrase "square envelope" appears in this result.
Computer
Participant
2-Source View
User 01
Square Envelope
Scott
Search
Searches page on Square Envelope. Finds nonmachinable.
Computer
Participant
2-Source View
User 01
Square Envelope
Scott
Navigation
A square envelope has an aspect ratio of one. So how much does that cost?
Computer
Participant
2-Source View
User 01
Square Envelope
Scott
Navigation
So there is a surcharge, I just don't know how much that is.
Computer
Participant
2-Source View
User 01
Cookies
Scott
N/A
Reads scenario materials.
Computer
Participant
2-Source View
User 01
Cookies
Scott
N/A
Reading scenario.
Computer
Participant
2-Source View
User 01
Cookies
Scott
N/A
ClickNShip looks awsome, it's the first thing there, but I don't want to make an account.
Computer
Participant
2-Source View
User 01
Cookies
Scott
Terminology
(Main menu, clicks Send) Send, so I'm looking to send a box.
Computer
Participant
2-Source View
User 01
Cookies
Scott
Terminology
Clicks sending packages.
Computer
Participant
2-Source View
User 01
Cookies
Scott
N/A
So I have my box. (Clicks into Pick your Box)
Computer
Participant
2-Source View
User 01
Cookies
Scott
N/A
You know, I've seen these pre-printed USPS boxes like the stuff eBay stuff comes in.
Computer
Participant
2-Source View
User 01
Cookies
Scott
Workflow
Wouldn't it be nice if they had links to those standard sized USPS boxes? (Looking at How to Ship a Package > Step 1: Pick Your Box)
Computer
Participant
2-Source View
User 01
Cookies
Scott
Workflow
I don't want to order supplies, I want to ship.
Computer
Participant
2-Source View
User 01
Cookies
Scott
Terminology
Priority Mail, Priority Mail Express... wow.
Computer
Participant
2-Source View
User 01
Cookies
Scott
Layout
hovering over different box types.
Computer
Participant
2-Source View
User 01
Cookies
Scott
Workflow
Clicks Prioriy Mail. So i think this would hold my cookies. I don't know how medium compares to other sizes. Is there a small? I don't know how much it would cost to ship.
Computer
Participant
2-Source View
User 02
Cookies
Main Logger
N/A
Recording Started
Computer
Participant
2-Source View
User 02
Cookies
Main Logger
Introduction
Quicklog: Introduction
Computer
Participant
2-Source View
User 02
Cookies
Main Logger
N/A
Reads scenario materials.
Computer
Participant
2-Source View
User 02
Cookies
Main Logger
Layout
That's a lot of icons and they are big. So these are the nine most important things I can do with the postal service.
Computer
Participant
2-Source View
User 02
Cookies
Main Logger
Terminology
ClickNShip. I'm gonna click and I'm gonna ship. Oh, so this should be called "click and logon." OK, so that doesn't do it for me.
Computer
Participant
2-Source View
User 02
Cookies
Main Logger
Terminology
I saw boxes and I got excited (but it's PO Boxes)
Computer
Participant
2-Source View
User 02
Cookies
Main Logger
Workflow
(Clicks Calculate a Price) (Types ZIPs) I don't see why time matters for this.
Computer
Participant
2-Source View
User 02
Cookies
Main Logger
Layout
Ah, flat rate boxes. I like that. (click in)
Computer
Participant
2-Source View
User 02
Cookies
Main Logger
Layout
(Looking at details for medium box) USPS produced box -- these are weird differences in dimensions. Why are these two medium boxes so different in size? I don't understand this.
Computer
Participant
2-Source View
User 02
Cookies
Main Logger
N/A
(Selects medium box) Finds tracking option. I don't know how to use tracking.
Computer
Participant
2-Source View
User 02
Square Envelope
Main Logger
N/A
Reads scenario materials
Computer
Participant
2-Source View
User 02
Square Envelope
Main Logger
Layout
Huh, what happened to the menu bar that was up here before? (on the Retail Postage Price Calculator page)
Computer
Participant
2-Source View
User 02
Square Envelope
Main Logger
Layout
How do I get home? (Click logo) That? Nice. I wonder why that wasn't up at the top.
Computer
Participant
2-Source View
User 02
Square Envelope
Main Logger
Navigation
Mail, I'd love to send mail. Send (menu) and mail. That's great.
Computer
Participant
2-Source View
User 02
Square Envelope
Main Logger
Workflow
(On sending mail, reads...) you can send mail easily. I guess I'll be the judge of that.
Computer
Participant
2-Source View
User 02
Square Envelope
Main Logger
Layout
(Choose an envelope or postcard. Clicks in) So I'm going to choose a square envelope. (Reads about rectangular envelopes.) But you haven't told me what happens if they are not rectangular.
Computer
Participant
2-Source View
User 02
Square Envelope
Main Logger
Navigation
Frustration here would be high. You tease me with the idea that you understand that there are rectangular and other shapes. I don't think square would be a weird shape to talk about at this point.
Computer
Participant
2-Source View
User 02
#10 Business
Main Logger
N/A
Reads scenario materials.
Computer
Participant
2-Source View
User 02
#10 Business
Main Logger
Terminology
You know, I'm already here (on the How To Mail A Letter page) and I'm looking at rectangular envelopes, boom. (Clicks size and weight requirements.) but they say "requirements" so I bet they aren't going to talk about cost.
Computer
Participant
2-Source View
User 02
#10 Business
Main Logger
Navigation
(on what are you mailing, domestic) Do I see dollar signs or cent signs anywhere? No.
Computer
Participant
2-Source View
User 02
#10 Business
Main Logger
N/A
Uses information to verify weight is ok.
Computer
Participant
2-Source View
User 02
#10 Business
Main Logger
Navigation
Maybe Choosing a Service for Mailing will discuss cost.
Computer
Participant
2-Source View
User 02
#10 Business
Main Logger
Layout
Sees First-Class Mail. Nice. Is this a secret? It's like you're kidding me. (Clicks dollar sign) Would it have killed you to tell me how much it costs? That's frustrating.
Computer
Participant
2-Source View
User 03
Square Envelope
Main Logger
N/A
Recording Started
Computer
Participant
2-Source View
User 03
Square Envelope
Main Logger
Introduction
Quicklog: Introduction
Computer
Participant
2-Source View
User 03
Square Envelope
Main Logger
N/A
Reads scenario material
Computer
Participant
2-Source View
User 03
Square Envelope
Main Logger
Layout
Wow that is nine things. These are the most important nine things.
Computer
Participant
2-Source View
User 03
Square Envelope
Main Logger
Search
Searches on "square letter"
Computer
Participant
2-Source View
User 03
Square Envelope
Main Logger
N/A
I would go into buy stamps, but I don't suspect there is a "square envelope" stamp.
Computer
Participant
2-Source View
User 03
Square Envelope
Main Logger
N/A
I never knew there was an SLA on first class mail.
Computer
Participant
2-Source View
User 03
Square Envelope
Main Logger
Terminology
Clicks first class mail and postage.
Computer
Participant
2-Source View
User 03
Square Envelope
Main Logger
N/A
(Reads Prices shaped on Shape and Weight) If I had not scrolled down, I would not have seen this. I would have clicked these links and been gone.
Computer
Participant
2-Source View
User 03
Square Envelope
Main Logger
N/A
Square, 75 cents. I'm shocked and successful.
Computer
Participant
2-Source View
User 03
#10 Business
Main Logger
N/A
Reads scenario material.
Computer
Participant
2-Source View
User 03
#10 Business
Main Logger
N/A
Let's close the browser and see if I can find this info from starting over because it was on the page I was at.
Computer
Participant
2-Source View
User 03
#10 Business
Main Logger
N/A
If I search on Square Envelope I know I'll get the link again.
Computer
Participant
2-Source View
User 03
#10 Business
Main Logger
N/A
What did I click last time?
Computer
Participant
2-Source View
User 03
#10 Business
Main Logger
Search
Business Envelope
Computer
Participant
2-Source View
User 03
#10 Business
Main Logger
N/A
Looking to see if First Class Rates and Postage is in results for Business Envelope.
Computer
Participant
2-Source View
User 03
#10 Business
Main Logger
Terminology
"First-Class Mail and Postage" and "Postage Rates & Prices". One is a child to the other.
Computer
Participant
2-Source View
User 03
#10 Business
Main Logger
N/A
Wow (it's not there)
Computer
Participant
2-Source View
User 03
#10 Business
Main Logger
Navigation
Alt-back arrow. Doesn't go back. Do I not have history?
Computer
Participant
2-Source View
User 03
#10 Business
Main Logger
Search
"Square Letter" and "Square Envelope" yield very different search results.
Computer
Participant
2-Source View
User 03
#10 Business
Main Logger
N/A
It's not on the first scrolling list for business envelope.
Computer
Participant
2-Source View
User 03
#10 Business
Main Logger
Search
Searches again on Business Letter
Computer
Participant
2-Source View
User 03
#10 Business
Main Logger
Search
And it's a #10 Envelope, which really puts us in the weeds
Computer
Participant
2-Source View
User 03
#10 Business
Main Logger
Navigation
Clicks First Class Mail and Postage
Computer
Participant
2-Source View
User 03
#10 Business
Main Logger
Navigation
Clicks First Class Mail Prices
Computer
Participant
2-Source View
User 03
#10 Business
Main Logger
N/A
weight not over 1oz, 55 cents. There we go.
Computer
Participant
2-Source View
User 03
Cookies
Main Logger
N/A
Reads scenario materials.
Computer
Participant
2-Source View
User 03
Cookies
Main Logger
Layout
Clicks into Calculate a Price.
Computer
Participant
2-Source View
User 03
Cookies
Main Logger
Workflow
Clicks flat rate boxes.
Computer
Participant
2-Source View
User 03
Cookies
Main Logger
N/A
Small Medium Large... OK. Small is too small
Computer
Participant
2-Source View
User 03
Cookies
Main Logger
Layout
(Looking at Med box) This is some really weird... I don't know... box? What the or for? These are two very different boxes.
Computer
Participant
2-Source View
User 03
Cookies
Main Logger
Layout
Does drop off date drive price? I'm not going to click back and forth a gazillion times to exercise that either.
Computer
Participant
2-Source View
User 03
Cookies
Main Logger
N/A
The name is probably just Priority Mail 2 Day.
Computer
Participant
2-Source View
User 03
Cookies
Main Logger
N/A
UPS tracking is there.
Computer
Participant
2-Source View
User 03
Cookies
Main Logger
Terminology
I guess "retail" means drop it off at the post office.
Computer
Participant
2-Source View
User 03
Cookies
Main Logger
Workflow
I think I'm done, I have tracking. I don't know how to USE tracking.
Computer
Participant
2-Source View
Square Letter Postage

Participants had difficulty finding:

  • that square envelopes are priced differently than rectangular envelopes
  • what that price is.

Time
Participant
Task
Observer
Marker
Note
User 02
Square Envelope
Main Logger
Layout
(Choose an envelope or postcard. Clicks in) So I'm going to choose a square envelope. (Reads about rectangular envelopes.) But you haven't told me what happens if they are not rectangular.
Computer
Participant
2-Source View
User 02
Square Envelope
Main Logger
Navigation
Frustration here would be high. You tease me with the idea that you understand that there are rectangular and other shapes. I don't think square would be a weird shape to talk about at this point.
Computer
Participant
2-Source View
User 01
Square Envelope
Scott
Navigation
A square envelope has an aspect ratio of one. So how much does that cost?
Computer
Participant
2-Source View
User 01
Square Envelope
Scott
Search
Square corners. It's funny that they did not make the connection that the entire phrase "square envelope" appears in this result.
Computer
Participant
2-Source View
User 03
Square Envelope
Main Logger
N/A
I would go into buy stamps, but I don't suspect there is a "square envelope" stamp.
Computer
Participant
2-Source View
First Class Envelope

Participants had difficulty finding the specifications and cost for First Class mail. Here is what search results look like, as an example of what it is like to search for this information.

Time
Participant
Task
Observer
Marker
Note
User 02
#10 Business
Main Logger
Terminology
You know, I'm already here (on the How To Mail A Letter page) and I'm looking at rectangular envelopes, boom. (Clicks size and weight requirements.) but they say "requirements" so I bet they aren't going to talk about cost.
Computer
Participant
2-Source View
User 02
#10 Business
Main Logger
Layout
Sees First-Class Mail. Nice. Is this a secret? It's like you're kidding me. (Clicks dollar sign) Would it have killed you to tell me how much it costs? That's frustrating.
Computer
Participant
2-Source View
User 02
#10 Business
Main Logger
Navigation
(on what are you mailing, domestic) Do I see dollar signs or cent signs anywhere? No.
Computer
Participant
2-Source View
User 01
#10 Business
Scott
N/A
I didn't find out cost to mail letter. Assumes his letter is OK for first class. Sees Forever Stamp and .55 cents.
Computer
Participant
2-Source View
User 03
#10 Business
Main Logger
N/A
Looking to see if First Class Rates and Postage is in results for Business Envelope.
Computer
Participant
2-Source View
User 03
#10 Business
Main Logger
N/A
Wow (it's not there)
Computer
Participant
2-Source View
Search Terminology Letter v. Envelope

Participants frequently used search and encountered unexpected behavior in search results:

  • When they searched by "#10" the pound sign was stripped and ignored.
  • Results for "Business Letter" and "Business Envelope" were drastically different.

Time
Participant
Task
Observer
Marker
Note
User 03
#10 Business
Main Logger
Search
"Square Letter" and "Square Envelope" yield very different search results.
Computer
Participant
2-Source View
User 03
Square Envelope
Main Logger
Search
Searches on "square letter"
Computer
Participant
2-Source View
User 01
Square Envelope
Scott
Search
Searches page on Square Envelope. Finds nonmachinable.
Computer
Participant
2-Source View
User 03
#10 Business
Main Logger
Search
And it's a #10 Envelope, which really puts us in the weeds
Computer
Participant
2-Source View
Click N' Ship

Participants did not know what to do with Click n' Ship.

Time
Participant
Task
Observer
Marker
Note
User 01
#10 Business
Scott
Mental Model
Click and Ship. OK that's got to be wrong because I don't have an account. I'm 50 and I've never had an account with the post office.
Computer
Participant
2-Source View
User 01
Cookies
Scott
N/A
ClickNShip looks awsome, it's the first thing there, but I don't want to make an account.
Computer
Participant
2-Source View
User 02
Cookies
Main Logger
Terminology
ClickNShip. I'm gonna click and I'm gonna ship. Oh, so this should be called "click and logon." OK, so that doesn't do it for me.
Computer
Participant
2-Source View
Links between Format, Service, and Cost

In general, participants had difficulty finding the connection between levels of service, format of envelope/parcel, and price.

Time
Participant
Task
Observer
Marker
Note
User 01
Cookies
Scott
Workflow
Clicks Prioriy Mail. So i think this would hold my cookies. I don't know how medium compares to other sizes. Is there a small? I don't know how much it would cost to ship.
Computer
Participant
2-Source View
User 01
Cookies
Scott
Workflow
Wouldn't it be nice if they had links to those standard sized USPS boxes? (Looking at How to Ship a Package > Step 1: Pick Your Box)
Computer
Participant
2-Source View
User 02
#10 Business
Main Logger
Navigation
Maybe Choosing a Service for Mailing will discuss cost.
Computer
Participant
2-Source View
0
X
Swap
Swap