Evaluate your test results

What method(s) did you use to evaluate the results of your usability tests?
How did you evaluate the results?

We created Diagrams with Google Docs and compared the different user ratings from the After Scenario Quest (ASQ) and the time they needed for the tasks. We calculated the average for the time and every post-task question and included them in the diagrams.

Project description

Prototype

Link to the final prototype: https://www.figma.com/proto/uyswzIlrEDZOnSqmzk08te/Sleepy-Heads-HiFi-Prototype?node-id=17%3A11&scaling=scale-down&page-id=0%3A1

Unique in our prototype is that we have a design for the night and for the day.

Tagline

Sleepy Heads – Good night and sleep well. 🙂

Group members

Description

Many people suffer from sleeping problems and need help with falling asleep, having a well-rested sleep and waking up. Sleep is very important so that people can handle their everyday life. There are a lot of factors that can affect the sleep and many people aren’t really aware of this. For example: We all know that we should avoid screentime before going to bed, but we often do it nonetheless. That’s why we decided that our app should have a blue light filter and we chose to avoid the colour blue and its variations. This is the reason why our mobile application is only using warm colours.

Our goal is to increase the quality of sleep and improve understanding of the persons sleep schedule. Everything will be conveniently recordable in one app and there is no need for paperwork. By recording sleeping patterns, it provides the capabilities to improve the sleep quality and allow doctors to diagnose faster, easier and more accurately.

Additional Pictures

(1) Improving our high-fidelity prototype

We continued working on our prototype and mostly fine-tuned it so that we were as happy as possible with the latest version we gave to the evaluators in our partnering group. This was important for us in order to get the most out of the evaluation and really be able to make use of the incoming feedback.

Here is the latest version of our high-fidelity prototype:

https://www.figma.com/proto/uyswzIlrEDZOnSqmzk08te/Sleepy-Heads-HiFi-Prototype?node-id=17%3A11&scaling=scale-down&page-id=0%3A1

(2) Conducting the first and second phases of a heuristic evaluation

Phase 1: Preparation

Tasks For the Evaluators

1. You are looking for help in the app, where would you find it?
2. Pick a relaxation method.
3. Pick a sleeping sound.
4. You have a headache and muscle soreness, choose these factors in the app.
5. Add your own custom factor.
6. Activate the option to record your sound during sleep.
7. Start the sleeping mode.

Our evaluation form

We used the evaluation form template in order to make our own for the evaluation process.

https://docs.google.com/forms/d/e/1FAIpQLSe_-bejHJuFllO1uwi9hQ1E8uUpEFpXDQEg1q9P8z5QNIGDKw/viewform?usp=sf_link

Screenshots

We each used Google Drive to collect our screenshots individually and share their URLs in the evaluation form.

Phase 2: Evaluation

While conducting our individual inspections, we all benefitted from the following sources in order to better understand the Nielsen’s Heuristics:

• https://media.nngroup.com/media/articles/attachments/Heuristic_Summary1_A4_compressed.pdf
• https://www.nngroup.com/articles/ten-usability-heuristics/

Summary of our evaluation processes

We all found the tasks that were given to us very well written and had no problems understanding what was wanted from us. Each of us carried out the tasks and noted the problems we came across while doing so. We then used the sources stated above to link the issues we had found with the corresponding heuristics. Once we knew which heuristic the issue belonged to, we filled out the evaluation form and supported our explanation with a linked screenshot. Once we had all finished our individual inspections, we met up to briefly exchange our experiences during the heuristic evaluation process.

Our heuristic evaluation report for Group 1:

https://docs.google.com/presentation/d/15QT9JuhXICgxco4mfTA4s6zQVY1cxVwdC5XPKEWonXc/edit?usp=sharing

What framework or tools are you going to use? Why?

Figma will remain our main prototype tool. This time we will use design kits like the one from Material UI to improve our prototype and give it feeling of an almost finished app. Because of the former assignments we improved our interaction with Figma and can now build an even more complex prototype.

Timeline

(1) Continue to develop (or start a new) paper prototype based on new insights or feedback from your peers.

What and why you have changed your prototype?

We used the feedback we got from our first feedback round, for example to have a dream diary with voice recording.
In the first version of our prototype we build functions that are important before a potential user goes to sleep (the night mode) but in order to test our prototype with its full functionality we decided to extend our prototype with the missing functions (the morning mode).

This is our new version of the prototype:

https://www.figma.com/proto/qH6nfjcUHLOE83Zun9yMlr/Prototype-ver2?node-id=0%3A1&scaling=scale-down&page-id=0%3A1

How you expect this will improve the prototype?

We hope that with the extension of our prototype the user will easier understand how the mobile application should be used than only with the first version.

(2) Conduct (at least) 3 formative usability tests.

Develop a script for your usability test.

(1) Notes for the User
We test our app and not you! Please ask questions and I do my best to answer them. Please think aloud and describe your interaction with our prototype.
We will record your screen and this interview as described in the Consent Form.

(2) Context of the application
Sleepy Heads is a mobile application that can track your sleep.
It was developed to help people to monitor and improve their sleeping problems. Furthermore you have a dream diary, relaxation methods and your sleep will be evaluated.
As part of the usertest we want to test our mobile application and get your feedback.

(3) Questions
How old are you?
Did or do you have sleeping problems?
Did or do you go to a doctor because of your sleeping problems?
What helps you falling asleep?

(4) Szenario(s)
You went to the doctor because of your sleeping problems and he recommended the mobile application ‚Sleepy Heads‘ to you.
You owned a smartwatch and you tracked your sleeping schedule with it until now. You want some more help and thats why you want to use ‚Sleepy Heads‘.
Now you want to go through a relaxation method before going to sleep and listen to a nature sound while falling asleep.
After you wake up you want to write down a dream you had and rate your sleep quality.

(5) Tasks
1. Familiarize yourself with the application and describe your first impression.
2. Please choose a Relaxation method.
3. Please search for sleep sound.
4. Please add a new factor.
5. Please choose the option to record your voice while you are sleeping.
6. Start the sleep modus.
7. Please stop the sleep modus.
8. Please add a new dream in your dream diary. Where would you navigate?
9. Please rate your sleep quality.

(6) Final questions
1. What are pros and cons of our mobile application?
2. Were all of the icons that we used understandable?
3. How did you find it that you had sleep methods and sleep music in the same category?
4. What did you find very good?
5. What would you changed?

(7) Final
Do you have any more questions regarding our mobile application?
Thanks for your time and participation!

Document who is taking what role.

Because of the fact that everyone of us invited one friend for the usertest we decided that the person who knows the interviewee will be the moderator and the other two will document the interview.

Decide if you want to record your test session and how you take notes during the test sessions.

We want to record the voice and the screen of the interviewee but it depends on the person if they agree to it or not.

Document who you are inviting for a test session and how long the session lasted.

1. Usertest: woman, 21 years old, no sleeping problems, but it takes a long time to fall asleep. The interview took 25 minutes.

2. Usertest: woman, 28 years old, has sleeping problems and went to a doctor, melatonin helps her to sleep better. The interview took 18 minutes.

3. Usertest: woman, 24 years old, has problems to wake up because her sleep is too deep, in order to fall asleep she listens to music or podcasts. The interview took 18 minutes.

Conceptual model for task analysis

We chose the BPMN model to visualize the task flow of our mobile application to get a better understanding how the user would navigate through our app.

Moodboard

Affinity Diagramming

For this task we used the website Google Jamboard
Here is an explanation for the different colours we used:

1. Share Data
   At first we put each information on a sticky note and sorted them by the different data gathering method we used namely interview and markt analysis:

2. Clustering
Then we organzied them into different categories…

3. Headlines
…and gave them headlines:

4. Super Headlines
Then we discussed if we could combine some categories or not and gave
these new categories headlines too.

There you will see that some of our undercategories changed a bit due to our
constant discussions about the informations that we gathered.

5. Prioritize
In this final step we prioritized each category as to whether it is super
important for designing our mobile application or not.

Our Plan

1. What are the goals of our data gathering session? What do we want to find out?

Our goal is to find out the requirements of the target group for such an application and what possible functions there are that we haven’t taken into consideration yet. We would also like to find out about the process at sleep labs and how the data collection is done.

2. What kind of people do we want to gather data from? Who are our participants?

Our participant is a person from our target group, who has been struggling with sleeping problems for many years and has also previously had experience with sleeping labs. Our participant is 27 years old and is not currently studying or working.

3. What is our data gathering method?

Which method will we be using and why?
We will be gathering data by conducting an in-depth interview with a person from our target group. As an additional data gathering method, we will be analysing existing sleep-tracking apps.

What type of interview?
Semi-structured

What kind of data do we want to gather?
Qualitative detailed data

4. How we decided to handle the topics pilot study & data recording?

We will conduct a pilot study with someone from a closer circle (as in a friend or family member) in order to make sure all questions are clear and understandable.

The data recording will be done via 1 interviewer and 2 recording clerks. If possible, the interview will also be recorded.

Preparation and conduction of our in-depth interview

1. Where to find our interview script?

You can find our interview script on page 2 of this blog post.

2. How we decided to conduct the interview?

Who took notes?
Angelika and Yasemin took notes. (We had decided to have 2 recording clerks instead of 1 to make sure we don’t miss out on any information if the interview is not recorded.)

Who asked the questions?
Tanita, as she knows the interviewee, we decided this would be best in order to make the interviewee feel as comfortable as possible during the interview.

3. Where to find the collected data?

You can find a summary of the collected data on page 2 of this blog post.

Conduction of one other data gathering method

1. What method did we choose and what type of data did we decide to gather?

We decided to conduct a market analysis to collect further data on our topic.
The questions that most interested us during this analysis where what our competition is doing and what we could do to outshine them.

2. Where to find the collected data?

You can find a summary of the collected data on page 2 of this blog post.

We are Team 5, now also known as Sleepy Heads.

Our Team

Our project

1. Who is our user group?

Persons with insomnia or persons that want to record their sleeping schedule and behaviour are our user group.

Additional stakeholders are doctors and therapists.

What person will we consult?
We will consult a friend of Tanita, that has already experience with a sleeping lab and is interested in tracking her sleep.

2. What is the exact problem?

Patients with insomnia often have to fill out many paper sheets and may lose or mislabel one of those sheets, which can make it harder for doctors to make a diagnoses.

3. Where is our user group interacting with our software?

At home (in their bed).

4. When is the user group interacting with our software?

Before going to sleep and after waking up.

5. Why do our users need this software?

Our software is designed to increase the efficiency of recording the sleep schedule of a person. Everything is conveniently recordable in one app and there is no need for paperwork.

By recording sleeping patterns, it should provide the capabilities to improve the sleep quality and allow doctors to diagnose faster, easier and more accurately.

6. How do we want to solve the problem?

We want to create a mobile application where people can track their sleep (manually or via external trackers like Google Fit, Fitbit, Apple Health, etc) and also have the possibility to write down their dreams.

The app analyses the given data and provides additional suggestions and support for a better sleep. All data can be exported to PDF format and shared with doctors and therapists.