[T3, A11] Analyzing the test results and the whole project

(1) Summarize the project goals, context, and stakeholders

Introduction:

The SmokeFree app was really complicated, and its features weren’t easy to understand right away. The lack of clear explanations during the onboarding process made it even more difficult for users to understand the functionalities. To address these issues and enhance user experience, our project focuses on creating a comprehensive tutorial that simplifies the onboarding process for users. With this tutorial, we want to make Smokefree Diga easier to use and help users understand how it works. We believe that a seamless and straightforward onboarding process is crucial in motivating users to engage with the app and stay committed to their journey towards a smoke-free life.

Our main stakeholders are current people who want to quit smoking. The redesigned onboarding process aims to motivate and support them in their journey towards a smoke-free life with the SmokeFree Diga app.

Activities:

In this project, our main goal was to improve the onboarding process of the SmokeFree Diga app. We started by researching and choosing the app because that had room for improvement. Then, we looked at the challenges and the different people who use the app (stakeholders) – current smokers and those trying to quit.

To understand the users better, we collected data about smoking behavior and interviewed three smokers to hear about their experiences and needs.

With this information, we created fictional characters (personas) to represent typical users. This helped us design with users in mind. In addition to creating personas, we also crafted a scenario for one of the personas to provide a realistic context for the user’s journey within the app. The scenario helped us visualize how the app would be used in a specific situation, making the design process more user-centered.

To focus our efforts, we wrote a clear problem statement about the issues users face during onboarding. Then, we made a hypothesis statement to propose solutions and how they could help users.

Problem statement (related to our persona Sabine):

Sabine needs a way to register easily in the app and to get a simple and understandable introduction because she decided to quit smoking, but due to her limited experience with smartphones, she could be overwhelmed by the many features of the app. We know that this is achieved when Sabine regularly uses the app and its functions.

Hypothesis statement (related to our persona Sabine):

We believe that by creating a good introduction for Sabine, we will achieve an easier onboarding for users so that more of them start using it regularly with understanding of basic app functionality.

Using sketches, we explored different ideas and got feedback. We also created a Low-Fidelity-Prototyp with Figma. This helped us refine our designs.

We made two main user journeys to show how users would go through the app’s key features. With that, we created a high-fidelity prototype that looked more like the final app.

To check the usability without user involvment, we used Nielsen’s usability heuristics to see if there were any problems with the design.

Finally, we tested the new onboarding process with five users in a usability test to see how they interacted with it and if it worked well for them. We used UEQ in its short version as the questionnaire. We also asked the testers to apply the think-aloud method during the usability tests.

(2) Summarize your test results.

  • What method(s) did you use to evaluate the results of your usability tests?

We decided to use Thematic Analysis to evaluate the usability tests. This method is designed to handle large and unstructured amount of data. With this approach, we can efficiently organize and categorize the diverse feedback and observations gathered during the usability testing phase. By using Thematic Analysis, we aim to gain valuable insight into user experiences and identify current issues as well as opportunities for future improvement. In our case we grouped the feedback into the topics: problems with dashbord, separation of tutorial from the app, issues with original app design, and other comments.

  • How did you evaluate the results?

By collaboratively analyzing the data with the project team, we identified recurring usability challenges, potential improvement opportunities, and key insights into users‘ experiences.

  • What did you learn from the testing? What are your main takeaways? Please list these takeaways.
  1. Usability test is not the best way to test for our problem
  2. The consistancy of our prototype is not as good as we thought
  3. Tutorial design is not as easy as we thought
  4. Description texts need to be clearer and give more background information
  5. Tutorial can be seen as a different mode of the app
  6. A lot of usability tests back to back are exhausting
  7. A lot of functionalty of the original app is confusing, but we didnt want to change because it was not a directly related to our prototype and goal of the whole project

Affinity diagram

(3) Compare your results with the defined problem (problem statement) you wanted to solve.

  • To what extent have you been able to meet your goals? Discuss it.

We think that we were only able to partially meet our goal. The usability tests were not the best method to test the effectiveness of the tutorial. We did not test how much of the tutorial the users were able to understand and jow much they could recall. But we think we are at a good starting point to design a good tutorial. We also overerstimated the effort for the foundation of the design

  • What would be your next steps? How would you proceed with this project?

If we would continue our project we would continue to improve our prototype. The usability tests helped to give us a new perspective on tutorials as a design option. Especially how and when the introductions to parts of the app will happen e.g. the tutorial is not one „block“ in the beginning but opens when you use a function for the first time. We would like to compare different kinds of tutorial and do more reasearch into tutorials. Furthermore we would like to do more tests concerning the quality of out tutorial.

[T3, A9] Prepare usability test

High Fidelity-Prototyp: https://www.figma.com/proto/vKlEYIkcpOjpLLNpAIOY6F/HCI-first-prototype?type=design&node-id=1-3&scaling=min-zoom&page-id=0%3A1&starting-point-node-id=1%3A3

Heuristic Evaluation Form: https://docs.google.com/forms/d/e/1FAIpQLSfYjptnWvuVZwWnj5XQnChkWsiv-P8ktnyN5SowfupHybUPiA/viewform

(1) Reflect on the results of the heuristic evaluation you conducted in class.

Bei der durchgeführten heuristischen Evaluation wurden verschiedene Bereiche identifiziert, in denen Verbesserungsbedarf besteht. Die Ergebnisse zeigen, dass das Design nicht auf allen Seiten konsistent ist. Das Design einiger Seiten weicht voneinander ab. Die Navigation auf der ersten Seite erfolgt durch Wischen anstelle von Klicken, was ebenfalls eine Inkonsistenz darstellt. In allen anderen Frames ist zum Navigieren bzw. Interagieren lediglich ein Klicken erforderlich. Die Kennzeichnung des der markierten Navigationselemente im Tutorial ist nicht auffällig genug und die Platzierung der Sprechblase des Roboters nicht auf allen Frames identisch. Die Position der „Weiter“-Buttons variiert auf den Sprechblasen der verschiedenen Screens und sollte einheitlicher gestaltet werden. Die „Weiter“-Buttons haben die gleiche Farbe wie andere Elemente, was ihre Sichtbarkeit verringert. Der „Zurück“-Button sollte eine relative Rücknavigation ermöglichen und nicht zu einem bestimmten Bildschirm führen. Des Weiteren wurden Kritikpunkte an der Gestaltung der Roboter-Grafik geäußert, da sie viel Platz auf dem Bildschirm einnimmt, jedoch keine zusätzlichen Informationen bietet. Bei Formularen ist der „Submit“-Button immer aktiv, obwohl keine Daten eingegeben wurden, was den Nutzer verwirren kann. Die Benutzerführung weist ebenfalls Schwächen auf. Bspw. wird der Nutzer beim Klicken auf „Skip Tutorial“ auf der ersten Seite nach keiner erneuten Bestätigung gefragt. Ebenfalls sollte eine Option zum Wiederholen des Tutorials vorhanden sein.

Zusammenfassend erfordern die Ergebnisse der heuristischen Evaluation Verbesserungen in verschiedenen Bereichen wie Designkonsistenz, Platzierung von UI-Elementen, Sichtbarkeit von Elementen und Benutzerführung. Diese Punkte sollten bei der weiteren Vorgehensweise berücksichtigt werden, um eine bessere Benutzererfahrung zu gewährleisten.

(2) Continue to improve your high-fidelity prototype.

Improvements / Areas we worked on:

CommentDone
First page is still in the old designx
Navigation on page 1 is different to all other pages (swipe instead of click)x
Marking of the footer icon is not eye catching enoughx
The speech bubble of the robot is too far left in comparisson with all other screensx
Robot graphics take up a large part of the screen without providing any additional information.x
Submit button in forms is active when there is no datax
Chosen point in diagram needs to be biggerx
Mission screen is not the same as in the original app. desing should not be touchedx
The „continue“ buttons are placed in different places each time, which can be a bit trickyx
The „continue“ buttons have the same color as most other elements. Not that visiblex
Back button should be a real back button (navigation = back and not to a specific screen)x
When you click „Skip Tutorial“ on the first screen, you are not asked again if you are sure you want to cancel the tutorial.x
Welcome text should have correct interpunctationx
No skip buttonx
The Continue button is somewhat lost here, since it is in a different place than on the other screens. It is also x
The Continue button has the same shade of green as the bar chart and is therefore not directly noticed.x
Cancel button x is not centered in headerx
Text in continue button should be bigger and better readablex
„Back“ button doesn’t feel like „Back“ buttonx
You can see pixels in robot icon. Need to be a bigger iconx
No question if we are sure we want to skip the tutorial in the beginningx
After finishing the tutorial, there is no possibility to repeat it.x
„Do you want to finish the tutorial?“ is separate page, doesn’t feel like warningx
Too sharp corners, rounded would look betterx
Oval as a Speech Bubble looks a bit primitivex

https://www.figma.com/proto/vKlEYIkcpOjpLLNpAIOY6F/HCI-first-prototype?type=design&node-id=1-3&scaling=min-zoom&page-id=0%3A1&starting-point-node-id=1%3A3

(3) Prepare another round of heuristic evaluation  (this time you test the interface of another group)

(4) Reflection

Wer hat welchen Beitrag geleistet?

Wir haben uns die einzelnen Punkte, die in unserem High-Fidelity-Prototyp verbessert werden müssen, untereinander aufgeteilt, sodass jeder ungefähr den gleichen Arbeitsaufwand hat.

Was habt ihr gelernt?

Wir haben gelernt was bei der Erstellung eines High-Fidelity-Prototyps wichtig ist und wie man ein Formular für eine Heuristische Evaluation entwirft.

Was lief gut?

Die Kommunikation im Team lief sehr gut. Wir haben und regelmäßig getroffen, um uns auszutauschen und an der Bearbeitung der Aufgaben weiterzuarbeiten.

Was möchtet ihr verbessern?

Wir möchten unser Zeitmanagement weiter verbessern, um zum einen die uns zur Verfügung stehende Zeit effizienter zu nutzen und zum anderen genügend Zeit für unerwartete Änderungen zu haben.

[A3, P3] Assignment 3

Define the goals of your data collection session.
What do you want to find out? These goals should be based on your project goals.

Enhance the user guidance and support features of the „Smoke free“ app by integrating the next features:
– a short guide of using the app
– daily tracking with motivation section, and advice

Therefore we want to find out more about our target group. For example what are their reasons for quiting, if they quit smoking before and why they went back to smoking. Furthermore we want to find out if they know about apps that support you or if they already tried them.

Based on your goal, derive the stakeholders you want to collect data from.
What are your stakeholders? Whom do you need to approach (i.e., who is indispensable for your achieving project goals)?

Our main target group are people who want to quit smoking, but it would also be interessting to talk to people who quit before (successfully or not) to see patterns that helped them or reasons why they were unsuccessfull.

Start with collecting information without approaching your stakeholders (so-called secondary data collection). Use existing websites, app information, and experience the app by yourself. Describe the app you are using in detail (who has developed the app, why, what functionality is available, how can people access the app, etc.)

Prepare an interview study with your primary stakeholders.
Name the stakeholders and explain, why they need to be interviewed. Explain, how you want to contact them.

Our main stakeholders are current smokers and smokers that want to quit. They need to be interviewed because we need to understand the reasoning for quitting and failed quitting attemps so we can fill the gap with the app. Furthermore we need to see if they see an app as a possible support of their quitting journe. Also we need to see if our stakeholder group is correct or if we need to talk to a other subgroup (e.g. longterm smokers or smokers in a certain age)

Justify the type of interview (structured, unstructured, semi-structured).

In our case, a semi-structured interview is the best choice as it allows us to get some general info about person’s smoking routine as well as to focus on each person’s case individually.

Prepare the introduction and the closing session as well as the consent form as well as the questions (the interview should be 20 to 30 min long)
Describe how you plan to conduct the interview (Who is taking notes? How do you take notes? Who is asking questions?)

Einleitung:

Guten Tag Herr/Frau XY,

herzlich Willkommen und vielen Dank, dass sie sich die Zeit für dieses Interview nehmen. Wir sind … und möchten dieses Interview für unseren Unikurs „Human-Computer Interaction“ an der Freien Universität Berlin führen. Im Kurs beschäftigen wir uns mit Digitalen Gesundheitsanwendungen, so genannten DIGAs. Die DiGA, die wir untersuchen möchten ist eine App zur Unterstützung der Rauchentwöhnung und daher möchten wir ihnen einige Fragen zu den Themen Rauchen und Rauchentwöhnung stellen. Damit wir die Daten aus diesem Interview auch verwenden dürfen, brauchen wir bitte die unterschriebene EInverständniserklärung. Außerdem möchten wir das Gespräch, das ca. 20 bis 30 Minuten in Anspruch nehmen wird, gerne aufzeichnen. Ist das für Sie in Ordnung?

Einverständniserklärung

Demographische Fragen:

  • Wie viel rauchen Sie pro Tag/pro Woche/pro Monat?
  • Welche Tabakprodukte konsumieren Sie?
  • Wann haben Sie mit dem Rauchen angefangen
  • Warum haben Sie mit dem rauchen angefangen?

Fragen zur Rauchentwöhnung:

  • Haben Sie schon einmal mit dem Rauchen aufgehört oder versucht mit dem Rauchen aufzuhören?
    • Wenn ja: Welche Hilfsmittel (Apps, Nikotinpflaster ect.) oder Unterstützungsangebote(Gruppensitzungen, Raucherberatungen) haben Sie bei der Rauchentwöhnung wahrgenommen?
    • Was waren Ihre größten Probleme bei der Rauchentwöhnung?
  • Welche Hilfsmittel und Hilfsangebote zur Rauchentwöhnung kennen Sie?
  • Kennen Sie Apps, die bei der Rauchentwöhnung unterstützen können?

Outro:

Vielen Dank das Sie sich für das Interview zeitgenommen haben. Wir werden Ihre Antworten in den kommenden Wochen auswerten und in unser Projekt einbinden. Dürfen wir sie bei weiteren Fragen zu einem zweiten Interview einladen?

Reflexion

Wer hat welchen Beitrag geleistet?

We decided all together on the questions for the interview. Henry created the consent form, Anna answered the starting questions for the blogpost and Alexandra did the research concerning smoking-statistics (in germany, in the EU and worldwide) and wrote the introduction and closing session for the interview as well as the blog post

Was habt ihr gelernt?

We learned how to set up a semi-structured interview and how to formulate the proper questions for it. Furthermore, we learned how to create a consent form for an interview. We also continued to work with the app and became familiar with its functions.

Was lief gut?

The communication in the team worked very well. We were able to distribute the tasks evenly so that everyone had the same amount of work. We were also able to set up questions for an interview using the methods we had learned in the lecture. Furthermore, we were able to successfully create a consent form in which all data protection-related questions are clarified.

Was möchtet ihr verbessern?

We need to improve our time management so that there is enough time to review the answers and correct mistakes.