Evaluation of test results

What method(s) did you use to evaluate the results of your usability tests?
How did you evaluate the results?

Zum einem haben wir alle Kritiken und Verbesserungsvorschläge herausgeschrieben, zusammengefasst und diskutiert, bei welchen es sinnvoll ist auf sie einzugehen und die Verbesserungsvorschläge umzusetzen.
Außerdem haben wir zur Bewertung der App die Tester/innen den “System Usability Scale” (SUS) ausfüllen lassen und ausgewertet. Unsere Ergebnisse waren 80, 85 und 90 Punkte, was alles in die Kategorie Excellent fällt. Dies freut uns natürlich sehr, allerdings sind wir auch der Meinung, dass unsere Testproband:innen nett waren und uns möglicherweise etwas zu freundlich bewertet haben.

What did you learn from the testing?

Es macht tatsächlich einen großen Unterschied, ob man als Entwickler:in Entwürfe oder Systeme noch einmal durchgeht und guckt, ob alle nötigen Funktionen eingebaut sind, diese verständlich sind, oder ob das jemand tut, der sich damit noch überhaupt nicht beschäftigt hat und ganz offen ohne Vorwissen die Sache angeht.
Es sind ein paar Probleme aufgetreten, bzw. Funktionsweisen nicht erkannt worden, mit denen wir nicht gerechnet hätten und es kamen auch einige sehr gute Anmerkungen und Verbesserungsvorschläge, auf die wir nicht gekommen wären, da wir zu sehr in der Thematik drin waren und so keinen offenen Blick mehr hatten, für was noch alles möglich/vielleicht sogar notwendig ist.

What are your main takeaways?

Testen ist unglaublich wichtig und sollte am Besten schon relativ früh erfolgen, damit es noch möglich ist, die vorgeschlagenen Veränderungen und Verbesserungen einzubauen, ohne dass dies einen enormen zusätzlichen Arbeitsaufwand nötig macht.
Außerdem sollten mehrere Tester/innen genutzt werden, da unterschiedliche Menschen einen unterschiedlichen Fokus haben und damit auch unterschiedliche Ideen und Verbesserungsvorschläge. Außerdem zeigt sich so auch, ob etwas, was beim ersten Tester/ der ersten Testerin nicht funktioniert hat, ein generelles Problem ist, oder dies nur ein Einzelfall bei dieser Testperson war und somit gar nicht behoben werden muss.

Project description

1. One image (1000×500 px, png), which shows your prototype.

2. One image (200×200 px, png), which shows some unique part.

3. Name of your project +tagline:
Scenic Route – find your perfect walking route!

4. Name of all group members:

Milos Budimir, Freie Universität Berlin, Master Informatik
Marc Oprisiu, Freie Univerisät Berlin, Bachelor Informatik
Sebastian Wullrich, Freie Universität Berlin, Bachelor Informatik

5. Project description text:

Last year showed us the importance of staying healthy in times of pandemic. Walking with one’s closest friends turned out to be the most important form of socialization during the lockdown, but also has a positive effect on the psyche of each individual. Scenic Route was developed to emphasize awareness of one’s surroundings. Unlike conventional route planning, it does not calculate the fastest route, but the one with the most sights or the most unusual attractions. You get relevant information about interesting buildings and the surrounding area while walking around, so that you become an expert of your city! Choose whatever sort of gastronomy or attraction you would like to see on your route and enjoy the walk while being provided with interesting background knowledge based on experiences of a (soon to be) huge community. Functionally our prototype does still not provide real time route guidance. In the actual implementation open source map data such as OpenStreetMap could be used.

6. Additional images:

• 

• 

• 

7. LINK TO OUR PROTOTYPE

Reflexion

WHO MADE WHAT CONTRIBUTION?

We worked together on this Assignment at a WebEx meeting.

WHAT DID YOU LEARN?

We have learnt how to evaluate testing results, and also where we made mistakes, where there has been place for improvement of our prototype, how it could be done better next time.

WHAT WENT WELL?

Whole process went really well. We were happy to work together on another assignment. 

WHAT WOULD YOU LIKE TO IMPROVE?

There is nothing we would like to improve, since we are pretty much satisfied with how it went, both content wise and atmosphere wise.

We continued to improve our Prototype. We added info square about the interesting places nearby while on the route.

Heuristic evaluation

Phase 1:

We are going to use Nielsen’s heuristics.

The results of our individual evaluation are available here.

We tested the prototype of group Veritas.

We used Imgur to upload our screenshots.

A summary of important issues can be found here.

Here is also the Anwendungsfall:

Anwendungsfall 1 – Starte eine Routenführung:

Herr Mustermann ist vor wenigen Wochen für sein Studium nach Berlin gezogen und möchte jetzt seine Nachbarschaft erkunden. Er kennt sich nicht sehr gut in der Umgebung aus und möchte an seinem freien Tag einen Spaziergang mit kulturellen Entdeckungen verbinden. Er hat für den restlichen Tag keine Verpflichtungen mehr und möchte den Tag draußen verbringen. 

Aufgaben:

1. Erstellen Sie sich zunächst einen Account für die App.
2. Starten Sie eine neue Route und füllen Sie die notwendigen Felder bzgl. dieser gemäß Ihrer Präferenzen aus.
3. Wählen Sie eine geeignete Route durch swipen aus (links: ja, rechts: nein)
4. Starten Sie die Navigation
5. Unterwegs haben Sie einen interessanten Ort gefunden, welchen Sie gerne als Ort speichern möchten. Fügen Sie diesen in der App hinzu.

Phase 2:

Evaluation of Group 7’s prototype by each group member separately.


REFLEXION:

WHO MADE WHAT CONTRIBUTION?

For Phase 1 we met in a WebEx meeting. We made some improvements to our prototype. We all together discussed what we are going to send to group 7. We wrote the Anwendungsfall and made a Google form which will serve as a heuristic evaluation which members of group 7 would have to fulfill for us. In the form we also included our prototype and Anwendungsfall. Once we receive the form from Group 7, each of us separately goes through their Figma prototype and conducts heuristic evaluation for group 7.

WHAT DID YOU LEARN?

We have learnt about heuristic evaluation, specifically Nielsen’s heuristics. Apart from getting ourselves theoretically familiar with the concept, we also had hands-on experience of the whole process and we find that very useful. It makes it easier to understand. 

WHAT WENT WELL?

We like the teamwork and load balance we share between ourselves. We are still satisfied with how we do the task division – mostly we do it together and if an individual tasks appear – we afterwards share opinions and thoughts.

WHAT WOULD YOU LIKE TO IMPROVE?

There is nothing we would like to improve, since we are pretty much satisfied with how it went, both content wise and atmosphere wise. 

Hypothesis and problem statement

Melanie needs a way to easily find proposals of nice routes when she wants to go for a walk because she wants to stay health and in the same time she wants to know more about cultural and natural monuments in the city she lives in. We will know this to be true when we see that she successfully takes different routes and meets new parts of her city.

We believe that by building the application for personalized walking routes,
for Melanie, we will achieve improvement of Melanie’s walking habits by making her walks more independent and giving her an easy opportunity to explore the city where she lives in her free time.

Moodboard

Conceptual models for task analysis

We chose Business Process Modeling Notation (BPMN) as a way to graphically represent our task flow. In our opinion it gives us certain advantages over other techniques such as diagrammatical Hierarchical task analysis and other models in contextual design, since BPMN is the most thorough and takes in consideration the underlying technologies we are using (App) in the most detailed way. It helps us determine cycles in our concept and what steps a user might go through using our app.

  

Sketches

Storyboard

Reflexion

WHO MADE WHAT CONTRIBUTION?
We were working simultaneously on this assignment but decided to separate our paths from Task 3 to be able to do the tasks individually. Afterwards we combined our ideas and voted for the most important ones.

WHAT DID YOU LEARN?
These methods are a good way to start a project, because you get fast results, that will help develop your project in the long run.
Especially the task flow with BPMN is a good way to understand better, what needs to be included and might be important to contemplate. 

WHAT WENT WELL?
Everybody had a lot of ideas and the atmosphere was relaxed, so nobody had to worry about an idea maybe being not that good. We were surprised about how many ideas we came up with because we were really into developing our own ideas at first. Later on, after combining the ideas, we found similarities and felt like seeing the ideas and sketches of the other Team Members has been very inspirational for us.

WHAT WOULD YOU LIKE TO IMPROVE?
Sebastian would like to improve his drawing skills, because he isn´t even able to draw something even remotely close to an hand. We are thinking of opening a GoFundMe page for some drawing classes.
Also, the next time we wouldn’t want to spend so much time on finding the right drawing tool for the second Task (diagrams in general).

Affinity Diagramming

Step 1: Share Data

Step 2: Clustering

Step 3: Headlines

Steps 4 and 5: Super Headers and Priorities

Primary Persona

Melanie Mayer, 25 Jahre alt, Medizinstudentin

Extreme Character

Thomas Schneider, 29 Jahre alt, U-Bahnfahrer bei der BVG

Scenario

Melanie steht morgens um 9 Uhr auf und stellt die Kaffeemaschine an, während sie duscht. Im Anschluss schaut Sie sich ihre heutige Online-Vorlesung an, während Sie frühstückt und ihren Kaffee trinkt.
Ihre arbeit für die Uni ist gegen 17 Uhr in der Regel erledigt und sie geht, nachdem sie sich eine halbe Stunde bei Youtube entspannt hat, spazieren.
Dafür nimmt sie ihre Kopfhörer, ihren Hausschlüssel, etwas Bargeld, aber nicht die ganze Brieftasche, und selbstverständlich ihre Maske mit.
Meistens zieht es sie in Richtung des nahegelegenen Parks, indem sie gerne spazieren geht. Auch die in der Nähe liegende Schrebergartenkolonie ist eine gern gewählte Anlaufstelle.
An dem heutigen, recht sonnigen, Tag wandert es sie allerdings ziellos durch die Straßen ihres Viertels, da sie sich nach etwas Abwechslung sehnt und auch ein wenig danach ein paar andere Gesichter zu sehen.
Sie nutzt den Spaziergang um sich bei einer nahe gelegenen Rösterei einen Kaffee zu holen und sich auf eine Bank zu setzen. Dabei hört sie ihren Lieblingspodcast.
Auf der gegenüberliegenden Straßenseite fällt ihr ein sehr schönes Gebäude auf. Sie vermutet es stammt aus der Renaissance ist sich aber nicht sicher. Das Gebäude sieht recht offiziell aus und sie fragt sich, ob es öffentliche genutzt wird. Sie probiert das Googeln kommt aber nicht weiter und bricht die Suche nach 3 Minuten ab.
Anschließend geht sie einen anderen Weg zurück nach hause und macht sich Abendessen.
Es gibt Nudeln mit einer selbstgemachten Tomatensoße a la Arrabiata.

UML Use Cases

Use Case 1: Routenvorschlag hinzufügen

Use Case 2: Routenplanung starten 

Assignment reflection

WHO MADE WHAT CONTRIBUTION?
We together conducted the Affinity Diagramming, defined persona and came up with a scenario. Marc and Sebastian also worked on the UML Diagramming while Milos finished the Blogpost as a whole.

WHAT DID YOU LEARN?
In this Assignment we dived deeper into our app ideas. While working we learnt how to practically implement personas and scenarios using the data we previously collected. We also have a better understanding now why such analysis make sense before jumping into coding.

WHAT WENT WELL?
We really liked doing that and found it quite interesting, because we had to think about actual usages of our application. Creating a persona went really good, so we decided to also create an extreme character as an additional element. 

WHAT WOULD YOU LIKE TO IMPROVE?
On the other hand, we did not find Affinity Diagramming quite useful neither for our quantitative survey nor for our qualitative interview data, because it was "too obvious". Probably larger amounts of data would be more senseful for such analysis. Other than that there is nothing we would like to improve specifically, since we are quite satisfied with out team flow and app idea.

1. Define the goals of your data gathering session.
   What do you want to find out?
   
   Find out if there is a need for this App. Get feedback from interviewees, get more ideas.
   Have deeper understanding of their current behaviour and what do they expect: how would this App improve their everyday interactions with physical activities and free time outside walks?
   How often would they use it?
   Could they picture themselves using the App?
   Do they feel insecure about using the App while walking?
   Would they like to have an offline mode which pre-saves some data in order to be able to use the app without internet?
   
2. Based on your goal, derive the kind of people you want to gather data from.
   What are your participants?
   Walking enthusiasts, that are new to the city.
   Walking enthusiasts, that live here, but doesn’t know the city too well.
   
   Who are you interviewing?
   We will be interviewing one friend who lives in Berlin for a bit more than one year. He used to go to the gym regularly, but during lockdown his new favourite activity became daily hiking/walking around Berlin and sightseeing.
   
3. Decide on your data gathering method.
   Which methods are you using and why?
   1. Interview
      An individual point of view is important and an interviewee might have some ideas on his/her own, what features are important for her.
   2. Questionnaire
      A good way to generate more data ones we know what are the most important topics that need to be acquainted.
      
4. What type of interview (structured, unstructured, semi-structured)?
   1. Interview is semi-structured. 
   2. Questionnaire is semi-structured.
      
5. What kind of data do you want to gather?
   We want to gather valuable information that would help us better understand needs and requirements the future users of our App might have. This we do via two methods: one in-depth interview with one person from our target group, but also a short quantitative survey which would be filled by another chunk of our desired user group.
   
6. Decide how you handle the topics pilot study & data recording
   We would not record the interview, but rather write notes while we conduct the talk. The questionnaire will be an online formular which we will disseminate to our targeted group, so we will then have organized data in a table, for further analysis. 

Interview:

• How would you describe the scope of your sporty everyday activities?

• How would you describe your general health at this moment?

• What is your motivation for walking? Why?

• Who do you usually walk with?

• Has your behaviour concerning walks changed during the pandemic?

• What distances are you usually walking?

• How often do you take a walk per week?

• How difficult would you say the routes generally are?

• How does your walking route usually look like? Do they always differ?

• Do you listen to music while walking or are you focussed on the route?

• How far from home do you usually go to take a walk? Do you start your walk straight from where you live or do you go somewhere (by car, bike, public transport) specifically in order to have a walk?

• On a scale from 1 to 10 how well are you informed about your surroundings (natural, cultural…)? Where do you feel you could use more data? 

• Do you search on the internet in order to get to know your area and the city you live in?

• Do you usually look up or prepare the route or are you open to discover new areas?

• Do you use some apps in order to orientate yourself when going for a walk? Which ones?

• How do you feel about the idea of having an app which provides routes with nice sceneries for walking?

• What would you change it the Koomot app to make it better for your usage?
  
• Would you like to receive interesting facts, in the form of push notification, about your surroundings while walking?

• Is there anything else you would like to add?

The interview took around 30min. Those questions were just a starting point for a conversation. The answers were collected by writing notes during the interview, and improving them afterwards.

Questionnaire:

This is the questionnaire that we have disseminated in our community: https://docs.google.com/forms/d/e/1FAIpQLSfTPrB_u4pLkw10Ci1R9CT3WBkAz2iArutgk6ejpvcr3y-1ew/viewform

We have 56 answers, 50% of which are from those who identify as male, and 50% female.

Assignment reflection:
In this assignment we learnt more about different techniques of data gathering for the project. The three of us, together, came up with the interview questions and questionnaire, meaning we decided in team on the main points and ideas we want to get from this step of our research pipeline. One team member conducted the interview alone, since, due to the pandemic, it was not possible otherwise. We do not see any things that went wrongly. In our opinion we got what we wanted, we learnt some valuable information for our app from both interview and the questionnaire.

TEAM NAME: The scenic way of life

1. Who is your user group?
   Our user group are people in big cities who want to spend their free time outside hiking in the most beautiful landscape possible.

2. What is the exact problem?
   In this corona time, people cannot do many sporting activities, but they still have free time and wanna stay active in the best way possible, so they often choose having a walk. The problem is that existing maps applications suggest the fastest/shortest way to a location and not the most scenic one – the one person would love to spend their free time on.

3. Where is your user group interacting with your software?
   The user is interacting with the software outside, in parks, on the street – in big cities.

4. When is the user group interacting with your software?
   The user group is interacting whenever they have spare time (e.g. when going for an afternoon walk, on their way home, etc)

5. Why do your users need this software?
   Hiking in metropolitan areas has been proven to be dangerous and stressful for pedestrians as well as for bikers and others involved in urban traffic. Users need our software to improve their quality of life and safety. Furthermore they are able to involve in cultural and educational aspects.

6. How do you want to solve the problem?
   We want to create a mobile app that gives a route suggestion based on user’s preferences: 
   1. The level of hardness: easy, medium or advanced routes
   2. How much time user wants to spend
   3. Where does user want to start the route
   4. Where does user want to end the route

Reflexion:

Wer hat welchen Beitrag geleistet?
We developed the general idea together. After Milos generated the Blogpost, Marc prepared the slides for the presentation and Sebastian thought of the name and designed the group picture.
Afterwards we prepared the presentation.

Was habt ihr gelernt?
A sort of think tank to throw ideas around is a fun and productive way to develop and map out an idea.

Was lief gut?
We developed the idea quite fast and all of us were happy with it. Also the distribution of work wasn’t a problem and everyone was happy doing his part.

Was möchtet ihr verbessern?
One group member was a bit late to the meeting, so the other ones had to wait for him. This shouldn’t happen again. Also we should figure out if it is possible to edit other person’s Blogpost.