Set of four distinct data sets constructed by a statistician Francin Anscombe in 1973. All four sets have almost the same statistical observations, included mean, variance, correlation and number of elements (11), however they have different distributions which results in distinguishments in plotting on the graph.
Although particular points on the graph of each set are not corresponding, each dataset produces the same statistical properties and is represented with the similar linear function. This quartet model is used to show importance of data visualization and plotting datasets on the graph. Even though each dataset generate similar statistical properties, in fact it looks different on the graph. Thus, datasets should be firstly analyzed graphically, before performing relationship analyse and interpretation.
It is not exactly known how Francinc Anscombe constructed this model, however nowadays there are more models with similar properties, which is identical statistics and dissimilar graphics.
[1] Matejka, J., Fitzmarice, G., (2017). Same Stats, Different Graphs: Generating Datasets with Varied Appearance and Identical Statistics through Simulated Annealing
This is the biggest problem because it deviates from the expectations of the user, thus potentially confuses them.
How will you solve it?
We followed their recommendation and made the state constient by removing the forgotten link we still had active in figma.
How do you expect the user will behave, after the problem is solved?
Thanks to the added tutorial at the beginning of the interference with our app, the user is guided how to start using the app. The layout of our diagrams is consistent now and follow general guidelines on which we decided.
Decide what usability issues you are going to fix in your prototype and what features you are going to implement next.
We implemented all of the three proposed fixes (add tour through the app at start, change one icon, fix one inconsistent screen), as they were small in scope and we saw their potential for improving the usability and understandability of our app.
(2) Preparation for a summative evaluation
What are things you would like to do differently this time?
The tasks were formulated too specific. For example, 2 b) was:
“Schaue dir die Statistik zu diesem Thema an”
We wanted the user to acknowledge the “stats” tab but now we just told them to click on it. Instead, we should have formulated it like “get an overview of the number of articles” or similarly, so that it’s an actual behavior which might be observed in real users.
Additionally, there were too many tasks compared to the small scope of the prototype.
What documents do you need?
Consent form:
6.1 Pilot Study Forms
Think-Aloud and Interview Pilot Study – Consent Form Sheet #1
I agree to participate in the study conducted by the Human-Centered Computing Lab of the Freie Universitat Berlin. Participation in this study is voluntary and I agree to immediately raise any concerns or areas of discomfort during the session with the study administrators.
I understand that all data gathered in this test, including sound records, will be anonymized. I am aware that results from these tests might be published. Please sign below to indicate that you have read and you understand the information on this form.
Date: ____________________ Please print your name: ____________________ Please sign your name: ____________________
Thank you!
The Task Observation Form
We updated our script:
(1) Hinweise für den User
Small-talk
Disclaimer: “Wir testen die App, nicht dich! Fragen bitte gerne stellen, ich kann aber nicht versprechen, alle Fragen direkt zu beantworten! Bitte laut denken (Think aloud).”
(2) Kontext der Anwendung
Veritas ist eine App, die für alle gesellschaftspolitisch interessierten Menschen entwickelt wurde. Veritas bietet Usern die Möglichkeit, URLs oder Themen einzugeben. Den Usern wird dann ein umfangreicher Überblick über das Thema gezeigt.
Wie interessiert bist du an gesellschaftspolitischen Themen?
(3) Szenario(s) und
Du bist auf der Website der NYTimes. Du liest einen spannenden Kommentar über Joe Bidens Waffenpolitik. Du findest den Artikel nicht sehr ausgewogen und würdest gerne andere Perspektiven über dieses Thema einholen.
Benutze die veritas-App, um ähnliche Artikel zu finden.
(4.1) Aufgaben
Lies einen Artikel, der eine gegensätzliche Meinung zu dem NY-Times Artikel hat
Verschaffe dir einen Überblick, wie die Medien über dieses Thema berichten
Which standardized questionnaire did you choose and why?
We used the UMUX questionnaire and adopted it to our needs: Link
(3) REFLEXION
WHO MADE WHAT CONTRIBUTION?
We met all together and analyzed the evaluation that we received from group 3 and decided what we need to change in our prototype. Daniel played the biggest role in working on the Figma prototype On another meeting we conducted preparations for summative evaluation. On all of that we worked together during the meetings.
WHAT DID YOU LEARN?
We could retrospect assignment #6. After gaining more experience and knowledge about our project, we changed our approach in testing and restructured the tasks for the next testing session.
WHAT WENT WELL?
As we were quite happy with our prototype already, with tips from group 3 and our own conclusions, we made our prototype even better.
WHAT WOULD YOU LIKE TO IMPROVE?
We could have split up the work as some of the points we did during our meetings didn’t require all of us at the same time.
What framework or tools are you going to use? Why?
We used Figma to prepare our Hi-Fi prototype. It is a framework that we were familiar with from our paper prototype and there was no issues with working with use of it before. Figma allowed us to use components based on styled guidelines compatible with official specs.
List your functional and non-functional requirements (features) you are planning to include in your High-Fidelity Prototype.
Non-functional requirements:
Simple to use (mobile + busy environments)
Explain the political compass
Functional Requirements:
Search articles via an URL
Search articles via topic search
List view of articles
View for a single article
View for the Topic(s)
View of the political compass
Statistics view for a topic
Allow feedback for article placement (article is not e.g. “left-authoritarian”)
Allow feedback for topic clusters (articles aren’t about same topic)
Please answer the following question: How did you handle the topics: menu, UI consoles, and Design patterns?
What menu type did you choose and why?
We decided on slide menu which is inline with material design. As our user group is going to use mainly the platform on their phones, this type of menu will fit the best to scroll through the list of trending topics and article list.
We applied also top navigation for possibility to go to different options after searching an article by URL. Instead of putting everything on one screen, the user can choose what steps to take next.
Which UI controls are appropriate for your application and why?
As for UI controls we used buttons so the user can explore our platform easily and navigate through it. An example would be search button, which then opens text input field. This is another control in our prototype. Another example of a button is menu button that navigates to submenu.
By using top navigation we implemented tabs for features choose. Links allow user to go to chosen article and read it either on our platform or go specific website.
Radio buttons were used for feedback option to rate the articles placement.
Which design patterns are suitable for your application and which ones have you implemented or used? Why?
While designing the prototype we were basing mainly on the guideline that we accessed from Figma. These are icons used for the buttons and general layouts. Political compass and all the diagrams were designed by ourselves.
Reflexion
Who made what contribution?
We worked together on the HiFi prototype. Firstly we planned what components needs to be designed and applied. Then we assigned the tasks, so parts of our prototype, and combined them again together. While everyone contributed to the prototype, Daniel and Arne focused more on working with Figma, Clemens did a Gantt chart and Mateusz summed up our work in the blog post.
What did you learn?
We definitely improved our skills in Figma and be able to use more advanced tools included there. There was more components that we need than we expected, but this allowed us to go into detail with our work.
What went well?
Communication with the team went well. We are happy with the direction that we are going to with the prototype. There is of course still field for improvements, but our target should and will be achieved.
What would you like to improve?
After two weeks’ time our prototype could be looking better so we could be further with the advancement of the prototype. It still looks quite basic, but as mentioned before, this will be improved.
(1) Continue to develop (or start a new) paper prototype based on new insights or feedback from your peers.
As there was no testing during last tutorial, we couldn’t gather any feedback to develop our paper prototype. However, we implemented a few changes compare to last week.
This means couple of more slides on our paper prototype, so the idea and process of working with the political compass by the users is more clear.
Disclaimer: “Wir testen die App, nicht dich! Fragen bitte gerne stellen, ich kann aber nicht versprechen, alle Fragen direkt zu beantworten! Bitte laut denken (Think aloud).”
(2) Kontext der Anwendung
Veritas ist eine App, die für alle gesellschaftspolitisch interessierten Menschen entwickelt wurde. Veritas bietet Usern die Möglichkeit, URLs oder Themen einzugeben. Den Usern wird dann ein umfangreicher Überblick über das Thema gezeigt.
(3) evtl. kurzer Fragebogen (Fragen zur Person)
Wie alt bist du?
Was studierst du?
(4) Szenario(s) und
Du bist auf der Website der NYTimes. Du liest einen spannenden Kommentar zu einem Thema, das dich interessiert. Du findest den Artikel nicht sehr ausgewogen und würdest gerne andere Perspektiven auf dieses Thema einholen.
Benutze die veritas-App, um ähnliche Artikel zu finden.
(5) Aufgaben
Suchen mittels der Artikel URL aus dem Zwischenspeicher weitere Artikel
Beschaffe dir eine Übersicht zu einem Thema
Sortiere die Artikel nach “Opposite first”
Schaue dir die Statistik zu diesem Thema an
Finde andere Artikel indem du den politischen Kompass verwendest
Kehre zur Startseite zurück
Entdecke weitere Themen
(6) Abschließende Fragen
Hast du noch irgendwelche Fragen oder Anregungen?
(7) Abschluss
Danke für deine Zeit
ROLES
In terms of roles of moderator and observer during the tests we were changing positions each test. The tests were conducted in single sessions, so there was no confusion about that for our participants. We decided that Daniel, Arne and Clemens will be moderators for each test, as they are German speakers. This way the participants would feel more comfortable as they were German speakers and there would be less problems with understanding.
RECORDING
As our prototype is in digital form, all three tests took place online, through Discord platform, where our participants could easily share their screen with the prototype that was provided to them through link during the session. All of us could see how they go through our prototype, while thinking aloud, so there was no issue with taking notes during the test.
WHO AND HOW LONG?
Participant
Yingshan
Charlotte
Ela
Age
23
29
23
Occupation
Student of Biochemistry
Finished her history master and is now looking for work
Student of Chemistry
Facilitator-role
Clemens
Arne
Daniel
Observer
Arne, Daniel
Mateusz, Daniel
Mateusz, Clemens
Time
15 min
16 min
14 min
(3) Document and evaluate the results of your testing.
What method did you use to evaluate the results of your usability tests? How did you evaluate the results?
After conducting the usability tests we met for analyse of the results. We went through the notes together and thoroughly talked about them. There were also comments from our side, what we saw during the tests and how do we feel about usage of our prototype. From all these data we gathered, we extracted the takeaways.
What did you learn from the testing? What are your main takeaways?
It occurred that there were some issues with understanding the context of political compass and political spectrum. Some users might be not familiar with those terms and their functionality. As this appears to be a problem, political compass should be designed in the clearest possible way, so there would be no misunderstanding or confusion for the users. Providing this tool is crucial for functionality of the platform, in order for users to extract the most during building their unbiased view on the topic.
One of the good ideas from our participants was to gather feedback from the users on the articles. Users could rate the positioning and effectiveness of political spectrum. This would help us in developing the platform.
Another problem appeared with the home screen of the platform. Our participants stated that without context it is hard to determine what is it about. This should be improved, however we believe that our users now beforehand what this platform is about and they go for our tool with an idea what do they want to achieve. Our “paper” prototype might have not been clear enough due to our manual skills with creating it.
In our paper prototype we focused mainly on the use case, when the user searches an article by an URL to explore particular topic. Our participants of the usability test pointed that the list of topics to explore should be developed in the prototype. This showed us that users would be interested also in this use case and it is worth to focus on that.
Other takeaways that were discovered from the usability test:
– On what basis is the topic shown on the explore view?
– Clicking on the small political compass on articles should be maybe also implemented in the political compass view
– Articles in Article list take too much space if there were too many articles
– Political spectrum in articles overview seen as key differentiator
Reflexion
Who made what contribution?
The script was prepared during last tutorial and we improved it afterwards before conducting usability tests. On our weekly meeting after tutorial we discussed the prototype again and Daniel made necessary changes. We conducted usability tests in separate sessions, the roles were pointed before in the blog post. We discussed and summarized the data from the tests together and Mateusz wrote the blog post.
What did you learn?
The usability test gave us different perspective on our prototype. For us, everything was clear and understandable as it is our idea and we are working on it for quite some time. As for the person that have first contact with this topic it appeared that some aspects are not that obvious. It gave us a lesson that we need to focus more on presentation of our idea. We do not want the future users to be discouraged by first interaction with our platform.
What went well?
This week there were no delays and lack of communication, so we are definitely happy about that. We improved ourselves on this aspect.
What would you like to improve?
The results from usability tests showed us what we need to work on. As mentioned above, users need to get better idea of the context of the platform.
Moore’s Law is based on an observation of transistors in an integrated circuits. Its statement is that the number of those transistors on a macrochip double every year. It is rather an emperical observation than physics low.
Moore’s Law refers to technological development and increasing performance capabilities of computers. While computetional progress is increasing, the costs of it are decreasing. The annual growth rate of this progress is counted at 41%.
Moore’s Law was used for over 50 years to conduct long-term planning and set goals for research in developments of computetional industry. Advancements, such as increase in memory capacity or number of pixels in digital cameras, are strongly linked to Moore’s Law.
1. Summarize the feedback you received regarding your storyboard
From the feedback, that we received from our peers, we gathered some clues that we did not think about before.
The political compass should be explained. The users may not know what political compass actually is and as they are introduced with this idea on our app, they should be informer or guided through it so they can get the most of the experience.
In our storyboard we showed many features of our app and what user can do but we did not specify what is the landing page when the app is opened. As from this point the whole interaction with our app starts, we should emphasize it more in our prototype.
The distribution of article count should be two dimensional. On our storyboard we showed general count of how many articles refers to the topic, but it would be more useful to visualize, i.e. which side of the political compass has more articles . We could use a heatmap for this. It gives overall view which side talks more about particular topic.
In our prototyping process we had a look on the feedback we received from our peers. We decided that we will base on what we did in the storyboard. With this as the fundamental part we could include clues from feedback and create a prototype basing on the use case from previous assignment. Instead of many features shown individually on the storyboard we condensed it to simple prototype with Figma web application.
the use case and/or model (task analysis from last assignment) this prototype relates to.
In terms of use case we do not lead users what to do, but rather give them options to move around our platform. We do not want to make them read particular articular. After entering their article they get an overview of political compass through tab “Stats”. From there they can choose if they want to read an article and which.
how the storyboard is reflected in your prototype.
While storyboard showed overall of our apps features, in prototype we condensed it to the case when the user wants to use the app by typing in the article URL and what can happen after that. The protype is strongly based on the storyboard and contains features implemented there.
self-assessment of potential strengths and weaknesses of this first step into your design space.
We believe that the idea of our app is quite straight-forward: user types in an article, get an overview of the situation on political compass and reads other articles. As our goal is to achieve unbiased opinion by the user, the app can not be biased in any way to influence the user. With the prototype that we created so far, we are meeting this requirement.
On the other hand it is hard to determine how accurate political compass positioning can be. The question is how to evaluate articles truly unbiased. Also our drawing skills may not be perfect ones, so it is hard to express our idea fully in this type of prototype.
3. Design rationales
To design our rationales for our questions we used structure-oriented technique, QOS.
Reflexion
Who made what contribution?
We met on our weekly meeting straight after the tutorial on Tuesday. We went through assignments together and discussed about the tasks. We gathered all the ideas and prepared draft of the prototype and design rationales. As only Clemens attended the tutorial, he was responsible for gathering the feedback from other students. Daniel prepared the design of prototype, Clemens and Arne prepared visualization for design rationales and Mateusz gathered all together and prepared the blog post.
What did you learn?
Sometimes we have the feeling that we could already go to the next steps and some of the tasks are repetitive. However, we are learning how to be patient and we see advantages of low-fidelity prototyping. We could focus in trivial aspects of our domain and thanks to feedback from other students make changes to our project that did not required stepping back in the design process. At this level we can still form our design and it does not cost us in terms of time and effort already taken in the project.
What went well?
We are happy with how our idea is developing and our understanding of it through different assignments. Apps that are linked in the assignments are helpful and it is easier to visualize what we have in minds.
What would you like to improve?
There was a serious lack of communication last Tuesday and in result only Clemens was present on the tutorial. This definitely can not happen again. Through different aspects we divided our work not evenly on last two assignments and we did not have problem with at a time. However, for our own best and learning the skills from assignments, we need to distribute work better and every week each of us needs to have the same involvement in the project.
Define the goals of your data gathering session. The goal of our study is to gather information on how people of age group 50+ form their opinions on political topics, i.e., how many and which sources are used. Additionally, we attempt to find out whether they consider their own opinion biased or balanced. We want to also know how they life in social media looks like and their daily usage of news.
Based on your goal, derive the kind of people you want to gather data from. We needed people of age 50+ for our interviews. Survey, however, was sent to wider range of people to get bigger picture on the problem. Through this survey we were also looking for participants over 50 years old.
Decide on your data gathering method. We use two methods. First, we conduct a questionnaire targeted at the general public, which we will post on forums and mailing lists. In parallel, partially based on preliminary results from the questionnaire we conduct in-depth interviews with multiple participants of our target group of age 50+.
Our interviews are semi-structured. We decided, since we are going to talk about politics, we need to give our interviewed a space for longer responses and to share their opinion on the problem.
Decide how you handle the topics pilot study & data recording. For data recording we used: Audio recording and notes for personal interviews and video recording for a remote interview. Achieved transcription from the interviews will be used as a main source for us.
We use Google Forms as questionnaire software. They will provide us with categorized and counted solutions in the form of Excel sheet. We will be working from there on the next assignments.
Reflection:
We took our time to rethink thouroughly the questions for participants of our survey and interview. There was a lot of brain storming to decide what data do we want to gather from the public. Working together on the file was very helpful and time saving.
What surprised us, was the response for our survey. We got more replies than we expected and we hope that this is going to make our work more clear in next assignments. Our topic is definitely a wide area and a controversive one. We hope for more questions to answer and more solutions to provide.
In terms of how the interviews went, we are happy how did it go and with data we were able to gather. Some information and ideas came up during them, that we didn’t predict before. The biggest struggle was avoiding leading questions, as the main topic is politics and media, where each person can have it’s own, original opinion.
Also thanks to the survey we conducted, we could get a feedback from our responses about the questions and the problem. Opened questions in the survery, allowed our responders to state their opinion and some of them we followed up by the discussion on our meeting.
Who is your user group? Anyone interested in politics and news.
What is the exact problem? Today’s news is often inherently biased and lacks nuanced journalism. Some more, such as breitbart news, and some less, such as reuters. With the recent surge of social media, a large share of unsuspecting readers fall into a so called filter bubble, i.e., they only see news of specific political backgrounds, which they are anticipated to like. Some think, this may lead to political extremism [https://www.connectsafely.org/social-media-can-lead-to-political-extremism/], further deepening preconceived opinions and ideas. Anyone interested in politics and news.
Where is your user group interacting with your software? When the general public are using, e.g., social media, and want to fact check an article or get a different view on the topic the article is about.
When is the user group interacting with your software? The “check” might be both 5 minutes for the particular single news and also longer time, depending on how much the user wants to learn about different sources.
Why do your users need this software? We believe that the majority of readers in filter bubbles are simply unaware of other political views, but when notified of their biased opinions, are willing to get an overview of the entire political spectrum. Usually, people seek to find out the truth, making this software particularly interesting.
How do you want to solve the problem? We want to provide a web page where users can provide an url of an article which, then, gets placed onto a political compass. Other articles of the same topic will be shown on the compass too.
Reflection
On our first meeting we decided to give ourselves some time to come up with ideas and look for possible interviewee. We presented each idea and after thorough discussion agreed on the final one that we will work on during the semester. We considered all the pros and coins and general details that we need at this stage. Formulating the pitch was quite straightforward after that. As our users group is quite wide we need to definitely work on narrowing this group of interests before starting gathering the data.
