Gathering data using bing, chatGPT and Bard

Our goal was to explore possible the usefulness of LLM’s for finding information about the ePA’s usability and accessibility. We looked at the free version chatGPT provided on https://chat.openai.com, at the bing chat mode and at the recently released LLM by google, Bard.

We soon found that google’s Bard is far from remembering the context of a conversation as good as the GPT based chatbots. We were not able to keep up a useful conversation.

ChatGPT’s major flaw for our use case is its lack of recent information (up until 2021) and internet access.

Bings chatbot, which is based on GPT4, does not have either limitation, but a conversation is limited to 20 prompts. However, bing states it is not able „to find any specific information about the usability of the German ePA (elektronische Patientenakte) or its accessibility for people with impaired vision.„

Indeed, our research we did for exercise 2 showed that there is not much information about this on the internet. Additionally, the ability of LLM’s to find niche information, which might only occur a handful of times in the entire internet, is very limited.

Creative tasks seem to be more suited than finding rare information. Bing is able to generate personas and scenarios based on the context that I created when prompting bing for information about the usability of the ePA.

https://sl.bing.net/bpYTyTkKyzY (3 personas, 3 scenarios)

Using Reddit to gain insights

Status der elektronische Patienten Akte (ePA)?
by u/simsonsa in de_EDV

Out of curiosity, we published a post on a German developer thread and received plenty of feedback and strong opinions.

The opinions varied from strong rejection due to data security and data centralisation concerns to people welcoming the change and digitalisation of the German health system. People complained about the registration process of the TK app while others shared very positive experience about the ePA and the TK App.

An interesting idea was to contact journalists writing about the ePA and related topics to gain more insights and potentially find an interview partner.

Persona (generated by bing)

Name: Martin Schmidt

Age: 60

Occupation: Retired

Location: Hamburg

Health status: Glaucoma (increased pressure within eyeball), has severe vision impairment and needs a cane to walk, can use digital services with difficulty and assistance

Goals: To maintain his independence and dignity, to get support and care when needed, to stay informed and connected with his family and friends, to enjoy his hobbies and interests

Frustrations: Low vision aids and devices, inaccessible or complex websites and apps, lack of empathy or awareness from others, social isolation or stigma, loss of mobility or freedom

Motivations: Accessibility, simplicity, reliability, safety, comfort

EPA usage: Martin uses the ePA app on his tablet with a large font size and high contrast mode. He also uses a text-to-speech software and a speech-to-text software to interact with the app. He uploads his eye pressure measurements and medication schedule to his ePA occasionally. He also downloads his test results and doctor’s notes from his ePA and listens to them with headphones. He grants access to his ePA data to his ophthalmologist, his general practitioner, and his son. He dislikes the ePA app because it has many usability issues for him, such as small buttons, unclear icons, long texts, confusing menus, and inconsistent layouts. He also complains that the ePA app does not work well with his assistive software and sometimes crashes or freezes. He wishes the ePA app could be more user-friendly and accessible for people with low vision. He also hopes the ePA app could offer some features like voice control, audio feedback, zoom function, and help function.

Scenario

Martin has an appointment with his ophthalmologist next week. He wants to check his eye pressure and see if his medication needs to be adjusted. He decides to use the ePA app to prepare for his visit. He opens the app on his tablet and turns on his text-to-speech and speech-to-text software. He says “Login” and the app asks him to enter his PIN. He says his PIN and the app confirms that he is logged in. He says “Show me my documents” and the app displays a list of documents that he has in his ePA. He says “Sort by date” and the app sorts the documents from newest to oldest. He says “Open the latest document” and the app opens a PDF file that contains his last test results from his general practitioner. He says “Read aloud” and the app reads the document to him with a synthetic voice. He listens carefully and tries to understand the numbers and terms. He says “Stop” and the app stops reading. He says “Go back” and the app returns to the list of documents. He says “Open the second latest document” and the app opens another PDF file that contains his last eye pressure measurements from his home device. He says “Read aloud” and the app reads the document to him. He listens and compares his current measurements with his previous ones. He says “Stop” and the app stops reading. He says “Go back” and the app returns to the list of documents.

He wants to share these two documents with his ophthalmologist before his appointment, so he can review them and give him some feedback. He says “Select the latest document” and the app selects the document. He says “Select the second latest document” and the app selects another document. He says “Share with my ophthalmologist” and the app asks him to confirm his choice. He says “Yes” and the app asks him to enter a time limit for the access. He says “One week” and the app confirms that he has granted access to his ophthalmologist for one week. He says “Send a message to my ophthalmologist” and the app opens a message window. He says “Hello, I have shared my latest test results and eye pressure measurements with you via ePA. Please take a look at them and let me know if you have any questions or concerns. Thank you.” The app transcribes his speech into text and shows it on the screen. He says “Send” and the app sends the message to his ophthalmologist.

He hopes that his ophthalmologist will reply soon and give him some advice on how to manage his glaucoma better. He says “Logout” and the app logs him out of his ePA account. He closes the app and puts away his tablet.