A user holds Josy in her hands and operates the tray.

Virtual companion "Josy" cares about elderly people

forschung leben – the magazine of the University of Stuttgart (Issue March 2022)

A virtual assistance technology system designed to help older people in their everyday lives. The developers exchange ideas with interested parties in a series of workshops.
[Photo: Ludmilla Parsyak]

“She” is ticklish, loves to laugh, and drinks regularly: “Josy” is a virtual assistance technology system designed to help older people in their everyday lives. The system was developed by Franziska Braun and her colleagues at the University of Stuttgart's Institute of Human Factors and Technology Management (IAT) and the Fraunhofer Institute for Industrial Engineering (IAO).

“Josy acts as a friend that older people like to talk to and include in their daily lives,” Braun explains. For example, the virtual companion regularly takes a drink, which is something that older people in particular often forget to do. The system can also recognize when someone feels lonely or sad, whereupon it asks: “Would you like to call your relatives, or should we send them a picture of you?” The technology is intended as an aid in preventing loneliness and social isolation.

Josy incorporates an app based on facial recognition technology, whereby the built-in camera scans a face and compares the resulting data with an emotion recognition library. If the corners of the mouth point downwards then the system assumes that person is sad and responds by suggesting something along the lines of looking at a virtual photograph album together. Relatives can select the images beforehand and store them in the app. The fact that Josy can be personalized is important to Braun and her team: “Our aim is to create a personalized solution that can create an emotional connection. Everyone likes it when something comes from the heart.” That's why the researchers also offer sewing instructions for making a cover for Josy, which relatives can use to customize the companion even more.

The focus is on the user perspective

The project is part of the High Performance Center for Mass Personalization (LZMP), where research is focused on a user-centric approach: “We've been engaging a lot with our potential users, always listening to what they're thinking and trying to understand their thought processes.” In the process, we discovered that older people are often worried that they might make mistakes or break something when using digital devices. “That's why we programmed Josy to be as simple as possible to use,” says Braun: “No one will be able to accidentally delete the Internet or buy anything by mistake.”

The Josy assistance system greets the user Gerda with a "Good morning". Large buttons with symbols enable easy handling.
Personal support: virtual assistance technology should also touch people on an emotional level.

A code has to be entered in order to access the device settings area, which relatives and friends can use to add photos or change the viewing settings, for example. Among other things, Braun and her team acquired these insights in a series of online workshops, the aim of which is to present Josy as an assistance solution to interested persons, but a lot of it is also about dialog.

We programmed Josy to be as simple as possible to use. No one will be able to accidentally delete the Internet or buy anything by mistake.

Franziska Braun

The workshops are aimed at a range of very different target groups including managers who are interested in innovative production processes, care facilities that want to learn more about the system, companies that are interested in the product, and, of course, people with elderly relatives.

Research benefits from public dialog

Braun makes a point of ensuring that the training seminars do not resemble a traditional classroom teaching session and is pursuing a community-based approach with the project: “Engaging in dialog with interested parties and the public is hugely important to us,” she explains. “We explicitly invite public participation and plan to incorporate the feedback into our research, which will ultimately benefit us and the project.”

Franziska Braun stands on a staircase and smiles at the camera.
Josy’s developer, Franziska Braun, is working on the development of virtual companions.

”For example, the people’s reactions during their initial contact with Josy are extremely revealing. Does a person say “wow!” or do they make some skeptical comment about privacy? One of the results that came out of the public dialog was that people view the hand-sewn covers as a very positive thing. All interested parties also liked the fact that Josy reacts when touched, exclaiming, for example: “Hey, quit tickling me.” Open desiderata include, for example, that Josy's facial feature recognition capabilities should be even better. Going forward, Braun and her team plan to continue offering workshops and seeking dialog with the public, whereby the focus will always be on people's wishes and needs.

High Performance Center for Mass Personalization

Scientists from nine University of Stuttgart Institutes and four Fraunhofer Institutes in Stuttgart are collaborating with future-oriented industrial partners to explore ways of manufacturing customized products for the mass market. The concept of “Mass Personalization” takes account of the individual needs of users right from the product development stage.

Editor: Bettina Wind

Franziska Brau, E-Mail, Phone: +49 711 970 2384 

To the top of the page