Youths invent sign language learning app for family, friends of those with hearing loss

SINGAPORE — After watching a number of television dramas and movies which featured individuals with hearing loss and how they interact with each other,  Ms Anastasia Patricks was inspired to learn more about sign language.

Ms Patricks, 24, who was searching for a "meaningful" final year project for her engineering course at Nanyang Technological University (NTU), decided to research sign language more with two of her peers, Richardson Qiu and Steven Rachman.

They soon realised that while there was a set way of communicating within the deaf community, that ability to communicate did not always extend to the families and friends of those who were deaf or had hearing loss. 

This was because learning sign language is not easy, and they found that many existing sign language apps are typically not interactive and do not allow users to practice in a conversational manner. 

This lead the trio, who recently graduated, to create Sign2Sign, a sign language learning app with interactive and bite-sized lesson structures similar to popular language learning app, Duolingo.

Their efforts landed landed them as the Asian regional winner for the Lifestyle category at at Microsoft's Imagine Cup 2022, an annual global competition for students to create inspiring and inclusive projects that began last September and culminated in May.

Microsoft said there were applications from “thousands of students” from over 160 countries for this year's competition.

The team won US$2,500 (S$3,499) in cash and a top up of US$2,500 in additional funding by Microsoft to further develop Sign2Sign.

Along with easy-to-follow lessons, the app also offers a 3D avatar that demonstrates signing movements from different angles to facilitate a user’s learning.

The team are also using a Artificial Intelligence recognition feature that can detect whether a user is signing correctly. 

To refine their product, they sought feedback from the Singapore Asssociation for the Deaf as well as a volunteer group under the NTU Welfare Services Club that regularly organises initiatives supporting the deaf community. 

“We received positive feedback on our approach of focusing our lessons on simple daily conversations as the lessons are simple enough for beginners to learn and the content is relevant for everyday interactions,” said Mr Rachman, adding that this made the learning process more accessible.

The team do not have a timeline yet on when the app will be available to the public.

KITCHENWARE FOR VISUALLY IMPAIRED HITS THE SHELVES

Another innovation by a youth to help individuals with disability is now in the market, four years after the design was picked as the Singapore National Winner at the 2018 James Dyson Award. 

Folks Kitchenware by designer Kevin Chiam was launched for sale in March by collectibles retailer Mighty Jaxx, with proceeds channelled to Guide Dogs Singapore, a non-profit organisation that helps people with vision loss

Like the NTU trio, 30-year-old Mr Chiam was also looking for a meaningful way to cap off his bachelors’ degree journey at the National University of Singapore.

Inspiration came when he watched how a chef with visual impairment Christine Ha was crowned champion in the popular cooking competition, MasterChef.

He found out that MasterChef Ha’s speediness in the kitchen was achieved through long and rigorous practice, which can be demoralising at the start.

He said that this “seeded the concept” of helping a person with visual disability more safely work in the kitchen.

Mr Chiam got feedback from people who experience vision loss to ensure that potential solutions were really suitable for them and not merely based on his assumptions

This was how he came to realise that one of his initial ideas of having a motorised blade to “keep users away from the knife as far as possible” was not feasible, as it did not offer control and flexibility a user needed.

He went through 50 or 60 different designs in about three months before arriving at a viable iteration of the products.

The current Folks Kitchenware collection comprises two items. First is a knife with a retractable guard at the blade that guides the user’s fingers and prevents them from getting accidentally cut.

Also in the set is a chopping board with modular arrangement of trays that facilitates transfer of ingredients without spillage.

Even after Mighty Jaxx approached him in late 2018 to manufacture the kitchenware line, Mr Chiam continued to refine the designs after receiving feedback from potential users

The process was challenging not only for Mr Chiam, but for Mighty Jaxx as it was the firm’s first foray into manufacturing kitchenware. 

Mr Chiam’s pursuing of further studies and the Covid-19 pandemic further extended the manufacturing timeline, but these did not deter Mighty Jaxx, which felt a particular affinity for Made in Singapore projects.

“When we see local stories and local creators, it reminds us of how we started as well. So we wanted to support fellow Singaporeans (in doing something meaningful),” said Mr Bryan Tan, chief strategic officer of Mighty Jaxx.

A total of 500 sets were manufactured, said Mr Tan, with half of them up for sale at US$99 and "selling fast". For each set that is sold, a set is donated to a beneficiary at Guide Dogs Singapore.

One user who benefited from the product is Ms Nuraziana Mohd Said, who has been suffering deteriorating vision loss since she was a child and is now left with less than 10 per cent vision.

She said she relied heavily on sense of touch and memory when cutting in the kitchen, and had to take more care when cutting bigger and harder items that may require more force.

“Now I am able to cut more confidently as well as faster. Furthermore, I am now able to cut the slices more evenly with the help of the knife,” said the 38-year-old, who lives with her husband who also has a vision impairment.

“In the future, I hope designer Kevin Chiam and Mighty Jaxx can collaborate again to come up with knives of different sizes and more kitchen wares that are safe and efficient for people with vision loss so we can cook more safely."