I

The main aim of this product is to have designed and produced a cross-sensory accessory and interface system that converts gesture and sign language to speech. This product is targeted toward hearing-impaired individuals and can ultimately be expanded to an audience of non-verbal individuals. The SLCG is proposed to break the barrier of communication between verbal and non-verbal people, allowing non-verbal individuals to communicate without the need for a translator.

The relevance to interaction design is significant, as the communication glove would need to be designed in a way that allows for intuitive and seamless communication between the user and others. The design should take into account the physical limitations of the user, such as limited mobility or sensitivity in their hands, as well as any sensory limitations, such as difficulty with touch or vibration.

Additionally, the design should consider the social and cultural context in which the glove will be used, including the variety of sign languages that exist and how they are used in different countries and communities. The interface should be user-friendly and accessible to all, regardless of their technical expertise or familiarity with technology.

Initial research and designs for the SLCG

II

We initially started off with 3 iterations of the SLCG in order to come up with ideas for the final prototype.

The first design features a half sleeve, containing all the sensors on the lower back of the hand and a sensor at the wrist as well. This allows for the fingers to move freely, ultimately allowing the sensors to pick up on the movement as well.

The second iteration features another half-sleeve design, also containing sensors at the back, however, there are sensors on the finger that are also picked up for added accuracy and functionality.

The final iteration combines both the first and second designs to compose a full glove, detecting the entire movement of the hands up until the fingertips, again for added functionality and comfort due to the fact it is one full cover.

A glove is an everyday piece of fashion, we wanted to provide not only a functional glove but one that is aesthetically pleasing as well to go with an everyday outfit.

The backhand sensor technology inspiring the SLCG is referenced here: https://www.youtube.com/watch?v=6BxMtLkZ_ks

Backhand sensors shown as well as on the wrists for inital drawings

First iteration of the glove - have sleeve design with backhand sensors and free movement of fingers allowed

Second iteration of the glove featuring the same half sleeve concept and added finger sensors for increased functionality

Third iteration forming a full glove with sensors along the fingers and fingertips for added functionality, accuracy and comfort.

III

The new design of SLCG should afford the following:

Intuitive and seamless communication between the user and others, with minimal barriers and delays in interpreting and conveying messages.
Compatibility with various sign languages used around the world, with the ability to adapt to local dialects and variations.
Comfort and ease of use for the user, with consideration of their physical limitations, such as hand size, dexterity, and sensitivity.
Flexibility and adaptability to different environments and contexts, such as outdoor or indoor use, noisy or quiet environments, and social or professional settings.

Aspects of visual, auditory, and haptic perception that can inform the design decisions include:

Visual: The design should make use of visual cues to convey information, such as sign language symbols and gestures, along with visual feedback to confirm message reception and comprehension.
Auditory: The glove would incorporate audio feedback, such as spoken words or sounds, to supplement visual cues and assist with message interpretation and comprehension.

We chose to go for these cross-sensory modalities due to:

Accessibility: Cross-sensory modalities can enhance the accessibility of sign language communication for individuals with different types and levels of sensory impairments. For example, individuals who are both deaf and blind may benefit from additional sensory cues beyond visual or tactile cues. Incorporating auditory or vibrotactile feedback in the glove can provide additional cues for such individuals to understand and produce sign language effectively.
Multimodality: Sign language is a multimodal language that combines hand gestures, facial expressions, body movements, and sometimes even vocalizations. By incorporating cross-sensory modalities, a sign language communication glove can provide a more holistic and comprehensive communication experience that closely resembles the richness and complexity of sign language as it is used in everyday communication.
Learning and Training: Cross-sensory modalities can be used to facilitate learning and training of sign language. For example, incorporating auditory feedback in the glove can help beginners to associate the signs they are making with corresponding spoken words, which can aid in language acquisition. Similarly, incorporating vibrotactile feedback can help users to learn the correct handshapes and movements for different signs.
Feedback and Correction: Cross-sensory modalities can provide real-time feedback and correction to users when they are using the sign language communication glove. For example, incorporating sensors that can detect handshapes, movements, and facial expressions can provide feedback on the accuracy and fluency of sign language production. This can help users to correct their signing errors and improve their signing skills.
Versatility: Cross-sensory modalities can enhance the versatility of a sign language communication glove by extending its functionalities beyond communication. For example, incorporating haptic feedback in the glove can enable users to receive notifications, alerts, or other types of information through touch, which can be particularly useful in situations where visual or auditory cues are not practical or safe, such as in noisy environments or during emergencies.

Gesture to speech allows a higher shared intentionality between people who don't know ASL and people who do, it bridges the gap between mutual knowledge needed to communicate with some who signs in ASL. This also lessens the need for spacial topological cues, in other words, you need to see ASL to interpret it. The SLCG will allow non-verbal individuals to communicate without needing others to see ASL ultimately making communication more effective. Temporal synchrony will have to be maintained.

To determine the success or improvement of the new design, user testing and feedback would be crucial. The design should be tested in various real-life scenarios and with users from different cultural and linguistic backgrounds to ensure its effectiveness and compatibility. Feedback from both users and experts in design and technology should be gathered to identify areas of improvement and ensure that the design meets the needs of the intended user group.

We decided to go for the third iteration. This not only provides full accessibility for the sensors to work but proves a full sleeve to match everyday clothing, making it suitable for everyday use. Materials we had in mind for this wood be wool or spandex. The reasons for this would be that both materials are stretchable, soft, breathable, flexible, and durable. We also need to remember to allow for conductivity.

Project