Peer Reviewed Journal via three different mandatory reviewing processes, since 2006, and, from September 2020, a fourth mandatory peer-editing has been added.
Lip movement of speaker is very informative for many
application of speech signal processing such as multi-modal
speech recognition and password authentication without
speech signal. However, in collecting multi-modal speech
information, we need a video camera, large amount of
memory, video interface, and high speed processor to
extract lip movement in real time. Such a system tends to be
expensive and large. This is one reasons of preventing the
use of multi-modal speech processing.
In this study, we have developed a simple infrared lip
movement sensor mounted on a headset, and made it
possible to acquire lip movement by PDA, mobile phone,
and notebook PC. The sensor consists of an infrared LED
and an infrared photo transistor, and measures the lip
movement by the reflected light from the mouth region.
From experiment, we achieved 66% successfully word
recognition rate only by lip movement features. This
experimental result shows that our developed sensor can be
utilized as a tool for multi-modal speech processing by
combining a microphone mounted on the headset.
