MAScreen – make

MAScreen: Augmenting Speech with Visual Cues of Lip Motions, Facial Expressions, and Text Using a Wearable Display

Personal protective equipment, particularly face masks, have become increasingly common with the rise of global health issues, such as fine-dust storms and pandemics. Face masks, however, also degrade speech intelligibility by effectively occluding visual cues, such as lip motions and facial expressions. In this paper, we propose MAScreen, a wearable LED display in the shape of a mask, which is capable of sensing lip motion and speech and provides real-time visual feedback of the mouth behind the mask.

Reference

@inproceedings{10.1145/3415255.3422886,
  author = {Lee, Hyein and Kim, Yoonji and Bianchi, Andrea},
  title = {MAScreen: Augmenting Speech with Visual Cues of Lip Motions, Facial Expressions, and Text Using a Wearable Display},
  year = {2020},
  isbn = {9781450381109},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  url = {https://doi.org/10.1145/3415255.3422886},
  doi = {10.1145/3415255.3422886},
  booktitle = {SIGGRAPH Asia 2020 Emerging Technologies},
  articleno = {2},
  numpages = {2},
  keywords = {wearable display, speech augmentation, lip motion, Face mask},
  location = {Virtual Event, Republic of Korea},
  series = {SA '20}
}

Back to projects