<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="viegas-etal-2023-including">
    <titleInfo>
        <title>Including Facial Expressions in Contextual Embeddings for Sign Language Generation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Carla</namePart>
        <namePart type="family">Viegas</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Mert</namePart>
        <namePart type="family">Inan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Lorna</namePart>
        <namePart type="family">Quandt</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Malihe</namePart>
        <namePart type="family">Alikhani</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Alexis</namePart>
            <namePart type="family">Palmer</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jose</namePart>
            <namePart type="family">Camacho-collados</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Toronto, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>State-of-the-art sign language generation frameworks lack expressivity and naturalness which is the result of only focusing manual signs, neglecting the affective, grammatical and semantic functions of facial expressions. The purpose of this work is to augment semantic representation of sign language through grounding facial expressions. We study the effect of modeling the relationship between text, gloss, and facial expressions on the performance of the sign generation systems. In particular, we propose a Dual Encoder Transformer able to generate manual signs as well as facial expressions by capturing the similarities and differences found in text and sign gloss annotation. We take into consideration the role of facial muscle activity to express intensities of manual signs by being the first to employ facial action units in sign language generation. We perform a series of experiments showing that our proposed model improves the quality of automatically generated sign language.</abstract>
    <identifier type="citekey">viegas-etal-2023-including</identifier>
    <identifier type="doi">10.18653/v1/2023.starsem-1.1</identifier>
    <location>
        <url>https://aclanthology.org/2023.starsem-1.1</url>
    </location>
    <part>
        <date>2023-07</date>
        <extent unit="page">
            <start>1</start>
            <end>10</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Including Facial Expressions in Contextual Embeddings for Sign Language Generation
%A Viegas, Carla
%A Inan, Mert
%A Quandt, Lorna
%A Alikhani, Malihe
%Y Palmer, Alexis
%Y Camacho-collados, Jose
%S Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023)
%D 2023
%8 July
%I Association for Computational Linguistics
%C Toronto, Canada
%F viegas-etal-2023-including
%X State-of-the-art sign language generation frameworks lack expressivity and naturalness which is the result of only focusing manual signs, neglecting the affective, grammatical and semantic functions of facial expressions. The purpose of this work is to augment semantic representation of sign language through grounding facial expressions. We study the effect of modeling the relationship between text, gloss, and facial expressions on the performance of the sign generation systems. In particular, we propose a Dual Encoder Transformer able to generate manual signs as well as facial expressions by capturing the similarities and differences found in text and sign gloss annotation. We take into consideration the role of facial muscle activity to express intensities of manual signs by being the first to employ facial action units in sign language generation. We perform a series of experiments showing that our proposed model improves the quality of automatically generated sign language.
%R 10.18653/v1/2023.starsem-1.1
%U https://aclanthology.org/2023.starsem-1.1
%U https://doi.org/10.18653/v1/2023.starsem-1.1
%P 1-10

Markdown (Informal)

[Including Facial Expressions in Contextual Embeddings for Sign Language Generation](https://aclanthology.org/2023.starsem-1.1) (Viegas et al., *SEM 2023)

• Including Facial Expressions in Contextual Embeddings for Sign Language Generation (Viegas et al., *SEM 2023)

ACL

• Carla Viegas, Mert Inan, Lorna Quandt, and Malihe Alikhani. 2023. Including Facial Expressions in Contextual Embeddings for Sign Language Generation. In Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023), pages 1–10, Toronto, Canada. Association for Computational Linguistics.