Werewolf Among Us: Multimodal Resources for Modeling Persuasion Behaviors in Social Deduction Games

ACL Findings 2023

Bolin Lai*1, Hongxin Zhang*2, Miao Liu*3, Aryan Pariani*1, Fiona Ryan1, Wenqi Jia1, Shirley Anugrah Hayati4, James M. Rehg1, Diyi Yang5,
1Georgia Institute of Technology    2Shanghai Jiao Tong University    3Meta AI
4University of Minnesota    5Stanford University
Paper Code Dataset
We have released new follow-up work with more annotations. More details can be found here:
1. Modeling Multimodal Social Interactions (CVPR 2024, Oral)
2. Social Gesture (Under Review)


Abstract

Persuasion modeling is a key building block for conversational agents. Existing works in this direction are limited to analyzing textual dialogue corpora. We argue that visual signals also play an important role in understanding human persuasive behaviors. In this paper, we introduce the first multimodal dataset for modeling persuasion behaviors. Our dataset includes 199 dialogue transcriptions and videos captured in a multi-player social deduction game setting, 26,647 utterance level annotations of persuasion strategy, and game level annotations of deduction game outcomes. We provide extensive experiments to show how dialogue context and visual signals benefit persuasion strategy prediction. We also explore the generalization ability of language models for persuasion modeling and the role of persuasion strategies in predicting social deduction game outcomes.

Six Persuasion Strategies

Utterance-level persuasion strategy annotations. AUL refers to the average utterance length in terms of the number of words in an utterance and α refers to Krippendorff’s alpha.



Model Architecture

Experiments

Ablation Study of Context Length

Domain Generalization

We report the testing performance on the Ego4D dataset using models trained only on YouTube data (w.o. Fine-tuning), and trained on YouTube data and further fine-tuned with Ego4D data (w. Fine-tuning). We also report the performance (refer to Table 3) of the models trained only on Ego4D dataset (Ego4D Only) as comparison.



SVM Weights Visualization for Vote Prediction

The connection between a strategy and 0 means this strategy contributes to the prediction of 0 (i.e. the voter doesn’t vote for the candidate). Likewise, the connection between a strategy and 1 denotes this strategy contributes to the prediction of 1 (i.e. the voter votes for the candidate). The transparency of lines corresponds to the weights of logistic regression. A less transparent line suggests a greater weight and more impact on the output.

BibTeX

@inproceedings{lai2023werewolf,
      title={Werewolf among us: Multimodal resources for modeling persuasion behaviors in social deduction games},
      author={Lai, Bolin and Zhang, Hongxin and Liu, Miao and Pariani, Aryan and Ryan, Fiona and Jia, Wenqi and Hayati, Shirley Anugrah and Rehg, James and Yang, Diyi},
      booktitle={Findings of the Association for Computational Linguistics: ACL 2023},
      pages={6570--6588},
      year={2023}}