publications

2025

1. MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation

   Weihao Xuan, Rui Yang, Heli Qi, and 15 more authors

   2025
2. JiraiBench: A Bilingual Benchmark for Evaluating Large Language Models’ Detection of Human Self-Destructive Behavior Content in Jirai Community

   Yunze Xiao, Tingyu He, Lionel Z. Wang, and 5 more authors

   2025

2024

1. Chinese Offensive Language Detection:Current Status and Future Directions

   Yunze Xiao, Houda Bouamor, and Wajdi Zaghouani

   2024
2. ToxiCloakCN: Evaluating Robustness of Offensive Language Detection in Chinese with Cloaking Perturbations

   Yunze Xiao, Yujia Hu, Kenny Tsu Wei Choo, and 1 more author

   2024
3. Verbing Weirds Language (Models): Evaluation of English Zero-Derivation in Five LLMs

   David R. Mortensen, Valentina Izrailevitch, Yunze Xiao, and 2 more authors

   2024
4. InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviews

   Xintao Wang, Yunze Xiao, Jen-tse Huang, and 10 more authors

   In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Aug 2024

   Abs

   Role-playing agents (RPAs), powered by large language models, have emerged as a flourishing field of applications. However, a key challenge lies in assessing whether RPAs accurately reproduce the personas of target characters, namely their character fidelity. Existing methods mainly focus on the knowledge and linguistic patterns of characters. This paper, instead, introduces a novel perspective to evaluate the personality fidelity of RPAs with psychological scales. Overcoming drawbacks of previous self-report assessments on RPAs, we propose InCharacter, namely **In**terviewing **Character** agents for personality tests. Experiments include various types of RPAs and LLMs, covering 32 distinct characters on 14 widely used psychological scales. The results validate the effectiveness of InCharacter in measuring RPA personalities. Then, with InCharacter, we show that state-of-the-art RPAs exhibit personalities highly aligned with the human-perceived personalities of the characters, achieving an accuracy up to 80.7%.

5. Surveying Attitudinal Alignment Between Large Language Models Vs. Humans Towards 17 Sustainable Development Goals

   Qingyang Wu, Ying Xu, Tingsong Xiao, and 8 more authors

   Aug 2024

2023

1. Nexus at ArAIEval Shared Task: Fine-Tuning Arabic Language Models for Propaganda and Disinformation Detection

   Yunze Xiao, and Firoj Alam

   In Proceedings of ArabicNLP 2023, Dec 2023

2022

1. A Transformer-based Attention Flow Model for Intelligent Question and Answering Chatbot

   Yunze Xiao

   In 2022 14th International Conference on Computer Research and Development (ICCRD), Dec 2022