Researchhighlights

Manling Li, Ruochen Xu, Shuohang Wang, Xudong Lin, Chenguang Zhu, Michael Zeng, Heng Ji, Shih-Fu Chang
- Open doors to introducing event semantic structures into vision-language understanding
- Bridges the gap between two modalities during V+L pretraining by optimizing structural alignment

Manling Li*, Alireza Zareian*, Qi Zeng, Spencer Whitehead, Di Lu, Heng Ji, Shih-Fu Chang
- Opens up a new research direction of event-centric multimedia information extraction
- Multimedia knowledge extraction is a new paradigm for information extraction, and has been well-noticed by the research community

Manling Li, Alireza Zareian*, Ying Lin, Xiaoman Pan, Spencer Whitehead, Brian Chen, Bo Wu, Heng Ji, Shih-Fu Chang, Clare R Voss, Dan Napierski, Marjorie Freedman
- The first open-source multimedia knowledge extraction system
- Used by various government agencies (e.g., ARL, DARPA, and IARPA)
- A top performer at the DARPA AIDA/NIST SM-KBP evaluation in each phase
- Received the ACL 2020 Best Demo Paper Award

Qingyun Wang, Manling Li, Xuan Wang, Nikolaus Parulian, Guangxing Han, Jiawei Ma, Jingxuan Tu, Ying Lin, Haoran Zhang, Weili Liu, Aabhas Chauhan, Yingjun Guan, Bangzheng Li, Ruisong Li, Xiangchen Song, Heng Ji, Jiawei Han, Shih-Fu Chang, James Pustejovsky, David Liem, Ahmed Elsayed, Martha Palmer, Jasmine Rah, Clare Voss, Cynthia Schneider, Boyan Onyshkevych
- A graph of genes, diseases, symptoms, chemicals, as well as their connections
- Awarded NAACL 2021 Best Demo Paper
- Generate a drug re-purposing report during collaborations with Prof David Liem from UCLA Data Science in Cardiovascular Medicine

Manling Li, Sha Li, Zhenhailong Wang, Lifu Huang, Kyunghyun Cho, Heng Ji, Jiawei Han
- Can input partially instantiated graphs to ``grow’’ the graph either forward or backward in time to predict missing events, arguments or relations, both from the past and in the future
- A different perspective, Models as Schemas, to output probabilistic schema models that can be probed on demand for event prediction
- A complicated graph including temporal dynamics and multiple paths involving entities that play important roles in a cohesive narrative - The first to capture event temporal dynamics probabilistically and perform global inference for event prediction

Manling Li, Qi Zeng, Ying Lin, Kyunghyun Cho, Heng Ji, Jonathan May, Nathanael Chambers, Clare Voss
- An auto-regressive language model trained on event-event paths, select paths showing how one event walks to another through entities
- Launchs a new step towards deep understanding

Zhenhailong Wang*, Manling Li*, Ruochen Xu, Luowei Zhou, Jie Lei, Xudong Lin, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Derek Hoiem, Shih-Fu Chang, Mohit Bansal, Heng Ji (* denotes equal contribution)
- Starts a new paradigm of leveraging langauge models to understand videos

Manling Li, Tengfei Ma, Mo Yu, Lingfei Wu, Tian Guo, Heng Ji, Kathleen McKeown
- Creates a new document-/corpus-level representation based on event graphs, and successfully supports zero-shot timeline summarization via graph compression.

Manling Li, Lingyu Zhang, Heng Ji, Rich Radke
- Leverages agenda-based topics to segment meeting transcripts
- The first to take advantage of multi-modal sensing of the meeting environment, such as cameras to capture each participant’s head pose and eye gaze.

Manling Li, Revanth Gangi Reddy, Ziqi Wang, Yi-Shyuan Chiang, Tuan M. Lai, Pengfei Yu, Zixuan Zhang, Heng Ji
- Provides claims with claimer and claimer epistemic status (refuting/supporting) for the first time, placing knowledge within a social context