I am a postdoctoral scholar in the EECS department at UC Berkeley advised by Prof. Trevor Darrell.
I received my PhD degree from the computer science department at Boston University in 2018 advised by Prof. Kate Saenko.
I interned at Disney Research, Pittsburgh with Prof. Leonid Sigal.
Before coming to US, I received a Master’s degree from Graduate University of Chinese Academy of Sciences in 2012, advised by Professor Hua Yu,
and a Bachelor’s degree in computer science from Hefei University of Technology in 2009.
My research focuses on deep learning, computer vision and natural language processing, particularly in the area of action understanding in video.
The goal of my research is building robust action understanding algorithms with human level structural knowledge and efficient multi-modal supervision.
To achieve this goal, my research touches the following two aspects: compositional action recognition with knowledge reasoning, and multi-modal action detection with less supervision.
I will join the CSE Department at Pennsylvania State University--University Park as
an assistant professor in Fall 2021.
Prospective students: We have openings for PhD students
and visiting scholars with background in computer vision, natural language processing, artificial intelligence, machine
learning, robotics or computer graphics. Interested candidates can send
your CV and transcripts to hkx5063@psu.edu
LoGAN: Latent Graph Co-Attention Network for Weakly-Supervised Video Moment Retrieval
Reuben Tan, Huijuan Xu, Kate Saenko, Bryan A. Plummer
IEEE Winter Conference on Applications of Computer Vision (WACV), 2021
Paper
Code
Auxiliary Task Reweighting for Minimum-data Learning
Baifeng Shi, Judy Hoffman, Kate Saenko, Trevor Darrell, Huijuan Xu
Neural Information Processing Systems (NeurIPS), 2020
Paper
Code
Project page
Weakly-Supervised Action Localization with Expectation-Maximization Multi-Instance Learning
Zhekun Luo, Devin Guillory, Baifeng Shi, Wei Ke, Fang Wan, Trevor Darrell, Huijuan Xu
European Conference on Computer Vision (ECCV), 2020
Paper
Code
Demo1
Demo2
Learning Canonical Representations for Scene Graph to Image Generation
Roei Herzig*, Amir Bar*, Huijuan Xu, Gal Chechik, Trevor Darrell, Amir Globerson
European Conference on Computer Vision (ECCV), 2020
Paper
Code
Project page
Something-Else: Compositional Action Recognition with Spatial-Temporal Interaction Networks
Joanna Materzynska, Tete Xiao, Roei Herzig, Huijuan Xu†, Xiaolong Wang†, Trevor Darrell†
Computer Vision and Pattern Recognition (CVPR), 2020
Paper
Dataset
Project page
TwoStreamVAN: Improving Motion Modeling in Video Generation
Ximeng Sun, Huijuan Xu, Kate Saenko
IEEE Winter Conference on Applications of Computer Vision (WACV), 2020
Paper
Code
Demo
Spatio-Temporal Action Graph Networks
Roei Herzig*, Elad Levi*, Huijuan Xu*, Hang Gao, Eli Brosh, Xiaolong Wang, Amir Globerson, Trevor Darrell
International Conference on Computer Vision Workshop (ICCVW), 2019
Paper
Code
Learning Instance Activation Maps for Weakly Supervised Instance Segmentation
Yi Zhu, Yanzhao Zhou, Huijuan Xu, Qixiang Ye, David Doermann, Jianbin Jiao
Computer Vision and Pattern Recognition (CVPR), 2019
Paper
Project page
Two-Stream Region Convolutional 3D Network for Temporal Activity Detection
Huijuan Xu, Abir Das, Kate Saenko
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2019
Paper
Code
Multilevel Language and Vision Integration for Text-to-Clip Retrieval
Huijuan Xu, Kun He, Bryan A. Plummer, Leonid Sigal, Stan Sclaroff, Kate Saenko
Thirty-Third AAAI Conference on Artificial Intelligence (AAAI), 2019
Paper
Code
Demo1
Demo2
Joint Event Detection and Description in Continuous Video Streams
Huijuan Xu, Boyang Li, Vasili Ramanishka, Leonid Sigal, Kate Saenko
IEEE Winter Conference on Applications of Computer Vision (WACV), 2019
Paper
Code
Demo1
Demo2
R-C3D: Region Convolutional 3D Network for Temporal Activity Detection
Huijuan Xu, Abir Das, Kate Saenko
International Conference on Computer Vision (ICCV), 2017
Paper
Code
Demo
Project page
Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering
Huijuan Xu, Kate Saenko
European Conference on Computer Vision (ECCV), 2016
Paper
Code
Video spotlight
Translating Videos to Natural Language Using Deep Recurrent Neural Networks
Subhashini Venugopalan, Huijuan Xu, Jeff Donahue, Marcus Rohrbach, Raymond Mooney, Kate Saenko
North American Chapter of the Association for Computational Linguistics (NAACL), 2015
Paper
Project page
Teaching Assistant (Spring 2015)
91.422/545: Machine Learning
Lab Instructor (Fall 2013, Spring 2014)
91.103: Computing I Lab
Disney Research, Pittsburgh, Summer 2017
Reviewer for journals:
Transactions on Pattern Analysis and Machine Intelligence (TPAMI),
International Journal of Computer Vision (IJCV),
ACM Transactions on Multimedia Activities Computing Communications and Applications (TOMM),
IEEE Transactions on Image Processing (TIP),
Pattern Recognition (PR),
IEEE/ASME Transactions on Mechatronics,
IEEE Transactions on Multimedia
Reviewer/program committee member for conferences:
ICCV2017, CVPR2018, ECCV2018, ACCV2018, CVPR2019,
NAACL-HLT2019, IJCAI2019, ICCV2019, AAAI2020, CVPR2020,
ECCV2020, IJCAI2020, ICML2020, CVPR2021, EMNLP2021, NeurIPS2021, ICLR2022
Reviewer/program committee member for workshops:
WiCV2018, WiCV2019, HADCV'20, DL-HAU2021