Huijuan Xu

Postdoc scholar
EECS, UC Berkeley
Email: huijuan@eecs.berkeley.edu
           hxu@bu.edu

CV Google Scholar















About Me

I am a postdoctoral scholar in the EECS department at UC Berkeley advised by Prof. Trevor Darrell. I received my PhD degree from the computer science department at Boston University in 2018 advised by Prof. Kate Saenko. I interned at Disney Research, Pittsburgh with Prof. Leonid Sigal. Before coming to US, I received a Master’s degree from Graduate University of Chinese Academy of Sciences in 2012, advised by Professor Hua Yu, and a Bachelor’s degree in computer science from Hefei University of Technology in 2009.

My research focuses on deep learning, computer vision and natural language processing, particularly in the area of action understanding in video. The goal of my research is building robust action understanding algorithms with human level structural knowledge and efficient multi-modal supervision. To achieve this goal, my research touches the following two aspects: compositional action recognition with knowledge reasoning, and multi-modal action detection with less supervision.

I am looking for tenure-track assistant professor in computer science or computer engineering.



News



Selected Publications




Auxiliary Task Reweighting for Minimum-data Learning

Baifeng Shi, Judy Hoffman, Kate Saenko, Trevor Darrell, Huijuan Xu.
Neural Information Processing Systems (NeurIPS), 2020.
Paper Code Project page





Weakly-Supervised Action Localization with Expectation-Maximization Multi-Instance Learning

Zhekun Luo, Devin Guillory, Baifeng Shi, Wei Ke, Fang Wan, Trevor Darrell, Huijuan Xu
European Conference on Computer Vision (ECCV), 2020.
Paper Code Demo1 Demo2





Learning Canonical Representations for Scene Graph to Image Generation.

Roei Herzig*, Amir Bar*, Huijuan Xu, Gal Chechik, Trevor Darrell, Amir Globerson
European Conference on Computer Vision (ECCV), 2020.
Paper Code Project page





Something-Else: Compositional Action Recognition with Spatial-Temporal Interaction Networks

Joanna Materzynska, Tete Xiao, Roei Herzig, Huijuan Xu†, Xiaolong Wang†, Trevor Darrell†.
Computer Vision and Pattern Recognition (CVPR), 2020.
Paper Dataset Project page





TwoStreamVAN: Improving Motion Modeling in Video Generation.

Ximeng Sun, Huijuan Xu, Kate Saenko
IEEE Winter Conference on Applications of Computer Vision (WACV), 2020.
Paper Code Demo





Spatio-Temporal Action Graph Networks.

Roei Herzig*, Elad Levi*, Huijuan Xu*, Hang Gao, Eli Brosh, Xiaolong Wang, Amir Globerson, Trevor Darrell
International Conference on Computer Vision Workshop (ICCVW), 2019.
Paper Code





Learning Instance Activation Maps for Weakly Supervised Instance Segmentation.

Yi Zhu, Yanzhao Zhou, Huijuan Xu, Qixiang Ye, David Doermann, Jianbin Jiao
Computer Vision and Pattern Recognition (CVPR), 2019.
Paper Project page





Multilevel Language and Vision Integration for Text-to-Clip Retrieval.

Huijuan Xu, Kun He, Bryan A. Plummer, Leonid Sigal, Stan Sclaroff, Kate Saenko.
Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19), 2019.
Paper Code Demo1 Demo2





Joint Event Detection and Description in Continuous Video Streams.

Huijuan Xu, Boyang Li, Vasili Ramanishka, Leonid Sigal, Kate Saenko.
IEEE Winter Conference on Applications of Computer Vision (WACV), 2019.
Paper Code Demo1 Demo2





Two-Stream Region Convolutional 3D Network for Temporal Activity Detection.

Huijuan Xu, Abir Das, Kate Saenko.
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2019.
Paper Code





R-C3D: Region Convolutional 3D Network for Temporal Activity Detection.

Huijuan Xu, Abir Das, Kate Saenko.
International Conference on Computer Vision (ICCV), 2017.
Paper Code Demo Project page





Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering.

Huijuan Xu, Kate Saenko.
European Conference on Computer Vision (ECCV), 2016.
Paper Code Video spotlight



Translating Videos to Natural Language Using Deep Recurrent Neural Networks.

Subhashini Venugopalan, Huijuan Xu, Jeff Donahue, Marcus Rohrbach, Raymond Mooney, Kate Saenko.
North American Chapter of the Association for Computational Linguistics (NAACL), 2015.
Paper Project page






Workshop Papers

Huijuan Xu and Kate Saenko. Dual Attention Network for Visual Question Answering. ECCV2016 2nd Workshop on Storytelling with Images and Videos (VisStory), 2016.
Huijuan Xu and Kate Saenko. Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering. VQA Challenge Workshop at CVPR 2016.
Huijuan Xu, Subhashini Venugopalan, Vasili Ramanishka, Marcus Rohrbach and Kate Saenko. A Multi-scale Multiple Instance Video Description Network. ICCV15 workshop on Closing the Loop Between Vision and Language (CLVL), 2015. (Abstract)


Teaching

Teaching Assistant, UMass Lowell(Spring 2015)
91.422/545: Machine Learning

Lab Instructor, UMass Lowell (Fall 2013, Spring 2014)
91.103: Computing I Lab


Research Internship

Disney Research, Pittsburgh, Summer 2017


Professional Activities

Reviewer for journals:

Transactions on Pattern Analysis and Machine Intelligence (TPAMI), International Journal of Computer Vision (IJCV), ACM Transactions on Multimedia Activities Computing Communications and Applications (TOMM), IEEE Transactions on Image Processing (TIP)


Reviewer/program committee member for conferences:

ICCV2017, CVPR2018, WiCV2018, ECCV2018, ACCV2018, CVPR2019, NAACL-HLT2019, IJCAI2019, ICCV2019, WiCV2019, AAAI2020, CVPR2020, HADCV'20, ECCV2020, IJCAI2020, ICML2020, CVPR2021