Hao Yu

Ph.D. student in Computer Science
Boston University
Email: haoyu [at] bu.edu
CV / Linkedin / Github / Google Scholar

Hi there! I am currently a PhD student in the Department of Computer Science at Boston University advised by Professor Margrit Betke. Prior to joining BU, I obtained my B.S. in Computer Science at Zhejiang University. 2019 Spring, I interned with Prof. Terence Sim at National University of Singapore. I have also worked with Prof. Yong Jae Lee as a summer research student at UC Davis in 2018.

My primary interests are in computer vision, human-computer interaction, and applications of machine learning, especially in facial image processing and analysis.

We're looking for participants to join our user study for the Camera Mouse research project. If you are interested, please fill out the google form. Your participation is greatly appreciated!

Teaching


  • Image and Video Computing (CS 585), Head Teaching Fellow, Boston University (2024 Spring)
  • Image and Video Computing (CS 585), Teaching Fellow, Boston University (2021 Spring, 2022 Spring)
  • Data Science Tools and Applications (CS 506), Teaching Fellow, Boston University (2024 Fall)
  • Machine Learning (CS 542), Teaching Fellow, Boston University (2023 Spring, 2023 Fall)
  • Intro to Computer Science (CS 112), Teaching Fellow, Boston University (2020 Spring, 2023 Summer)

    • Research Projects

    Affect Behavior Prediction: Using Transformers and Timing Information to Make Early Predictions of Student Exercise Outcome.
    In this work, we propose a novel approach for predicting the outcome of a student solving a mathematics problem in an intelligent tutor using early visual and tabular cues. Our approach analyzes only the first several seconds of a student’s problem-solving process captured in a video feed, along with timing information obtained from their learning log.

    COVES: A Cognitive-Affective Deep Model that Personalizes Math Problem Difficulty in Real Time and Improves Student Engagement with an Online Tutor.
    We propose a computer vision enhanced problem selector (COVES), a deep learning model to select a personalized difficulty level for each student. A combination of visual information and traditional log data is used to predict student-problem interactions, which are then used to guide problem difficulty selection in real time.

    Measuring and Integrating Facial Expressions and Gaze as Indicators of Engagement and Affect in Tutoring Systems
    In this work, we conducted two studies using computer vision techniques to measure students’ engagement and affective states from their head pose and facial expressions, as they use an online tutoring system, MathSpring.org.

    Leveraging Affect Transfer Learning for Behavior Prediction in an Intelligent Tutoring System
    We propose a video-based transfer learning approach for predicting problem outcomes of students working with an intelligent tutoring system (ITS). By analyzing a student’s face and gestures, our method predicts the outcome of a student answering a problem in an ITS from a video feed.

    The Effect of Television News Portrayals of Political Candidates
    This study employs state-of-the-art computer vision techniques to collect, process, and analyze a large-scale television video dataset about the six leading candidates of the 2020 Democratic Party presidential primaries in the United States. Combining a manual content analysis and deep learning methods, the study develops an automated facial expression recognition (FER) system that detects each candidate’s facial expression portrayed in television news.

    Hide-and-Seek: A Data Augmentation Technique for Weakly-Supervised Localization and Beyond
    We propose 'Hide-and-Seek', a general purpose data augmentation technique. The key idea is to hide patches in a training image randomly, in order to force the network to seek other relevant content when the most discriminative content is hidden. We perform extensive experiments to showcase the advantage of Hide-and-Seek on various visual recognition problems, such as image classification, semantic segmentation, and emotion recognition.

    Publications