CV

This is my CV. You can also download it as a PDF.

CV Preview

Basics

Name Keyu He
Label Master in NLP@CMU SCS || NLP Researcher
Email keyuhe@cmu.edu
Phone (213) 713-2973
Url https://keyu-he.github.io/
Summary Master of Intelligent Information Systems (MIIS) student at Carnegie Mellon University. Previously earned B.S. from USC with double major in Computer Science and Applied & Computational Mathematics. Passionate about socially intelligent AI systems and human-centered AI evaluation.

Education

  • 2025.08 - 2027.05

    Pittsburgh, PA

    Master's
    Carnegie Mellon University
    Master of Intelligent Information Systems (MIIS)
    GPA: 4.14/4.33
    • Advanced Natural Language Processing (A+)
    • Multimodal Machine Learning (A+)
    • Prompt Engineering (A+)
    • Introduction to Question Answering (A)
    • Independent Study in Language Technologies (A)
    • Directed Study in Language Technologies (A)
  • 2021.08 - 2025.05

    Los Angeles, CA

    Bachelor's
    University of Southern California
    Computer Science and Applied & Computational Mathematics
    GPA: 3.98/4.00
    Minor in Artificial Intelligence Applications. Specializations in Applied Analytics and Video Game Programming (USC Viterbi ITP). Dean's List, USC Dornsife and USC Viterbi (2021-2025).
    • Language Models in Natural Language Processing (A)
    • Applied Machine Learning for Natural Language Processing (A)
    • Applied Neural Networks (A)
    • Capstone: Design and Construction of Large Software Systems (A)
    • Mathematical Statistics (A)
    • Probability Theory (A)
    • Numerical Methods (A)

Publications

Research Experience

  • 2026 - Present
    Social RLVR
    Carnegie Mellon University; ongoing follow-up to Social Gym with Prof. Maarten Sap
    • Developing multi-agent social scenarios with rule-computed outcomes for reinforcement learning with verifiable rewards (RLVR).
  • 2026 - Present
    Social Gym and SPaRTan
    Carnegie Mellon University; with Prof. Maarten Sap
    • Built Social Gym, a Python, game-agnostic N-agent environment with finite-state game logic, partial observability, and rule-defined rewards across 21 games, enabling reproducible multi-turn evaluation without LLM judges.
    • Ran role-balanced tournaments for seven closed- and open-weight LLMs across 17 competitive and four cooperative games, covering 560+ episodes per model; fit Bradley–Terry/Elo ratings and role-conditioned capability profiles.
    • Designed SPaRTan, a training-free play–reflect–transfer loop and evaluated within-game iteration, cross-game and multigame transfer, and distillation into six student models.
  • 2025 - 2026
    RECAP
    Carnegie Mellon University; with Prof. Tongshuang Wu
    • Built an open-source VS Code instrumentation and replay platform in TypeScript that captures Copilot chats and fine-grained shadow-git edits, links AI suggestions to code, and supports extensible behavior analysis.
    • Deployed RECAP with a cloud data pipeline in a Spring 2026 CMU software engineering for machine learning course, capturing 2,034 prompts and 8,239 code edits from 41 students across 406 work sessions.
  • 2024 - 2024
    Attributing Culture-Conditioned Generations to Pretraining Corpora
    USC INK Lab; with Prof. Xiang Ren
    • Contributed large-scale corpus analysis, Python data engineering, and visualization to MEMOed, a framework for attributing culture-conditioned generations across 110 cultures.
    • Built Google APIs survey infrastructure and a routing service that directed participants to culture-specific forms for scalable data collection.
  • 2024 - 2025
    ELI-Why
    USC INK and DILL Labs; with Profs. Xiang Ren and Swabha Swayamdipta
    • Co-created a 13.4K-question benchmark for testing whether LLM explanations adapt to elementary, high-school, and graduate-level learners; led task, rubric, and prompt design and built the human-study web application.
    • Ran two human studies across 811 participants and multi-metric analyses, showing GPT-4 explanations matched their requested grade level 50% of the time versus 79% for human explanations.
  • 2024 - 2026
    Believing without Seeing
    USC DILL Lab; with Profs. Swabha Swayamdipta and Xiang Ren
    • Designed Visual Fidelity and Contrastiveness, two quality scores for VLM explanations, and evaluated their calibration against model correctness on A-OKVQA, VizWiz, and MMMU-Pro.
    • Designed and ran Prolific studies with 300 annotations per dataset and condition; the combined scores improved users' correctness judgments by 11.1 percentage points and reduced false belief in incorrect predictions by 15.4 points.

Work

  • 2026.05 - 2026.08

    San Jose, CA

    Machine Learning Engineer Intern
    Adobe
    • Shipped 5+ agent skills via an agent skill generator merged into the team's shared skill repo, turning rough developer requests into standards-compliant skills grounded in internal prior art and self-audited against the agentskills.io spec.
    • Developed the systematic quality measure for the team's agent recommendation pipeline, a reference-free 6-dimension rubric combining LLM-as-judge with deterministic checks, reproducible to 0.01–0.05 per-dimension variance across repeated runs and surfacing two systematic weaknesses for the team to act on.
    • Took that measure to production across the org: ported it onto the internal GenAI evaluation service as a reusable scorer and staged it end-to-end for two enterprise customers, on a four-stage Databricks pipeline built to run it.
  • 2025.05 - 2025.08

    Remote

    Data Engineer Intern
    CMind
    • Built a LangChain-based RAG prototype over an internal analytics DB (chunking, embeddings, FAISS indexing/retrieval, LLM answer synthesis) with metadata filters and notebook/API utilities.
    • Designed CEO-speech trait metrics and an auto-evaluation pipeline with custom rubrics, achieving Fleiss' κ of 0.89–0.91 against human annotations.

Teaching Experience

  • 2026.08 - Present

    Pittsburgh, PA

    Graduate Teaching Assistant, 11-711 Advanced Natural Language Processing
    Carnegie Mellon University, Language Technologies Institute
    • Supporting a graduate NLP research course covering modeling and learning algorithms and culminating in an original research project.
  • 2022.08 - 2025.05

    Los Angeles, CA

    Teaching & Grading Assistant
    University of Southern California
    • Coordinated logistics and grading for two computer science courses (CSCI-102 and CSCI-360), serving approximately 300 students per term.
    • Led weekly office hours and discussions for approximately 20 students, clarifying concepts and providing feedback on assignments.

Projects

  • 2026.02 - 2026.04
    VLM Hallucination Mitigation
    Training-free latent reflection for vision-language models, at Carnegie Mellon University.
    • Built a training-free latent reflection module that monitors decoding entropy and injects reflection steps when the model is uncertain.
    • Benchmarked against DAMO, VCD, OPERA, and DeGF on POPE/RePOPE/MMBench across LLaVA-1.5-7B, InstructBLIP-7B, and Qwen2-VL-7B.
    • On RePOPE, raised LLaVA-1.5-7B accuracy from 84.0% to 90.3% with the lowest calibration error (ECE 0.028) among all methods compared.
  • 2024.03 - 2024.04
    LLM Prompt Recovery
    Developed a system to recover user prompts using fine-tuned Mixtral models, achieving top 3.4% in a Kaggle competition.
    • Achieved a score of 0.65 using sentence-T5-base and sharpened cosine similarity.
    • Ranked 75/2175 globally in the Kaggle competition.
    • Published fine-tuned model on Kaggle for broader access.
  • 2024.11 - 2024.12
    AI-Based Career Advisor
    Built an AI tool to assist users in planning career paths based on skills and interests.
    • Developed a Streamlit-based interactive UI integrating GPT-4o for job suggestions.
    • Implemented cosine similarity search for skill-job matching.
    • Integrated Bing AI for real-time job application link retrieval.
  • 2023.08 - 2023.11
    Enhancing Debugging Skills of LLMs with Prompt Engineering
    Improved LLM debugging capabilities through advanced prompt engineering techniques.
    • Experimented with various prompting strategies to enhance debugging efficiency.
    • Achieved significant improvements in LLM performance on debugging tasks.
  • 2023.09 - 2023.12
    Automated Hate Speech Detection in Social Media
    Developed an advanced ML model for detecting hate speech, achieving 94% accuracy.
    • Fine-tuned BERT for classification tasks.
    • Enhanced online safety and inclusivity through robust model optimization.

Skills

Programming
C++
Python
Java
MySQL
HTML
CSS
JS
TS
x86-64 Assembly
Software
PyTorch
Pandas
NumPy
Git
AWS
LaTeX
Mathematica
Matlab
Areas of Expertise
Machine Learning
Natural Language Processing (NLP)
Large Language Models (LLMs)
Vision Language Models (VLMs)
Data Science / Data Engineering
Languages
Mandarin (native)
English (professional)

Awards