|
|
|
Sihyun Yu
sihyun.yu at kaist dot ac dot kr
yusihyunc at gmail dot com
Google Scholar |
Github |
X (Twitter) |
CV
I am a final-year Ph.D. student at KAIST, advised by Jinwoo Shin.
I also have interned at Google Research and NVIDIA Research.
My research mainly focuses on visual representation learning for efficient generative model training and inference,
and ultimately a representation applicable across all vision-related downstream tasks.
I am particularly interested in video generation, but any other complex modalities like 3D scenes/shapes are also in my interest.
I would love to have a chat with anyone who is interested in generative AI! Feel free to email me after checking my schedule.
|
News
Oct 2024: I gave a talk at BioImaging, Signal Processing & Learning Lab at KAIST.
Jun 2024: I am attending CVPR in person. Looking forward to seeing you in Seattle!
May 2024: I am attending ICLR in person. See you in Vienna!
May 2024: Hi-Mol is accepted to ICML 2024.
Feb 2024: We are organizing INRs for Vision Workshop at CVPR 2024! See more details
here.
Jan 2024: CMD is accepted to ICLR 2024.
Nov 2023: Two of my papers (PVDM and NVP) won Qualcomm Innovation Fellowship Korea 2023.
|
Publications
Selected
All
Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think
Sihyun Yu,
Sangkyung Kwak,
Huiwon Jang,
Jongheon Jeong,
Jonathan Huang,
Jinwoo Shin,
Saining Xie
preprint
paper |
project page |
code
MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation
Sihyun Yu,
Meera Hahn,
Dan Kondratyuk,
Jinwoo Shin,
Agrim Gupta,
José Lezama,
Irfan Essa,
David Ross,
Jonathan Huang
preprint
Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition
Sihyun Yu,
Weili Nie,
De-An Huang,
Boyi Li,
Jinwoo Shin,
Anima Anandkumar
ICLR 2024
paper |
project page |
code
Video Probabilistic Diffusion Models in Projected Latent Space
Sihyun Yu,
Kihyuk Sohn,
Subin Kim,
Jinwoo Shin
CVPR 2023
paper |
project page |
code
Scalable Neural Video Representations with Learnable Positional Features
Subin Kim*,
Sihyun Yu*,
Jaeho Lee,
Jinwoo Shin
NeurIPS 2022
paper |
project page |
code
Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks
Sihyun Yu*,
Jihoon Tack*,
Sangwoo Mo*,
Hyunsu Kim,
Junho Kim,
Jung-Woo Ha,
Jinwoo Shin
ICLR 2022
paper |
project page |
slide |
code
RoMA: Robust Model Adaptation for Offline Model-based Optimization
Sihyun Yu,
Sungsoo Ahn,
Le Song,
Jinwoo Shin
NeurIPS 2021
paper |
slide |
code
Abstract Reasoning via Logic-guided Generation
Sihyun Yu,
Sangwoo Mo,
Sungsoo Ahn,
Jinwoo Shin
ICML 2021 Workshop on Self-Supervised Learning for Reasoning and Perception (oral)
paper |
slide |
poster
Controllable Human Image Generation with Personalized Multi-Garments
Yisol Choi,
Sangkyung Kwak,
Sihyun Yu,
Hyungwon Choi,
Jinwoo Shin
preprint
paper |
project page |
code
Efficient Long Video Tokenization via Coordinate-based Patch Reconstruction
Huiwon Jang,
Sihyun Yu,
Jinwoo Shin,
Pieter Abbeel,
Younggyo Seo
preprint
paper |
project page |
code
Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think
Sihyun Yu,
Sangkyung Kwak,
Huiwon Jang,
Jongheon Jeong,
Jonathan Huang,
Jinwoo Shin,
Saining Xie
preprint
paper |
project page |
code
MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation
Sihyun Yu,
Meera Hahn,
Dan Kondratyuk,
Jinwoo Shin,
Agrim Gupta,
José Lezama,
Irfan Essa,
David Ross,
Jonathan Huang
preprint
MC2: Multi-view Consistent Depth Estimation via Coordinated Image-based Neural Rendering
Subin Kim,
Seong Hyeon Park,
Sihyun Yu,
Kihyuk Sohn,
Jinwoo Shin
CVPR 2024 Workshop on Neural Rendering Intelligence
paper |
project page
Data-Efficient Molecular Generation with Hierarchical Textual Inversion
Seojin Kim,
Jaehyun Nam,
Sihyun Yu,
Younghoon Shin,
Jinwoo Shin
ICML 2024
paper |
code
Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition
Sihyun Yu,
Weili Nie,
De-An Huang,
Boyi Li,
Jinwoo Shin,
Anima Anandkumar
ICLR 2024
paper |
project page |
code
Learning Large-scale Neural Fields via Context Pruned Meta-Learning
Jihoon Tack,
Subin Kim,
Sihyun Yu,
Jaeho Lee,
Jinwoo Shin,
Jonathan Richard Schwarz
NeurIPS 2023
paper |
code
Video Probabilistic Diffusion Models in Projected Latent Space
Sihyun Yu,
Kihyuk Sohn,
Subin Kim,
Jinwoo Shin
CVPR 2023
paper |
project page |
code
Enhancing Multiple Reliability Measures via Nuisance-extended Information Bottleneck
Jongheon Jeong,
Sihyun Yu,
Hankook Lee,
Jinwoo Shin
CVPR 2023
paper |
code
Scalable Neural Video Representations with Learnable Positional Features
Subin Kim*,
Sihyun Yu*,
Jaeho Lee,
Jinwoo Shin
NeurIPS 2022
paper |
project page |
code
Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks
Sihyun Yu*,
Jihoon Tack*,
Sangwoo Mo*,
Hyunsu Kim,
Junho Kim,
Jung-Woo Ha,
Jinwoo Shin
ICLR 2022
paper |
project page |
slide |
code
Consistency Regularization for Adversarial Robustness
Jihoon Tack,
Sihyun Yu,
Jongheon Jeong,
Minseon Kim,
Sung Ju Hwang,
Jinwoo Shin
AAAI 2022
paper |
slide |
code
RoMA: Robust Model Adaptation for Offline Model-based Optimization
Sihyun Yu,
Sungsoo Ahn,
Le Song,
Jinwoo Shin
NeurIPS 2021
paper |
slide |
code
Abstract Reasoning via Logic-guided Generation
Sihyun Yu,
Sangwoo Mo,
Sungsoo Ahn,
Jinwoo Shin
ICML 2021 Workshop on Self-Supervised Learning for Reasoning and Perception (oral)
paper |
slide |
poster
|
Work Experience
Google Research
Student Researcher (host: Jonathan Huang)
Oct 2023 - Apr 2024, Kirkland, WA
NVIDIA Research
Research Intern
(mentors: Weili Nie,
De-An Huang,
Boyi Li,
and Anima Anandkumar)
Mar 2023 - Sep 2023, Santa Clara, CA (remote)
Google Research
University Relation Program (host: Kihyuk Sohn)
Jul 2022 - Jan 2023, Bay Area, CA (remote)
|
Honors and Awards
Winner, Qualcomm Innovation Fellowship Korea 2023
Recipient, CVPR 2023 Travel Awards
Recipient, CVPR 2023 Google Conference Scholarships (APAC)
Top Reviewer, NeurIPS 2022
Best Paper Awards, Korean Artificial Intelligence Association, 2021
Recipient, KAIST Presidental Fellowship
Recipient, 2019 Qualcomm-KAIST Innovation Awards
Recipient, National Presidental Scholarship for Science
Recipient, Hansung Scholarship for Gifted Students
|
Invited Talks
Training Diffusion Transformers Is Easier Than You Think
Oct 2024; BioImaging, Signal Processing & Learning Lab @ KAIST (Seoul, South Korea)
Efficient Latent Video Diffusion Models via Triplane Encoding
Nov 2023; Google Research (Kirkland, WA)
Video Probabilistic Diffusion Models in Projected Latent Space
Jun 2023; Innerverz (remote)
Efficient Generative Models for Videos
Mar 2023; LG AI Research (Seoul, South Korea)
RoMA: Robust Model Adaptation for Offline Model-based Optimization
Jun 2022; Samsung Electronics (Suwon, South Korea)
Scaling Video Generation via Implicit Neural Representations
Jun 2022; Pohang University of Science and Technology (Pohang, South Korea)
Abstract Reasoning via Logic-guided Generation
Jun 2021; ICML Workshop on Self-Supervised Learning for Reasoning and Perception 2021 (remote)
|
Academic Services
Conference reviewer: ICML'22-24, NeurIPS'22-24, ICLR'23-25, CVPR'23-24, ICCV'23, ECCV'24, WACV'25, AAAI'25, AISTATS'25
Workshop organizer: INR-V@CVPR'24
Workshop reviewer: AI4CC@CVPR'22, Neural-Fields@ICLR'23
|
Teaching
Teaching Assistant, Deep Learning, Fall 2020
Teaching Assistant, Samsung Electronics AI-Expert Program, Summer 2020
Peer Tutor, System Programming, Spring 2019
Teaching Assistant, Linear Algebra, Spring 2019
Teaching Assistant, Calculus 1, Spring 2018
|
|
