Sergey Tulyakov

Welcome! I'm a Director of Research, leading the Creative Vision at Snap Inc. We build large generative models, make them efficient, and personalized.

Research

At Creative Vision, we strive to transform anyone into a creator. To achieve this, we focus on three core elements: skill, efficiency, and personalization. Our large generative models for images, videos, 3D, and 4D significantly boost skill. But that’s not enough! The creative process demands instantaneous feedback. To do so, we push efficiency to the edge, enabling our models to run on mobile phones at nearly real-time speeds while utilizing only a fraction of the size of larger models. Yet, this is still not enough. Each creator possesses a unique style and personality. Therefore, we not only build models that are efficient, but also make them personalized. Since the inception of our team, we have contributed substantially to products, used by hundreds of millions of Snapchatters everyday!

If our mission resonates with you, please send us an email. We are constantly in search of interns, collaborators, and researchers.

Experience

Jul 2024

Director of Research

Mar 2022

Principal Research Scientist, Senior Manager

Mar 2021

Principal Research Scientist, Manager

Dec 2018

Lead Research Scientist

Jul 2017

Joined Snap Inc. as Senior Research Scientist

Jan 2017 - Apr 2017

Research Intern at NVIDIA Research, Santa Clara, CA

Aug 2016 - Nov 2016

Research Intern at Microsoft Research, Cambridge, UK

Sept 2010 - Feb 2015

Research intern at the Robotics Institute, Carnegie-Mellon University

2012-2017

PhD at University of Trento, Italy

2010

MSC at Belorusian State University of Informatics and Radioelectronics

2009

B.Eng at Belorusian State University of Informatics and Radioelectronics

Community Service

I served as a technical program comittee member for all major computer vision, graphics and machine learning conferences: CVPR, ECCV, ICCV, NeurIPS, ICLR, SIGGRAPH, SIGGRAPH Asia, ICML. Since 2022 I serve as an AC for CVPR, ICML, NeurIPS, ICLR, WACV, 3DV, ECCV, ICCV. Since June 2024 I'm serving as an Associate Editor for TPAMI.

Our team organizes tutorials, teaches courses, and gives keynotes.

A week-long course on "Teaching Computers to Imagine with Deep Generative Models"

Universty of Trento'2019

A tutorial on "Unlocking Creativity with Computer Vision: Representations for Animation, Stylization and Manipulation"

CVPR'2020

A tutorial on "Video Synthesis: Early Days and New Developments"

ECCV'2022

A tutorial on "Efficient Neural Networks: From Algorithm Design to Practical Mobile Deployments"

CVPR'2023

Publications

This is an incomplete list. Please see my google scholar.

NeurIPS
SF-V: Single Forward Video Generation Model

Zhixing Zhang, Yanyu Li, Yushu Wu, Yanwu Xu, Anil Kag, Ivan Skorokhodov, Willi Menapace, Aliaksandr Siarohin, Junli Cao, Dimitris Metaxas, Sergey Tulyakov, Jian Ren

Neural Information Processing Systems, NeurIPS’2024

Project Paper

NeurIPS
BitsFusion: 1.99 bits Weight Quantization of Diffusion Model

Yang Sui, Yanyu Li, Anil Kag, Yerlan Idelbayev, Junli Cao, Ju Hu, Dhritiman Sagar, Bo Yuan, Sergey Tulyakov, Jian Ren

Neural Information Processing Systems, NeurIPS’2024

Project Paper

Pre-print
Lightweight Predictive 3D Gaussian Splats

Junli Cao, Vidit Goel, Chaoyang Wang, Anil Kag, Ju Hu, Sergei Korolev, Chenfanfu Jiang, Sergey Tulyakov, Jian Ren

Project Paper

Pre-print
Taming Data and Transformers for Audio Generation

Moayed Haji-Ali, Willi Menapace, Aliaksandr Siarohin, Guha Balakrishnan, Sergey Tulyakov, Vicente Ordonez

Project Paper

NeurIPS
4Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models

Heng Yu, Chaoyang Wang, Peiye Zhuang, Willi Menapace, Aliaksandr Siarohin, Junli Cao, Laszlo A Jeni, Sergey Tulyakov, Hsin-Ying Lee

Neural Information Processing Systems, NeurIPS’2024

Project Paper

Pre-print
GTR: Improving Large 3D Reconstruction Models through Geometry and Texture Refinement

Peiye Zhuang, Songfang Han, Chaoyang Wang, Aliaksandr Siarohin, Jiaxu Zou, Michael Vasilkovsky, Vladislav Shakhrai, Sergey Korolev, Sergey Tulyakov, Hsin-Ying Lee

Project Paper

SIGGRAPH Asia
MoA : Mixture-of-Attention for Subject-Context Disentanglement in Personalized Image Generation

Kuan-Chieh (Jackson) Wang, Daniil Ostashev, Yuwei Fang, Sergey Tulyakov, Kfir Aberman

Transactions on Graphics, SIGGRAPH Asia'2024

Project Paper

ECCV
TC4D: Trajectory-Conditioned Text-to-4D Generation

Sherwin Bahmani, Xian Liu, Yifan Wang, Ivan Skorokhodov, Victor Rong, Ziwei Liu, Xihui Liu, Jeong Joon Park, Sergey Tulyakov, Gordon Wetzstein, Andrea Tagliasacchi, David B. Lindell

European Conveference on Computer Vision, ECCV'2024

Project Paper

ECCV
UpFusion: Novel View Diffusion from Unposed Sparse View Observations

Bharath Raj Nagoor Kani, Hsin-Ying Lee, Sergey Tulyakov, Shubham Tulsiani

European Conveference on Computer Vision, ECCV'2024

Project Paper

ECCV
MyVLM: Personalizing VLMs for User-Specific Queries

Yuval Alaluf, Elad Richardson, Sergey Tulyakov, Kfir Aberman, Daniel Cohen-Or

European Conveference on Computer Vision, ECCV’2024

Project Paper

CVPR
Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers

Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang, Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, Sergey Tulyakov

Computer Vision and Patter Recognition, CVPR’2024

Project Paper

CVPR
Snap Video: Scaled Spatiotemporal Transformers for Text-to-video Synthesis

Willi Menapace, Aliaksandr Siarohin, Ivan Skorokhodov, Ekaterina Deyneka, Tsai-Shien Chen, Anil Kag, Yuwei Fang, Aleksei Stoliar, Elisa Ricci, Jian Ren, Sergey Tulyakov

Computer Vision and Patter Recognition, CVPR’2024 Highlight

Project Paper

CVPR
4D-fy: Text-to-4d Generation using Hybrid Score Distillation Sampling

Sherwin Bahmani, Ivan Skorokhodov, Victor Rong, Gordon Wetzstein, Leonidas Guibas, Peter Wonka, Sergey Tulyakov, Jeong Joon Park, Andrea Tagliasacchi, David B Lindell

Computer Vision and Patter Recognition, CVPR’2024

Project Paper

CVPR
SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priors

Dave Zhenyu Chen, Haoxuan Li, Hsin-Ying Lee, Sergey Tulyakov, Matthias Nießner

Computer Vision and Patter Recognition, CVPR’2024 Highlight

Project Paper

CVPR
Towards Text-guided 3D Scene Composition

Qihang Zhang, Chaoyang Wang, Aliaksandr Siarohin, Peiye Zhuang, Yinghao Xu, Ceyuan Yang, Dahua Lin, Bolei Zhou, Sergey Tulyakov, Hsin-Ying Lee

Computer Vision and Patter Recognition, CVPR’2024

Project Paper

CVPR
TextCraftor: Your Text Encoder Can Be Image Guality Controller

Yanyu Li, Xian Liu, Anil Kag, Ju Hu, Yerlan Idelbayev, Dhritiman Sagar, Yanzhi Wang, Sergey Tulyakov, Jian Ren

Computer Vision and Patter Recognition, CVPR’2024

Project Paper

CVPR
Hierarchical Patch Diffusion Models for High-Resolution Video Generation

Ivan Skorokhodov, Willi Menapace, Aliaksandr Siarohin, Sergey Tulyakov

Computer Vision and Patter Recognition, CVPR’2024

Project Paper

CVPR
SPAD: Spatially Aware Multi-View Diffusers

Yash Kant, Aliaksandr Siarohin, Ziyi Wu, Michael Vasilkovsky, Guocheng Qian, Jian Ren, Riza Alp Guler, Bernard Ghanem, Sergey Tulyakov, Igor Gilitschenski

Computer Vision and Patter Recognition, CVPR’2024

Project Paper

Transactions on Grahpics
Promptable Game Models: Text-guided Game Simulation via Masked Diffusion Models

Willi Menapace, Aliaksandr Siarohin, Stéphane Lathuilière, Panos Achlioptas, Vladislav Golyanik, Sergey Tulyakov, Elisa Ricci

Transactions on Grahpics, TOG’2024

Project Paper

ICLR
Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors

Guocheng Qian, Jinjie Mai, Abdullah Hamdi, Jian Ren, Aliaksandr Siarohin, Bing Li, Hsin-Ying Lee, Ivan Skorokhodov, Peter Wonka, Sergey Tulyakov, Bernard Ghanem

International Conference on Learning Representations, ICLR’2024

Project Paper

ICLR
HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusion

Xian Liu, Jian Ren, Aliaksandr Siarohin, Ivan Skorokhodov, Yanyu Li, Dahua Lin, Xihui Liu, Ziwei Liu, Sergey Tulyakov

International Conference on Learning Representations, ICLR’2024

Project Paper

SIGGRAPH Asia
Text-Guided Synthesis of Eulerian Cinemagraphs

Aniruddha Mahapatra, Aliaksandr Siarohin, Hsin-Ying Lee, Sergey Tulyakov, Jun-Yan Zhu

Transactions on Graphics, SIGGRAPH Asia'2023

Project Paper

NeurIPS - World's fastest mobile diffusion!
SnapFusion: Text-to-image Giffusion Model on Mobile Devices within Two Seconds

Yanyu Li, Huan Wang, Qing Jin, Ju Hu, Pavlo Chemerys, Yun Fu, Yanzhi Wang, Sergey Tulyakov, Jian Ren

Neural Information Processing Systems, NeurIPS’2023

Project Paper

NeurIPS
Autodecoding latent 3d diffusion models

Evangelos Ntavelis, Aliaksandr Siarohin, Kyle Olszewski, Chaoyang Wang, Luc Van Gool, Sergey Tulyakov

Neural Information Processing Systems, NeurIPS’2023

Project Paper

ICCV
Rethinking Vision Transformers for MobileNet Size and Speed

Yanyu Li, Ju Hu, Yang Wen, Georgios Evangelidis, Kamyar Salahi, Yanzhi Wang, Sergey Tulyakov, Jian Ren

International Conference on Computer Vision, ICCV’2023

Project Paper

ICCV
Text2Tex: Text-driven Texture Synthesis via Diffusion Models

Dave Zhenyu Chen, Yawar Siddiqui, Hsin-Ying Lee, Sergey Tulyakov, Matthias Nießner

International Conference on Computer Vision, ICCV’2023

Project Paper

ICCV
InfiniCity: Infinite-Scale City Synthesis

Chieh Hubert Lin, Hsin-Ying Lee, Willi Menapace, Menglei Chai, Aliaksandr Siarohin, Ming-Hsuan Yang, Sergey Tulyakov

International Conference on Computer Vision, ICCV’2023

Project Paper

CVPR
Unsupervised Volumetric Animation

Aliaksandr Siarohin, Willi Menapace, Ivan Skorokhodov, Kyle Olszewski, Jian Ren, Hsin-Ying Lee, Menglei Chai, Sergey Tulyakov

Computer Vision and Pattern Recognition, CVPR’2023

Project Paper

CVPR
Affection: Learning Affective Explanations for Real-World Visual Data

Panos Achlioptas, Maks Ovsjanikov, Leonidas Guibas, Sergey Tulyakov

Computer Vision and Pattern Recognition, CVPR’2023

Project Paper

CVPR
Real-Time Neural Light Field on Mobile Devices

Junli Cao, Huan Wang, Pavlo Chemerys, Vladislav Shakhrai, Ju Hu, Yun Fu, Denys Makoviichuk, Sergey Tulyakov, Jian Ren

Computer Vision and Pattern Recognition, CVPR’2023

Project Paper

CVPR
3DAvatarGAN: Bridging Domains for Personalized Editable Avatars

Rameen Abdal, Hsin-Ying Lee, Peihao Zhu, Menglei Chai, Aliaksandr Siarohin, Peter Wonka, Sergey Tulyakov

Computer Vision and Pattern Recognition, CVPR’2023

Project Paper

CVPR
SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation

Yen-Chi Cheng, Hsin-Ying Lee, Sergey Tulyakov, Alex Schwing, Liangyan Gui

Computer Vision and Pattern Recognition, CVPR’2023

Project Paper

CVPR Highlight
DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-aware Scene Synthesis

Yinghao Xu, Menglei Chai, Zifan Shi, Sida Peng, Ivan Skorokhodov, Aliaksandr Siarohin, Ceyuan Yang, Yujun Shen, Hsin-Ying Lee, Bolei Zhou, Sergey Tulyakov

Computer Vision and Pattern Recognition, CVPR’2023 Highlight

Project Paper

ICLR Oral
3D Generation on ImageNet

Ivan Skorokhodov, Aliaksandr Siarohin, Yinghao Xu, Jian Ren, Hsin-Ying Lee, Peter Wonka, Sergey Tulyakov

International Conference on Learning Representations, ICLR’2023 Oral

Project Paper

ICLR
Discrete Contrastive Diffusion for Cross-Modal Music and Image Generation

Ye Zhu, Yu Wu, Kyle Olszewski, Jian Ren, Sergey Tulyakov, Yan Yan

International Conference on Learning Representations, ICLR’2023

Project Paper

NeurIPS
EpiGRAF: Rethinking Training of 3D GANs

Ivan Skorokhodov, Sergey Tulyakov, Yiqun Wang, Peter Wonka

Neural Information Processing Systems, NeurIPS’2022

Project Paper

NeurIPS
EfficientFormer: Vision Transformers at MobileNet Speed

Yanyu Li, Geng Yuan, Yang Wen, Eric Hu, Georgios Evangelidis, Sergey Tulyakov, Yanzhi Wang, Jian Ren

Neural Information Processing Systems, NeurIPS’2022

Project Paper

ECCV
R2L: Distilling Neural Radiance Field to Neural Light Field for Efficient Novel View Synthesis

Huan Wang, Jian Ren, Zeng Huang, Menglei Chai, Kyle Olszewski, Yun Fu, Sergey Tulyakov

European Conference on Computer Vision, ECCV’2022

Project Paper

SIGGRAPH
NeROIC: Neural Rendering of Objects from Online Image Collections

Zhengfei Kuang, Kyle Olszewski, Menglei Chai, Zeng Huang, Panos Achlioptas, Sergey Tulyakov

Transactions on Graphics, SIGGRAPH’2022

Project Paper

CVPR
Playable Environments: Video Manipulation in Space and Time

Willi Menapace, Stéphane Lathuilière, Aliaksandr Siarohin, Christian Theobalt, Sergey Tulyakov, Vladislav Golyanik, Elisa Ricci

Computer Vision and Pattern Recognition, CVPR’2022

Project Paper

CVPR
Show Me What and Tell Me How: Video Synthesis via Multimodal Conditioning

Ligong Han, Jian Ren, Hsin-Ying Lee, Francesco Barbieri, Kyle Olszewski, Shervin Minaee, Dimitris Metaxas, Sergey Tulyakov

Computer Vision and Pattern Recognition, CVPR’2022

Project Paper

CVPR
StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2

Ivan Skorokhodov, Sergey Tulyakov, Mohamed Elhoseiny

Computer Vision and Pattern Recognition, CVPR’2022

Project Paper

CVPR
Motion Representations for Articulated Animation

Aliaksandr Siarohin, Oliver Woodford, Jian Ren, Menglei Chai, Sergey Tulyakov

Computer Vision and Pattern Recognition, CVPR’2021

Project Paper

CVPR Oral
Playable Video Generation

Willi Menapace, Stéphane Lathuilière, Sergey Tulyakov, Aliaksandr Siarohin, Elisa Ricci

Computer Vision and Pattern Recognition, CVPR’2021 Oral

Project Paper

ICLR Spotlight
A Good Image Generator Is What You Need for High-Resolution Video Synthesis

Yu Tian, Jian Ren, Menglei Chai, Kyle Olszewski, Xi Peng, Dimitris N. Metaxas, Sergey Tulyakov

International Conference on Learning Representations, ICLR’2021 Spotlight

Project Paper

NeurIPS
First Order Motion Model for Image Animation

Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, Nicu Sebe

Neural Information Processing Systems, NeurIPS’2019

Project Paper

CVPR
MoCoGAN: Decomposing Motion and Content for Video Generation

Sergey Tulyakov, Ming-Yu Liu, Xiaodong Yang, Jan Kautz

Computer Vision and Pattern Recognition, CVPR’2018

Project Paper

CVPR Oral
Self-Adaptive Matrix Completion for Heart Rate Estimation from Face Videos under Realistic Conditions

Sergey Tulyakov, Xavier Alameda Pineda, Elisa Ricci, Jijun Yin, Jeffrey Cohn, Nicu Sebe

Computer Vision and Pattern Recognition, CVPR’2016 Oral

Project Paper