Welcome! I am a Principal Research Scientist heading the Creative Vision team at Snap Research. My work focuses on creating methods for manipulating the world via computer vision and machine learning. This includes 2D and 3D methods for photorealistic object manipulation and animation, video synthesis, prediction and retargeting. My work has been published as 30+ top conference papers, journals and patents resulting in multiple tech transfers, including Snapchat Pet Tracking and Real-time Neural Lenses (gender swap, baby face, real-time try-on and many others).
Before joining Snap Inc., I built the real-time face tracking and reconstruction engine behind facify.me. 2016, I was a research intern at Microsoft Research, Cambridge, UK, and worked with Sebastian Nowozin and Andrew Fitzgibbon on improving generative models. In 2017, I interned at NVIDIA with Ming-Yu Liu, Xiaodong Yang and Jan Kautz and worked on video generation.
I am a PhD graduate from the University of Trento, Italy, where I worked with Nicu Sebe at the Multimedia and Human Understanding Group MHUG. Before joining the MHUG group, I worked as a software engineer for six years.
I am looking for prospective interns and collaborators. If you are interested send me an email.
Media coverage:
Contact: stulyakov@snap.com, sergey@stulyakov.com
The ultimate goal of the Creative Vision group is to develop cutting edge computer vision, machine learning, graphics and natural language technology to unlock creativity of users and creators in a variety of products at Snap. Our work enables machines to see, understand and manipulate the world.
We focus on foundational models for image, video, 3D, 4D synthesis, animation, manipulation and foundational technology to make these models small, fast, accurate and personalized. We uses images, audio, videos, text, sketches as inputs and outputs.
I'm always looking for motivated and driven researchers! Feel free to reach out to me to know more.
July 2017 - Present:
Principal Research Scientist at Snap Research, Santa Monica, CA
Jan 2017 - Apr 2017:
Research Intern at NVIDIA Research, Santa Clara, CA
Aug 2016 - Nov 2016:
Research Intern at Microsoft Research, Cambridge, UK
July 2010 - Sept 2012:
Team and Project Lead at HiQo-Solutions, Minsk, Belarus
June 2006 - June 2010:
Software Engineer at Todes Ltd, Minsk, Belarus
Nov 2012 - Apr 2017:
PhD Canditate at The Univesity of Trento, Italy
Sept 2014 - Feb 2015:
Research intern at the Robotics Institute, Carnegie-Mellon University
Sept 2009 - July 2010:
MSc at the Belarusian State University of Intormatics and Radioelectronics
Sept 2004 - July 2009:
B.Eng at the Belarusian State University of Intormatics and Radioelectronics
University of Trento, Italy. November 19-26, 2019
June 20, CVPR Virtual 2021
October 20, ECCV 2022, Tel-Aviv, Israel
Unsupervised Volumetric Animation
Aliaksandr Siarohin, Willi Menapace, Ivan Skorokhodov, Kyle Olszewski, Jian Ren, Hsin-Ying Lee, Menglei Chai, Sergey Tulyakov
Computer Vision and Pattern Recognition, 2023
Project page Paper Code
Affection: Learning Affective Explanations for Real-World Visual Data
Panos Achlioptas, Maks Ovsjanikov, Leonidas Guibas, Sergey Tulyakov
Computer Vision and Pattern Recognition, 2023
Project page Paper
Make-A-Story: Visual Memory Conditioned Consistent Story Generation
Tanzila Rahman, Hsin-Ying Lee, Jian Ren, Sergey Tulyakov, Shweta Mahajan, Leonid Sigal
Computer Vision and Pattern Recognition, 2023
Paper
Real-Time Neural Light Field on Mobile Devices
Junli Cao, Huan Wang, Pavlo Chemerys, Vladislav Shakhrai, Ju Hu, Yun Fu, Denys Makoviichuk, Sergey Tulyakov, Jian Ren
Computer Vision and Pattern Recognition, 2023
Project page Paper Code
3DAvatarGAN: Bridging Domains for Personalized Editable Avatars
Rameen Abdal, Hsin-Ying Lee, Peihao Zhu, Menglei Chai, Aliaksandr Siarohin, Peter Wonka, Sergey Tulyakov
Computer Vision and Pattern Recognition, 2023
Project page Paper Code (Coming Soon)
SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation
Yen-Chi Cheng, Hsin-Ying Lee, Sergey Tulyakov, Alex Schwing, Liangyan Gui
Computer Vision and Pattern Recognition, 2023
Project page Paper Code
Invertible Neural Skinning
Yash Kant, Aliaksandr Siarohin, Riza Alp Guler, Menglei Chai, Jian Ren, Sergey Tulyakov, Igor Gilitschenski
Computer Vision and Pattern Recognition, 2023
Project page Paper Code (Coming Soon)
DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-aware Scene Synthesis
Yinghao Xu, Menglei Chai, Zifan Shi, Sida Peng, Ivan Skorokhodov, Aliaksandr Siarohin, Ceyuan Yang, Yujun Shen, Hsin-Ying Lee, Bolei Zhou, Sergey Tulyakov
Computer Vision and Pattern Recognition, 2023
Project page Paper Code
3D Generation on ImageNet
Ivan Skorokhodov, Aliaksandr Siarohin, Yinghao Xu, Jian Ren, Hsin-Ying Lee, Peter Wonka, Sergey Tulyakov
International Conference on Learning Representations, 2023
Top 5%, Oral Presentation
Project page Paper Code
Discrete Contrastive Diffusion for Cross-Modal Music and Image Generation
Ye Zhu, Yu Wu, Kyle Olszewski, Jian Ren, Sergey Tulyakov, Yan Yan
International Conference on Learning Representations, 2023
Project page Paper Code
EpiGRAF: Rethinking training of 3D GANs
Ivan Skorokhodov, Sergey Tulyakov, Yiqun Wang, Peter Wonka
Neural Information Processing Systems, 2022
Project page Paper Code
R2L: Distilling Neural Radiance Field to Neural Light Field for Efficient Novel View Synthesis
Huan Wang, Jian Ren, Zeng Huang, Menglei Chai, Kyle Olszewski, Yun Fu, Sergey Tulyakov
European Conference on Computer Vision, 2022
Project page Paper Code
Cross-Modal 3D Shape Generation and Manipulation
Zezhou Cheng, Menglei Chai, Jian Ren, Hsin-Ying Lee, Kyle Olszewski, Zeng Huang, Subhransu Maji, Sergey Tulyakov
European Conference on Computer Vision, 2022
Project page Paper Code
NeROIC: Neural Rendering of Objects from Online Image Collections
Zhengfei Kuang, Kyle Olszewski, Menglei Chai, Zeng Huang, Panos Achlioptas, Sergey Tulyakov
ACM Transactions on Graphics (SIGGRAPH), 2022
Project page Paper Code
Playable Environments: Video Manipulation in Space and Time
Willi Menapace, Stéphane Lathuilière, Aliaksandr Siarohin, Christian Theobalt, Sergey Tulyakov, Vladislav Golyanik, Elisa Ricci
Computer Vision and Pattern Recognition, 2022
Project page Paper Code
Show Me What and Tell Me How: Video Synthesis via Multimodal Conditioning
Ligong Han, Jian Ren, Hsin-Ying Lee, Francesco Barbieri, Kyle Olszewski, Shervin Minaee, Dimitris Metaxas, Sergey Tulyakov
Computer Vision and Pattern Recognition, 2022
Project page Paper Code
StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2
Ivan Skorokhodov, Sergey Tulyakov, Mohamed Elhoseiny
Computer Vision and Pattern Recognition, 2022
Project page Paper
In&Out: Diverse Image Outpainting via GAN Inversion
Yen-Chi Cheng, Chieh Hubert Lin, Hsin-Ying Lee, Jian Ren, Sergey Tulyakov, Ming-Hsuan Yang
Computer Vision and Pattern Recognition, 2022
Project page Paper Code
F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantization
Qing Jin, Jian Ren, Richard Zhuang, Sumant Hanumante, Zhengang Li, Zhiyu Chen, Yanzhi Wang, Kaiyuan Yang, Sergey Tulyakov
International Conference on Learning Representations, 2022
Oral Presentation
Openreview Paper Code
InfinityGAN: Towards Infinite-Resolution Image Synthesis
Chieh Hubert Lin, Hsin-Ying Lee, Yen-Chi Cheng, Sergey Tulyakov, Ming-Hsuan Yang
International Conference on Learning Representations, 2022
Project page Paper Code
Flow Guided Transformable Bottleneck Networks for Motion Retargeting
Jian Ren, Menglei Chai, Oliver Woodford, Kyle Olszewski, Sergey Tulyakov
Computer Vision and Pattern Recognition, 2021
Paper
Motion Representations for Articulated Animation
Aliaksandr Siarohin, Oliver Woodford, Jian Ren, Menglei Chai, Sergey Tulyakov
Computer Vision and Pattern Recognition, 2021
Project page Paper Code
Playable Video Generation
Willi Menapace, Stéphane Lathuilière, Sergey Tulyakov, Aliaksandr Siarohin, Elisa Ricci
Computer Vision and Pattern Recognition, 2021
Oral Presentation
Project page Paper Code Demo
A Good Image Generator Is What You Need for High-Resolution Video Synthesis
Yu Tian, Jian Ren, Menglei Chai, Kyle Olszewski, Xi Peng, Dimitris N. Metaxas, Sergey Tulyakov
International Conference on Learning Representations, 2021
Spotlight Presentation
Project page Paper Code
Interactive video stylization using few-shot patch-based training Best in Show Award at SIGGRAPH Real-Time Live!
ACM Transactions on Grahpics (SIGGRAPH), 2020
Paper Code Project page
Neural Hair Rendering
Menglei Chai, Jian Ren, Sergey Tulyakov
European Conference on Computer Vision, 2020
PaperTowards Photo-Realistic Facial Expression Manipulation
Zhenglin Geng, Chen Cao, Sergey Tulyakov
International Journal of Computer Vision, 2020
Paper
Human Motion Transfer from Poses in the Wild
Jian Ren, Menglei Chai, Sergey Tulyakov, Chen Fang, Xiaohui Shen, Jianchao Yang
Image Manipulation Workshop, Computer Vision and Pattern Recognition, 2020
Paper
Laplace Landmark Localization
Joseph P Robinson, Yuncheng Li, Ning Zhang, Yun Fu, and Sergey Tulyakov
International Conference on Computer Vision, 2019
Paper
3D Guided Fine-Grained Face Manipulation
Zhenglin Geng, Chen Cao, Sergey Tulyakov
Computer Vision and Pattern Recognition, 2019
Paper
Train One Get One Free: Partially Supervised Neural Network for Bug Report Duplicate Detection and Clustering
Lahari Poddar, Leonardo Neves, William Brendel, Luis Marujo, Sergey Tulyakov, Pradeep Karuturi
North American Chapter of the Association for Computational Linguistics, 2019
Paper
Recurrent Convoltutional Shape Regression
Wei Wang, Sergey Tulyakov and Nicu Sebe
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
Paper
Hybrid VAE: Improving Deep Generative Models using Partial Observations
Sergey Tulyakov, Andrew Fitzgibbon, Sebastian Nowozin
Neural Information Processing Systems Workshops, 2017
Paper
Viewpoint-consistent 3D Face Alignment
Sergey Tulyakov, Laszlo A. Jeni, Jeffrey F. Cohn and Nicu Sebe
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
Paper
Recurrent Convolutional Face Alignment
Wei Wang, Sergey Tulyakov and Nicu Sebe
Asian Conference on Computer Vision, 2016
Oral Presentation
PaperSergey Tulyakov, Xavier Alameda Pineda, Elisa Ricci, Jijun Yin, Jeffrey Cohn and Nicu Sebe
Computer Vision and Pattern Recognition, 2016
Oral Presentation
Read moreRegressing a 3D Face Shape from a Single Image
Sergey Tulyakov and Nicu Sebe
International Conference on Computer Vision, 2015
Read moreFaceCept3D: Real Time 3D Face Tracking and Analysis
Sergey Tulyakov, Radu Vieriu, Enver Sangineto and Nicu Sebe
International Conference on Computer Vision Workshops, 2015
Paper