Education

PhD - Massachusetts Institute of Technology

• Ph.D. in Media Arts and Sciences (Bioengineering), GPA 5.0/5.0
• Cambridge, MA, July 2021- Dec 2024
• Supervised by Dr. Ed Boyden and Dr. Joe Jacobson.
• Selected Courses: Advances in Computer Vision, Advanced Natural Language Processing, Dynamic Programming and Reinforcement Learning, Tissue vs Silicon in Machine Learning, Brain Algorithms

Master - Massachusetts Institute of Technology

• M.S. in Media Arts and Sciences (Bioengineering), GPA 4.85/5.0
• Cambridge, MA, Sept 2019 - June 2021
• Supervised by Dr. Ed Boyden and Dr. Joe Jacobson.
• Selected Courses: Machine Learning, Information and Inference, Computational Biology: Genomes, Networks, Evolution, Fields, Forces, and Flows in Biological systems, Nucleic Acids

Bachelor - Shanghai Jiao Tong University

• Bachelor in Life Science, Zhiyuan Honor Program, GPA 93/100 ( Rank 1/7 )
• Shanghai, China, Sept 2015 - July 2019
• Zhiyuan Honor Program selects top 10% students of different majors, featuring interdisciplinary training and small classroom teaching.
• Selected Courses: Mathematical Analysis, Linear Algebra, Physics, Physics Laboratory, Chemical Principle, Chemistry Laboratory, Biochemistry, Neurobiology, Genetics, Digital Signal Processing, Biostatiscs, Cell Biology, Biology Laboratory

Exchange student - St John’s College, University of Cambridge

• Lent term exchange student, sponsored by St John’s College
• Cambridge, UK, 2018.1-2018.3

Work Experience

Semantic Perception Team, XR, Google LLC Mountain View, CA

• Software Engineer Dec 2024 - Present
• Enhanced text readability of an AI conversational agent on Google Glass to match human visual acuity, enabling low-vision users to access text more easily; featured in the Astra Aira Visual Interpreter at Google I/O 2025.
• Designed and implemented a unified egocentric dialogue data synthesis pipeline to support for effective tool invocation during Gemini post-training, enabling advanced text-reading tasks.

AR Perception Team, XR, Google LLC. Mountain View, CA

• Software Engineer Intern May 2023 - Aug 2023
• Developed a system for 3D object generation conditioned on text prompts and user-provided sketches, leveraging diffusion model architecture with multimodal inputs. Trained an encoder to capture 3D object representations and employed neural radiance fields (NeRFs) to render high-resolution outputs. This work has strong applications in AR/VR content creation and digital asset generation.

Research Project

Mapping Spatial Transcriptome of an Entire Vertebrate Brain

• [PhD thesis] MIT, Brain and Cognitive Sciences
• Jan. 2020 - Dec. 2024
• Produced and analyzed 30 TB of in situ RNA sequencing imaging data that involves image registration, stitching, customized puncta extraction and point cloud registration in Python.
• Bioinformatic analysis of zebrafish single nuclei RNA sequencing dataset which identifies cell type markers for ~50 neuronal clusters and correlative studies with spatial transcript localization information.

Instance Segmentation of Unlabeled Modalities via Cyclic Segmentation GAN

• Feb. 2022 - Dec. 2022
• Supervised by Dr. Donglai Wei
• Provided data for benchmarking a novel Cyclic Segmentation Generative Adversarial Network (CySGAN) that conducts image translation and instance segmentation jointly using a unified framework. Our model achieves 3D neuronal nuclei segmentation of unlabeled expansion microscopy data with annotated electron microscopy images.

AlphaTracker: A Multi-Animal Tracking and Behavioral Analysis Tool

• [Undergrad thesis] MIT, Brain and Cognitive Sciences
• Apr. 2018-July. 2022
• Spearheaded applying cutting-edge human pose estimation algorithm AlphaPose on tracking multiple mice
• Automated the analysis of their social interaction and individual behavioral motif with unsupervised clustering.
• Supervised the development of a web-based UI for inspecting tracking and behavioral clustering results.

Cortical ensembles orchestrate social competition through hypothalamic outputs

• MIT, Brain and Cognitive Sciences
• Apr. 2018 - Feb. 2019
• [Undergrad thesis] Supervised by Dr. Nancy Padilla and Dr. Kay Tye
• Pioneered the discovery that the prefrontal-hypothalamic circuit regulates social dominance behavior with optogenetic approaches(involving stereotaxic brain surgeries), which leads to a major publication on Nature.
• Built a MATLAB GUI for scoring animal behavior with a video playback function.

The electrophysiological properties of maturing hippocampal neurons

• University of Cambridge, Dept. of Physiology, Development and Neuroscience, Cambridge, UK
• Jan. 2018 - Mar. 2018
• Supervised by Ole Paulsen, MD, PhD
• Studied the electrophisiology of hippocampal neurons of different cell types and developmental stages from both hemisphere with whole-cell patch clamping.
• Developed an algorithm in MATLAB to analyze neural electrophysiological properties from collected electrophysiological data.

Large-scale imaging and data processing of zebrafish brain with calcium sensor and voltage-sensitive dye

• Dept. of Biomedical Engineering, Duke University
• July. 2017 - Aug. 2017
• Supervised by Dr. Yiyang Gong
• Adapted the algorithm principle/independence component analysis and constrained non-negative matrix factorization to process the calcium and voltage imaging data on the larval zebrafish brain.

Functions of miRNA-7 and miR-17-92 in the arcuate nucleus in the obesity phenotype

• School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, China
• Feb. 2016 - Jan. 2018
• Supervised by Dr. Tao Sun
• Performed qPCRs on the predicted targets of certain microRNAs and validated the identified targets with immunohistology.
• Developed MATLAB scripts to automate and accelerate cell counting from fluorescent images acquired from confocal microscopy.

Course Project

Multi-View 6D Object Pose Refinement with Differentiable Rendering


• [6.s980 Machine Learning for Inverse Graphics] MIT, EECS
• Sept. 2022 - Dec. 2023
• Designed loss function for refining object pose prediction with gradient descent that widens the basin of convergence by 1.5 fold. The loss function combines single-view level feature descriptors like photometric loss, structural similarity loss and perceptual loss, and object level pose differences for bundle adjustment.

Towards photo-realistic face swapped videos

• [6.869 Computer Vision] MIT, EECS
• Feb. 2022 - May. 2022
• Implemented a face swapping pipeline to protect the identities of Asian woman interviewees. The project involved synthesizing artificial faces with styleGAN, applying face swapping with SimSwap, followed by various video super resolution methods to enhance the spatial resolution and generate photo-realistic face-swapped videos.

Solving Put Option Prices with Dynamic Programming

• [6.231 Dynamic programming and reinforcement learning] MIT, EECS
• Feb.2022 - May.2022
• Devised the approximate value iteration and approximate policy iteration algorithm to compute the optimal stopping policy for the American put option which shows significant improvement in performance than the vanilla algorithm. Compared the performance of different base functions and multi-layer perceptrons in approximating value / Q function in dynamical programming. Demonstrated its generalizability to option pricing for multi-stock indexes.

Affect Encoding in Word Embeddings

• [6.864 Natural Language Processing] MIT, EECS
• Sept.2021 - Nov.2021
• Investigated how different NLP models (eg. contextualized BERT) encode affect-related meaning and relate to human judgement with statistical tools (PCA, Spearman correlation, and logistic regression).
• Demonstrated a contextualized BERT model fine-tuned with affect-value regression task outperformed vanilla models in the transfer learning and the few-shot learning scheme.

Skills

• Research skills Stereotaxic surgery, Immunohistochemistry, Confocal imaging, qPCR, Patch clamping on cell culture, Optogenetic stimulation during behavioral assays, Biostatistics, Calcium/voltage imaging data analysis, Digital signal processing, GUI building for video scoring
• Coding Language: Python, MATLAB, C++
• Other skills: HTML, JavaScript; Adobe Illustrator, Adobe Photoshop, DaVinci, Final Cut Pro, Adobe Premiere, CapCut

Activities

• Organizer of the Generative AI for Visual Arts Workshop at CVPR 2025.
• Organizer of the third MIT AI Film Hack 2025.
• Organizer of the Generative AI for Visual Arts Workshop at CVPR 2024.
• Chief organizer of the second MIT AI for Filmmaking Hackathon, 2024
• Chief organizer of the first MIT AI for Filmmaking Hackathon, 2023
• Founder and President of the MIT Film Makers Association, 2022
• Director of a documentary about MIT student life Decode MIT, received more than 176,000 views on Bilibili, 2022.
• Director of a sci-fi film Your Poetic AlphaFriend, received more than 40,000 views, 2019.

Awards & Scholarships

• 2013 National Olympiad in Informatics in Province, First prize, Jiangsu Province
• 2014 China High School Biology Olympiad, First prize, Jiangsu Province
• 2014 Chinese Mathematical Olympiad in Senior, First prize, Jiangsu Province
• 2014 Chinese Chemistry Olympiad，Bronze medal, National Final I was the first student from my high school in the past 10 years to enter the National Final
• 2016, 2017 National Scholarship Awarded to students who are top 10% in GPA in Zhiyuan College, the rate national wide is 2/1000
• 2016, 2017 Zhiyuan Honor Scholarship
• 2016 - present Tanglixin Scholarship A very competitive scholarship sponsored by a Chinese entrepreneur Mr. Lixin Tang. Once awarded, the student will be sponsored annually until graduation.

Publication

• Zhang, R., Yu, B., Min, J., et al. (2025). Generative AI for film creation: A survey of recent advances. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 6267–6279).
• Zhang, R. (2025). Mapping the spatial transcriptome across whole organisms (Doctoral dissertation, Massachusetts Institute of Technology).
• Besier E., Zhang, R., et al, 2024. RnR-ExM: Robust Non-rigid Registration Challenge for Expansion Microscopy Volumes. IEEE Tranactions on Medical Imaging 2024.
• Chen, Z., Zhang, R., et. al., 2023. AlphaTracker: a multi-animal tracking and behavioral analysis tool. Frontiers in Behavioral Neuroscience, 17, p.1111908.
• Lauenburg, L., Lin, Z., Zhang, R., et al, 2023. 3D Domain Adaptive Instance Segmentation Via Cyclic Segmentation GANs. IEEE Journal of Biomedical and Health Informatics.
• Linghu, C., An, B., Shpokayte M., Celiker O.T., Shmoel N., Zhang R., et al. Recording of cellular physiological histories along optically readable self-assembling protein chains. Nat Biotechnol (2023).
• Zhang, Y., Chen, W., Zhang, R., & Zhang, X, 2023. Representing affect information in word embeddings. Experiments in Linguistic Meaning, 2, pp.310-321.
• Goodwin, D.R., Vaughan, A., Leibel, D., Alon, S., Henry, G.L., Cheng, A., Chen, X., Zhang, R., et al, 2022. Expansion Sequencing of RNA Barcoded Neurons in the Mammalian Brain: Progress and Implications for Molecularly Annotated Connectomics. bioRxiv, pp.2022-07.
• Zhang, R., 2021. Towards Mapping Spatial Transcriptome of an entire vertebrate brain. Master thesis. DSpace@MIT.
• Padilla-Coreano, N., Batra, K., Patarino, M., Chen, Z., Rock, R.R., Zhang, R. , et. al. 2022. Cortical ensembles orchestrate social competition through hypothalamic outputs. Nature, 603(7902), pp.667-671.
• Gao, Y., Li, J., Zhang, Z., Zhang, R., et.al., 2019. MicroRNA miR-7 and miR-17-92 in the arcuate nucleus of Mouse Hypothalamus regulate sex-specific diet-induced obesity. Molecular Neurobiology, 56(11), pp.7508-7521.