Zhanpeng Zhou (周展鹏)

Ph.D. candidate in Computer Science at Shanghai Jiao Tong University, advised by Prof. Junchi Yan. My research interests focus on deep learning theory and science of large language models.

I am a Pre-Career Scholar at SII and currently lead optimizer research at StepFun. Prior to that, I worked on pretraining at Tencent Hunyuan, ByteDance Seed, and MiniMax.

Email: zzp1012 [at] sjtu.edu.cn  /  zhanpeng.personal [at] gmail.com

CV  /  Google Scholar  /  GitHub  / 

profile photo
Publications   (* indicates equal contributions; † indicates correspondence.)
How Does Local Landscape Geometry Evolve in Language Model Pre-Training?   [arXiv]
Zhanpeng Zhou*, Yuhan Sun*, Bingrui Li, Jinbo Wang, Huaijin Wu, Lei Wu, Junchi Yan
In Submission
Looped Transformers as Optimizers   [arXiv]
Yulong Huang*, Chen Jiang*, Zhanpeng Zhou, Hongtao Zhang, Tianyu Li, Tianyu He, Xiangyu Zhang, Bojun Cheng
In Submission
Efficient Hyperparameter Tuning via Trajectory Invariance Principle   [arXiv]
Bingrui Li, Jiaxin Wen, Zhanpeng Zhou, Jun Zhu, Jianfei Chen
In Submission
From LR to ELR: A Better Heuristic for Pretraining Dynamics   [Post]   [GitHub]
Zhanpeng Zhou*, Wenjie Zhou*, Yufei Gu*, Juqiu Wang, Guanwei Zhang, Ruobing Xie, Xianbiao Qi, Yushi Bai, Kaifeng Lyu, Shunyu Yao
Hunyuan Research
How Transformers Learn to Plan via Multi-Token Prediction   [arXiv]
Jianhao Huang*, Zhanpeng Zhou*, Renqiu Xia, Baharan Mirzasoleiman, Weijie Su, Wei Huang
COLM 2026
The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence   [arXiv]   [Hugging Face]
MiniMax (Co-author)
Technical Report
More Sail than Ballast: Addressing Harmful Knowledge Leakage in the Expansive Reasoning Space of LRMs   [arXiv]
Qibing Ren, Xinhao Song, Ke Fan, Lijun Li, Zhanpeng Zhou, Gongshen Liu, Junchi Yan, Lizhuang Ma, Jing Shao
ICML 2026
On Path to Multimodal Historical Reasoning: HistBench and HistAgent   [arXiv]   [GitHub]
Qiu et al. (Co-author)
ICML 2026
Fast Catch-Up, Late Switching: Optimal Batch Size Scheduling via Functional Scaling Laws   [arXiv]
Jinbo Wang, Binghui Li, Zhanpeng Zhou, Mingze Wang, Yuxuan Sun, Jiaqi Zhang, Xunliang Cai, Lei Wu
ICLR 2026
Achieving Low-bit Muon Optimization through Subspace Preservation and Grid Quantization   [arXiv]   [GitHub]
Huaijin Wu, Bingrui Li, Yebin Yang, Yi Tu, Zhanpeng Zhou, Jianfei Chen, Junchi Yan
ICLR 2026
IGU-LoRA: Adaptive Rank Allocation via Integrated Gradients and Uncertainty-Aware Scoring   [arXiv]   [GitHub]
Xuan Cui, Huiyue Li, Run Zeng, Yunfei Zhao, Jinrui Qian, Wei Duan†, Bo Liu†, Zhanpeng Zhou†
ICLR 2026
On the Surprising Effectiveness of a Single Global Merging in Decentralized Learning   [arXiv]   [GitHub]   [Slides]   [Post]
Tongtian Zhu, Tianyu Zhang, Mingze Wang, Zhanpeng Zhou†, Can Wang
ICLR 2026 (Oral)
On the Learning Dynamics of Two-layer Linear Networks with Label Noise SGD   [arXiv]   [GitHub]
Tongcheng Zhang*, Zhanpeng Zhou*, Mingze Wang, Andi Han, Wei Huang, Taiji Suzuki, Junchi Yan
AAAI 2026 (Oral)
New Evidence of the Two-Phase Learning Dynamics of Neural Networks   [arXiv]
Zhanpeng Zhou, Yongyi Yang, Mahito Sugiyama, Junchi Yan
HiLD 2025
On the Role of Label Noise in the Feature Learning Process   [arXiv]   [GitHub]
Andi Han*†, Wei Huang*†, Zhanpeng Zhou*†, Gang Niu, Wuyang Chen, Junchi Yan, Akiko Takeda, Taiji Suzuki
ICML 2025
The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training   [arXiv]   [GitHub]
Jinbo Wang*, Mingze Wang*, Zhanpeng Zhou*, Junchi Yan, Weinan E, Lei Wu
ICML 2025
On the Cone Effect in the Learning Dynamics   [arXiv]
Zhanpeng Zhou, Yongyi Yang, Jie Ren, Mahito Sugiyama, Junchi Yan
ICLR 2025 Workshop
SE-Merging: A Self-Enhanced Approach for Dynamic Model Merging   [arXiv]
Zijun Chen*, Zhanpeng Zhou*, Bo Zhang, Weinan Zhang, Xi Sun, Junchi Yan
IJCNN 2025
Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late in Training   [arXiv]   [GitHub]   [Slides]
Zhanpeng Zhou*†, Mingze Wang*, Yuchen Mao, Bingrui Li, Junchi Yan†
ICLR 2025 (Spotlight)
On the Optimization and Generalization of Two-layer Transformers with Sign Gradient Descent   [arXiv]
Bingrui Li, Wei Huang, Andi Han, Zhanpeng Zhou, Taiji Suzuki, Jun Zhu, Jianfei Chen
ICLR 2025 (Spotlight)
On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning Paradigm   [arXiv]   [GitHub]   [Slides]
Zhanpeng Zhou*, Zijun Chen*, Yilan Chen, Bo Zhang, Junchi Yan
ICML 2024
Going Beyond Neural Network Feature Similarity: The Network Feature Complexity and Its Interpretation Using Category Theory   [arXiv]   [GitHub]
Yiting Chen, Zhanpeng Zhou, Junchi Yan
ICLR 2024
Going Beyond Linear Mode Connectivity: The Layerwise Linear Feature Connectivity   [arXiv]   [GitHub]   [Slides]   [Post]
Zhanpeng Zhou, Yongyi Yang, Xiaojiang Yang, Junchi Yan, Wei Hu
NeurIPS 2023
Defects of Convolutional Decoder Networks in Frequency Representation   [arXiv]   [GitHub]
Ling Tang*, Wen Shen*, Zhanpeng Zhou, Quanshi Zhang
ICML 2023
Batch Normalization Is Blind to the First and Second Derivatives of the Loss   [arXiv]   [GitHub]
Zhanpeng Zhou*, Wen Shen*, Huixin Chen*, Ling Tang, Quanshi Zhang
AAAI 2024 (Oral)
Can We Faithfully Represent Absence States to Compute Shapley Values on a DNN?   [arXiv]   [GitHub]
Jie Ren, Zhanpeng Zhou, Qirui Chen, Quanshi Zhang
ICLR 2023
A Unified Game-Theoretic Interpretation of Adversarial Robustness   [arXiv]   [Github]
Jie Ren*, Die Zhang*, Yisen Wang*, Lu Chen, Zhanpeng Zhou, Yiting Chen, Xu Cheng, Xin Wang, Meng Zhou, Jie Shi, Quanshi Zhang
NeurIPS 2021
Blogs
A Simple Proof of the Functional Scaling Law
Zhanpeng Zhou   20-min read
The Choices of Kronecker factors: A New AdaGrad Variant
Zhanpeng Zhou   10-min read
The Counterpart of Gradient Accumulation: A Sharpness View
Zhanpeng Zhou   10-min read
Removing Square-Roots in Adaptive Optimization: Resolve Scaling Mismatch First
Zhanpeng Zhou   5-min read
Estimating Diagnoal Hessian: A Hand-crafted Derivation
Zhanpeng Zhou   5-min read
PMuon: Preconditioning Muon with Gradient Covariance Before the Polar Step
Zhanpeng Zhou   5-min read
Deriving and Explaining Muon: A Directional Sharpness Perspective
Zhanpeng Zhou   10-min read
Directional Feature Learning and Insights into Manifold Muon
Zhanpeng Zhou   5-min read
Services

[Area Chair] CPAL '26, ICLR '26 DeLTa, ICML '26 FoGen

[Conference Reviewer] ICML ('22, '24-26), NeurIPS ('22-26), ICLR ('24-27)

[Journal Reviewer] IEEE T-PAMI


Thanks Jon Barron for this template.