Contributions

T5X was developed as part of the T5 Infrastructure effort at Google Research.

Adam Roberts founded and leads the project, designed and wrote much of seqio and t5x, and co-authored the T5X and SeqIO paper. Hyung Won Chung designed and wrote much of t5x, led its open sourcing, and co-authored the paper. Anselm Levskaya built the initial prototype for t5x and wrote much of the code. Gaurav Mishra leads seqio, implemented deterministic pipelines, and co-authored the paper. James Bradbury implemented partitioning in t5x and co-wrote the paper.

Daniel Andor, Sharan Narang, Brian Lester, Colin Gaffney, Afroz Mohiuddin, Curtis Hawthorne, Aitor Lewkowycz, Alex Salcianu, Marc van Zee, Jacob Austin, Sebastian Good-man, Livio Baldini Soares, Haitang Hu, Sasha Tsvyashchenko, Aakanksha Chowdhery, Jasmijn Bastings, Jannis Bulian, Xavier Garcia, Jianmo Ni, Andrew Chen, Kathleen Kenealy, Kehang Han, Jonathan H. Clark, Stephan Lee, Dan Garrette, and James Lee-Thorp made substantial code contributions.

Colin Raffel and Noam Shazeer helped design seqio. Marvin Ritter advised on deterministic pipelines and the use of CLU Metrics. Maarten Bosma helped design deterministic pipelines. Jeremy Maitin-Shepard advised on the use of TensorStore. Alexandre Passos and Ryan Sepassi advised on overall technical design.

Noah Fiedel is a member of the leadership team, contributed to the high level design and roadmap, and co-wrote the paper. Mark Omernick, Brennan Saeta, Ryan Sepassi, Alexander Spiridonov (Product Manager), and Josh Newlan (Technical Program Manager) are members of the leadership team and co-wrote the paper. Andrea Gesmundo is a member of the leadership team and contributed to the internal infrastructure component.

Thanks to the many other contributors to the project: Ian Simon, Reiner Pope, Vincent Zhao, Pierre Ruyssen, Linting Xue, Junwhan Ahn, Barret Zoph, David Dohan, Masumi Parekh, Chang Lan, Frederick Liu, Julien Amelot, Luheng He, Fede Lebron, RebeccaChen, Anosh Raj, Mandy Guo, Ethan Dyer, Mihai Tiuca, Hongkun Yu, Kevin Brooks, David Soergel, Kelvin Guu, Joshua Ainslie, Luyao Xu, Ji Ma, Josh Gardner, Daphne Ippolito, Peter Hawkins, Bo Pang, Marc Rasi, Wei Li, Wenhu Chen, Iulia Turc, John Wieting, Alex Passos, Zonglin Li, Katie Everett, Olivier Bachem, Francesco Piccinno, Jakub Adamek, Jonathan Heek, Parker Schuh, Hexiang Hu, Du Phan, Max Moroz, David Miller, Ryan Doherty, David Elworthy, Alfonso Casta ̃no, Julian Eisenschlos, Vlad-Doru Ion, Lucas Dixon, Ron Shapiro, Dinghua Li, Aaron Parisi, Xi Chen, Nan Ding, Chung-ching Chang, Timothy Dozat, Natalia Ponomareva, Delesley Hutchins, Ankush Garg, Yu-Han Liu, Mehrdad Khatir, Costanza Conforti, Philipp Keck, Rapha ̈el Marinier, Marie Pellat, Raghuram Vadapalli, Joshua Maynez, Yi Tay, Xihui Wu, David Belanger, Luke Metz, Dan Zheng, Deepti Bhatia, Hariharan Shanmugavadivel, Rewon Child, Rigel Swavely, Mihir Sanjay Kale, Arash Afkanpour, Roberto Rama, Juro Gottweis, Jonathan Herzig, Yilei Yang, Elias Mizan, Pedram Pejman, Jiayu Ye, Smit Sanghavi, Rahul Joshi, Ziqiang Feng, Charles Sutton, Weikang Zhou, Liam Fedus, Shanqing Cai, Ginger Perng, Yash Katariya, Urvashi Khandelwal, Sebastian Gehrmann, Edward Loper, Tianze Shi, Luke Vilnis, Amelia Archer, Tom Weingarten, David Zats, Murtaza Dhuliawala, Xin Xie, Sahil Dua, Andr ́e SusanoPinto, Piotr Padlewski, Sascha Rothe, Erik Aas, Felix Stahlberg, Ken Durden, Christina Sorokin, Jaehoon Lee, Roy Frostig, Jacob Devlin, Jorge Gonzalez Mendez, Deepak Ramachandran, Santiago Ontanon, Karthik Raman, Yi Sun, Ali Elqursh, Reuben La Haye,Adam Fahrenkopf, Alex Polozov, Vinay Ramasesh, Ian Tenney.

Thanks to NVIDIA for GPU contributions: Sahil Jain, Terry Kong, Yu-Hang Tang, Ming Huang, Frederic Bastien, Sharath Turuvekere Sreenivas, Xiaowei Ren, Ryan Jeng, Reese Wang

Thanks to Douglas Eck and Zoubin Ghahramani for sponsoring the project.