T5X

T5X is a modular, composable, research-friendly framework for high-performance, configurable, self-service training, evaluation, and inference of sequence models (starting with language) at many scales.

It is essentially a new and improved implementation of the T5 codebase (based on Mesh TensorFlow) in JAX and Flax. To learn more, see the T5X Paper.

Table of Contents

• Quick Start
  • Quickstart (Recommended)
  • GPU Usage
  • Installation
  • Example: English to German translation
  • Custom components
  • Checkpoints
  • Citing T5X
  • Note
• Tutorials
  • Overview
  • How to Use These Guides
  • T5X Guides
• Usage Guides
  • Pretraining a model
  • Fine Tuning a Model
  • Evaluating a Model
  • Running Inference on a Model
  • Auxiliary Job
  • Decoding
  • Metrics Overview
  • Data, Model, and Activation Partitioning
  • Gin Primer
• Models
  • Selecting a model:
  • Public Research Models
• API Reference
  • Binaries
  • Training
  • Inference
  • Models
  • Checkpointing
  • Paritioning
  • Config
  • Utils
• Contributions