HuggingFace 🤗 on Trainium

A deep dive into training 🤗 Transformers on AWS Trainium — AWS’s custom chip designed for high-performance deep learning.

📢 UPDATE: 🤗 on Trainium is now part of the AWS Iberia AI/ML workshops

Inspired by Julien Simon’s excellent post on Accelerate Transformer training with AWS Trainium 🙌, I built a project with Terraform configurations and PyTorch training scripts to get you up and running immediately.

The goal? Demonstrate how minimal the code changes are when moving training jobs to Trn1 instances, using Hugging Face’s fine-tuning tutorial as the baseline.

This is still a work in progress 🚧, so feedback and suggestions are always welcome.

📚 Want to learn more about AWS Trainium? Check out the References section in the repo docs.