PyTorch and free GPUs

PyTorch is the default in research and most industry work, and the Hugging Face ecosystem is built around it. JAX is strong for large-scale research, and Keras 3 can run on several backends. Learn PyTorch first. The ideas transfer.

The core pieces are few. Tensors are arrays that can live on a GPU. An nn.Module holds layers and weights. A loss function and an optimizer such as AdamW do the learning. Autograd computes gradients for you.

Every PyTorch loop has the same five lines at heart. For each batch: run the model to get predictions, compute the loss, call loss.backward() to get gradients, call optimizer.step() to update weights, and call optimizer.zero_grad() to clear them for the next batch.

Around that, you move the model and data to the GPU with .to("cuda"), switch to model.eval() and torch.no_grad() when measuring validation loss, and save checkpoints often. Libraries like Hugging Face Transformers or PyTorch Lightning wrap this loop, but write it by hand at least once.

Google Colab gives free notebooks with a GPU, often an NVIDIA T4. GPU access is not guaranteed, sessions disconnect when idle and have a maximum length, and files vanish when the session ends unless you save to Google Drive.

Kaggle Notebooks give a weekly GPU quota, roughly 30 hours, which can change, with sessions capped at several hours. Both are plenty for learning, fine-tuning small models and the projects in this track. They are not enough to pretrain a large model. Always check the current limits on each site.

Overfit a single batch first: if the model cannot drive loss near zero on 32 examples, you have a bug. Set a random seed. Print tensor shapes when something breaks, since shape mismatches are the most common error. Save checkpoints so a disconnect does not cost you the run.