PyTorch Monarch, Meta's single-controller distributed training framework, has been ported to AMD Instinct GPUs running ROCm. The port involved converting CUDA/NCCL code to HIP/RCCL using hipify_torch, handling dynamic linking differences, and adding a Rust compatibility shim to avoid platform-specific branching. Integrated with TorchTitan and TorchFT, the system enables checkpoint-less fault-tolerant training: when a GPU process crashes, healthy replicas continue training while the failed replica restarts and receives a peer checkpoint transfer from a donor replica. Validation on a 16-node SLURM cluster (128 MI300 GPUs) and a 32-node Kubernetes cluster (256 MI355 GPUs) showed stable loss convergence despite frequent injected failures, with no full job restarts required.
