Outperform OpenAI GPT-3 with SetFit for text-classification
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
SetFit is a few-shot learning approach from Intel Labs, UKP Lab, and Hugging Face that outperforms GPT-3 on 7 out of 11 text-classification tasks while being 1600x smaller. This tutorial walks through building a text-classification model using only 32 labeled samples (8 per class) on the AG News dataset. It covers environment setup, dataset creation, fine-tuning a Sentence Transformer with contrastive training via SetFitTrainer, and using hyperparameter search with Optuna to reach ~87.3% accuracy. The result: 375x less data with only ~7% accuracy loss compared to a fully supervised model, trainable in under an hour on a single NVIDIA T4 GPU for under $1.
