--- title: OSFT explained: Prevent catastrophic forgetting in LLM... description: Orthogonal Subspace Fine-Tuning (OSFT) is a technique that addresses catastrophic forgetting — the phenomenon where fine-tuning an LLM on new tasks causes it... canonical: https://daily.dev/posts/osft-explained-prevent-catastrophic-forgetting-in-llm-fine-tuning-w5nzecsi8 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: OSFT explained: Prevent catastrophic forgetting in LLM fine-tuning | daily.dev og:description: Orthogonal Subspace Fine-Tuning (OSFT) is a technique that addresses catastrophic forgetting — the phenomenon where fine-tuning an LLM on new tasks causes it... og:url: https://daily.dev/posts/osft-explained-prevent-catastrophic-forgetting-in-llm-fine-tuning-w5nzecsi8 og:image: https://api.daily.dev/og/posts/W5NzECSI8.png og:image:alt: OSFT explained: Prevent catastrophic forgetting in LLM fine-tuning og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # OSFT explained: Prevent catastrophic forgetting in LLM fine-tuning **[Red Hat Developer](https://daily.dev/sources/rhdev)** · 5 min read · 0 upvotes · 0 comments ## Summary Orthogonal Subspace Fine-Tuning (OSFT) is a technique that addresses catastrophic forgetting — the phenomenon where fine-tuning an LLM on new tasks causes it to lose previously acquired capabilities. Unlike SFT and LoRA, which have no mechanism to protect existing model knowledge, OSFT identifies which parts of the model encode current skills and restricts training to orthogonal (non-interfering) subspaces. Benchmarks show OSFT reaching 48.4% average accuracy on a multi-task benchmark versus ~23% for standard fine-tuning. Red Hat's Training Hub library supports OSFT with a simple API change from `sft` to `osft`, plus a single `unfreeze_rank_ratio` parameter controlling how much of the model is open to new learning. It supports Llama, Qwen, Granite, Gemma, Phi, and other model families, and integrates with Red Hat OpenShift AI for production-scale training. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Post-training methods for language models](https://daily.dev/posts/post-training-methods-for-language-models-ghah2wf18) · Red Hat Developer · 1 upvotes · 0 comments - [Researchers propose a self-distillation fix for ‘catastrophic forgetting’ in LLMs](https://daily.dev/posts/researchers-propose-a-self-distillation-fix-for-catastrophic-forgetting-in-llms-cfeweepzh) · InfoWorld · 0 upvotes · 0 comments - [Post-Training Generative Recommenders with Advantage-Weighted Supervised Finetuning](https://daily.dev/posts/post-training-generative-recommenders-with-advantage-weighted-supervised-finetuning-aaj5ljika) · Netflix TechBlog · 1 upvotes · 0 comments - [Fine-Tune an Open-Source LLM on Your Own Data](https://daily.dev/posts/fine-tune-an-open-source-llm-on-your-own-data-fl1zitmwk) · Medium · 2 upvotes · 0 comments --- Tags: [#python](https://daily.dev/tags/python), [#lora](https://daily.dev/tags/lora) [View this post on daily.dev](https://daily.dev/posts/osft-explained-prevent-catastrophic-forgetting-in-llm-fine-tuning-w5nzecsi8) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"OSFT explained: Prevent catastrophic forgetting in LLM fine-tuning","url":"https://daily.dev/posts/osft-explained-prevent-catastrophic-forgetting-in-llm-fine-tuning-w5nzecsi8","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/osft-explained-prevent-catastrophic-forgetting-in-llm-fine-tuning-w5nzecsi8"},"datePublished":"2026-07-28T07:31:00.104Z","dateModified":"2026-07-28T07:31:23.033Z","description":"Orthogonal Subspace Fine-Tuning (OSFT) is a technique that addresses catastrophic forgetting — the phenomenon where fine-tuning an LLM on new tasks causes it...","image":"https://media.daily.dev/image/upload/s--P4t4XyoV--/f_auto/v1722860399/public/Placeholder%2001","thumbnailUrl":"https://media.daily.dev/image/upload/s--P4t4XyoV--/f_auto/v1722860399/public/Placeholder%2001","isAccessibleForFree":true,"articleSection":"Red Hat Developer","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Red Hat Developer","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/2a8895f3216c4900bdc455585a4ad3e4","url":"https://daily.dev/sources/rhdev"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/osft-explained-prevent-catastrophic-forgetting-in-llm-fine-tuning-w5nzecsi8","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"python,lora","timeRequired":"PT5M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Red Hat Developer","item":"https://daily.dev/sources/rhdev"},{"@type":"ListItem","position":3,"name":"OSFT explained: Prevent catastrophic forgetting in LLM fine-tuning"}]} ```