---
title: "Speeding up ViTs using Block Sparsity"
url: https://daily.dev/posts/speeding-up-vits-using-block-sparsity-umq7gw9ns
source_url: https://pytorch.org/blog/speeding-up-vits/
type: article
source: "PyTorch"
published: 2024-05-14T17:16:31.462Z
updated: 2024-05-14T17:16:30.341Z
tags: ["deep-learning", "pytorch"]
reading_time: 6
upvotes: 1
comments: 0
language: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Speeding up ViTs using Block Sparsity

**[PyTorch](https://daily.dev/sources/pytorch)** · 6 min read · 1 upvotes · 0 comments

## Summary

The post discusses the application of block sparsity on MLP modules in vision transformers, showing promising speedups with minimal accuracy drop. It explains the training and inference steps, provides microbenchmarking results, and showcases the speedup and accuracy achieved on a specific ViT model. Future steps and potential optimizations for block sparsity on vision transformers are also mentioned.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://pytorch.org/blog/speeding-up-vits/>

## Similar posts on daily.dev

- [Don’t just attend KubeCon \+ CloudNativeCon, Merge Forward your experience\!](https://daily.dev/posts/don-t-just-attend-kubecon-cloudnativecon-merge-forward-your-experience--l0rpp73x8) · CNCF · 0 upvotes · 0 comments
- [Announcing H2 2026 KCDs](https://daily.dev/posts/announcing-h2-2026-kcds-m96goajm1) · CNCF · 1 upvotes · 0 comments
- [Two months of Open Community Groups](https://daily.dev/posts/two-months-of-open-community-groups-asf52zhbs) · CNCF · 0 upvotes · 0 comments

---

Tags: [#deep-learning](https://daily.dev/tags/deep-learning), [#pytorch](https://daily.dev/tags/pytorch)

[View this post on daily.dev](https://daily.dev/posts/speeding-up-vits-using-block-sparsity-umq7gw9ns)