--- title: "AI Fundamentals: Understanding Activation Functions (Part 1)" url: https://daily.dev/posts/ai-fundamentals-understanding-activation-functions-part-1--nznnqfpvz source_url: https://pub.towardsai.net/ai-fundamentals-understanding-activation-functions-part-1-fde15d65c735 type: article source: "Medium" published: 2026-08-05T20:51:50.585Z updated: 2026-08-05T20:52:27.381Z tags: ["machine-learning", "deep-learning", "neural-networks"] reading_time: 8 upvotes: 2 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # AI Fundamentals: Understanding Activation Functions (Part 1) **[Medium](https://daily.dev/sources/medium_js)** · 8 min read · 2 upvotes · 0 comments ## Summary Without non-linear activation functions, stacking any number of neural network layers collapses mathematically into a single affine transformation — a phenomenon called linear collapse. Activation functions like ReLU, sigmoid, or tanh break this algebraic collapse by introducing non-linearity after each layer's linear step, enabling networks to learn complex, curved decision boundaries rather than just straight lines. The Universal Approximation Theorem (UAT) formalizes this: a feedforward network with a non-linear activation and a sufficiently large hidden layer can approximate any continuous function on a bounded domain. The UAT is explained intuitively through the concept of localized 'towers' built from pairs of neurons, where hidden-layer weights and biases control where a tower is active and output weights control its height. Importantly, the UAT is an existence proof, not a training recipe — it guarantees representational capacity but says nothing about whether gradient descent can find the right weights or how well the result generalizes. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [An Intuitive Guide to Non-Linearity of ReLU](https://daily.dev/posts/an-intuitive-guide-to-non-linearity-of-relu-yc47oozis) · Daily Dose of Data Science \| Avi Chawla \| Substack · 1 upvotes · 0 comments - [AI Fundamentals: What is a Neuron?](https://daily.dev/posts/ai-fundamentals-what-is-a-neuron--grwe15zt3) · Medium · 1 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#deep-learning](https://daily.dev/tags/deep-learning), [#neural-networks](https://daily.dev/tags/neural-networks) [View this post on daily.dev](https://daily.dev/posts/ai-fundamentals-understanding-activation-functions-part-1--nznnqfpvz)