This post explores the concept of abstraction in neural networks and its importance in cognition. Abstraction is the act of ignoring irrelevant details and focusing on the relevant parts. Neural networks implement abstraction by using non-injective mappings and discarding irrelevant features. Abstraction is closely related to generalization, where more abstract assumptions lead to more widely applicable conclusions. Abstraction plays a crucial role in our ability to understand and learn from the world around us.
Table of contents
Understanding Abstractions in Neural NetworksAbstraction, Abstractly DefinedDuality of Abstraction and GeneralizationAbstractions in Neural NetworksBringing in Information TheoryA Unifying PerspectiveThe Core of CognitionConclusions50 Impressions