--- title: With 86% of the words, you might understand only 38% of... description: Knowing most of the words spoken in an anime doesn't mean you can follow most of its lines. In Frieren, the 500 most common words cover 85.9% of spoken words... canonical: https://daily.dev/posts/with-86-of-the-words-you-might-understand-only-38-of-an-anime-s-lines-mea5owbbb twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: With 86% of the words, you might understand only 38% of an anime's lines | daily.dev og:description: Knowing most of the words spoken in an anime doesn't mean you can follow most of its lines. In Frieren, the 500 most common words cover 85.9% of spoken words... og:url: https://daily.dev/posts/with-86-of-the-words-you-might-understand-only-38-of-an-anime-s-lines-mea5owbbb og:image: https://api.daily.dev/og/posts/Mea5oWBbb.png og:image:alt: With 86% of the words, you might understand only 38% of an anime's lines og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # With 86% of the words, you might understand only 38% of an anime's lines **[Hacker News](https://daily.dev/sources/hn)** · 11 min read · 0 upvotes · 0 comments ## Summary Knowing most of the words spoken in an anime doesn't mean you can follow most of its lines. In Frieren, the 500 most common words cover 85.9% of spoken words but let you fully understand only 38.4% of the first 3,000 lines, since you need every word in a line to follow it. The gap holds across eight anime tested: Serial Experiments Lain needs the fewest words (about 400 for half the lines), while Jujutsu Kaisen needs 980. Going from 500 to 1,000 known words raises coverage by just 6.5 points but lines understood by 22.1 points. The analysis used the SudachiPy 0.6.11 tokenizer with SudachiDict-core, and the full Python script (29 lines) is included. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why does knowing 86% of the words in a show not mean I understand 86% of the dialogue? Because understanding a line requires knowing every word in it, not just most words overall. In Frieren, the 500 most common words give 85.9% word coverage (repeats counted) but only 38.4% of the first 3,000 lines are fully understood, since an unknown word anywhere in a line blocks comprehension of that whole line. _Anyone picking vocabulary targets for a show can track analyses like this one on daily.dev._ ### How many Japanese words do I need to know to understand half the lines in an anime like Jujutsu Kaisen versus Serial Experiments Lain? Jujutsu Kaisen needs about 980 known words to reach 50% of lines understood, while Serial Experiments Lain needs only about 401 words for the same threshold, measured across each show's first 3,000 subtitle lines using dictionary-form word counts from the Sudachi tokenizer in split mode C. _Developers comparing language-learning datasets across shows can follow breakdowns like this on daily.dev._ ### How do you tokenize Japanese subtitles in Python to count word frequency for vocabulary analysis? Use SudachiPy with SudachiDict-core in split mode C, convert each token to its dictionary form, and skip punctuation, proper names, and numbers using the tokenizer's part-of-speech tags. A documented approach uses SudachiPy 0.6.11, SudachiDict-core 20260428, and Python 3.13.7, counting word occurrences with collections.Counter in under 30 lines of code. _Developers building Japanese NLP tooling can find tokenizer workflows like this through daily.dev._ ## Community take How the wider developer community reacted, aggregated from 1 discussion and 41 comments across hackernews (as of 2026-11-01). **TL;DR:** Discussion largely pushes back on the post's framing, arguing that context, redundancy in language, and visual cues let people understand lines even without knowing every word, while some share personal anecdotes about the gap between vocabulary and comprehension. **Sentiment:** 10% positive · 55% mixed · 35% skeptical **The case for** - Several people found the core finding plausible or relatable, citing similar experiences learning other languages like German or Latgallian. - The methodology of using per-show word frequency rather than a generic corpus was seen as a reasonable approach. **The pushback** - Many argued the analysis ignores that context, visuals, and redundancy in language let you infer meaning without knowing every word. - Some disputed the claim that rare words carry the most information, pointing out common words like 'not' can be load-bearing. - The use of only word-level knowledge was seen as ignoring grammar, kanji compounds, and speaking speed as other barriers to comprehension. - The writing and framing were criticized as LLM-generated and bloated with an unnecessary marketing pitch. - One commenter noted that for shows like Neon Genesis Evangelion, understanding every word still doesn't guarantee understanding the content, since the ambiguity is intentional. **By community** - hackernews (mixed): Commenters mostly contest the post's framing with counterarguments about context and redundancy, while a smaller group shares supportive anecdotes and tangents about language learning. **Hottest debate:** Whether understanding every word in a sentence is actually necessary to understand its meaning, given context, redundancy, and visual information. **Open questions** - Would accounting for context and visual cues significantly change the 'lines understood' figures? - How would the analysis differ if grammar and kanji compound knowledge were factored in rather than just word knowledge? **Highlights** > Human languages are massively redundant specifically to enable this kind of "fill the gaps" learning process. Observe an unknown word used a few times in different contexts, and you'll get a very good bead on what it means. It's the "natural" way of learning a language. No one gets every word explained to them in detail - most words you know in your own first language are the ones you picked up on the go. I think this "structural redundancy" might be a big part of what enables LLMs to learn human languages "from nothing". > — [ACCount39 on hackernews](https://news.ycombinator.com/item?id=50320713) > I think it is more that the common words in the language are the scaffolding (like hedging with "I think...", "like" for setting up an example) and the subject of the scaffold is normally unique and highly context specific. Looking at the first Frieren example: "Live without [fear]", "Live without [bread]", "Live without [sunlight]", "Live without [honour]". The common words are more than enough to set up a generic sentence for lack of something but then what someone lacks can be literally any object, concept or verb. There are a lot more than 500 of those in the world. > — [roenxi on hackernews](https://news.ycombinator.com/item?id=50321386) > I’d disagree with that. For example, in many sentences, the most load-bearing (heh) word is “not”, which is also one of the most common words. > — [xigoi on hackernews · 1 comments](https://news.ycombinator.com/item?id=50321401) > "How we did it" (Claude and him). Some bad writing examples I liked. > By the third pause to look up a word, your tea has gone cold. And hear I thought that water had one of the highest specific heat capacities > Both numbers are true What is both numbers? Presumably the two percentages, (based on the title, but there is no framing right away that these are the key numbers), and proximate to the line are several unrounded numbers. > is a flashcard app that teaches Japanese from anime. Each card uses a line where you know everything except the one new word, which is why we started counting lines. The figures above count each word once, however many meanings it has. That wouldn't work in the app: knowing あれ as "that" doesn't help when someone says あれ？ (huh?). These are two separate cards. It doesn't catch every such word yet, we're working on it. During early access it offers one show, Hunter x Hunter. Highly advise cutting down your marketing pitch for your product, it's just too long to grab people. > — [ethanj8011 on hackernews](https://news.ycombinator.com/item?id=50321708) > One thing that seem to be improving Japanese literacy across the world is Inmu gay porn meme contents. I believe it hasn't reached the US, but it is becoming eerily well known in East/Southeast Asian countries, including mainland China. It uses tiny set of vocabulary in highly repetitive manner, a la kid's picture books, but in the way relatable and addicting to social media users rather than e.g. portraying a day in the life of a cute crayon drawn caterpillar. IMO, what that tells us is that a completely dumb repetition of small set of simple concepts to just set up the pathway can be an effective means to kickstart language acquisition process. Most language learning materials attempt to establish a compact set of rules to be expanded with a long tail of modular add-ons, even many that aims to create an alternative and less stressful approaches like Duolingo. The Inmu dictionary has less than 500 known expression, all unusable in regular communications and inappropriate in most settings, and the way it is used is simplistic, childish, disrespectful, and dumb. And I think that might be how "start with picture books" advice look like if rebuilt from first principles. > — [numpad0 on hackernews](https://news.ycombinator.com/item?id=50320705) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=50320028) · 61 points · 41 comments ## Similar posts on daily.dev - [How Specializing the Vocabulary of SLMs Can Cut Inference Cost by 77% in Real Applications](https://daily.dev/posts/how-specializing-the-vocabulary-of-slms-can-cut-inference-cost-by-77-in-real-applications-a4vqrjdnh) · Hugging Face · 22 upvotes · 1 comments - [The Eleven Languages My App Can Describe But Not Pronounce](https://daily.dev/posts/the-eleven-languages-my-app-can-describe-but-not-pronounce-v1a1ctrqe) · Medium · 0 upvotes · 0 comments --- Tags: [#python](https://daily.dev/tags/python), [#nlp](https://daily.dev/tags/nlp), [#data-analysis](https://daily.dev/tags/data-analysis) [View this post on daily.dev](https://daily.dev/posts/with-86-of-the-words-you-might-understand-only-38-of-an-anime-s-lines-mea5owbbb) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"With 86% of the words, you might understand only 38% of an anime's lines","url":"https://daily.dev/posts/with-86-of-the-words-you-might-understand-only-38-of-an-anime-s-lines-mea5owbbb","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/with-86-of-the-words-you-might-understand-only-38-of-an-anime-s-lines-mea5owbbb"},"datePublished":"2026-10-31T23:25:51.660Z","dateModified":"2026-11-01T01:26:39.929Z","description":"Knowing most of the words spoken in an anime doesn't mean you can follow most of its lines. In Frieren, the 500 most common words cover 85.9% of spoken words...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/814c23777a6c04f7346d22b41aa7d23c?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/814c23777a6c04f7346d22b41aa7d23c?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/with-86-of-the-words-you-might-understand-only-38-of-an-anime-s-lines-mea5owbbb","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"python,nlp,data-analysis","timeRequired":"PT11M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"With 86% of the words, you might understand only 38% of an anime's lines"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/with-86-of-the-words-you-might-understand-only-38-of-an-anime-s-lines-mea5owbbb#faq","mainEntity":[{"@type":"Question","name":"Why does knowing 86% of the words in a show not mean I understand 86% of the dialogue?","acceptedAnswer":{"@type":"Answer","text":"Because understanding a line requires knowing every word in it, not just most words overall. In Frieren, the 500 most common words give 85.9% word coverage (repeats counted) but only 38.4% of the first 3,000 lines are fully understood, since an unknown word anywhere in a line blocks comprehension of that whole line. Anyone picking vocabulary targets for a show can track analyses like this one on daily.dev."}},{"@type":"Question","name":"How many Japanese words do I need to know to understand half the lines in an anime like Jujutsu Kaisen versus Serial Experiments Lain?","acceptedAnswer":{"@type":"Answer","text":"Jujutsu Kaisen needs about 980 known words to reach 50% of lines understood, while Serial Experiments Lain needs only about 401 words for the same threshold, measured across each show's first 3,000 subtitle lines using dictionary-form word counts from the Sudachi tokenizer in split mode C. Developers comparing language-learning datasets across shows can follow breakdowns like this on daily.dev."}},{"@type":"Question","name":"How do you tokenize Japanese subtitles in Python to count word frequency for vocabulary analysis?","acceptedAnswer":{"@type":"Answer","text":"Use SudachiPy with SudachiDict-core in split mode C, convert each token to its dictionary form, and skip punctuation, proper names, and numbers using the tokenizer's part-of-speech tags. A documented approach uses SudachiPy 0.6.11, SudachiDict-core 20260428, and Python 3.13.7, counting word occurrences with collections.Counter in under 30 lines of code. Developers building Japanese NLP tooling can find tokenizer workflows like this through daily.dev."}}]} ```