--- title: Detecting LLM-Generated Web Fiction with "Classical"... description: A developer built a classical ML-based detector for LLM-generated Chinese web fiction using scikit-learn's TF-IDF and LinearSVC pipeline. Motivated by... canonical: https://daily.dev/posts/detecting-llm-generated-web-fiction-with-classical-machine-learning-aigc-text-detection--cublctgyq twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Detecting LLM-Generated Web Fiction with "Classical" Machine Learning (AIGC Text Detection) | daily.dev og:description: A developer built a classical ML-based detector for LLM-generated Chinese web fiction using scikit-learn's TF-IDF and LinearSVC pipeline. Motivated by... og:url: https://daily.dev/posts/detecting-llm-generated-web-fiction-with-classical-machine-learning-aigc-text-detection--cublctgyq og:image: https://api.daily.dev/og/posts/CUblcTgYq.png og:image:alt: Detecting LLM-Generated Web Fiction with "Classical" Machine Learning (AIGC Text Detection) og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Detecting LLM-Generated Web Fiction with "Classical" Machine Learning (AIGC Text Detection) **[Hacker News](https://daily.dev/sources/hn)** · 12 min read · 2 upvotes · 0 comments ## Summary A developer built a classical ML-based detector for LLM-generated Chinese web fiction using scikit-learn's TF-IDF and LinearSVC pipeline. Motivated by AI-flooded fanfic platforms, they generated ~10,000 human and ~70,000 LLM-written samples across 7 models, trained 7 binary classifiers with majority voting, and achieved ~85% per-sentence accuracy. The model generalizes well to unseen LLMs (GPT-4o, MiMo-V2) with ~70–90% detection rates. A JavaScript browser demo was built for serverless deployment. Testing on 10,000 pre-2022 Lofter fanfics showed a false positive rate under 0.04% at 60% threshold, while 32% of current trending Lofter posts scored above 50% AI. Common bypass techniques (translation roundtrips, anti-AI prompts) reduced detection rates only marginally. The author concludes that LLM-generated text has surprisingly strong statistical fingerprints detectable with simple classical ML. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How accurate is a TF-IDF plus LinearSVC classifier at detecting LLM-generated text compared to perplexity-based methods? A TF-IDF plus LinearSVC classifier achieved roughly 85% sentence-level accuracy on a test set of Chinese web fiction, using seven separate binary classifiers trained on outputs from Gemini, Qwen, GLM, Kimi, Deepseek, and Doubao, combined via majority voting. This outperformed a perplexity-based detection approach, which suffered from false positives, false negatives, high inference cost, and poor generalization across models. _Anyone weighing detection approaches for AI-generated content can compare real accuracy trade-offs like these on daily.dev._ ### What percentage of trending Lofter fanfiction is likely AI-generated without disclosure? Sampling articles from Lofter's Android top 20 trending weekly tags found that 32.22% scored above 50% on an AI-generated text detector, and none of the flagged articles had disclosed AI use. A separate random sample of 10,000 pre-2022 high-engagement fanfics gave a false positive rate as low as 0.04% at a 60% detection threshold, supporting confidence in the trending-tag findings. _Readers tracking AI content saturation on platforms can dig into findings like this on daily.dev._ ### Do translation round-tripping or anti-AI-detection prompts actually evade AI text detectors? Not effectively. Google Translate round-tripping (Chinese to English to Chinese) only dropped a detection score from 89.9% to 85.0%, Youdao to 79.2%, and Sogou to 86.0%. Prompting an LLM to rewrite text and minimize AI flavor with simple or complex prompts only reduced scores to 83.0% and 79.3% respectively, still leaving the text clearly flagged as AI-generated by a TF-IDF and SVM based classifier. _Developers evaluating AI content detection robustness can track results like these on daily.dev._ ## Similar posts on daily.dev - [AI wins Imitation Game: Readers prefer ChatGPT fan fiction](https://daily.dev/posts/ai-wins-imitation-game-readers-prefer-chatgpt-fan-fiction-dcrov8kav) · The Register · 1 upvotes · 2 comments - [Scikit-LLM vs. Traditional Text Classifiers: When Should You Use an LLM?](https://daily.dev/posts/scikit-llm-vs-traditional-text-classifiers-when-should-you-use-an-llm--p02kqzqrv) · Machine Learning Mastery · 0 upvotes · 0 comments - [Is This Slop? Detecting AI-Generated Content Without a Model](https://daily.dev/posts/is-this-slop-detecting-ai-generated-content-without-a-model-bn7hlke8u) · Towards Data Science · 1 upvotes · 0 comments - [AI detection tools cannot prove that text is AI-generated](https://daily.dev/posts/ai-detection-tools-cannot-prove-that-text-is-ai-generated-fvd95pteg) · sean goedecke · 2 upvotes · 0 comments - [Researchers discover a shortcoming that makes LLMs less reliable](https://daily.dev/posts/researchers-discover-a-shortcoming-that-makes-llms-less-reliable-wiiiapmcn) · MIT News · 3 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#llm](https://daily.dev/tags/llm), [#nlp](https://daily.dev/tags/nlp), [#scikit](https://daily.dev/tags/scikit) [View this post on daily.dev](https://daily.dev/posts/detecting-llm-generated-web-fiction-with-classical-machine-learning-aigc-text-detection--cublctgyq) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Detecting LLM-Generated Web Fiction with \"Classical\" Machine Learning (AIGC Text Detection)","url":"https://daily.dev/posts/detecting-llm-generated-web-fiction-with-classical-machine-learning-aigc-text-detection--cublctgyq","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/detecting-llm-generated-web-fiction-with-classical-machine-learning-aigc-text-detection--cublctgyq"},"datePublished":"2026-07-16T17:45:29.291Z","dateModified":"2026-09-14T08:15:47.326Z","description":"A developer built a classical ML-based detector for LLM-generated Chinese web fiction using scikit-learn's TF-IDF and LinearSVC pipeline. Motivated by...","image":"https://media.daily.dev/image/upload/s--ZrL_HSsR--/f_auto/v1722860399/public/Placeholder%2006","thumbnailUrl":"https://media.daily.dev/image/upload/s--ZrL_HSsR--/f_auto/v1722860399/public/Placeholder%2006","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/detecting-llm-generated-web-fiction-with-classical-machine-learning-aigc-text-detection--cublctgyq","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,llm,nlp,scikit","timeRequired":"PT12M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"Detecting LLM-Generated Web Fiction with \"Classical\" Machine Learning (AIGC Text Detection)"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/detecting-llm-generated-web-fiction-with-classical-machine-learning-aigc-text-detection--cublctgyq#faq","mainEntity":[{"@type":"Question","name":"How accurate is a TF-IDF plus LinearSVC classifier at detecting LLM-generated text compared to perplexity-based methods?","acceptedAnswer":{"@type":"Answer","text":"A TF-IDF plus LinearSVC classifier achieved roughly 85% sentence-level accuracy on a test set of Chinese web fiction, using seven separate binary classifiers trained on outputs from Gemini, Qwen, GLM, Kimi, Deepseek, and Doubao, combined via majority voting. This outperformed a perplexity-based detection approach, which suffered from false positives, false negatives, high inference cost, and poor generalization across models. Anyone weighing detection approaches for AI-generated content can compare real accuracy trade-offs like these on daily.dev."}},{"@type":"Question","name":"What percentage of trending Lofter fanfiction is likely AI-generated without disclosure?","acceptedAnswer":{"@type":"Answer","text":"Sampling articles from Lofter's Android top 20 trending weekly tags found that 32.22% scored above 50% on an AI-generated text detector, and none of the flagged articles had disclosed AI use. A separate random sample of 10,000 pre-2022 high-engagement fanfics gave a false positive rate as low as 0.04% at a 60% detection threshold, supporting confidence in the trending-tag findings. Readers tracking AI content saturation on platforms can dig into findings like this on daily.dev."}},{"@type":"Question","name":"Do translation round-tripping or anti-AI-detection prompts actually evade AI text detectors?","acceptedAnswer":{"@type":"Answer","text":"Not effectively. Google Translate round-tripping (Chinese to English to Chinese) only dropped a detection score from 89.9% to 85.0%, Youdao to 79.2%, and Sogou to 86.0%. Prompting an LLM to rewrite text and minimize AI flavor with simple or complex prompts only reduced scores to 83.0% and 79.3% respectively, still leaving the text clearly flagged as AI-generated by a TF-IDF and SVM based classifier. Developers evaluating AI content detection robustness can track results like these on daily.dev."}}]} ```