--- title: An LLM tagged 24 SEC filings with the wrong company, and... description: An engineer describes a bug in an LLM-based pipeline that tags SEC Form 4 insider-trading filings with tickers. Because a Form 4 names both a reporting person... canonical: https://daily.dev/posts/an-llm-tagged-24-sec-filings-with-the-wrong-company-and-the-right-answer-was-in-the-same-row-2vix0mny8 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: An LLM tagged 24 SEC filings with the wrong company, and the right answer was in the same row | daily.dev og:description: An engineer describes a bug in an LLM-based pipeline that tags SEC Form 4 insider-trading filings with tickers. Because a Form 4 names both a reporting person... og:url: https://daily.dev/posts/an-llm-tagged-24-sec-filings-with-the-wrong-company-and-the-right-answer-was-in-the-same-row-2vix0mny8 og:image: https://api.daily.dev/og/posts/2vIX0MNy8.png og:image:alt: An LLM tagged 24 SEC filings with the wrong company, and the right answer was in the same row og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # An LLM tagged 24 SEC filings with the wrong company, and the right answer was in the same row **[AlphAI](https://daily.dev/sources/alphai)** · [@mikhailmakeev](https://daily.dev/mikhailmakeev) · 2 min read · 1 upvotes · 0 comments ## Summary An engineer describes a bug in an LLM-based pipeline that tags SEC Form 4 insider-trading filings with tickers. Because a Form 4 names both a reporting person and an issuer, the model frequently latched onto the more prominent name (the filer) rather than the issuer whose shares actually traded, mistagging 24 of 15,445 filings over 90 days, plus 81 filings that got no ticker at all and were dropped from the feed since it requires at least one ticker. Existing hallucination checks didn't catch this because they only verify a name appears somewhere in the text. The fix pins the ticker to the issuer resolved by CIK at ingest, keeping a model-suggested ticker only when it represents another share class of that same issuer. Interestingly, applying the same fix to 8-K filings made results worse, since 8-Ks name exact securities (preferred shares, ETNs, units) while the model's guess of the liquid ticker is often what readers actually want, so 8-K rows keep the model's original tags instead. ## Content A query for HOOD insider trades returned five rows. Four of them were sales in a different company: Robinhood Ventures Fund I, a separate issuer that Robinhood Markets files for. Every SEC Form 4 in our pipeline goes through an LLM that returns tickers, a category and a relevance score. Category and score are both reconciled against the filing afterwards. Tickers were not. Whatever the model emitted got stored. A Form 4 has a reporting person and an issuer, and the most prominent company name in the text is the reporting person. So the model tagged the filer instead of the company whose shares actually moved. Our hallucination check couldn't catch it either, because all it asks is whether the company name appears in the body, and "Robinhood Markets, Inc." genuinely does. The issuer had been sitting in the same row the whole time, resolved by CIK at ingest. Across 90 days and 15,445 filings: 24 rows pointing at the wrong company, and 81 with no ticker at all, which made them invisible since the feed requires at least one. The fix pins the ticker to the issuer from the filing and keeps a model ticker only if it is that issuer's other share class. What I didn't expect: the same pin measured on 8-K filings makes them worse. There the filing names the exact security, some preferred series or an ETN or a unit, while the model names the liquid ticker a reader actually follows. So 8-K rows keep the model's tags and Form 4 rows don't, decided on 90 days of our own rows rather than on which source sounds more trustworthy. The detector for it reads a different table than the one the fix writes, otherwise it would just agree with itself. The insider feed is free to poke at if you want to check your own tickers, 100 calls a day without a card: alphai.io/developers ## Similar posts on daily.dev - [I Built a Hybrid RAG Pipeline Over SEC Filings — Here’s What Broke, and Why That Was the Point](https://daily.dev/posts/i-built-a-hybrid-rag-pipeline-over-sec-filings-here-s-what-broke-and-why-that-was-the-point-nuh5kchyt) · Medium · 0 upvotes · 0 comments - [SEC What Changed: 10-K Filing Snapshot for 16 June 2026](https://daily.dev/posts/sec-what-changed-10-k-filing-snapshot-for-16-june-2026-c7vh3awvq) · R-bloggers · 0 upvotes · 0 comments - [Why Do LLMs Lie?](https://daily.dev/posts/why-do-llms-lie--ntzewhtr7) · ByteByteGo · 2 upvotes · 1 comments - [Ten Is Not a Hundred](https://daily.dev/posts/ten-is-not-a-hundred-wlrkzwmrm) · Towards Data Science · 4 upvotes · 1 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#backend](https://daily.dev/tags/backend) [View this post on daily.dev](https://daily.dev/posts/an-llm-tagged-24-sec-filings-with-the-wrong-company-and-the-right-answer-was-in-the-same-row-2vix0mny8) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"DiscussionForumPosting","mainEntityOfPage":"https://daily.dev/posts/an-llm-tagged-24-sec-filings-with-the-wrong-company-and-the-right-answer-was-in-the-same-row-2vix0mny8","headline":"An LLM tagged 24 SEC filings with the wrong company, and the right answer was in the same row","text":"An engineer describes a bug in an LLM-based pipeline that tags SEC Form 4 insider-trading filings with tickers. Because a Form 4 names both a reporting person and an issuer, the model frequently latched onto the more prominent name (the filer) rather than the issuer whose shares actually traded, mistagging 24 of 15,445 filings over 90 days, plus 81 filings that got no ticker at all and were dropped from the feed since it requires at least one ticker. Existing hallucination checks didn't catch this because they only verify a name appears somewhere in the text. The fix pins the ticker to the issuer resolved by CIK at ingest, keeping a model-suggested ticker only when it represents another share class of that same issuer. Interestingly, applying the same fix to 8-K filings made results worse, since 8-Ks name exact securities (preferred shares, ETNs, units) while the model's guess of the liquid ticker is often what readers actually want, so 8-K rows keep the model's original tags instead.","url":"https://daily.dev/posts/an-llm-tagged-24-sec-filings-with-the-wrong-company-and-the-right-answer-was-in-the-same-row-2vix0mny8","datePublished":"2026-08-17T06:13:30.905Z","dateModified":"2026-09-13T20:49:09.401Z","author":{"@type":"Person","name":"Mikhail Makeev","url":"https://daily.dev/mikhailmakeev","image":"https://lh3.googleusercontent.com/a/ACg8ocIjcvDU1gmJ9-KJDeTYqY2rMdl5QxcWYtLTGSXrj8ubF4nwar_w=s96-c","description":"Owner of alphai.io","interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"EndorseAction"},"userInteractionCount":4050}},"image":"https://media.daily.dev/image/upload/s--Dodg-2SN--/f_auto/v1789277258/posts/2vIX0MNy8?_a=BAMAMicg0","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"isPartOf":{"@type":"WebPage","url":"https://daily.dev/squads/alphai","name":"AlphAI"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"AlphAI","item":"https://daily.dev/squads/alphai"},{"@type":"ListItem","position":3,"name":"An LLM tagged 24 SEC filings with the wrong company, and the right answer was in the same row"}]} ```