--- title: Beyond Text Splitting: Building a Chunking Strategy... description: A C# External Logic library for OutSystems Developer Cloud (ODC) that provides four document chunking strategies for RAG pipelines: ChunkByCharacter,... canonical: https://daily.dev/posts/beyond-text-splitting-building-a-chunking-strategy-library-for-odc-nlpaupgtr twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Beyond Text Splitting: Building a Chunking Strategy Library for ODC | daily.dev og:description: A C# External Logic library for OutSystems Developer Cloud (ODC) that provides four document chunking strategies for RAG pipelines: ChunkByCharacter,... og:url: https://daily.dev/posts/beyond-text-splitting-building-a-chunking-strategy-library-for-odc-nlpaupgtr og:image: https://api.daily.dev/og/posts/NLpauPGtR.png og:image:alt: Beyond Text Splitting: Building a Chunking Strategy Library for ODC og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Beyond Text Splitting: Building a Chunking Strategy Library for ODC **[ITNEXT](https://daily.dev/sources/itnext)** · 12 min read · 0 upvotes · 0 comments ## Summary A C# External Logic library for OutSystems Developer Cloud (ODC) that provides four document chunking strategies for RAG pipelines: ChunkByCharacter, ChunkRecursively, ChunkBySentence, and ChunkMarkdown. The library addresses the core problem that poor chunking of structured documents (code blocks, tables, nested headings) degrades retrieval quality before embeddings are even computed. ChunkMarkdown preserves heading hierarchy as breadcrumb context in each chunk, while all strategies protect code blocks and tables as atomic units. The post also recommends standardizing on Markdown as the intermediate format for document ingestion pipelines, with upstream ODC Forge components available for DOCX/PPTX and PDF conversion. The library has 143 tests and is available on the OutSystems Forge and GitHub. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How do I prevent code blocks and Markdown tables from being split across chunks in a RAG pipeline? Fenced code blocks and Markdown tables should be replaced with placeholder tokens before any splitting cascade runs, then restored whole after splitting completes. If a restored block produces an oversized chunk, atomicity takes priority over the size limit. This approach is used in both ChunkRecursively and ChunkBySentence strategies, ensuring neither code blocks nor table rows are fragmented across chunk boundaries. _Developers building document ingestion pipelines track chunking patterns like this on daily.dev._ ### Why does my RAG system return 'I cannot find the answer' even when the document contains the right information? Poor chunking is a common cause. When a chunk is split mid-sentence, mid-function, or mid-table, the resulting vector is a fragment whose similarity score falls below the retrieval threshold. The vector store faithfully returns broken context — it cannot repair a chunk that lost its heading, table structure, or code block boundary. The problem must be solved at the chunking step, before embedding or retrieval runs. _Teams debugging RAG retrieval misses find relevant chunking strategies and discussions on daily.dev._ ### Should I convert PDFs to Markdown before chunking for a RAG pipeline? Yes. Converting PDFs to Markdown before chunking produces significantly better results. Processing the same document as Markdown versus plain text yields populated heading paths, preserved code blocks, and intact tables in the Markdown version, versus none of those in the plain text version. Weaviate and LangChain both recommend this approach; LangChain's MarkdownHeaderTextSplitter carries the full header path as metadata into every chunk. _Developers choosing between PDF extraction approaches for RAG compare options on daily.dev._ ## Similar posts on daily.dev - [Beyond Structure: Semantic Chunking Inside ODC](https://daily.dev/posts/beyond-structure-semantic-chunking-inside-odc-8jkbjujyq) · ITNEXT · 0 upvotes · 0 comments - [Your Chunks Failed Your RAG in Production](https://daily.dev/posts/your-chunks-failed-your-rag-in-production-xkq4ggftq) · Towards Data Science · 6 upvotes · 0 comments - [RAG Chunking Strategies 2026: Production Patterns That Work](https://daily.dev/posts/rag-chunking-strategies-2026-production-patterns-that-work-dmnsqdzyl) · Alex CloudStar · 1 upvotes · 0 comments - [Essential Chunking Techniques for Building Better LLM Applications](https://daily.dev/posts/essential-chunking-techniques-for-building-better-llm-applications-hr2idjqb4) · Machine Learning Mastery · 2 upvotes · 0 comments - [We Stopped Splitting Code Like Text: Code Aware Chunking Unlocked Better RAG](https://daily.dev/posts/we-stopped-splitting-code-like-text-code-aware-chunking-unlocked-better-rag-ihd6ilnbv) · Trendyol Tech · 1 upvotes · 0 comments --- Tags: [#rag](https://daily.dev/tags/rag), [#vector-search](https://daily.dev/tags/vector-search), [#markdown](https://daily.dev/tags/markdown) [View this post on daily.dev](https://daily.dev/posts/beyond-text-splitting-building-a-chunking-strategy-library-for-odc-nlpaupgtr) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Beyond Text Splitting: Building a Chunking Strategy Library for ODC","url":"https://daily.dev/posts/beyond-text-splitting-building-a-chunking-strategy-library-for-odc-nlpaupgtr","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/beyond-text-splitting-building-a-chunking-strategy-library-for-odc-nlpaupgtr"},"datePublished":"2026-08-07T08:06:47.482Z","dateModified":"2026-08-07T08:07:21.603Z","description":"A C# External Logic library for OutSystems Developer Cloud (ODC) that provides four document chunking strategies for RAG pipelines: ChunkByCharacter,...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/e19f8fb456ea4619e6ac225b4233b9b4?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/e19f8fb456ea4619e6ac225b4233b9b4?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"ITNEXT","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"ITNEXT","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/itnext2","url":"https://daily.dev/sources/itnext"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/beyond-text-splitting-building-a-chunking-strategy-library-for-odc-nlpaupgtr","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"rag,vector-search,markdown","timeRequired":"PT12M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"ITNEXT","item":"https://daily.dev/sources/itnext"},{"@type":"ListItem","position":3,"name":"Beyond Text Splitting: Building a Chunking Strategy Library for ODC"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/beyond-text-splitting-building-a-chunking-strategy-library-for-odc-nlpaupgtr#faq","mainEntity":[{"@type":"Question","name":"How do I prevent code blocks and Markdown tables from being split across chunks in a RAG pipeline?","acceptedAnswer":{"@type":"Answer","text":"Fenced code blocks and Markdown tables should be replaced with placeholder tokens before any splitting cascade runs, then restored whole after splitting completes. If a restored block produces an oversized chunk, atomicity takes priority over the size limit. This approach is used in both ChunkRecursively and ChunkBySentence strategies, ensuring neither code blocks nor table rows are fragmented across chunk boundaries. Developers building document ingestion pipelines track chunking patterns like this on daily.dev."}},{"@type":"Question","name":"Why does my RAG system return 'I cannot find the answer' even when the document contains the right information?","acceptedAnswer":{"@type":"Answer","text":"Poor chunking is a common cause. When a chunk is split mid-sentence, mid-function, or mid-table, the resulting vector is a fragment whose similarity score falls below the retrieval threshold. The vector store faithfully returns broken context — it cannot repair a chunk that lost its heading, table structure, or code block boundary. The problem must be solved at the chunking step, before embedding or retrieval runs. Teams debugging RAG retrieval misses find relevant chunking strategies and discussions on daily.dev."}},{"@type":"Question","name":"Should I convert PDFs to Markdown before chunking for a RAG pipeline?","acceptedAnswer":{"@type":"Answer","text":"Yes. Converting PDFs to Markdown before chunking produces significantly better results. Processing the same document as Markdown versus plain text yields populated heading paths, preserved code blocks, and intact tables in the Markdown version, versus none of those in the plain text version. Weaviate and LangChain both recommend this approach; LangChain's MarkdownHeaderTextSplitter carries the full header path as metadata into every chunk. Developers choosing between PDF extraction approaches for RAG compare options on daily.dev."}}]} ```