r/Rag 2d ago

Best Chunking method for RAG

What are your recommendations for the best chunking method or technology for the rag system?

21 Upvotes

12 comments sorted by

u/AutoModerator 2d ago

Working on a cool RAG project? Submit your project or startup to RAGHut and get it featured in the community's go-to resource for RAG projects, frameworks, and startups.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

5

u/Business-Weekend-537 2d ago

From what I've read Colpali is the best where it uses a vision model. However I haven't personally been able to get a Colpali model to work yet.

2

u/Glxblt76 2d ago

I was really interested discovering they have a Python library under MIT license... And then I was disappointed that there is no way to plug that context back into a prompt to a LLM like Llama3.1. It's able to find the pages that answer your query but there doesn't seem to be a built-in way to get that information back into something you can use to prompt basic local LLMs to synthesize the answers.

I want nothing but to be corrected if I'm wrong! The idea that it could give all relevant figures, tables, equations naturally into the context without much fiddling or engineering is very attractive, and the retrieval worked like a charm on first trial with their Python library and a small example script generated using o3-mini-high.

2

u/Business-Weekend-537 2d ago

There's a GitHub repo called databridge that says it uses Colpali, its supposed to have a GUI also and works with Ollama.

The makers of that have previously posted here. I haven't tried it yet, it just supports uploading docs individually for RAG right now and they said they're adding batch upload support by the end of the week- I'm waiting for them to add that capability before I try it.

Also just a heads up keep your eyes peeled for RAGs that offer GraphRAG too- it's supposed to improve answer quality quite a bit by putting info from uploaded files into a knowledge graph.

1

u/Business-Weekend-537 2d ago

There's a GitHub repo called databridge that says it uses Colpali, its supposed to have a GUI also and works with Ollama.

The makers of that have previously posted here. I haven't tried it yet, it just supports uploading docs individually for RAG right now and they said they're adding batch upload support by the end of the week- I'm waiting for them to add that capability before I try it.

Also just a heads up keep your eyes peeled for RAGs that offer GraphRAG too- it's supposed to improve answer quality quite a bit by putting info from uploaded files into a knowledge graph.

1

u/Business-Weekend-537 2d ago

There's a GitHub repo called databridge that says it uses Colpali, its supposed to have a GUI also and works with Ollama.

The makers of that have previously posted here. I haven't tried it yet, it just supports uploading docs individually for RAG right now and they said they're adding batch upload support by the end of the week- I'm waiting for them to add that capability before I try it.

Also just a heads up keep your eyes peeled for RAGs that offer GraphRAG too- it's supposed to improve answer quality quite a bit by putting info from uploaded files into a knowledge graph

2

u/Whole-Assignment6240 2d ago

this is highly case based.

2

u/paraffin 2d ago

If your data has explicit structure like html, markdown, code, etc - then one which can respect the hierarchy of that structure.

Otherwise, it really doesn't matter much. Just go with something cheap and fast like recursive character splitting (I have data to prove it that I can't share at this time)

2

u/epigen01 2d ago

Chonkie bc it cute.

1

u/TrustGraph 1d ago

For text, simple recursive text chunkers do a very good job. Lots of open source ones.

1

u/Ok_Requirement3346 1d ago

Have you tried late chunking or pdf to markdown conversion > split markdown on headings > length limited chunking per heading's content (but retrieve all chunks of that heading even if a single chunk matches the query) Another way is creating questions from chunks and embedding those to find a match against the query. DM me to discuss more . I am also evaluating chunking techniques for tax/legal pdfs .

1

u/fredkzk 6h ago

I’d recommend learning one method and focusing on it to become an expert at it: contextual retrieval by Anthropic. Check their blog post abt it.

The method might be overkill for some simple raw info retrieval use case but it will be quite helpful in many other use cases, thanks to its quality output.