Model Comparisons

My hydropower benchmark revealed the differences between agentic retrieval-augmented generation and plain hybrid retrieval-augmented generation.

Agentic RAG for hydropower, run locally on Ollama, raised retrieval precision but added latency, with mixed faithfulness versus plain RAG.

My hydropower benchmark revealed the differences between agentic retrieval-augmented generation and plain hybrid retrieval-augmented generation.

Key Takeaways

  • The project builds a local, open-weight Agentic RAG system for small hydropower engineering using Ollama, avoiding paid inference APIs for document processing, embeddings, retrieval, generation, and runtime checks.
  • Benchmark results are mixed: agentic retrieval improves precision, but latency rises substantially and faithfulness does not automatically exceed a well-designed plain RAG pipeline.
  • Conventional RAG struggles with hydropower questions that span multiple manuals, tables, calculations, or engineering recommendations, while agentic workflows attempt to assess context sufficiency, use tools, and flag unsupported claims.
  • Mixed-format hydropower documents require a multi-tier parsing pipeline for Office files, PDFs, scanned pages, tables, and images, with metadata tracking and caching, because poor parsing directly degrades downstream retrieval and generation.
  • Chunking uses a rule-based Markdown-aware hierarchical pipeline that leverages headings, paragraphs, sentence boundaries, and conditional overlap rather than LLM or embedding-based semantic segmentation.

A developer has built an Agentic RAG system for Small Hydropower Plant (SHP) engineering that runs entirely on local, open-weight models through Ollama, with no paid inference APIs required for document processing, embeddings, retrieval, answer generation, or runtime quality checks. The project tests whether a multi-agent retrieval workflow can improve answers drawn from private technical documents. The early verdict is mixed: retrieval precision improves, but latency rises substantially, and the system does not automatically beat a well-designed plain RAG pipeline on faithfulness.

My hydropower benchmark revealed the differences between agentic retrieval-augmented generation and plain hybrid retriev

The work targets learners and independent students who cannot absorb recurring API bills. Anyone with compatible hardware can download the models and dependencies once, then experiment without paying per question or per token. For offline benchmarking, the author used Gemini and Groq free tiers to score answers against a golden dataset; a local Ollama judge can substitute. Free-tier quotas and availability can change, so they are not an unlimited resource. Hardware, electricity, and inference time still carry cost.

Why standard RAG struggles with engineering documents

A conventional pipeline retrieves a few documents, sends them to an LLM, and generates an answer. That suffices for simple lookup questions. It weakens when a question spans multiple manuals, requires numerical reasoning, contains tables, or asks for an engineering recommendation grounded in plant documentation.

SHP projects mix technical manuals, regulatory guidelines, turbine-selection references, hydrology documents, financial policies, scanned reports, tables, and drawings. A single question may need several kinds of evidence at once: which turbine suits a given head and discharge, which regulatory requirements apply, which maintenance practices reduce silt erosion, what the power output is from provided values, and whether the answer is actually supported by retrieved documents. Traditional RAG can retrieve passages but has no mechanism to judge whether context is sufficient, whether a calculation needs a tool, or whether the generated answer contains unsupported claims. The agentic approach targets exactly that gap.

Parsing: turning mixed formats into usable evidence

Hydropower knowledge arrives as digital PDFs, scanned reports, Word documents, presentations, spreadsheets, and images. The project uses a multi-tier parsing pipeline that converts files to Markdown before chunking and embedding.

Tier 1 handles Office documents and text with dedicated parsers, avoiding expensive OCR for files that already have accessible structure. Tier 2 sends PDFs through pymupdf4llm to extract Markdown while aiming to retain headings and table structure. Output of at least 80 characters is accepted; shorter output or failed extraction falls through to Docling. That threshold is a practical routing heuristic with a known limitation: a PDF mixing digital text and scanned pages may pass even when some pages need OCR. Page-level quality checks would improve it.

My hydropower benchmark revealed the differences between agentic retrieval-augmented generation and plain hybrid retriev

Tier 3 processes scanned PDFs and failed extractions with Docling, enabling OCR and table-structure recovery when the required backend is available. Images also enter the Docling path. The PDF OCR configuration prefers EasyOCR with RapidOCR as an alternative, using CUDA acceleration when available and falling back to CPU otherwise. Complex drawings, dense tables, and formulas remain difficult, so extracted Markdown should not be assumed to reproduce every detail correctly.

Each parse records source information: filename, file type, content hash, parsing duration, and page count where available. Failures are recorded explicitly. The content hash supports caching, so an unchanged document reuses its existing Markdown output. One caveat: binary SHA-256 deduplication prevents wasting heavy parsing and OCR compute on identical files, but for the same converted file, binary deduplication does not apply—the system treats them as two completely separate files.

A real metadata record shows the shape of this output: source path `D:\GenAi_Code\Agentic_RAG\data\raw\AK - Hydrology of Ungauged Catchment.pdf`, file type PDF, file hash `127653ce9e3151fd6732fd83bfc66e570ab762bdafb22d568fcef524ce0d19bc`, page count 0, parse duration 0.0 seconds, and no error. Parsing quality directly affects everything downstream: if a table loses its units or OCR misreads a formula, retrieval and generation work with incorrect evidence.

Chunking: a four-pass, rule-based pipeline

The Markdown-aware hierarchical chunker uses document structure and text length for boundaries rather than an LLM or embedding-based semantic segmentation model. It runs four passes.

Pass 1 scans the whole document and divides it into structural sections, walking line by line and tracking headings (#, ##, ###, ####) like an outline. A heading or horizontal divider (---) closes the current section. When a heading includes a label such as "Slide 12" or "Page 8," that location is captured as metadata—though page references depend on what parsing preserved and are not automatically available for every chunk.

Pass 2 splits sections into target-sized chunks of 1,800 characters, dividing long text at paragraph boundaries and oversized paragraphs at sentence boundaries (., !, ?). Overlap is conditional rather than a fixed sliding window: the previous paragraph is carried over only if it is 150 characters or fewer. The defaults are `max_chars = 1800` and `overlap_chars = 150`, and the 1,800-character setting is a target, not a strict maximum—long individual sentences, table rows, fragment merging, and added source metadata can produce larger chunks.

My hydropower benchmark revealed the differences between agentic retrieval-augmented generation and plain hybrid retriev

A worked example clarifies the logic. A section has four paragraphs: 900 characters on Francis turbine runner design, 800 on guide vanes and water flow, 85 stating "Cavitation occurs when local water pressure drops below vapor pressure," and 600 on preventative coatings and blade maintenance. Chunk 1 takes the first three paragraphs at 1,785 characters. Adding the fourth would reach 2,385, exceeding 1,800, so Chunk 1 is saved. Because the last paragraph is 85 characters—within the 150 allowance—it carries over as a natural bridge, and Chunk 2 opens with the cavitation sentence before the coatings text. Had that paragraph been 600 characters, repeating it would waste a third of Chunk 2's capacity, so the condition fails and Chunk 2 starts cleanly with the fourth paragraph alone.

Tables get separate treatment. The chunker detects Markdown tables (`| col1 | col2 |`), slices large tables row by row, and repeats the header and divider row at the top of every slice so numbers retain their meaning. A 40-row table exceeding 1,800 characters becomes smaller tables that each carry column names. Slices are tagged `is_table = True` so the agent can run table-specific filtered searches.

Pass 3 merges tiny leftover fragments under 150 characters. Pass 4 enriches every chunk by injecting a citation banner and computing token counts. The banner addresses a real failure mode: an isolated chunk reading "The operating head is 45 meters" gives an embedding model or LLM no idea which plant or turbine it applies to. The injected context looks like `[Source: Turbine_Manual.pdf | Location: Slide/Page 12 | Section: Turbine Manual > Selection Guidelines > Francis Turbines]`, placed directly above the text. Final chunks are written to `chunks.jsonl`.

What the evaluation shows and what comes next

The pipeline's full implementation is published as Agentic RAG for Small Hydropower. Based on evaluation so far, the multi-agent workflow improves retrieval precision but carries a substantial latency cost and does not automatically outperform a well-designed plain RAG pipeline on faithfulness. That nuance matters for anyone deciding whether to adopt agents: better retrieval does not guarantee better grounded answers, and the runtime quality checks built into the system exist precisely because unsupported claims remain a risk.

The project's broader value is as a learning scaffold. It covers document ingestion, hybrid retrieval, agent orchestration, tool use, and evaluation on a limited budget—skills that transfer well beyond hydropower. The open questions are concrete: page-level parsing quality checks to catch mixed digital-scanned PDFs, and closing the faithfulness gap without adding latency. For readers building private-document RAG, the takeaway is to invest first in parsing and chunking fidelity, then measure whether agentic orchestration earns its cost.

Comments (0)

  1. No comments yet. Be the first to share what worked for you.

Leave a comment

Comments are reviewed before they appear. Your email address is not published.