Unlock the hidden treasure in your factory by rescuing dark data without spending thousands of dollars.

Meta description: A manufacturing data test used Llama 3.1 and Ollama to recover dark data from PDF reports, reaching 99.6 percent data retrieval on a standard local computer.

Tags: Dark Data, Local AI, Ollama, Manufacturing, PDF extraction

Featured image: abstract friendly illustration of factory PDF reports turning into structured rows, no people, no logos, no text; the source image was credited to GPT 5.6 luna.

For manufacturing teams that cannot justify thousands of dollars to upgrade legacy equipment, a local AI demonstration for dark data used Llama 3.1 (8B) and Ollama on a standard personal computer to process 20 fictional PDF reports. The test reported a 99.6% recovery rate; the 0.4% failure came from an intentionally omitted end date that the model had to estimate, while 100% of the data actually contained inside the reports was retrieved.

Dark Data Accumulates in Legacy Factory Reports

Manufacturing plants often operate a mix of modern equipment connected to Internet of Things ecosystems and robust legacy machines that have run for years while remaining isolated from company systems. When improvement, automation or monitoring plans are designed, those veteran machines are often left out of the budget because updating proprietary software through the original manufacturer can cost thousands of dollars. Their visibility inside the technology stack may be low, but their effect on production remains high.

Technology consultancies call this problem dark data. Estimates cited in the article put unanalyzed or unusable enterprise data between 60% and 73%. A 2020 IDC survey of 1,500 business leaders reported that 68% of available enterprise data goes unutilized. Seagate, Forrester and Forbes have discussed the issue, and many vendors offer reliable but expensive solutions.

The trapped material is concrete. Reports include average batch values, which matter for product quality over time; working parameters, which can reveal component wear; processing times, alarms and stops, which are fundamental for calculating real productivity. If proprietary software can export CSV before a PDF is printed, a Python script can process the data almost for free. In practice, the standard output is often a PDF that production and quality teams review, then archive in a folder.

PDF Layouts Defeat Traditional Extraction

Veteran equipment usually runs reliable software inside a closed environment, but the cost of connectivity upgrades and the downtime they require keep the machines separated from the rest of the plant. The PDFs also change shape depending on batch duration, number of alarms and operator modifications. Traditional extraction tools, including standard OCR, struggle when tables shift, pages break unexpectedly, text lengths change or values differ from what the parser expects.

The result is familiar: thousands of reports on local servers and computers, with no practical way to analyze them manually. The data is not missing; it is hidden inside documents that were never designed for automated reading.

Local Llama 3.1 Turns PDFs Into Structured Data

The reported solution was a Python script backed by an open-source large language model running 100% locally and privately on a standard computer. The test used Llama 3.1 (8B) through Ollama, so factory data did not leave the plant. The script recovered report values and exported them daily to an updated Excel sheet. The article recommends placing PDFs in a folder or computer outside the equipment so extraction does not risk production failures.

To create a worst-case scenario, the author generated 20 fake unstructured PDF reports with different layouts, abbreviations, mixed units and semantic traps such as compression pressure versus pre-compression pressure. The figures in the article showed fictional reports and the resulting Excel output, whose headers were in Spanish from the local test.

Four implementation choices made the extraction possible.

First, the script connected to a local AI instead of paid cloud APIs. It used the OpenAI Python library pointed at an Ollama server at http://localhost:11434/v1. Ollama can run without a GPU when lightweight models are used, and all tests ran on a standard CPU laptop.

Second, the script preserved spatial relationships. pypdf2 worked well for text-only reports but created problems with tables. pdfplumber with layout=True kept the physical spacing of the document, allowing the model to understand columns and rows.

Third, the script forced strict JSON output. Setting temperature=0.0 reduced fabricated data, and using a json_object response format required the result to follow the requested structure. Missing fields were returned as "Not found".

Fourth, the prompt was made very specific. The example prompt was for a pharmaceutical tablet compression report and asked for batch, start date and time, end date and time, average weight as a numeric value in mg with no units, compression pressure as a numeric value in kN with no units, conforming units in a fixed format such as 1958600, and alarms as literal text. The prompt also warned about format differences such as 250000 versus 250.000, because those can break file output.

The script processed the 20 PDFs with a 99.6% recovery rate. The 0.4% failure occurred when an end date was intentionally omitted, forcing the AI to estimate it. The article states that 100% of the data inside the reports was retrieved. One spreadsheet value highlighted in green did not appear in the report and was incorrectly inferred. The alarm list was complete, although the Excel output format did not make that obvious.

Structured Rows Unlock Overall Equipment Effectiveness, Quality and Maintenance Trends

By converting PDFs into Excel or SQL structured formats, the test points to three Industry 4.0 capabilities that could be reached with very little hardware investment.

The first is real Overall Equipment Effectiveness visibility. Extracting exact batch durations, real machine speeds, stops and conforming units from reports removes the bias of manual data entry. Overall Equipment Effectiveness can then be analyzed at the end of a batch or day on machines that usually lack real-time measurement, turning it from an estimate into a calculated metric.

The second is long-term quality control. Monitoring the trend of critical values across hundreds of batches can reveal deviations and prevent process drift. The test spreadsheet calculated batch averages, parameters and Overall Equipment Effectiveness from process time and conforming units. The source notes that the positive and negative trends in the charts came from deliberately generated test data, not real production patterns.

The third is data-driven preventive maintenance. The real value lies in trends in working parameters. A slight increase in motor workload or pressure can reveal component wear, allowing maintenance teams to act before a catastrophic stop in production.

The larger lesson is not that old machines must be replaced. Sometimes the most practical industrial upgrade is to read what those machines have already been writing in PDF format for years.

Next Steps Aim at Scheduled Extraction and AI Summaries

The article identifies two next phases. The first is automation. Most factory equipment runs on Windows, so the proposed method is a .bat file that starts Ollama, waits a few seconds for it to load into RAM, runs the .py script, then closes Ollama to avoid wasting computer resources. The .bat file can be scheduled through Task Scheduler by creating a basic task, choosing a daily time and selecting Start a program with the .bat file as the action. To avoid reprocessing the same files, the script can track processed reports in a .txt file and check each stored PDF against that list.

The second phase is reporting. The article recalls a remark from a discussion about dark data: "If nobody has time to recover the data, nobody will have time to study it." That comment pointed toward a new goal: complete automation of extraction plus an AI that generates a brief report after the data is aggregated. Before summarization, raw extracted data would need to pass through calculations such as means, sums and other calculations, then be given to the AI for a concise daily or batch-level review.