Enterprise AI Bootcamp Demo 1

Rebuild the ingestion pipeline

Three changes, applied together because they are one change: layout-aware parsing instead of a text dump, parent-child chunking instead of fixed windows, and product family, OEM, model code and revision date carried on every chunk instead of discarded.

Naive chunks
5,434
1,000-character windows
Improved chunks
12,590
750-character children of 7,734 section parents
Tables recovered
411
kept as a grid, never split
Pages
3,146
across 25 documents

Run it

This spawns scripts/build_index.py as a subprocess on three documents and streams its progress. It is the same script that built the full index; the only argument that differs is the document list.






What actually changed in the chunk
The naive chunk below is a thousand characters of a page. The improved chunk is a table kept
whole, carrying the heading path that says which section it came from. Retrieval scores the
child; generation reads the parent section. That single heading is what separates the
reciprocating interval from the rotary-screw interval two pages away.




  
A naive chunk

    
tm-9-4310-241-15:n0 · page 1
    · no section, no family, no revision

    
SR a ea TR Gy Se AR tT ST EAE CR AL LRN IRIE HELE Sa a eee 
.43)/0-2y]-/S— 
TM 9-4310-241-15 
DEPARTMENT OF THE ARMY TECHNICAL MANUAL 
OPERATOR, ORGANIZATIONAL, FIELD, 
AND DEPOT MAINTENANCE MANUAL 
COMPRESSOR, RECIPROCATING: AIR: 5 CFM: 175 PSI: 
HAND TRUCK MOUNTED: GASOLINE DRIVEN (CHAMPION 
PNEUMATIC MODEL LP-512-ENG) LESS ENGINE 
FSN 4310-861 -9820 
RECEIVER MOUNTED; ELECTRIC DRIVEN (CHAMPION 
PNEUMATIC MODEL OFH-34-60-ENG-1) 
FSN 4310-861 -9823 
This copy is a reprint which includes current 
pages from Changes 1 through 4. 
} HEADQUARTERS, DEPARTMENT OF THE ARMY 
JANUARY 1963
eee 
SAFETY PRECAUTIONS f 
Before Operation 
When handling gasoline, always provide a metal-to-metal contact between the container and 
tank. This will prevent a spark from being generated as gasoline flows over the metallic surface. 
When operating a compressor with an electric motor, de-energize the input power leads at 
their source before changing connections or performing maintenanc

  

  
An improved chunk

    
tm-9-4310-241-15:c60 · page 15
    · section MODEL OEH-34-60-ENG-2). › AGO 7889A · kind table

    
MODEL OEH-34-60-ENG-2). › AGO 7889A
LAMBERT, |  |  | 
LAMBERT,: General, Un; ited States Army,
LAMBERT,: djutant Gene; ral.
LAMBERT,: ion :
LAMBERT,: e Army:; USAOSA (2) AMS (3)
LAMBERT,: USASA (2) DCSLOG (1); Dist Engr (2) Div Engr (2)
LAMBERT,: CNGB (1); Engr Fld Maint Sh; ops
LAMBERT,: ACSI (1) CSigO (1); USAERDL (3) Engr Cen (5)
LAMBERT,: CofT (1) TSG (1) CofEngrs (3) USA Maint Bd; (1); ESCO (10) Chicago Proc Ofc USA Mbl Spt Cen; (10) (36)
LAMBERT,: USAARTYBD USAARMBD USAIB (2) USARADBD USAAESWBD; (2) (2) (2) (2); Fid Comd, DASA USACOMZEUR ( USAREUR Engr USAREUR Engr USA Corps (1) MAAG (1); (8) 2) Sup Proc
LAMBERT,: USAAVNBD USCONARC OS Maj Comd USARJ (10 USAMOCOM; (2) (3) (5) except ) (2); JBUSMC (1) Units org under fo 3-500 (EA) (2) 5-48 (2); l TO
LAMBERT,: USASMCOM; (1); 5-237 (5)
LAMBERT,: MDW (1) Armies (2); 5-262 (5) 5-267 (1)
LAMBERT,: Corps (2) Div (2); 5-278 (5) 5-279 (2)
LAMBERT,: Engr Bde (1); 10-407 (2)
LAMBERT,: Engr Sec, GE; NDEP (10); 10-500 (HH, HI; ) (
LAMBERT,: Engr Dep (OS; ) (10); 29-51 (2)
LAMBERT,: Army Dep (2; ); 29-52 (1)
LAMBERT,: Sve Colleges; (2); 29-56 (2)
LAMBERT,: Br Sve Sch (2; ) except; 29-57 (2)
LAMBERT,: USAES (2; 0); 29-105 (2)
LAMBERT,: USMA (2); 29-107 (2)
LAMBERT,: USA Tml Com; d (2); 57-100 (2)
LAMBERT,: Army Tm! (1; )
LAMBERT,: State AG (3; ).
LAMBERT,: AR: Same as; Active Army except allowance is one copy to; each unit.
LAMBERT,: explanation o; f abbreviations used, see AR 320-50.

  





Index built by scripts/build_index.py at 2026-08-10 01:44:34 in 711s: 25 documents, 3,146 pages, 12,590 improved chunks / 5,434 naive chunks. Embeddings: Snowflake/snowflake-arctic-embed-s (int8 ONNX, Apache-2.0). Reranker: cross-encoder/ms-marco-MiniLM-L-6-v2 (ONNX, Apache-2.0).