Bounding box annotation is the foundational step in building document AI models, yet its technical mechanics are poorly understood outside specialist circles. This deep-dive covers coordinate systems (COCO, YOLO, LayoutLM's 0–1000 normalized format), the structure of fully annotated document JSON output including confidence scores, reading order, and hierarchical relationships, and how models like LayoutLMv3 consume this data. Key topics include Intersection over Union (IoU) as the primary quality metric, document-specific annotation challenges (multi-line blocks, tables, overlapping regions, multi-page coordinate normalization), PyTorch and HuggingFace pipeline integration, quality control practices like inter-annotator agreement and confidence thresholding, and how annotation errors directly degrade model mAP. Auto-labeling workflows and dataset design decisions (taxonomy granularity, DPI, source diversity) are also addressed.
