A practical guide to building LLM evaluation pipelines without complex infrastructure. Inspired by G-EVAL and Self-Rewarding Language Models research, the approach uses additive scoring, chain-of-thought reasoning steps, few-shot examples, and structured JSON output to create an LLM-as-a-Judge setup. A worked example evaluates a RAG application on NVIDIA SEC filings using Meta-Llama-3-70B-Instruct via the Hugging Face Inference API and the AsyncOpenAI client. Key limitations discussed include LLM bias toward AI-generated text, prompt sensitivity, and context window constraints.
