A podcast conversation with Suneet Malhotra, Senior Manager of Test Engineering at Motorola Solutions, explores building an end-to-end agentic software development lifecycle using multiple coordinated agents (MCPs). Topics include using Cohen's kappa to statistically evaluate multiple LLMs acting as judges, and improving requirements quality by adding a specification enrichment stage right after the design phase to shift QA left. A five-agent SDLC implementation and companion code are shared on GitHub.
Questions this post answers
What is Cohen's kappa used for when evaluating multiple LLMs as judges?
Cohen's kappa is used as a statistical measure of agreement between multiple large language models acting as judges, helping teams determine whether different LLM evaluators are consistently rating outputs the same way rather than diverging. This approach was applied within a five-agent agentic software development lifecycle built to automate parts of test engineering. Developers building llm-as-judge evaluation pipelines can track emerging practices like this on daily.dev.
What does shifting QA left mean in an agentic SDLC with a specification enrichment stage?
Shifting QA left in this context means adding a specification enrichment stage immediately after the design phase, so requirements are clarified and strengthened by an agent before development begins, catching ambiguity earlier rather than discovering it during testing. This was part of a five-agent agentic SDLC pipeline built for test engineering at Motorola Solutions. Teams designing agentic QA workflows can follow practical approaches like this on daily.dev.