Researchers at LG CNS have introduced KOPA-Bench, a new benchmark of 145 real-world tasks that test whether LLM agents can chain multiple tool calls across live Korean government APIs, alongside EDGE, a data-synthesis pipeline designed to close the performance gap they measured. The work was submitted to arXiv on 4 September 2026 and accepted to the EMNLP 2026 Industry Track.
The results are early-stage research, not a shipping product. But the headline finding is notable: a 9B open-source model fine-tuned on EDGE-generated data nearly matches the untuned 27B model from the same family, with gains that also carry over to the out-of-distribution BFCL benchmark.
Why Korean public APIs are a hard testbed
Data-sovereignty regulations increasingly push public institutions toward open-source, on-premise models rather than commercial cloud services. Those models must operate over the laws, corporate disclosures, and transportation data that agencies expose through open APIs — a setting where, the authors write, open-source models consistently underperform.
The paper identifies two recurring failure patterns. First, frequent entity code-lookups turn simple questions into dependent chains, where one tool's output must feed the next tool's input. Second, many endpoints return high-cardinality responses — long, multi-record results that require fan-out across pages or reduction before an answer is possible. Untuned models tend to skip prerequisite lookups or answer prematurely from the first page of a paginated response.
KOPA-Bench encodes exactly these constraints: 145 multi-step tasks spanning 10 platforms and six domains of live Korean public APIs, where every task requires chaining dependent calls and many involve high-cardinality intermediate results.
How EDGE synthesizes training data
EDGE — Execution-grounded Dynamic Graph for tool-calling data synthesis — starts by building a graph of how each tool's output could feed another tool's input. The key step is verification: each LLM-proposed dependency is actually executed against the live APIs, and links that repeatedly fail are pruned, keeping the graph dynamic as endpoints change.
From the verified graph, EDGE synthesizes executable multi-step trajectories. Each output-to-input handoff is labeled with the number of distinct returned values, which lets the pipeline convert high-cardinality responses into valid sequential trajectories through bounded fan-out or deterministic reduction. The resulting dataset is used for GRPO fine-tuning.
Results
Trained on EDGE-generated data, small Qwen3.5 models improved substantially: pass@1 on KOPA-Bench rose by 13 percentage points for the 4B model and 10 points for the 9B model. The 9B model nearly matched the untuned 27B model from the same family, and the gains extended to BFCL, an out-of-distribution tool-calling benchmark.
The paper is 30 pages with 26 tables, including ablations on the training objective, trajectory diversification, data filtering, and the execution-grounded graph itself, plus a data-contamination audit with a held-out platform evaluation. Code and data are available on GitHub.
Limitations
The authors flag several scope constraints. The pipeline relies on live endpoints, which can change or fail; the benchmark covers Korean public APIs specifically, and generalization to other national API ecosystems is future work; and the model comparison is limited to one open-source family. The work is research-stage — KOPA-Bench measures a real gap, but no production deployment is claimed.
Our take
The more transferable idea here is not the benchmark but the verification loop: letting live execution, not LLM judgment, decide which tool dependencies are real. Most synthetic tool-calling datasets trust the generator's plausibility; EDGE prunes by ground truth from the API itself. If that recipe holds up beyond Korean government endpoints, it gives any institution with a stable API catalog a way to manufacture training data that matches its own failure modes — which is precisely what data-sovereignty rules demand and generic datasets cannot supply.
