Translating COBOL into Java, Kotlin, Python, Rust or C++ was almost hopeless for GPT-5.6 Sol: 5.4% accuracy. GPT-6.1 Sol hits 86.6% at low reasoning effort and 97.9% when you turn the effort up. Those are the numbers OpenAI is using to show how far its models have come on languages with little training code available, such as COBOL, Fortran and Visual Basic.
What OpenAI measured
These languages have a data problem: there is far less public COBOL than Python, yet according to IBM, as cited by OpenAI, COBOL still powers more than 40% of online banking and 95% of ATM transactions. OpenAI built two evaluations to measure progress:
| Eval | Model | Result | Previous reference |
|---|---|---|---|
| COBOL migration to C++, Java, Kotlin, Python or Rust | GPT-6.1 Sol (low effort) | 86.6% | GPT-5.6 Sol: 5.4% |
| COBOL migration | GPT-6.1 Sol (high effort) | 97.9% | — |
| MultiLangBench, average over 20 languages | GPT-6 Astra (max effort) | 87.4% of tests | GPT-5.6 Terra: 37.9% |
| MultiLangBench | GPT-6 Luna | 40.3% | GPT-5.6 Terra: 37.9% |
GPT-6 Luna barely edges out GPT-5.6 Terra, but at a much lower cost: the quality jump lives in the big model with reasoning maxed out.
How MultiLangBench works
The most interesting part is that MultiLangBench is public, Apache 2.0 licensed, and demanding by design. The model has to build a SQLite 3.40.1-compatible engine from scratch in the requested language, with the official documentation as its only help: no internet access, and every copy of SQLite scrubbed from the environment.
The tests come from SQLite's TCL suite. Of its 51,000 tests, about 24,000 could be converted into a three-step format: a starting database, a query the model's engine runs against it, and a validation query that real SQLite runs on the resulting file. If the output doesn't match exactly, or the program crashes or takes longer than 60 seconds, the test fails. Write in another language, or mix several, and the score is zero.
The score isn't a single number but a pair: API cost and test pass rate. That's a smart call, because it forces you to report what each point costs, which is exactly what separates Astra from Luna in the table.
What it means if you maintain legacy code
For a team sitting on a COBOL core, going from 5.4% to 86.6% turns AI-assisted migration from a throwaway experiment into something worth a pilot. And 97.9% with more reasoning suggests compute cost, not capability, is becoming the constraint.
The real caveat is a different one: neither eval looks much like an actual migration. MultiLangBench checks whether a model can build a database engine that is documented in minute detail, and a migration test checks equivalence against known cases. A 1980s banking batch job carries business rules nobody ever wrote down, homegrown file formats and mainframe dependencies no test captures. The bottleneck there is still having a regression suite for the original system before touching a single line, whether a person or a model does the translation.
Sources
- OpenAI says GPT-6 is much better with legacy programming languages such as COBOL and Fortran, The Decoder, October 20, 2026.
- Low-resource programming languages, OpenAI, October 2026.
- MultiLangBench, OpenAI on GitHub.