OpenAI reports stronger coding results for COBOL and Fortran

OpenAI has published new evaluation results for its GPT-6 models on coding tasks in low-resource programming languages, including COBOL, Fortran, OCaml and Visual Basic. In its MultiLangBench test, GPT-6 Astra achieved an average pass rate of 87.4% across 20 languages, compared with 26.8% for GPT-5.6 Sol. GPT-6.1 Sol also reached 86.6% accuracy on OpenAI’s COBOL migration evaluation at low reasoning effort, versus 5.4% for GPT-5.6 Sol.

The announcement addresses languages for which models have comparatively less training code and documentation than for Python or JavaScript. OpenAI notes that these languages remain important to essential services: COBOL supports more than 40% of online banking systems, 80% of in-person credit-card transactions and 95% of ATM transactions. Fortran remains used in areas including weather forecasting and nuclear record safety analysis.

Testing software construction across 20 languages

MultiLangBench asks a model to build a SQLite-compatible database engine in a specified language. The work is not limited to producing isolated snippets: the model must organise a codebase, manage state, handle errors, add features and preserve existing behaviour during an extended coding session.

To make implementations comparable, the benchmark uses database files as a common interface. A candidate program receives an initial database and a SQL query, then outputs a modified database. OpenAI checks the resulting contents against the reference SQLite implementation, using adapted tests from SQLite’s test suite.

At maximum reasoning effort, Astra improved in every language tested. The reported pass rates rose from 10.9% to 72.1% in COBOL, from 25.6% to 85.8% in OCaml, from 21.4% to 83.5% in Fortran, and from 27.5% to 89.4% in Visual Basic. Astra scored above 83% in 19 of the 20 languages, while COBOL remained the most difficult at 72%.

OpenAI also positioned GPT-6 Luna as a lower-cost alternative. Luna reached a 40.3% average pass rate at maximum reasoning effort, exceeding GPT-5.6 Sol in every tested language. Its estimated average API cost was $0.77 per sample, compared with $3.48 for GPT-5.6 Sol, or approximately 78% lower.

Measuring whether COBOL migrations preserve behaviour

The separate migration evaluation asks models to convert COBOL programs to C++, Java, Kotlin, Python or Rust while retaining their observable behaviour. OpenAI uses hidden tests requiring byte-for-byte parity in outputs, and counts a migration as successful only when every test passes.

This requirement is significant for systems whose business rules may be expressed only in source code. OpenAI highlights COBOL fixed-point arithmetic as an example: excess decimal digits are discarded unless rounding is explicitly requested. A replacement program that rounds by default might execute successfully but produce different financial results.

At low reasoning effort, GPT-6.1 Sol achieved 86.6% accuracy while using 61% of the API cost of GPT-5.6 Sol, which recorded 5.4% under the same setting. At higher reasoning effort, OpenAI reported accuracy of 97.9%, close to saturation for this evaluation.

What teams should take from the results

OpenAI is releasing the MultiLangBench setup and test suite on GitHub so developers can inspect the test coverage and language-level results. The company says the evaluations are intended to assess both new software construction and behaviour preservation in existing codebases.

For enterprises maintaining legacy applications, the reported gains may make AI-assisted investigation, remediation and migration more practical. They do not remove the need for expert review: OpenAI explicitly says production work should be tested against the system’s business rules and dependencies. The practical implication is to use models alongside controlled builds, regression tests and domain validation rather than treating a generated migration as production-ready code.