freenode
Databases & Infrastructure

Postgres pytest rewrite draws speed gains and cleanup doubts

A second proposal to replace Perl TAP tests with Python is faster in early ports, but reviewers want leftover servers, coverage gaps, and CI noise fixed before any migration.

PostgreSQL may get a pytest framework for writing tests that start servers and run queries, but the second attempt from Jelte Fennema-Nio is still a proposal, not a migration. Fennema-Nio reworked the design after using a language model to convert about half of the existing Perl tests, which exposed awkward APIs. Sample conversions are meant only to show the framework. They are not proposed for commit until a separate discussion decides whether, and how, to move the suite.

The pitch is practical. Ports of a handful of tests were shorter, easier to write, and faster. Andres Freund, author of one of the Perl tests Fennema-Nio singled out as unreadable, said he hated writing that test. A first review by newcomer Pavlo Sukhanov, on Ubuntu containers, found the API easy to pick up. All seven ported tests passed under Meson and autoconf. Against the Perl originals, the set was about 13 percent faster overall, with large gains on two tests and little change on others.

Reviewers are not treating speed as enough. Freund rejected a change that would print far more CI log lines, arguing that unreadably verbose pytest failures should be fixed rather than dumped onto the terminal. He also opposed installing 32-bit Python in the CI images, calling it a maintenance burden. Fennema-Nio later shortened failure output, capped server logs, and aligned server settings with the Perl tests, including wal_level set to minimal by default. Daniel Gustafsson asked that bugs found while rewriting tests be filed on their own so they are not stuck behind the framework debate. Fennema-Nio did that.

Coverage is the harder open question. Freund asked for a systematic comparison, including branch coverage, so a rewrite does not quietly drop checks. Shihao Zhong's script, comparing one async I/O test, found the unique Perl hits were mostly in psql and the simple-query path, already covered elsewhere, plus a WAL-skip path the Python harness missed because it did not match Perl's default wal_level. That default is now aligned.

Nikolay Samokhvalov, using an AI harness he had not fully reviewed by hand, reported that a timeout on a background query could mark the result consumed and reuse the connection while the query still ran. Fennema-Nio called it a small bug and fixed it differently in a later revision. Samokhvalov also flagged a snapshot leak in a pg_visibility conversion that was never submitted for review. Fennema-Nio declined to add that.

Sukhanov's review found a sharper operational gap. A Meson timeout or SIGTERM can leave primary and standby servers running, with WAL sender and receiver still up. The Perl harness cleans those up. On CI and the buildfarm, leftovers mean occupied ports, shared memory, and data directories. Ctrl-C does clean up. He also found authentication differing between a copied template cluster and a fresh initdb. Andrey Borodin had briefly parked the Commitfest entry as waiting on the author. Fennema-Nio moved it back to needs review, saying the core still needs eyes.