hq (hep-queue)
A pull-based task queue for particle physics analysis. I gave it TLS, a way to return results, and a coffea executor, then ran a CMS(CMS Open Data ttbar) analysis on it.
hq is Peter Fackeldey's (CERN) pull-based task queue for High Energy Physics analysis workloads: nothing assigns work, an idle worker asks for the next task and takes it.
I spent the summer of 2026 on it as an IRIS-HEP fellow, making it deployable and teaching it to talk to coffea. TLS, a private queue per run, a way to return results, an executor. Then I ran the CMS Open Data ttbar analysis on it end to end: 9.8 million events, no failed tasks, histograms identical bin by bin to coffea's own.
It is slightly slower than Dask on one machine. A fresh Python process per task costs 1.7 seconds and the run pays that fifty times, which is what buys a software environment per task and per worker heartbeat nothing can stall. Whether it pays off at thousands of workers is the part I have not tested. Next step is to deploy on the Coffea-Casa Analysis Facility, to test at scale on HTCondor nodes.