LLM4VV Published at High Performance Computing (HiPC) 2025

We are proud to announce that our paper - LLM4VV: Evaluating Cutting-Edge LLMs for Generation and Evaluation of Directive-Based Parallel Programming Model Compiler Tests - was the only undergraduate paper accepted and published at High Performance Computing (HiPC) 2025. We would like to thank Arron Jarmusch for presenting the research on behalf of Zachariah Sollenberger, Rahul Patel, Saieda Ali Zada at the HiPC conference.

This paper introduced a Dual-Agent framework that consists of a generative LLM and a discriminative LLM to generate and validate OpenACC and OpenMP compiler tests, respectively. In this paper, a student benchmark tested multiple LLMS. The results concluded with Deepseek-Coder-33B-Instruct having a Pass@1 score of .434 and Qwen2.50-Coder-32B-Instruct having an F1-Score of .735 and an MCC score of .447. Both of these LLMs demonstrated the best scores. The future work includes fine-tuning these two LLMs.