Publications
Calibration and Correctness of Language Models for Code
conference
Claudio Spiess, David Gros, Kunal Suresh Pai, Michael Pradel, Md Rafiqul Islam Rabin, Amin Alipour, Sushmit Jha, Premkumar Devanbu, Toufique Ahmed
ICSE 2025
Machine learning models often produce incorrect outputs, making reliable confidence measures essential for determining the trustworthiness of these outputs. This paper introduces a framework to evaluate and improve the calibration of code-generating models, finding that these models are generally poorly calibrated initially but can be improved using methods like Platt scaling, thereby enhancing decision-making in software engineering.
Software EngineeringMachine LearningNaturalness of Software
Potential and Limitation of High-Frequency Cores and Caches
poster
Kunal Pai, Anusheel Nand, Jason Lowe-Power
ModSim 2024: Workshop on Modeling & Simulation of Systems and Applications
The poster presentation explores the potential and limitations of high-frequency in-order and out-of-order cores and caches in modern processors, highlighting the trade-offs between speedups and bandwidth.
Computer ArchitectureCryogenic ComputingSuperconducting
Automatic semantic augmentation of language model prompts (for code summarization)
conference
Toufique Ahmed, Kunal Suresh Pai, Premkumar Devanbu, Earl T. Barr
ICSE 2024
Adding explicit semantic facts as prompts to Large Language Models improves their performance in code summarization tasks, with notable improvements exceeding 2 BLEU and, in some cases, even surpassing 30 BLEU, demonstrating the effectiveness of this approach in enhancing code analysis and extraction of essential information.
Software EngineeringMachine LearningNaturalness of Software
Validating Hardware and SimPoints with gem5: A RISC-V Board Case Study
poster
Kunal Pai, Zhantong Qiu, Jason Lowe-Power
ISCA 2023: gem5 Workshop
The poster discusses the development of a RISC-V board model (RISCVMatched) in gem5, along with a methodology for fine-tuning gem5 configurations to closely match real-life systems, resulting in more accurate hardware validation and simulation capabilities.
Computer Architecturegem5
gem5 Vision
poster
Parth Shah, Kunal Pai, Harshil Patel, Arslan Ali
ISCA 2023: gem5 Workshop
The gem5 Vision Project seeks to improve user-friendliness and accessibility by introducing advanced search functionality, comprehensive resource categorization, and expanded database support within the gem5 ecosystem for researchers and developers.
Computer Architecturegem5