Fault Localization on Witnesses

Dirk Beyer, Matthias Kettl, and Thomas Lemberger

This work was accepted at SPIN 2024. A preprint is available here


Abstract

When verifiers report an alarm, they export a violation witness that helps validate the reachability of that alarm. Conventional wisdom says that this violation witness should be very precise: the ideal witness describes a single error path for the validator to check. But we claim that verifiers overshoot and produce large witnesses with information that makes validation unnecessarily difficult. To check our hypothesis, we reduce violation witnesses to that information that automated fault-localization approaches deem relevant for triggering the reported alarm in the program. We perform a large experimental evaluation on the witnesses produced in the International Competition on Software Verification (SV-COMP 2023). It shows that our reduction shrinks the witnesses considerably and enables the confirmation of verification results that were not confirmable before.


Tool and Reproduction Information

A reproduction package of this work is available on Zenodo (DOI). The implementation of Flow can be found in our Git repository. The implementation of FaultY is part of CPAchecker. A tutorial explaining the usage and the features of FaultY is archived on Zenodo (DOI)


Full Results

The experimental results reported in the paper can be found here. Under each research question, you can find the plots and tables that are referenced in the paper. The plots and tables are generated from the raw data that can be found in our reproduction package. We also include all validators and omitted plots here.