Sunday, May 10, 2020

Infinite try-catch loops can crash the JVM

Previously I discovered that the Java compiler sometimes emits exception table entries set to "handle" an exception by jumping back to the start of the covered region. This led to deadlock if an exception was indeed thrown in that region, usually due to bytecode rewriting inserting instructions. It was manageable to fix during bytecode rewriting with ASM by dropping exception table entries (not calling super.visitTryCatchBlock) that had the same start and handler label. Unfortunately there are other circumstances, particularly with while loops inside synchronized blocks, in which the compiler emits an entry with the handler in the middle of the covered region. I saw this crash the JVM entirely, citing an "invalid class file" but providing no additional information with -Xverify:all. Based on the access-violation error code, it seemed to be a stack overflow in the JVM itself. Avoiding this problem is more difficult because ASM doesn't provide a way to know the order of labels until all the exception table entries have been visited. I had to make the ClassReader first accept a visitor that recorded the indexes of problematic try-catch blocks, then use that information during the real transformation that dropped the offending entries.

No comments:

Post a Comment