I have spent the last three days debugging an ugly crash in the upcoming Zorp 3.1. First I had some problems with the core files produced with Linux 2.6.12, as the register values proved to be invalid, thus the backtrace was even more unusable than it is usual with heap corruptions.
I could get access to the original register values as Zorp dumps part of its stack when a fatal signal is encountered. Using that information I could locate the stack frame of the signal handler and luckily Linux passes a "struct sigcontext" to each signal handler as parameter which contains register information. But nevertheless it made analyzing the core files difficult.
After a post to the gdb mailing list it turned out to be a kernel problem rather than a gdb problem and with the help of my collegue Krisztián Kovács (of Netfilter ct_sync fame) we could solve the problem by backporting a fix from 2.6.15, so core files are now ok.
The problem however seems to be difficult, I have already studied the libc malloc implementation, disassembled and annotated the _int_malloc and _int_free functions, I'm now able to read hexdumps of heap areas fluently but I still don't have a fix for the problem. Lucky us Zorp restarts itself in this situation and the scenario where this problem occurs is not frequently used.
My suspicion is that the SSL error state for threads are the cause of the problem as I have evidence that the freed heap block is overwritten by ERR_clear_state(), which destroys the next and prev pointers in the freed memory block, thus resulting in the crash. The error states are supposedly thread-specific variables, but the way the allocation is done is suspicious.
I hope I can finally find this problem tomorrow.
I could get access to the original register values as Zorp dumps part of its stack when a fatal signal is encountered. Using that information I could locate the stack frame of the signal handler and luckily Linux passes a "struct sigcontext" to each signal handler as parameter which contains register information. But nevertheless it made analyzing the core files difficult.
After a post to the gdb mailing list it turned out to be a kernel problem rather than a gdb problem and with the help of my collegue Krisztián Kovács (of Netfilter ct_sync fame) we could solve the problem by backporting a fix from 2.6.15, so core files are now ok.
The problem however seems to be difficult, I have already studied the libc malloc implementation, disassembled and annotated the _int_malloc and _int_free functions, I'm now able to read hexdumps of heap areas fluently but I still don't have a fix for the problem. Lucky us Zorp restarts itself in this situation and the scenario where this problem occurs is not frequently used.
My suspicion is that the SSL error state for threads are the cause of the problem as I have evidence that the freed heap block is overwritten by ERR_clear_state(), which destroys the next and prev pointers in the freed memory block, thus resulting in the crash. The error states are supposedly thread-specific variables, but the way the allocation is done is suspicious.
I hope I can finally find this problem tomorrow.
Comments