Debugging Lesson: Take a Look Back To Go Forward

Over the years you learn a few lessons and make a few rules. You swear to yourself that you’ll always follow them but over time you get lazy or you just forget. Especially when debugging, you get in that panicky state where all that goes out the window. Urgency beats out common sense.

Usually when you find a crasher, you end up going at it head-on, in a brute force manner. Run it through the debugger. Carpet bomb the code with log output. Bang your head against the desk. Rinse. Repeat.

The lesson here is obvious (maybe painfully so) but I don’t see it applied enough in practice. I find that it’s commonly forgotten in the heat of the moment and is only heeded as a last resort. Basically, it is this:

If it worked before and is now broken, go back and look at what has changed.

Simple enough. Machines are deterministic and most systems you code are as well. Code usually doesn’t degrade and break on its own; something must have changed to cause the problem. Go back and see if you can isolate the build where the thing broke. If you have QA people, you can parallelize and have them isolate the build while you debug. Now look at the commit logs, starting at that build and go back. Many times, this will be the “a-ha” moment you needed. Even if the changed code is not the actual cause, it’s usually a trigger or it somehow exposes the problem. Now debug, focusing on the areas of those changes.

Now, this doesn’t always work. Maybe the code just acts differently after a certain date for some odd reason (Y2K anyone?) or there’s a hardware glitch or your program is getting input it never received before. But these cases are the exception. I have repeatedly regretted not checking the change logs but not once regretted doing it.

So, now that you know the rule, how do you make sure you follow it next time you’re looking down the barrel of a EXC_BAD_ACCESS error? Try this: set a timer for, say, 10 minutes and then start debugging. When the timer goes off, if you do not have a good sense of what’s going wrong, then stop and check the commit logs. How do you remember to set the timer? Actually, I don’t have good answer for that. Maybe you can write a hook into CrashReporter to start it for you. Or maybe you hire a magical pixie that appears whenever your program crashes and reminds you. Or you combine the two and have CrashReporter somehow wake up the magical pixie who will then remind you to set the timer. Then, just work in a bowling ball and a cuckoo clock and you’re golden.

Category: Debugging, Programming 2 comments »

2 Responses to “Debugging Lesson: Take a Look Back To Go Forward”

  1. Michael

    Or you can just use a back in time debugger. All you have to do is collect a trace log of the history from within the debugger or externally, load that log, and step backwards to figure out what happened.
    No manual logging
    No reproducing problems
    No more headaches like the ones you describe.
    Check it out: http://www.ghs.com/products/timemachine.html

  2. T. Moeller

    … or just use Leopard’s dtrace …
    http://www.theregister.co.uk/2004/07/08/dtrace_user_take
    http://prefetch.net/articles/dtracecookbook.html
    http://prefetch.net/articles/solaris.dtracetopten.html


Leave a Reply



Back to top