I ran into this problem recently and wrote up a post about the particular case in our code that was causing this problem.
http://cappuccino.org/discuss/2010/03/01/internet-explorer-global-variables-and-stack-overflows/
The quick summary is: recursion that passes through the host global object is limited to a stack depth of 13. In other words, if the reference your function call is using (not necessarily the function itself) was defined with some form window.foo = function, then recursing through foo is limited to a depth of 13.