Understanding the behavior of C’s preprocessor when a macro indirectly expands itself

Macro expansion is a complex process that is really only understandable by understanding the steps that occur.

1. When a macro with arguments is recognized (macro name token followed by ( token), the following tokens up to the matching ) are scanned and split (on , tokens). No macro expansion happens while this is happening (so the ,s and ) must be present in the input stream directly and cannot be in other macros).

2. Each macro argument whose name appears in the macro body not preceeded by # or ## or followed by ## is “prescanned” for macros to expand — any macros entirely within the argument will be recursively expanded before substituting into the macro body.

3. The resulting macro argument token streams are substituted into the body of the macro. Arguments involved in # or ## operations are modified (stringized or pasted) and substituted based on the original parser tokens from step 1 (step 2 does not occur for these).

4. The resulting macro body token stream is scanned again for macros to expand, but ignoring the macro currently being expanded¹. At this point further tokens in the input (after what was scanned and parsed in step 1) may be included as part of any macros recognized.

The important thing is that there are TWO DIFFERENT recursive expansions that occur (step 2 and step 4 above) and ONLY the one in step 4 ignores recursive macro expansions of the same macro. The recursive expansion in step 2 DOES NOT ignore the current macro, so can expand it recursively.

So for your example above, lets see what happens. For the input

EXPAND(TEST PARENTHESIS())

• step 1 sees the macro EXPAND and scans in argument list TEST PARENTHESIS() for X
• step 2 does not recognize TEST as a macro (no following (), but does recognize PARENTHESIS:
  • step 1 (nested) gets an empty token sequence for the argument
  • step 2 scans that empty sequence and does nothing
  • step 3 inserts into the macro body () yielding just that: ()
  • step 4 scans () for macros and doesn’t find any
• so the final value for X after step 2 is TEST ()
• step 3 inserts that into the body, giving TEST ()
• step 4 suppresses EXPAND and scans the result of step 3 for more macros, finding TEST
  • step 1 gets an empty sequence for the argument
  • step 2 does nothing
  • step 3 substitutes into the body giving EXPAND(0)
  • step 4 recursive expands that, suppressing TEST. At this point, both EXPAND and TEST are suppressed (due to being in the step 4 expansion), so nothing happens

Your other example EXPAND(TEST()) is different

• step 1 EXPAND is recognized as a macro, and TEST() is parsed as the argument X
• step 2, this stream is recursively parsed. Note that since this is step 2, EXPAND is NOT SUPPRESSED
  • step 1 TEST is recognized as a macro with an empty sequence argument
  • step 2 — nothing (no macros in an empty token sequence)
  • step 3, substituted into the body giving EXPAND(0)
  • step 4, TEST is suppressed and the result recursively expanded
    • step 1, EXPAND is recognized as a macro (remember, at this point only TEST is suppressed by step 4 recursion — EXPAND is in the step 2 recursion so is not suppressed) with 0 as its argument
    • step 2, 0 is scanned and nothing happens to it
    • step 3, substitute into the body giving 0
    • step 4, 0 is scanned again for macros (and again nothing happens)
• step 3, the 0 is substituted as the argument X into the body of the first EXPAND
• step 4, 0 is scanned again for macros (and again nothing happens)

so the final result here is 0

¹_{Exactly how this happens appears to vary between implementations. Some implementations will merely not expand the macro at this point, but might expand it later if a subsequent rescan for another macro sees it again. Other implementations will ‘mark’ the token internally so that they don’t expand the macro even with subsequent rescans. Either method can be seen as conforming to the spec, which is unclear on this detail}