On Fri, Jun 18, 2021 at 1:20 PM Gabriel Dos Reis <gdr@microsoft.com> wrote:

Also notice that while a dependency scanner needs to chase #included or imported files (via the appropriate implementation-defined mapping), it actually does not need  to fully preprocess the source code found therein; 

the spec has been carefully crafted to that effect.

It doesn't need to preprocess the complete file, but it needs to significantly preprocess the early sections of the file. While the spec has carefully prevented conditional specification of the keywords, it still allows macros to be used, which requires support for conditional evaluation, variable expansion, and string concatenation.

In other words, the only optimization granted by the standard today is on the discovery of whether or not a source file needs to participate on the construction of the DAG for processing modules, but you cannot discover the graph with a simple lexer, you need the full expression evaluation logic of the preprocessor.