“If my build system has already encoded all of my dependencies in a secondary layer (my build files), do I care about any of these concerns? “
Channeling Tom Honermann, but yes, your other tools care. Your IDEs and tidying tools and anything else that needs to consume and understand your source without building it will care. If you want types and values to have different colors in your IDE, then your IDE needs to be able to get from your file to the source file that declared those entities.
“I'm reasonably sure from past experience is that a proper modular and parallized build requires that secondary layer. (To say nothing of performance concerns: rescanning constantly is a waste of computation.)”
It may be the case that such concessions are necessary. However, I will argue that a significant portion of C++ projects don’t currently specify in their build files that foo.cpp depends on foo.h, boost/spirit.h, and the other hundred headers that boost/spirit.h drags in. I will go further and say that a significant portion of C++ projects rely on file globbing to even gather the list of .cpp files to compile. We should at least be aware of this cost and consider it, before requiring everyone to duplicate the build information that already exists in their source files and file system.