Date: Sun, 11 Aug 2019 14:57:10 -0700
On Sun, Aug 11, 2019 at 2:07 PM Lyberta via SG15 <sg15_at_[hidden]>
wrote:
> David Blaikie:
> > Oh, no, the build system doesn't do anything here other than tell the
> > compiler the path to the BMI - the compiler does the memory mapping -
> given
> > a path to the BMI, the compiler doesn't read the bytes with a stream - it
> > memory maps the file directly, so it can lazily load/reference components
> > of the file. Clang's BMI consists of efficient on-disk hash tables and
> > things, so only the necessary portions of a BMI are ever read (always
> room
> > for improvement, all software has bugs, etc - but that's the general
> goal).
>
> So does the OS cache memory maps between compiler invocations? Even if
> all file descriptors are closed between those?
>
I don't know much about this sort of thing - but my understanding is that
filesystems cache files in RAM when it's otherwise unused (the classic
situation where folks on linux get confused by why the 'free' tool that
reports memory usage says they have no free memory - because the OS is
making use of any spare memory for disk caches (
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/5/html/tuning_and_optimizing_red_hat_enterprise_linux_for_oracle_9i_and_10g_databases/chap-oracle_9i_and_10g_tuning_guide-memory_usage_and_page_cache
))
- so when you memory map a file, it could be already in RAM, in might be
moved into RAM, or your memory mapped pages might be virtual memory
actually falling through to the actual hard drive (& the hard drive might
have its own caches).
>
> _______________________________________________
> SG15 mailing list
> SG15_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg15
>
wrote:
> David Blaikie:
> > Oh, no, the build system doesn't do anything here other than tell the
> > compiler the path to the BMI - the compiler does the memory mapping -
> given
> > a path to the BMI, the compiler doesn't read the bytes with a stream - it
> > memory maps the file directly, so it can lazily load/reference components
> > of the file. Clang's BMI consists of efficient on-disk hash tables and
> > things, so only the necessary portions of a BMI are ever read (always
> room
> > for improvement, all software has bugs, etc - but that's the general
> goal).
>
> So does the OS cache memory maps between compiler invocations? Even if
> all file descriptors are closed between those?
>
I don't know much about this sort of thing - but my understanding is that
filesystems cache files in RAM when it's otherwise unused (the classic
situation where folks on linux get confused by why the 'free' tool that
reports memory usage says they have no free memory - because the OS is
making use of any spare memory for disk caches (
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/5/html/tuning_and_optimizing_red_hat_enterprise_linux_for_oracle_9i_and_10g_databases/chap-oracle_9i_and_10g_tuning_guide-memory_usage_and_page_cache
))
- so when you memory map a file, it could be already in RAM, in might be
moved into RAM, or your memory mapped pages might be virtual memory
actually falling through to the actual hard drive (& the hard drive might
have its own caches).
>
> _______________________________________________
> SG15 mailing list
> SG15_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg15
>
Received on 2019-08-11 16:59:23