writing

A Quick Overview of mmap: Principles and iOS Practicemmap 原理与 iOS 实践速览

2018-09-01 · Performance & internals

Translated from Chinese — .

On this page

• What is mmap
• Principles
• Differences from read/write
• Usage
• pagesize
• Implementation in xnu
• Sample Code
• References
• Summary

mmap is an essential tool for performance optimization. This article briefly lists out some related information.

What is mmap

To put it simply, loosely, and imprecisely, mmap can directly establish a mapping between memory and a file. A process’s modifications to the memory can be directly synchronized as modifications to the file’s contents, with only a single copy from disk to memory.

Here is Wikipedia’s explanation:

In computing, mmap(2) is a POSIX-compliant Unix system call that maps files or devices into memory. It is a method of memory-mapped file I/O. It implements demand paging, because file contents are not read from disk initially and do not use physical RAM at all. The actual reads from disk are performed in a “lazy” manner, after a specific location is accessed.

Here is a Chinese explanation (not a translation of the above):

mmap是一种内存映射文件的方法，即将一个文件或者其它对象映射到进程的地址空间，实现文件磁盘地址和进程虚拟地址空间中一段虚拟地址的一一对映关系。实现这样的映射关系后，进程就可以采用指针的方式读写操作这一段内存，而系统会自动回写脏页面到对应的文件磁盘上，即完成了对文件的操作而不必再调用read,write等系统调用函数。相反，内核空间对这段区域的修改也直接反映用户空间，从而可以实现不同进程间的文件共享。

Principles

I recommend reading this article “认真分析mmap：是什么 为什么 怎么用” (Seriously Analyzing mmap: What, Why, How) https://www.cnblogs.com/huxiao-tee/p/4660352.html

Differences from read/write

1. Regular file operations like read/write need the page cache as a relay between the kernel and the application layer, so a single file operation requires two data copies (kernel to page cache, page cache to application layer), whereas mmap eliminates the page-cache relay, removing one data copy.
2. mmap automatically writes back to disk when memory is insufficient or the process crashes. (It is precisely because of this that WeChat’s xlog can ensure logs are not lost.)

For more information, refer to “从内核文件系统看文件读写过程” (Looking at the File Read/Write Process from the Kernel File System) http://www.cnblogs.com/huxiao-tee/p/4657851.html

Usage

I won’t go into the specific methods in detail; again, refer to the article in the “Principles” section above.

pagesize

I always thought the page size was 4K, but starting from arm64 it’s already 16K.

https://forums.developer.apple.com/thread/47532

Implementation in xnu

Sample Code

For the mmap usage, you can directly refer to the AppleTrace code https://github.com/everettjf/AppleTrace/blob/master/appletrace/appletrace/src/appletrace.mm

The pagesize test code https://github.com/everettjf/Yolo/tree/master/BukuzaoArchive/sample/MmapSample/MmapSample/FastRead.m

References

1. https://en.wikipedia.org/wiki/Mmap
2. https://en.wikipedia.org/wiki/Fork_(system_call)
3. https://www.safaribooksonline.com/library/view/linux-system-programming/0596009585/ch04s03.html
4. http://www.tutorialspoint.com/unix_system_calls/mmap.htm
5. https://en.wikipedia.org/wiki/Virtual_memory
6. https://en.wikipedia.org/wiki/Virtual_address_space
7. https://satanwoo.github.io/2017/07/30/xlog/
8. http://blog.desmondyao.com/mars-xlog/

Summary

mmap is a magical tool. A deeper understanding of the principles requires analyzing the kernel implementation. Emmmmmm… too noob, still need to study more.

文章目录

• mmap是什么
• 原理
• 与read/write区别
• 使用
• pagesize
• xnu中的实现
• 示例代码
• 参考
• 总结

mmap是性能优化的必备神器，这篇文章简单罗列下相关信息。

mmap是什么

简单通俗不精确的说，mmap可以直接建立内存与文件的映射，进程对内存的修改可直接同步到文件内容的修改，仅有一次磁盘到内存的拷贝过程。

这是Wikipedia的解释：

In computing, mmap(2) is a POSIX-compliant Unix system call that maps files or devices into memory. It is a method of memory-mapped file I/O. It implements demand paging, because file contents are not read from disk initially and do not use physical RAM at all. The actual reads from disk are performed in a “lazy” manner, after a specific location is accessed.

这是一个中文解释（不是上文的翻译）：

mmap是一种内存映射文件的方法，即将一个文件或者其它对象映射到进程的地址空间，实现文件磁盘地址和进程虚拟地址空间中一段虚拟地址的一一对映关系。实现这样的映射关系后，进程就可以采用指针的方式读写操作这一段内存，而系统会自动回写脏页面到对应的文件磁盘上，即完成了对文件的操作而不必再调用read,write等系统调用函数。相反，内核空间对这段区域的修改也直接反映用户空间，从而可以实现不同进程间的文件共享。

原理

推荐阅读这篇文章 《认真分析mmap：是什么 为什么 怎么用》https://www.cnblogs.com/huxiao-tee/p/4660352.html

与read/write区别

1. read/write等常规文件操作，需要页缓存作为内核和应用层的中转，因此一次文件操作需要两次数据拷贝（内核到页缓存，页缓存到应用层），而mmap省去了页缓存到中转，少了一次数据拷贝。
2. mmap在内存不足、进程Crash时会自动回写磁盘。（也就是由于这点，微信xlog可确保日志的不丢失）

更多信息参考 《从内核文件系统看文件读写过程》 http://www.cnblogs.com/huxiao-tee/p/4657851.html

使用

具体方法不再细说，仍然参考上文“原理”中的文章。

pagesize

一直以为page size是4K，但在arm64开始已经是16K。

https://forums.developer.apple.com/thread/47532

xnu中的实现

示例代码

mmap使用方法可直接参考 AppleTrace 的代码 https://github.com/everettjf/AppleTrace/blob/master/appletrace/appletrace/src/appletrace.mm

pagesize 的测试代码 https://github.com/everettjf/Yolo/tree/master/BukuzaoArchive/sample/MmapSample/MmapSample/FastRead.m

参考

1. https://en.wikipedia.org/wiki/Mmap
2. https://en.wikipedia.org/wiki/Fork_(system_call)
3. https://www.safaribooksonline.com/library/view/linux-system-programming/0596009585/ch04s03.html
4. http://www.tutorialspoint.com/unix_system_calls/mmap.htm
5. https://en.wikipedia.org/wiki/Virtual_memory
6. https://en.wikipedia.org/wiki/Virtual_address_space
7. https://satanwoo.github.io/2017/07/30/xlog/
8. http://blog.desmondyao.com/mars-xlog/

总结

mmap 是个神器。更深入的原理需要分析内核的实现，Emmmmmm… 太菜，还需学习。