Memory-Mapped Files (mmap)

Have you ever wondered why reading a file feels slower than accessing memory? When you read a file, your program makes a system call, the kernel reads from disk, copies the data to a kernel buffer, then copies it to your program's memory. That's a lot of copying and system calls.

What if you could treat a file like it was already in memory? What if you could access file data with simple pointer operations instead of read() and write() calls?

That's exactly what memory-mapped files do. They let you map a file (or part of it) directly into your process's virtual address space. Once mapped, you can access the file data just like any other memory - no system calls needed.

The Basic Idea: Files as Memory

Let's start with a simple example. You have a large configuration file that your program needs to read frequently. With traditional file I/O:

// Traditional approach
FILE *file = fopen("config.txt", "r");
char buffer[1024];
fread(buffer, 1, 1024, file);
// Process the data...
fclose(file);

Every time you need to read the file, you make system calls and copy data.

With memory mapping:

// Memory-mapped approach
int fd = open("config.txt", O_RDONLY);
char *data = mmap(NULL, file_size, PROT_READ, MAP_PRIVATE, fd, 0);
// Now 'data' points directly to the file contents
// Access it like any other memory: data[0], data[100], etc.
munmap(data, file_size);
close(fd);

Once mapped, accessing data[100] is just a memory access - no system call, no copying.

How Does It Work?

The key insight is that the kernel can map file pages into your virtual address space. When you access a memory address that's mapped to a file:

1. If the page is already in memory (cached), you get it instantly
2. If the page isn't in memory, the kernel loads it from disk (page fault)
3. Future accesses to that page are just memory operations

The kernel handles all the complexity of caching, prefetching, and disk I/O behind the scenes.

The Performance Advantage

Why is this faster? Let's trace through what happens:

Traditional read():

• Your program calls read()
• Kernel copies data from disk to kernel buffer
• Kernel copies data from kernel buffer to your program's memory
• Your program processes the data

Memory-mapped access:

• Your program accesses memory (no system call)
• If page is cached: instant access
• If page isn't cached: kernel loads it once, then it's cached

The key difference is that with memory mapping, the kernel can optimize access patterns. It can prefetch pages you're likely to need, and it can keep frequently accessed pages in memory.

Two Mapping Modes: Private vs Shared

Memory mapping comes in two flavors, and the choice matters a lot.

MAP_PRIVATE: Your Own Copy

When you map a file with MAP_PRIVATE, you get your own private copy. If you modify the data, the changes stay in your memory and don't affect the original file.

How it works:

• Initially, all processes mapping the same file share the same physical pages
• When you write to a page, the kernel creates a copy just for your process
• Other processes continue to see the original data
• Your changes are lost when you unmap the memory

Use cases:

• Reading configuration files you might modify temporarily
• Processing large datasets where you need to experiment with changes
• When you want to read a file but might need to modify it in memory

MAP_SHARED: Changes Go Back to File

With MAP_SHARED, any changes you make are written back to the file and visible to other processes mapping the same file.

How it works:

• Changes are written back to the file (eventually)
• Other processes mapping the same file can see your changes
• The kernel handles the synchronization of writes

Use cases:

• Database files that need to be updated
• Log files that multiple processes write to
• Shared configuration that needs to be updated atomically

Real-World Example: Database Access

Let's walk through a real example. You're building a simple key-value store that stores data in a file.

Traditional approach:

// To read a value
FILE *file = fopen("database.db", "r");
fseek(file, key_offset, SEEK_SET);
fread(&value, sizeof(value), 1, file);
fclose(file);

// To write a value
FILE *file = fopen("database.db", "r+");
fseek(file, key_offset, SEEK_SET);
fwrite(&value, sizeof(value), 1, file);
fclose(file);

Memory-mapped approach:

// Map the entire database file
int fd = open("database.db", O_RDWR);
char *db = mmap(NULL, file_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);

// Reading a value is just memory access
Value *value = (Value*)(db + key_offset);
printf("Value: %d\n", value->data);

// Writing a value is just memory assignment
value->data = new_value;
// Changes are automatically written back to the file

The memory-mapped version is much simpler and potentially faster, especially for random access patterns.

When Memory Mapping Makes Sense

Memory mapping isn't always the best choice. Here's when it makes sense:

Use memory mapping when:

• You need random access to file data
• The file is large and you only need parts of it
• Multiple processes need to access the same file
• You want to minimize system call overhead
• You're doing a lot of file I/O

Avoid memory mapping when:

• The file is small (setup overhead isn't worth it)
• You only do sequential reads or writes
• The file changes frequently from outside your process
• You're under memory pressure
• You need to support very old systems

Performance Considerations

Read Performance

Sequential reads: Memory mapping can be faster because the kernel can optimize access patterns and prefetch data. Random access: Memory mapping excels here. Traditional I/O requires seeking to different positions, while memory mapping provides direct pointer access. Memory pressure: Under memory pressure, the kernel may need to page out mapped pages, which can cause performance degradation.

Write Performance

Small writes: For small, frequent writes, traditional write() might be more efficient due to buffering. Large writes: For large writes, memory mapping can be more efficient, especially when you can modify data in-place. Synchronization: MAP_SHARED writes require kernel synchronization, which can add overhead.

Common Patterns

Once you understand the basics, you'll see these patterns emerge:

Read-only mapping: Map files you only need to read

• Configuration files
• Reference data
• Static content

Read-write mapping: Map files you need to modify

• Database files
• Log files
• Temporary files

Shared mapping: Map files that multiple processes need to access

• Shared configuration
• Inter-process communication
• Database files

Integration with Other IPC

Memory mapping is often used together with other IPC mechanisms:

mmap() + Pipes: Use memory mapping for bulk data, pipes for control messages

• Example: Large data file mapped, control commands via pipes

mmap() + Signals: Use memory mapping for data, signals for notifications

• Example: Configuration file mapped, "config changed" signals

mmap() + Sockets: Use memory mapping for local data, sockets for network communication

• Example: Local cache mapped, network requests via sockets

The Bottom Line

Memory mapping is a powerful technique that can significantly improve file I/O performance, especially for random access patterns. It eliminates the overhead of system calls and data copying, letting you treat files like memory.

The key is understanding when the benefits outweigh the complexity. For simple sequential I/O, traditional file operations might be simpler. For random access or when you need the performance, memory mapping is often the right choice.

In high-performance applications like databases, memory mapping is essential because it provides the fastest possible file access. In other applications, the complexity might not be worth the performance gain.

Questions

Q: What is the primary advantage of mmap() over read()/write()?

mmap() eliminates the need for multiple system calls by mapping file directly to memory.

Q: Which flag makes mmap() shared between processes?

MAP_SHARED allows the mapping to be shared between processes.

Q: What happens when you write to a MAP_PRIVATE mapping?

MAP_PRIVATE creates copy-on-write pages when modified.

Implement a simple database using mmap() that can store and retrieve key-value pairs. Use a fixed-size record structure.

#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <cstring>
#include <string>
#include <optional>

// Fixed-size record structure for key-value pairs
struct Record {
    char key[64];
    char value[256];
    bool in_use;
};

class MmapDatabase {
private:
    static constexpr size_t MAX_RECORDS = 100;
    static constexpr size_t DB_SIZE = sizeof(Record) * MAX_RECORDS;

    int fd;
    Record* records;
    std::string filepath;

public:
    // TODO: Implement constructor
    // Should:
    // 1. Store the filepath
    // 2. Open/create the file with read/write permissions