[Linux kernel] memory map 정리

리눅스 커널의 메모리 구조에 대해 정리해보겠다. 그냥 궁금한 내용을 docs를 읽고 정리한 거라서 틀린 내용이 있을 수도 있따. (틀린 내용은 댓글에 지적 부탁드립니다.)

전체 구조( 4-level paging 기준)

========================================================================================================================
    Start addr    |   Offset   |     End addr     |  Size   | VM area description
========================================================================================================================
                  |            |                  |         |
 0000000000000000 |    0       | 00007fffffffffff |  128 TB | user-space virtual memory, different per mm
__________________|____________|__________________|_________|___________________________________________________________
                  |            |                  |         |
 0000800000000000 | +128    TB | ffff7fffffffffff | ~16M TB | ... huge, almost 64 bits wide hole of non-canonical
                  |            |                  |         |     virtual memory addresses up to the -128 TB
                  |            |                  |         |     starting offset of kernel mappings.
__________________|____________|__________________|_________|___________________________________________________________
                                                            |
                                                            | Kernel-space virtual memory, shared between all processes:
____________________________________________________________|___________________________________________________________
                  |            |                  |         |
 ffff800000000000 | -128    TB | ffff87ffffffffff |    8 TB | ... guard hole, also reserved for hypervisor
 ffff880000000000 | -120    TB | ffff887fffffffff |  0.5 TB | LDT remap for PTI
 ffff888000000000 | -119.5  TB | ffffc87fffffffff |   64 TB | direct mapping of all physical memory (page_offset_base)
 ffffc88000000000 |  -55.5  TB | ffffc8ffffffffff |  0.5 TB | ... unused hole
 ffffc90000000000 |  -55    TB | ffffe8ffffffffff |   32 TB | vmalloc/ioremap space (vmalloc_base)
 ffffe90000000000 |  -23    TB | ffffe9ffffffffff |    1 TB | ... unused hole
 ffffea0000000000 |  -22    TB | ffffeaffffffffff |    1 TB | virtual memory map (vmemmap_base)
 ffffeb0000000000 |  -21    TB | ffffebffffffffff |    1 TB | ... unused hole
 ffffec0000000000 |  -20    TB | fffffbffffffffff |   16 TB | KASAN shadow memory
__________________|____________|__________________|_________|____________________________________________________________
                                                            |
                                                            | Identical layout to the 56-bit one from here on:
____________________________________________________________|____________________________________________________________
                  |            |                  |         |
 fffffc0000000000 |   -4    TB | fffffdffffffffff |    2 TB | ... unused hole
                  |            |                  |         | vaddr_end for KASLR
 fffffe0000000000 |   -2    TB | fffffe7fffffffff |  0.5 TB | cpu_entry_area mapping
 fffffe8000000000 |   -1.5  TB | fffffeffffffffff |  0.5 TB | ... unused hole
 ffffff0000000000 |   -1    TB | ffffff7fffffffff |  0.5 TB | %esp fixup stacks
 ffffff8000000000 | -512    GB | ffffffeeffffffff |  444 GB | ... unused hole
 ffffffef00000000 |  -68    GB | fffffffeffffffff |   64 GB | EFI region mapping space
 ffffffff00000000 |   -4    GB | ffffffff7fffffff |    2 GB | ... unused hole
 ffffffff80000000 |   -2    GB | ffffffff9fffffff |  512 MB | kernel text mapping, mapped to physical address 0
 ffffffff80000000 |-2048    MB |                  |         |
 ffffffffa0000000 |-1536    MB | fffffffffeffffff | 1520 MB | module mapping space
 ffffffffff000000 |  -16    MB |                  |         |
    FIXADDR_START | ~-11    MB | ffffffffff5fffff | ~0.5 MB | kernel-internal fixmap range, variable size and offset
 ffffffffff600000 |  -10    MB | ffffffffff600fff |    4 kB | legacy vsyscall ABI
 ffffffffffe00000 |   -2    MB | ffffffffffffffff |    2 MB | ... unused hole
__________________|____________|__________________|_________|___________________________________________________________

* 아래에서 설명할 때 나오는 주소들은 KASLR이 적용된 상태이다.

1. physmap

physmap(direct map) 은 커널이 시스템 RAM을 커널 가상주소에 선형(고정 오프셋)으로 매핑해 둔 영역이다. kmalloc/slab로 할당된 많은 커널 힙 오브젝트가 이 매핑을 통해 접근되며, 커널 힙 익스플로잇에서 자주 표적이 된다.

0xffffa02d00000000-0xffffa02d00099000 0x0000000000099000 [rw-] physmap
0xffffa02d00099000-0xffffa02d0009a000 0x0000000000001000 [r--] physmap
0xffffa02d0009a000-0xffffa02d0009b000 0x0000000000001000 [r-x] physmap
0xffffa02d0009b000-0xffffa02d69000000 0x0000000068f65000 [rw-] physmap
0xffffa02d69000000-0xffffa02d69c08000 0x0000000000c08000 [r--] physmap
0xffffa02d69c08000-0xffffa02d69e00000 0x00000000001f8000 [rw-] physmap
0xffffa02d69e00000-0xffffa02d6a1ea000 0x00000000003ea000 [r--] physmap
0xffffa02d6a1ea000-0xffffa02dbffe0000 0x0000000055df6000 [rw-] physmap
0xffffa02e00000000-0xffffa02e3a976000 0x000000003a976000 [rw-] physmap
0xffffa02e3a976000-0xffffa02e3a977000 0x0000000000001000 [r--] physmap
0xffffa02e3a977000-0xffffa02e3a978000 0x0000000000001000 [rw-] physmap
0xffffa02e3a978000-0xffffa02e3a979000 0x0000000000001000 [r--] physmap
0xffffa02e3a979000-0xffffa02e40000000 0x0000000005687000 [rw-] physmap

 

2. vmalloc

vmalloc 영역은 물리적으로 연속일 필요 없이, 커널이 가상 주소만 연속으로 보장하고 싶은 메모리를 매핑해 놓은 커널 가상 메모리 공간이다. vmalloc 함수를 통해 할당가능하며 kernel 스택이 존재한다. 

* 커널스택을 vmalloc영역에 두는 이유는 kernel stack overflow를 방지하기 쉽기 때문다. - vmalloc 스택 뒤에 guard page를 넣으면 stack overflow가 났을 때 page fault가 발생해 탐지 할 수 있다.

0xffffbb410040b000-0xffffbb410060b000 0x0000000000200000 [rw-] vmalloc
0xffffbb410060c000-0xffffbb410061c000 0x0000000000010000 [rw-] vmalloc
0xffffbb410061d000-0xffffbb410062d000 0x0000000000010000 [rw-] vmalloc
0xffffbb410062e000-0xffffbb410062f000 0x0000000000001000 [rw-] vmalloc
0xffffbb4100630000-0xffffbb4100634000 0x0000000000004000 [rw-] vmalloc, kstack PID:1 (init)
0xffffbb4100635000-0xffffbb4100636000 0x0000000000001000 [r--] vmalloc
0xffffbb4100638000-0xffffbb410063c000 0x0000000000004000 [rw-] vmalloc, kstack PID:2 (kthreadd)
0xffffbb410063d000-0xffffbb410063e000 0x0000000000001000 [rw-] vmalloc
0xffffbb4100640000-0xffffbb4100644000 0x0000000000004000 [rw-] vmalloc, kstack PID:3 (rcu_gp)
0xffffbb4100645000-0xffffbb4100646000 0x0000000000001000 [rw-] vmalloc
0xffffbb4100648000-0xffffbb410064c000 0x0000000000004000 [rw-] vmalloc, kstack PID:4 (rcu_par_gp)
0xffffbb4100650000-0xffffbb4100654000 0x0000000000004000 [rw-] vmalloc, kstack PID:5 (kworker/0:0)
0xffffbb4100658000-0xffffbb410065c000 0x0000000000004000 [rw-] vmalloc, kstack PID:6 (kworker/0:0H)
0xffffbb4100660000-0xffffbb4100664000 0x0000000000004000 [rw-] vmalloc, kstack PID:7 (kworker/u8:0)
0xffffbb4100668000-0xffffbb410066c000 0x0000000000004000 [rw-] vmalloc, kstack PID:8 (mm_percpu_wq)
0xffffbb4100670000-0xffffbb4100674000 0x0000000000004000 [rw-] vmalloc, kstack PID:9 (ksoftirqd/0)
0xffffbb4100678000-0xffffbb410067c000 0x0000000000004000 [rw-] vmalloc, kstack PID:10 (rcu_sched)
0xffffbb4100680000-0xffffbb4100684000 0x0000000000004000 [rw-] vmalloc, kstack PID:11 (migration/0)
0xffffbb4100688000-0xffffbb410068c000 0x0000000000004000 [rw-] vmalloc, kstack PID:12 (kworker/0:1)

 

3. vmemmap

PFN(Physical Frame Number)마다 1개씩 있는 struct page들의 배열을 커널 가상주소에 매핑해둔 영역이다. page 구조체는 다음과 같이 간략하게 나타낼 수 있다. (실제 소스코드를 보면 더 크다)

struct page {
    unsigned long flags;        
    atomic_t _refcount;        
    struct list_head lru;     
    struct address_space *mapping;
    pgoff_t index;            
    unsigned long private;    
};

pipe_buffer 구조체는 vmemmap을 가리키는데 이를 조작하면 aar, aaw를 얻을 수 있다.

0xfffff22700000000-0xfffff22703000000 0x0000000003000000 [rw-] vmemmap(=page[])
0xfffff22704000000-0xfffff22705000000 0x0000000001000000 [rw-] vmemmap(=page[])

4. cpu_entry

리눅스 커널에서 CPU가 유저→커널 진입 또는 예외/인터럽트 진입을 할 때 필요한 코드와 데이터를 담는 고정된 가상메모리 영역이다. 주소가 고정되어 있어서 kernel exploit 할 때 rw 영역등이 필요한 경우에 사용 할 수 있다.

0xfffffe0000000000-0xfffffe0000002000 0x0000000000002000 [r--] cpu_entry
0xfffffe0000002000-0xfffffe0000003000 0x0000000000001000 [rw-] cpu_entry
0xfffffe0000003000-0xfffffe0000006000 0x0000000000003000 [r--] cpu_entry
0xfffffe0000006000-0xfffffe000000c000 0x0000000000006000 [rw-] cpu_entry
0xfffffe000002c000-0xfffffe000002d000 0x0000000000001000 [r--] cpu_entry
0xfffffe000002d000-0xfffffe000002e000 0x0000000000001000 [rw-] cpu_entry
0xfffffe000002e000-0xfffffe0000031000 0x0000000000003000 [r--] cpu_entry
0xfffffe0000031000-0xfffffe0000037000 0x0000000000006000 [rw-] cpu_entry
0xfffffe0000057000-0xfffffe0000058000 0x0000000000001000 [r--] cpu_entry
0xfffffe0000058000-0xfffffe0000059000 0x0000000000001000 [rw-] cpu_entry
0xfffffe0000059000-0xfffffe000005c000 0x0000000000003000 [r--] cpu_entry
0xfffffe000005c000-0xfffffe0000062000 0x0000000000006000 [rw-] cpu_entry
0xfffffe0000082000-0xfffffe0000083000 0x0000000000001000 [r--] cpu_entry
0xfffffe0000083000-0xfffffe0000084000 0x0000000000001000 [rw-] cpu_entry
0xfffffe0000084000-0xfffffe0000087000 0x0000000000003000 [r--] cpu_entry
0xfffffe0000087000-0xfffffe000008d000 0x0000000000006000 [rw-] cpu_entry

5. kernel base

vmlinux의 .text/.rodata/.data/.bss가 매핑된 커널 주소 구간이다. 

0xffffffff86000000-0xffffffff86c01000 0x0000000000c01000 [r-x] kernel .text
0xffffffff86c01000-0xffffffff86c08000 0x0000000000007000 [r-x]
0xffffffff86c08000-0xffffffff86e00000 0x00000000001f8000 [rw-]
0xffffffff86e00000-0xffffffff871ea000 0x00000000003ea000 [r--] kernel .rodata
0xffffffff871ea000-0xffffffff87200000 0x0000000000016000 [rw-]
0xffffffff87200000-0xffffffff87204000 0x0000000000004000 [rw-] kernel .data, kstack PID:0 (swapper/0)
0xffffffff87204000-0xffffffff8737a000 0x0000000000176000 [rw-] kernel .data
0xffffffff8737a000-0xffffffff87e00000 0x0000000000a86000 [rw-]

6. modules

커널이 Loadable Kernel Module (LKM) 을 동적으로 로드할 때 사용하는 코드/데이터 매핑 공간이다.

0xffffffffc0045000-0xffffffffc0046000 0x0000000000001000 [r-x] modules

 

7. fixmap

fixmap은 커널이 특정 페이지를 고정된 가상주소 슬롯에 박아두는 커널 내부용 fixed mapping 영역이고 userland, fixmap는 레거시 코드가 쓰는 vsyscall(고정 주소 호출) ABI 호환용으로 유저에서도 보이는 고정 매핑이다.

0xffffffffff5fc000-0xffffffffff5fe000 0x0000000000002000 [rw-] fixmap
0xffffffffff600000-0xffffffffff601000 0x0000000000001000 [r--] userland, fixmap

 

 

이번 글에서는 리눅스 커널의 메모리 구조를 간략하게 정리해봤다. 위에 있는 것들은 kernel exploit을 할 때 kvammp 명령어를 통해 나온 공간들의 정체를 설명한 것이다. 위에서 설명 한 것 이외에도 더 많은 공간이 있으니 공식 docs를 읽고 이해해보자.

 

 

reference: https://docs.kernel.org/mm/physical_memory.html

https://docs.kernel.org/mm/vmalloced-kernel-stacks.html

https://www.kernel.org/doc/Documentation/vm/vmalloced-kernel-stacks.rst

https://www.kernel.org/doc/html/v5.8/x86/x86_64/mm.html#complete-virtual-memory-map-with-4-level-page-tables

https://elixir.bootlin.com/linux/v4.6/source/include/linux/mm_types.h#L44

https://unix.stackexchange.com/questions/476768/what-is-cpu-entry-area

https://docs.kernel.org/arch/x86/x86_64/mm.html