307 lines
14 KiB
Plaintext
307 lines
14 KiB
Plaintext
|
=================================
|
||
|
|
FR451 MMU LINUX MEMORY MANAGEMENT
|
||
|
|
=================================
|
||
|
|
|
||
|
|
============
|
||
|
|
MMU HARDWARE
|
||
|
|
============
|
||
|
|
|
||
|
|
FR451 MMU Linux puts the MMU into EDAT mode whilst running. This means that it uses both the SAT
|
||
|
|
registers and the DAT TLB to perform address translation.
|
||
|
|
|
||
|
|
There are 8 IAMLR/IAMPR register pairs and 16 DAMLR/DAMPR register pairs for SAT mode.
|
||
|
|
|
||
|
|
In DAT mode, there is also a TLB organised in cache format as 64 lines x 2 ways. Each line spans a
|
||
|
|
16KB range of addresses, but can match a larger region.
|
||
|
|
|
||
|
|
|
||
|
|
===========================
|
||
|
|
MEMORY MANAGEMENT REGISTERS
|
||
|
|
===========================
|
||
|
|
|
||
|
|
Certain control registers are used by the kernel memory management routines:
|
||
|
|
|
||
|
|
REGISTERS USAGE
|
||
|
|
====================== ==================================================
|
||
|
|
IAMR0, DAMR0 Kernel image and data mappings
|
||
|
|
IAMR1, DAMR1 First-chance TLB lookup mapping
|
||
|
|
DAMR2 Page attachment for cache flush by page
|
||
|
|
DAMR3 Current PGD mapping
|
||
|
|
SCR0, DAMR4 Instruction TLB PGE/PTD cache
|
||
|
|
SCR1, DAMR5 Data TLB PGE/PTD cache
|
||
|
|
DAMR6-10 kmap_atomic() mappings
|
||
|
|
DAMR11 I/O mapping
|
||
|
|
CXNR mm_struct context ID
|
||
|
|
TTBR Page directory (PGD) pointer (physical address)
|
||
|
|
|
||
|
|
|
||
|
|
=====================
|
||
|
|
GENERAL MEMORY LAYOUT
|
||
|
|
=====================
|
||
|
|
|
||
|
|
The physical memory layout is as follows:
|
||
|
|
|
||
|
|
PHYSICAL ADDRESS CONTROLLER DEVICE
|
||
|
|
=================== ============== =======================================
|
||
|
|
00000000 - BFFFFFFF SDRAM SDRAM area
|
||
|
|
E0000000 - EFFFFFFF L-BUS CS2# VDK SLBUS/PCI window
|
||
|
|
F0000000 - F0FFFFFF L-BUS CS5# MB93493 CSC area (DAV daughter board)
|
||
|
|
F1000000 - F1FFFFFF L-BUS CS7# (CB70 CPU-card PCMCIA port I/O space)
|
||
|
|
FC000000 - FC0FFFFF L-BUS CS1# VDK MB86943 config space
|
||
|
|
FC100000 - FC1FFFFF L-BUS CS6# DM9000 NIC I/O space
|
||
|
|
FC200000 - FC2FFFFF L-BUS CS3# MB93493 CSR area (DAV daughter board)
|
||
|
|
FD000000 - FDFFFFFF L-BUS CS4# (CB70 CPU-card extra flash space)
|
||
|
|
FE000000 - FEFFFFFF Internal CPU peripherals
|
||
|
|
FF000000 - FF1FFFFF L-BUS CS0# Flash 1
|
||
|
|
FF200000 - FF3FFFFF L-BUS CS0# Flash 2
|
||
|
|
FFC00000 - FFC0001F L-BUS CS0# FPGA
|
||
|
|
|
||
|
|
The virtual memory layout is:
|
||
|
|
|
||
|
|
VIRTUAL ADDRESS PHYSICAL TRANSLATOR FLAGS SIZE OCCUPATION
|
||
|
|
================= ======== ============== ======= ======= ===================================
|
||
|
|
00004000-BFFFFFFF various TLB,xAMR1 D-N-??V 3GB Userspace
|
||
|
|
C0000000-CFFFFFFF 00000000 xAMPR0 -L-S--V 256MB Kernel image and data
|
||
|
|
D0000000-D7FFFFFF various TLB,xAMR1 D-NS??V 128MB vmalloc area
|
||
|
|
D8000000-DBFFFFFF various TLB,xAMR1 D-NS??V 64MB kmap() area
|
||
|
|
DC000000-DCFFFFFF various TLB 1MB Secondary kmap_atomic() frame
|
||
|
|
DD000000-DD27FFFF various DAMR 160KB Primary kmap_atomic() frame
|
||
|
|
DD040000 DAMR2/IAMR2 -L-S--V page Page cache flush attachment point
|
||
|
|
DD080000 DAMR3 -L-SC-V page Page Directory (PGD)
|
||
|
|
DD0C0000 DAMR4 -L-SC-V page Cached insn TLB Page Table lookup
|
||
|
|
DD100000 DAMR5 -L-SC-V page Cached data TLB Page Table lookup
|
||
|
|
DD140000 DAMR6 -L-S--V page kmap_atomic(KM_BOUNCE_READ)
|
||
|
|
DD180000 DAMR7 -L-S--V page kmap_atomic(KM_SKB_SUNRPC_DATA)
|
||
|
|
DD1C0000 DAMR8 -L-S--V page kmap_atomic(KM_SKB_DATA_SOFTIRQ)
|
||
|
|
DD200000 DAMR9 -L-S--V page kmap_atomic(KM_USER0)
|
||
|
|
DD240000 DAMR10 -L-S--V page kmap_atomic(KM_USER1)
|
||
|
|
E0000000-FFFFFFFF E0000000 DAMR11 -L-SC-V 512MB I/O region
|
||
|
|
|
||
|
|
IAMPR1 and DAMPR1 are used as an extension to the TLB.
|
||
|
|
|
||
|
|
|
||
|
|
====================
|
||
|
|
KMAP AND KMAP_ATOMIC
|
||
|
|
====================
|
||
|
|
|
||
|
|
To access pages in the page cache (which may not be directly accessible if highmem is available),
|
||
|
|
the kernel calls kmap(), does the access and then calls kunmap(); or it calls kmap_atomic(), does
|
||
|
|
the access and then calls kunmap_atomic().
|
||
|
|
|
||
|
|
kmap() creates an attachment between an arbitrary inaccessible page and a range of virtual
|
||
|
|
addresses by installing a PTE in a special page table. The kernel can then access this page as it
|
||
|
|
wills. When it's finished, the kernel calls kunmap() to clear the PTE.
|
||
|
|
|
||
|
|
kmap_atomic() does something slightly different. In the interests of speed, it chooses one of two
|
||
|
|
strategies:
|
||
|
|
|
||
|
|
(1) If possible, kmap_atomic() attaches the requested page to one of DAMPR5 through DAMPR10
|
||
|
|
register pairs; and the matching kunmap_atomic() clears the DAMPR. This makes high memory
|
||
|
|
support really fast as there's no need to flush the TLB or modify the page tables. The DAMLR
|
||
|
|
registers being used for this are preset during boot and don't change over the lifetime of the
|
||
|
|
process. There's a direct mapping between the first few kmap_atomic() types, DAMR number and
|
||
|
|
virtual address slot.
|
||
|
|
|
||
|
|
However, there are more kmap_atomic() types defined than there are DAMR registers available,
|
||
|
|
so we fall back to:
|
||
|
|
|
||
|
|
(2) kmap_atomic() uses a slot in the secondary frame (determined by the type parameter), and then
|
||
|
|
locks an entry in the TLB to translate that slot to the specified page. The number of slots is
|
||
|
|
obviously limited, and their positions are controlled such that each slot is matched by a
|
||
|
|
different line in the TLB. kunmap() ejects the entry from the TLB.
|
||
|
|
|
||
|
|
Note that the first three kmap atomic types are really just declared as placeholders. The DAMPR
|
||
|
|
registers involved are actually modified directly.
|
||
|
|
|
||
|
|
Also note that kmap() itself may sleep, kmap_atomic() may never sleep and both always succeed;
|
||
|
|
furthermore, a driver using kmap() may sleep before calling kunmap(), but may not sleep before
|
||
|
|
calling kunmap_atomic() if it had previously called kmap_atomic().
|
||
|
|
|
||
|
|
|
||
|
|
===============================
|
||
|
|
USING MORE THAN 256MB OF MEMORY
|
||
|
|
===============================
|
||
|
|
|
||
|
|
The kernel cannot access more than 256MB of memory directly. The physical layout, however, permits
|
||
|
|
up to 3GB of SDRAM (possibly 3.25GB) to be made available. By using CONFIG_HIGHMEM, the kernel can
|
||
|
|
allow userspace (by way of page tables) and itself (by way of kmap) to deal with the memory
|
||
|
|
allocation.
|
||
|
|
|
||
|
|
External devices can, of course, still DMA to and from all of the SDRAM, even if the kernel can't
|
||
|
|
see it directly. The kernel translates page references into real addresses for communicating to the
|
||
|
|
devices.
|
||
|
|
|
||
|
|
|
||
|
|
===================
|
||
|
|
PAGE TABLE TOPOLOGY
|
||
|
|
===================
|
||
|
|
|
||
|
|
The page tables are arranged in 2-layer format. There is a middle layer (PMD) that would be used in
|
||
|
|
3-layer format tables but that is folded into the top layer (PGD) and so consumes no extra memory
|
||
|
|
or processing power.
|
||
|
|
|
||
|
|
+------+ PGD PMD
|
||
|
|
| TTBR |--->+-------------------+
|
||
|
|
+------+ | | : STE |
|
||
|
|
| PGE0 | PME0 : STE |
|
||
|
|
| | : STE |
|
||
|
|
+-------------------+ Page Table
|
||
|
|
| | : STE -------------->+--------+ +0x0000
|
||
|
|
| PGE1 | PME0 : STE -----------+ | PTE0 |
|
||
|
|
| | : STE -------+ | +--------+
|
||
|
|
+-------------------+ | | | PTE63 |
|
||
|
|
| | : STE | | +-->+--------+ +0x0100
|
||
|
|
| PGE2 | PME0 : STE | | | PTE64 |
|
||
|
|
| | : STE | | +--------+
|
||
|
|
+-------------------+ | | PTE127 |
|
||
|
|
| | : STE | +------>+--------+ +0x0200
|
||
|
|
| PGE3 | PME0 : STE | | PTE128 |
|
||
|
|
| | : STE | +--------+
|
||
|
|
+-------------------+ | PTE191 |
|
||
|
|
+--------+ +0x0300
|
||
|
|
|
||
|
|
Each Page Directory (PGD) is 16KB (page size) in size and is divided into 64 entries (PGEs). Each
|
||
|
|
PGE contains one Page Mid Directory (PMD).
|
||
|
|
|
||
|
|
Each PMD is 256 bytes in size and contains a single entry (PME). Each PME holds 64 FR451 MMU
|
||
|
|
segment table entries of 4 bytes apiece. Each PME "points to" a page table. In practice, each STE
|
||
|
|
points to a subset of the page table, the first to PT+0x0000, the second to PT+0x0100, the third to
|
||
|
|
PT+0x200, and so on.
|
||
|
|
|
||
|
|
Each PGE and PME covers 64MB of the total virtual address space.
|
||
|
|
|
||
|
|
Each Page Table (PTD) is 16KB (page size) in size, and is divided into 4096 entries (PTEs). Each
|
||
|
|
entry can point to one 16KB page. In practice, each Linux page table is subdivided into 64 FR451
|
||
|
|
MMU page tables. But they are all grouped together to make management easier, in particular rmap
|
||
|
|
support is then trivial.
|
||
|
|
|
||
|
|
Grouping page tables in this fashion makes PGE caching in SCR0/SCR1 more efficient because the
|
||
|
|
coverage of the cached item is greater.
|
||
|
|
|
||
|
|
Page tables for the vmalloc area are allocated at boot time and shared between all mm_structs.
|
||
|
|
|
||
|
|
|
||
|
|
=================
|
||
|
|
USER SPACE LAYOUT
|
||
|
|
=================
|
||
|
|
|
||
|
|
For MMU capable Linux, the regions userspace code are allowed to access are kept entirely separate
|
||
|
|
from those dedicated to the kernel:
|
||
|
|
|
||
|
|
VIRTUAL ADDRESS SIZE PURPOSE
|
||
|
|
================= ===== ===================================
|
||
|
|
00000000-00003fff 4KB NULL pointer access trap
|
||
|
|
00004000-01ffffff ~32MB lower mmap space (grows up)
|
||
|
|
02000000-021fffff 2MB Stack space (grows down from top)
|
||
|
|
02200000-nnnnnnnn Executable mapping
|
||
|
|
nnnnnnnn- brk space (grows up)
|
||
|
|
-bfffffff upper mmap space (grows down)
|
||
|
|
|
||
|
|
This is so arranged so as to make best use of the 16KB page tables and the way in which PGEs/PMEs
|
||
|
|
are cached by the TLB handler. The lower mmap space is filled first, and then the upper mmap space
|
||
|
|
is filled.
|
||
|
|
|
||
|
|
|
||
|
|
===============================
|
||
|
|
GDB-STUB MMU DEBUGGING SERVICES
|
||
|
|
===============================
|
||
|
|
|
||
|
|
The gdb-stub included in this kernel provides a number of services to aid in the debugging of MMU
|
||
|
|
related kernel services:
|
||
|
|
|
||
|
|
(*) Every time the kernel stops, certain state information is dumped into __debug_mmu. This
|
||
|
|
variable is defined in arch/frv/kernel/gdb-stub.c. Note that the gdbinit file in this
|
||
|
|
directory has some useful macros for dealing with this.
|
||
|
|
|
||
|
|
(*) __debug_mmu.tlb[]
|
||
|
|
|
||
|
|
This receives the current TLB contents. This can be viewed with the _tlb GDB macro:
|
||
|
|
|
||
|
|
(gdb) _tlb
|
||
|
|
tlb[0x00]: 01000005 00718203 01000002 00718203
|
||
|
|
tlb[0x01]: 01004002 006d4201 01004005 006d4203
|
||
|
|
tlb[0x02]: 01008002 006d0201 01008006 00004200
|
||
|
|
tlb[0x03]: 0100c006 007f4202 0100c002 0064c202
|
||
|
|
tlb[0x04]: 01110005 00774201 01110002 00774201
|
||
|
|
tlb[0x05]: 01114005 00770201 01114002 00770201
|
||
|
|
tlb[0x06]: 01118002 0076c201 01118005 0076c201
|
||
|
|
...
|
||
|
|
tlb[0x3d]: 010f4002 00790200 001f4002 0054ca02
|
||
|
|
tlb[0x3e]: 010f8005 0078c201 010f8002 0078c201
|
||
|
|
tlb[0x3f]: 001fc002 0056ca01 001fc005 00538a01
|
||
|
|
|
||
|
|
(*) __debug_mmu.iamr[]
|
||
|
|
(*) __debug_mmu.damr[]
|
||
|
|
|
||
|
|
These receive the current IAMR and DAMR contents. These can be viewed with the _amr
|
||
|
|
GDB macro:
|
||
|
|
|
||
|
|
(gdb) _amr
|
||
|
|
AMRx DAMR IAMR
|
||
|
|
==== ===================== =====================
|
||
|
|
amr0 : L:c0000000 P:00000cb9 : L:c0000000 P:000004b9
|
||
|
|
amr1 : L:01070005 P:006f9203 : L:0102c005 P:006a1201
|
||
|
|
amr2 : L:d8d00000 P:00000000 : L:d8d00000 P:00000000
|
||
|
|
amr3 : L:d8d04000 P:00534c0d : L:00000000 P:00000000
|
||
|
|
amr4 : L:d8d08000 P:00554c0d : L:00000000 P:00000000
|
||
|
|
amr5 : L:d8d0c000 P:00554c0d : L:00000000 P:00000000
|
||
|
|
amr6 : L:d8d10000 P:00000000 : L:00000000 P:00000000
|
||
|
|
amr7 : L:d8d14000 P:00000000 : L:00000000 P:00000000
|
||
|
|
amr8 : L:d8d18000 P:00000000
|
||
|
|
amr9 : L:d8d1c000 P:00000000
|
||
|
|
amr10: L:d8d20000 P:00000000
|
||
|
|
amr11: L:e0000000 P:e0000ccd
|
||
|
|
|
||
|
|
(*) The current task's page directory is bound to DAMR3.
|
||
|
|
|
||
|
|
This can be viewed with the _pgd GDB macro:
|
||
|
|
|
||
|
|
(gdb) _pgd
|
||
|
|
$3 = {{pge = {{ste = {0x554001, 0x554101, 0x554201, 0x554301, 0x554401,
|
||
|
|
0x554501, 0x554601, 0x554701, 0x554801, 0x554901, 0x554a01,
|
||
|
|
0x554b01, 0x554c01, 0x554d01, 0x554e01, 0x554f01, 0x555001,
|
||
|
|
0x555101, 0x555201, 0x555301, 0x555401, 0x555501, 0x555601,
|
||
|
|
0x555701, 0x555801, 0x555901, 0x555a01, 0x555b01, 0x555c01,
|
||
|
|
0x555d01, 0x555e01, 0x555f01, 0x556001, 0x556101, 0x556201,
|
||
|
|
0x556301, 0x556401, 0x556501, 0x556601, 0x556701, 0x556801,
|
||
|
|
0x556901, 0x556a01, 0x556b01, 0x556c01, 0x556d01, 0x556e01,
|
||
|
|
0x556f01, 0x557001, 0x557101, 0x557201, 0x557301, 0x557401,
|
||
|
|
0x557501, 0x557601, 0x557701, 0x557801, 0x557901, 0x557a01,
|
||
|
|
0x557b01, 0x557c01, 0x557d01, 0x557e01, 0x557f01}}}}, {pge = {{
|
||
|
|
ste = {0x0 <repeats 64 times>}}}} <repeats 51 times>, {pge = {{ste = {
|
||
|
|
0x248001, 0x248101, 0x248201, 0x248301, 0x248401, 0x248501,
|
||
|
|
0x248601, 0x248701, 0x248801, 0x248901, 0x248a01, 0x248b01,
|
||
|
|
0x248c01, 0x248d01, 0x248e01, 0x248f01, 0x249001, 0x249101,
|
||
|
|
0x249201, 0x249301, 0x249401, 0x249501, 0x249601, 0x249701,
|
||
|
|
0x249801, 0x249901, 0x249a01, 0x249b01, 0x249c01, 0x249d01,
|
||
|
|
0x249e01, 0x249f01, 0x24a001, 0x24a101, 0x24a201, 0x24a301,
|
||
|
|
0x24a401, 0x24a501, 0x24a601, 0x24a701, 0x24a801, 0x24a901,
|
||
|
|
0x24aa01, 0x24ab01, 0x24ac01, 0x24ad01, 0x24ae01, 0x24af01,
|
||
|
|
0x24b001, 0x24b101, 0x24b201, 0x24b301, 0x24b401, 0x24b501,
|
||
|
|
0x24b601, 0x24b701, 0x24b801, 0x24b901, 0x24ba01, 0x24bb01,
|
||
|
|
0x24bc01, 0x24bd01, 0x24be01, 0x24bf01}}}}, {pge = {{ste = {
|
||
|
|
0x0 <repeats 64 times>}}}} <repeats 11 times>}
|
||
|
|
|
||
|
|
(*) The PTD last used by the instruction TLB miss handler is attached to DAMR4.
|
||
|
|
(*) The PTD last used by the data TLB miss handler is attached to DAMR5.
|
||
|
|
|
||
|
|
These can be viewed with the _ptd_i and _ptd_d GDB macros:
|
||
|
|
|
||
|
|
(gdb) _ptd_d
|
||
|
|
$5 = {{pte = 0x0} <repeats 127 times>, {pte = 0x539b01}, {
|
||
|
|
pte = 0x0} <repeats 896 times>, {pte = 0x719303}, {pte = 0x6d5303}, {
|
||
|
|
pte = 0x0}, {pte = 0x0}, {pte = 0x0}, {pte = 0x0}, {pte = 0x0}, {
|
||
|
|
pte = 0x0}, {pte = 0x0}, {pte = 0x0}, {pte = 0x0}, {pte = 0x6a1303}, {
|
||
|
|
pte = 0x0} <repeats 12 times>, {pte = 0x709303}, {pte = 0x0}, {pte = 0x0},
|
||
|
|
{pte = 0x6fd303}, {pte = 0x6f9303}, {pte = 0x6f5303}, {pte = 0x0}, {
|
||
|
|
pte = 0x6ed303}, {pte = 0x531b01}, {pte = 0x50db01}, {
|
||
|
|
pte = 0x0} <repeats 13 times>, {pte = 0x5303}, {pte = 0x7f5303}, {
|
||
|
|
pte = 0x509b01}, {pte = 0x505b01}, {pte = 0x7c9303}, {pte = 0x7b9303}, {
|
||
|
|
pte = 0x7b5303}, {pte = 0x7b1303}, {pte = 0x7ad303}, {pte = 0x0}, {
|
||
|
|
pte = 0x0}, {pte = 0x7a1303}, {pte = 0x0}, {pte = 0x795303}, {pte = 0x0}, {
|
||
|
|
pte = 0x78d303}, {pte = 0x0}, {pte = 0x0}, {pte = 0x0}, {pte = 0x0}, {
|
||
|
|
pte = 0x0}, {pte = 0x775303}, {pte = 0x771303}, {pte = 0x76d303}, {
|
||
|
|
pte = 0x0}, {pte = 0x765303}, {pte = 0x7c5303}, {pte = 0x501b01}, {
|
||
|
|
pte = 0x4f1b01}, {pte = 0x4edb01}, {pte = 0x0}, {pte = 0x4f9b01}, {
|
||
|
|
pte = 0x4fdb01}, {pte = 0x0} <repeats 2992 times>}
|