Short Notes on Memory in Linux

Most of part of this blog will be containing notes from Robert Love's Linux Kernel Development. But I will try to add the general questions also in this blog.


The most basic unit of memory is page. Kernel use page structure to keep track of all the pages in the system. Through this kernel can be informed that whether page is used or free, if used then who is using it etc.

Zone : Because of hardware limitation the kernel can't treat all pages identical. So it is divided in 3 parts mostly : ZONE_DMA(upto 16 MB), ZONE_NORMAL(16 to 896 MB) and ZONE_HIGHMEM(896 & above).

Main limitation to divide pages in Zone are:
1. Some hardware devices can perform DMA to only certain memory address.
2. Some memory can't be permanently mapped into kernel address space.(eg: HIGMEM)

Page allocation interface :

1. struct page * alloc_pages(gfp_t gfp_mask, unsigned int order) : This returns 2 to the power order pages contiguous physical page. As this function returns a page pointer, to get the logical address of it, we use page_address(struct page * page) method.
2. unsigned long  __get_free_pages(gft_t gfp_mask, unsigned int order) : it returns the logical address.
3. unsigned long get_zeroed_pages(gfp_mask) : it is useful for page allocated for user space, as it zeroes the content of allocated page, so that sensitive data doesn't pass to user space.
4. void * kmalloc(size_t size, gfp_t flags): this function returns a pointer to a region of memory that is atleast size bytes in length. This memory is physically contiguous.
5. vamlloc() : it allocates virtually contiguous memory. It does this by allocating potentially non contiguous chunks of physical memory and fixing the page table to map the memory into a contiguous chunk of virtual memory.

Deallocation interface :

1. __free_pages(struct page *page, unsigned int order) : Always ensure you are deallocating only those page which you allocate.
2. kfree(const void *ptr)
3. vfree(const void *ptr)

Introduction to Slab Layer:
Allocation and deallocation memory is the most common operation in Kernel. Because of it, there is good chance of de-fragmentation. Which is complete waste of resources, as you are having the free memory but you cant use it as it is not contiguous.
To resolve this issue, Linux kernel came up with the slab layer concept. So most frequently used data structure is allocated through cache. At the start-up we create the caches for all major data structure. Cache is further divided in slab. And slab contains a page. Apart from that Cache also maintain 3 list, empty list, full list and partial list.
So whenever allocation request comes, Cache check whether any empty page is there on any of the list. It allocates there and then return the page. Once that structure is released, the corresponding page is also released and return back to the free list.

This was the basics, now lets try with questions which I faced, these questions may not in order of relevance:

1. What is the return address of kmalloc() ? Physical or Virtual?
Ans : it always return the virtual address, and the allocated memory region will be physically as well as virtually contiguous.

2. How memory is allocated for program ?
Ans : When ever program compilation is done. At the time of loading, First program is loaded in memory, then mapped it to the Virtual memory. So it will be having page table with no association with physical memory. At the time of use, means when you want some memory for operation, it goes to page table, there MMU find that no physical memory is allocated, so it do the page_fault(). Then the physical memory is allocated and corresponding entry is created in the page table.

3. logical address and virtual address ?
Ans : Most important thing is logical address comes in picture if you have segmentation unit in your system. If it is not there then there is no logical address. So the address translation happens like this :
           logical address => [segmentation unit] => virtual address => [paging unit] => physical address

4. Why physically contiguous memory region is more efficient than virtually contiguous memory?

Ans : As memory allocated is physically contiguous can use a concept called huge pages. With this higher page size can be used, so correspondingly there will be lesser page entries in the table.Huge pages can improve performance through reduced page faults (a single fault brings in a large chunk of memory at once) and by reducing the cost of virtual to physical address translation (fewer levels of page tables must be traversed to get to the physical address). 

5. How Shared memory is used?
Ans : For shared memory we use IPC like shmem. So whenever two process wants to share some memory. They use shmem IPC to get an id for the associated memory, further this shared memory is mapped to both processes's address space. So for those process it looks like local memory. Internally the vm_area_struct (virtual memory area) uses VM_SHARED flag to show this memory area as shared memory.
           Also when we create a child process with CLONE_VM flag, at the time of creation it skips the allocate_mm() call (which is actually responsible for memory allocation) and assign it's mm structure to its parent's mm structure. 

6. Where does the memory is allocated for kernel stack ?
Ans : Kernel stack is allocated in the kernel space, remember not is user space memory. But it is mapped to that Process's address space, not only that but to all other process's address space. As apart from kernel no body will be using this, so it no process is able to recognize it and they don't have permission to access it.

7. Can a process use whole 4 GB address ?
Ans : No, it can't. Remember, memory is divided in memory area. So a process can only access only those memory areas for which it has permission. Even to add or remove memory area, it has to request kernel.