Friday, September 26, 2014

Device Tree in Linux

In the series of short notes on kernel concepts. Here is my new post on Device Tree. Device Tree has become a important part of linux kernel now. So lets start with it in my usual way of writing :

What is Device Tree?
- Device Tree is a tree like structure, which is used to store hardware information.
- It is a description of machine's hardware that is readable by the OS, so that kernel code could separated from machine specific information.
- So using Device tree, you can get any information from it at any point of time by parsing it.

What is the benefit of Device Tree?
- It ensures a single board file for a chipset, rest of the device specific information can be moved to device tree.
- Platform devices are created at run-time by the kernel by parsing the device tree nodes.
- we can add the devices which are not present right now, but in future they will, so no need to write the code again at later stage.
- So overall it steps toward the single kernel binary for all variation of chipset, which can be clubbed with specific device tree blob to  create different binaries for different variation. This will reduce the time to support new platform for the chipset.

How to enable DT support ?
- First thing is create an entry in board file, between DT_MACHINE_START & DT_MACHINE_END. you need to look into a board file to understand it better. Remember you must have dt_compat  field with the appropriate dt_mach structure, which will be containing the chipset details.
- Add your device tree source(dts) file. It have the compatible , model and id field, so it should be filled with appropriate variable. [ check /arch/arm/boot/dts/  folder for .dts file]
- To enable building this tree, there are 2 ways :
a) Enable through menuconfig -> boot options -> flattened device tree
or
b) define 'select USE_OF ' in Kconfig entry in arch/arm/Kconfig

Structure of Device Tree:
 Device tree describes a machine's hardware with a combination of

1) Nodes: It represents the device or bus. Contains properties and child nodes. It is defined as node_name@unit_address.

2) Properties: provides information about a device. It consists of a name and a value. Properties can further be categorized as Standard or Non-Standard.

Standard Properties:
a) compatible : it is used to distinguish between different nodes. Recommended format for compatible property is "manufacturer, model". OS can use it for machine/device driver selection.

b) phandle : specifies an unique numerical identifier for a node with the device tree. Used by other nodes that need to refer to the node associated with the property.

c) status : Indicate the operational status of a device. Values for this could be "okay", "disabled", "fail".

d) #address-cells,#size-cells : May be used in any device node that has children in the device tree hierarchy. Describes how the child nodes should be addressed. #address-cells tells no of cells to encode the address field in child's reg property. #size-cells tells no of cells to encode the size field in child's reg property.

e)reg : Arbitrary number of pairs of address and size. It describes the address and the size od the device's resources. It tells no of cells to specify address and size are specified by the #address-cells and #size-cells properties in the parent of the device node.


These are the main things. If you want more details, I will suggest you to refer Documentation/devicetree in linux kernel.

Thursday, September 25, 2014

Short Notes on System Call

This post covers pretty much all main points which you need to know about System Call in Linux. These points are my notes which I covered through many blogs/sites for my study. So lets start it :

What is System Call ?
- As Linux doesn't allow user process to directly modify anything in kernel space. It has provided a way to achieve it, which is known as System Call. So basically system call provide a layer between the kernel and user space process. Using this, kernel performs user space process's task inside the kernel space. This way kernel have control on, what a user process can do.

Why you need a System Call ?
- As system call is the only legal entry point to kernel, sometimes you may need some information, which can only be provided with kernel space credentials, so Kernel developers give you some system call. Although user space doesn't use it directly(in general), they uses some API for the same. 
So the flow will go like 

read() ----> read() wrapper ------> system_call() ------> sys_read()

Here first two thing is in user space and last two is in kernel space.

How do they work ?
- Application program calls the API.
- A system library routine is called first.
- It transforms the call to the system standard and traps to the kernel.
- Control is taken by the kernel running in the system mode.
- According to the code, the call dispatcher invokes the responsible handler.
- Interrupt is disable during this handler.
- After call is finished, mode is changed from system mode to user mode and calling process execution resumes.

How it works at low level ?
-  So whenever you call any system call, it is translated to SWI .
- Whenever processor sees SWI, it interrupts whatever is running currently( except IRQ/FIQ), changes the mode to supervisor mode(SVC).
- Whatever parameters were there in the system call, are passed using registers. 
- Also PC is stored in LR of svc mode and CPSR is saved in SPSR_svc, so that we can recover the previous state while returning from SWI.
- On execution of SWI, processor looks for SWI_handler address, which was already defined in the vector table. 
- From this SWI handler, processor jumps to the specific system call handler. Here we use the system call number to get the corresponding handler.
- Once the system call handler finishes its execution. Processor change the mode to user, and recover PC from LR.

Type of System Call :
- Process control and IPC : fork(), exec(), pipe()
- Memory Management : malloc(), free()
- File and File System management : open(), read(), write()
- Device Management : ioctl()
- Others : kill(), signal()

Implementation of System Call :
For this you can refer my previous blog on creating system call.

Friday, September 19, 2014

Add a Custom System Call in Linux Kernel

Here I will be covering a small tutorial on creating new system call in Linux Kernel. I am using Latest Kernel version 3.16 ( It is latest at the time of blog written).
I added my call for 32 bit system only. 

So the whole process is divided in 5 step.

Step 1:  Open arch/x86/syscalls/syscall_32.tbl . Here go to the last line in the file. It will be containing a number in first column, this number tells that it is the last number used by system for system call. So lets say the number is 356, so your new system call will have number 357. Now just duplicate the last line, and change the number and name of the sys call. Let say it is "hello". So the whole line will look like :

357    i386    hello   sys_hello

Step 2: Add the syntax of syscall in  include/linux/syscalls.h. Suppose this sys call takes 2 int parameters. So for syscall "hello", your new line should be like this :

asmlinkage long sys_hello(int a, int b);

Step 3:  Now add the entry in /kernel/sys_ni.c. So entry will be like :

cond_syscall(sys_hello);

Step 4: Add the function definition for sys call. Open kernel/sys.c. you can add it at different place too.  Now as our sys call is having two parameters, so the function will look like this :

SYSCALL_DEFINE2(hello /*name of syscall */, int /*type of first parameter */,  a /*name of first parameter*/, int /*type of second parameter */,  a /*name of second parameter*/)
{
int error = -EINVAL;
// code for whatever you want to do in syscall 

return 0;
}

That's it, your system call is created, but to reflect it in your kernel, you have to build it. you can refer my blog 


Step 5:  Test the system call. create a userspace program. Don't forget to add sys/syscall.h header. 

Now to call our hello syscall. 

int call = syscall(357, 1,2); // here 357 is our system call number, 1 is val for a and 2 for b.


That will be all from my side. I hope it help you to understand syscall. 

Wednesday, July 30, 2014

Insertion Sort : In non increasing format

Here is the pseudo code for arranging number in non increasing format using Insertion sort. This is one of the exercise of Book Introduction to Algorithms by CLRS.

//N is array's length
for j = N-1 to 1
   key = A[j];
   i = j + 1;
 
   while i<=N && A[i] > key
            A[i-1] = A[i];
            i = i +1;
   A[i-1] = key;

Input : 5 4 3 2 6
Output : 5 4 3 6 2
              5 4 6 3 2
              5 6 4 3 2
              6 5 4 3 2 [Final]

Thursday, July 24, 2014

Short Notes on Memory in Linux

Most of part of this blog will be containing notes from Robert Love's Linux Kernel Development. But I will try to add the general questions also in this blog.


The most basic unit of memory is page. Kernel use page structure to keep track of all the pages in the system. Through this kernel can be informed that whether page is used or free, if used then who is using it etc.

Zone : Because of hardware limitation the kernel can't treat all pages identical. So it is divided in 3 parts mostly : ZONE_DMA(upto 16 MB), ZONE_NORMAL(16 to 896 MB) and ZONE_HIGHMEM(896 & above).

Main limitation to divide pages in Zone are:
1. Some hardware devices can perform DMA to only certain memory address.
2. Some memory can't be permanently mapped into kernel address space.(eg: HIGMEM)

Page allocation interface :

1. struct page * alloc_pages(gfp_t gfp_mask, unsigned int order) : This returns 2 to the power order pages contiguous physical page. As this function returns a page pointer, to get the logical address of it, we use page_address(struct page * page) method.
2. unsigned long  __get_free_pages(gft_t gfp_mask, unsigned int order) : it returns the logical address.
3. unsigned long get_zeroed_pages(gfp_mask) : it is useful for page allocated for user space, as it zeroes the content of allocated page, so that sensitive data doesn't pass to user space.
4. void * kmalloc(size_t size, gfp_t flags): this function returns a pointer to a region of memory that is atleast size bytes in length. This memory is physically contiguous.
5. vamlloc() : it allocates virtually contiguous memory. It does this by allocating potentially non contiguous chunks of physical memory and fixing the page table to map the memory into a contiguous chunk of virtual memory.

Deallocation interface :

1. __free_pages(struct page *page, unsigned int order) : Always ensure you are deallocating only those page which you allocate.
2. kfree(const void *ptr)
3. vfree(const void *ptr)

Introduction to Slab Layer:
Allocation and deallocation memory is the most common operation in Kernel. Because of it, there is good chance of de-fragmentation. Which is complete waste of resources, as you are having the free memory but you cant use it as it is not contiguous.
To resolve this issue, Linux kernel came up with the slab layer concept. So most frequently used data structure is allocated through cache. At the start-up we create the caches for all major data structure. Cache is further divided in slab. And slab contains a page. Apart from that Cache also maintain 3 list, empty list, full list and partial list.
So whenever allocation request comes, Cache check whether any empty page is there on any of the list. It allocates there and then return the page. Once that structure is released, the corresponding page is also released and return back to the free list.

This was the basics, now lets try with questions which I faced, these questions may not in order of relevance:

1. What is the return address of kmalloc() ? Physical or Virtual?
Ans : it always return the virtual address, and the allocated memory region will be physically as well as virtually contiguous.

2. How memory is allocated for program ?
Ans : When ever program compilation is done. At the time of loading, First program is loaded in memory, then mapped it to the Virtual memory. So it will be having page table with no association with physical memory. At the time of use, means when you want some memory for operation, it goes to page table, there MMU find that no physical memory is allocated, so it do the page_fault(). Then the physical memory is allocated and corresponding entry is created in the page table.

3. logical address and virtual address ?
Ans : Most important thing is logical address comes in picture if you have segmentation unit in your system. If it is not there then there is no logical address. So the address translation happens like this :
           logical address => [segmentation unit] => virtual address => [paging unit] => physical address

4. Why physically contiguous memory region is more efficient than virtually contiguous memory?
Ans : As memory allocated is physically contiguous can use a concept called huge pages. With this higher page size can be used, so correspondingly there will be lesser page entries in the table.Huge pages can improve performance through reduced page faults (a single fault brings in a large chunk of memory at once) and by reducing the cost of virtual to physical address translation (fewer levels of page tables must be traversed to get to the physical address). 

5. How Shared memory is used?
Ans : For shared memory we use IPC like shmem. So whenever two process wants to share some memory. They use shmem IPC to get an id for the associated memory, further this shared memory is mapped to both processes's address space. So for those process it looks like local memory. Internally the vm_area_struct (virtual memory area) uses VM_SHARED flag to show this memory area as shared memory.
           Also when we create a child process with CLONE_VM flag, at the time of creation it skips the allocate_mm() call (which is actually responsible for memory allocation) and assign it's mm structure to its parent's mm structure. 

6. Where does the memory is allocated for kernel stack ?
Ans : Kernel stack is allocated in the kernel space, remember not is user space memory. But it is mapped to that Process's address space, not only that but to all other process's address space. As apart from kernel no body will be using this, so it no process is able to recognize it and they don't have permission to access it.

7. Can a process use whole 4 GB address ?
Ans : No, it can't. Remember, memory is divided in memory area. So a process can only access only those memory areas for which it has permission. Even to add or remove memory area, it has to request kernel.




Sunday, June 29, 2014

How to Submit android app to Amazon app store

If you are new to android development and specifically planning to launch your app to Amazon app store. Then you must be feeling a little bit left behind, because there you won't find much over net apart from Amazon's own material. 

Although it is mentioned on their site clearly, I thought of repeating it, because it will just some one else to find the right thing :

So if you want to add the ad api so here is the way : 

1.  Download sdk from here.
2.  Now add the Jar in Eclipse using Install New Software. After that add this link as name Amazon.
3.  Now Import your android project, remove all the google play service api from it, and add Amazon api. To do that right click on your project, and click on Properties. And then in that click on Amazon Mobile app SDK. 
4. There you can select whatever you want to use, Mobile Ads, ADM, In-App purchasing etc.
5. Now it comes to the coding part. Wherever you want to use it, include the following code :

AdRegistration.setAppKey("37de83fd21234ed4811e241507876542F"); // put your Application id
    
   this.adView = (AdLayout) findViewById(R.id.adview);
   
   AdTargetingOptions adOptions = new AdTargetingOptions().enableGeoLocation(true);
   // Optional: Set ad targeting options here.
   this.adView.loadAd(adOptions); // Retrieves an ad on background thread

And you have to create corresponding entry in xml :

 
    android:id="@+id/adview"
    android:layout_width="match_parent" 
    android:layout_height="wrap_content"/>

6. And you are good to go. 
7. Use Test your app option on Amazon before uploading your apk to Amazon app store.

I hope this blog help you to publish your first App on Amazon app store.

Thursday, June 26, 2014

Android : How to Check whether device is rooted or not ?

Many time user go for rooting their phone to break the constraints put by operator and OEMs. 
So Rooting your phone opens a door of new features which could be really useful. 

To root your phone, I will suggest you to go to Xda developers. They have wide range of tutorial specifically for each devices. So once you have followed their steps, and just wanted to confirm whether your device has been rooted, Here is a simple app called Root Verifier Pro Launched by Bitdroid Devz

I will suggest you to try it, it worked fine for me.