Add a Custom System Call in Linux Kernel

Here I will be covering a small tutorial on creating new system call in Linux Kernel. I am using Latest Kernel version 3.16 ( It is latest at the time of blog written).

I added my call for 32 bit system only. 

So the whole process is divided in 5 step.

Step 1:  Open arch/x86/syscalls/syscall_32.tbl . Here go to the last line in the file. It will be containing a number in first column, this number tells that it is the last number used by system for system call. So lets say the number is 356, so your new system call will have number 357. Now just duplicate the last line, and change the number and name of the sys call. Let say it is "hello". So the whole line will look like :

357    i386    hello   sys_hello

Step 2: Add the syntax of syscall in  include/linux/syscalls.h. Suppose this sys call takes 2 int parameters. So for syscall "hello", your new line should be like this :

asmlinkage long sys_hello(int a, int b);

Step 3:  Now add the entry in /kernel/sys_ni.c. So entry will be like :

cond_syscall(sys_hello);

Step 4: Add the function definition for sys call. Open kernel/sys.c. you can add it at different place too.  Now as our sys call is having two parameters, so the function will look like this :

SYSCALL_DEFINE2(hello /*name of syscall */, int /*type of first parameter */,  a /*name of first parameter*/, int /*type of second parameter */,  a /*name of second parameter*/)

{

int error = -EINVAL;

// code for whatever you want to do in syscall 

return 0;

}

That's it, your system call is created, but to reflect it in your kernel, you have to build it. you can refer my blog 

Step 5:  Test the system call. create a userspace program. Don't forget to add sys/syscall.h header. 

Now to call our hello syscall. 

int call = syscall(357, 1,2); // here 357 is our system call number, 1 is val for a and 2 for b.

That will be all from my side. I hope it help you to understand syscall.