Welcome to the Linux Foundation Forum!

Beginning Kernel Hacking

cheesysam
cheesysam Posts: 4
in Kernel Development

I'm going to be doing some kernel hacking as part of a final year compsci project.

What is the best way of starting this, for instance is there some sort of sandbox mode for running the kernel. My current attempts have been with Arch and slackware linux running in a VM but has proven to be a nightmare.

Where do I start with this?

Comments

  • Set_Killer
    Set_Killer Posts: 31
    maybe try to recompile the kernel by hand. the best way to do that is to use Gentoo, but they are prepathcing the kernel before put it in the repository. i think you can do this with any other distro.

    after that read the code of some modules and try to find bugss or howto abuse them. but be sure that the kernel is not written from newbie programmers.

    am not so advanced. it would be interesting to hear the opiniion of the gurus.
  • ScHmItCh
    ScHmItCh Posts: 10
    Gentoo is a good place to start if you want to learn about the kernel. You can also do recompiles in other systems. I would recommend doing a few customizations before you go hacking away at the source code.
  • woboyle
    woboyle Posts: 501
    First, visit www.kernel.org and www.tldp.org for useful how-tos and other kernel documentation.

    Next, don't give up on running your experimental OS in a virtual machine. I have done a fair amount of kernel development for both Linux and Solaris with VirtualBox VM's. The important thing here is that the host needs to be a stable system - I use CentOS/RHEL 5 as my host (8 cores and 8gb ram) and run newer Linux clients (Gentoo, Ubuntu, et al) as well as Windows and Solaris x86 as guests.
  • timhoppen
    timhoppen Posts: 3
    Actually, in Gentoo, you can get a kernel.org kernel without any patches.
    There are a variety of kernel packages available. Here are 3:

    To use the kernel patched by Gentoo: 
    emerge gentoo-sources

    To use the latest released kernel from kernel.org 
    emerge vanilla-sources

    Or, to use the daily git snapshot from git.kernel.org 
    emerge git-sources


    http://www.gentoo.org/doc/en/gentoo-kernel.xml

    Or, as with any distro, you can just download the tarball from kernel.org and have fun. Gentoo (potentially) makes it easier because it is required to compile the kernel yourself anyway.

    In addition, make sure that you know what areas of the kernel you want to hack at. Honing your focus will help a ton. The kernel is a big piece of code!
  • ford
    ford Posts: 24
    I would take a look at GoboLinux. GoboLinux uses a kernel patch to hide the traditional filehierarchy's structure. Something that is rather simple like this would be a great start.

    You may also want to look at some projects online such as Linux ABI and Longene.
  • amnesia
    amnesia Posts: 60
    easiest thing to do would be get a linux distro with the config file exported to /proc as 'config.gz'

    (by default) some distributions already have this enabled in the kernel before they release (slackware) :)

    next step would be
    cd /usr/src
    wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.x.xx.x.tar.bz2
    tar xvjf linux-2.x.xx.x.tar.bz2
    cd linux-2.x.xx.x.tar.bz2
    zcat /proc/config.gz > $PWD/.config
    make oldconfig;make menuconfig
    >make oldconfig may or may not ask you if you'd like enable certain >new features
    make menuconfig and edit in our out what you need and dont need
    exit and save

    cd src modules or whatever and look around.
    change whatever you feel you can and are able to change really

    cd /usr/src/linux-whatever

    make modules; make modules_install

    cp System.map /boot;cd arch/i386/boot/;cp bZimage /boot/kernel-whatever

    add lines (if you use it) to lilo
    image = /boot/kernel-whatever
    root = /dev/hda1
    label = whatever
    read-only

    lilo

    reboot.

    (not saying you don't know how to recompile a kernel, as kernel hacking goes -- you've got alot of lines to go through, have fun :P)

    if anythings wrong, you'll know with a kernel panic :>
  • cheesysam
    cheesysam Posts: 4
    Hi everyone, I'm overwhelmed by the great responses, thanks!

    Rubberman - Thanks for your motivation to keep me with VirtualBox. I'm using OS X 10.5 as a host at the moment which is as stable OS as I know. Presumably if that becomes a problem I can just backup the machine image and use it in Virtualbox on a different host?
    Thanks for the links also. I'll get reading.

    N.D, Josh, Tim - Gentoo seems like a common suggestion so I'll get that and get it installed.

    Hi Bradford - Unfortunately I think hiding the file hierarchy will contradict the aims of the project but thanks for the suggestion. I will take a look at the projects you mentioned though :)

    Ajay - Thanks for the little walkthrough. I have managed to get as far as compiling the kernel in a ubuntu distro but nothing seemed to be different after the edit/recompile/install/reboot process. I will try again with your advice once I get this system set up and stable!

    My original post came across like i just want to throw in random bits of code and see what happens, not the case! Hopefully it will be a bit more planned than that.

    As for where I'm focusing attention, I plan to look into doing some VFS modification for the benefit of simplification for the user when browsing directories and such-like. Purely experimental for the project I mentioned. When I get something working as intended I can post back here ;)

    In the meantime, thanks for the advice. I've got plenty to be getting on with but I'm sure I'll be back here for some more help at some point!

Categories

Upcoming Training