Saturday, December 17, 2005

Debian HOW-TO: Custom Kernel Compile

Last revision : February 23th, 2007

You want to compile your very own linux kernel ? No ? Well you should ! Here is a simple receipt.

The steps described below work for both Debian and Ubuntu.

So, read carefully.


  • Preparing the system

Note that I use sudo to get root privileges, refer to this tutorial for more info about that.

First install the required packages :

sudo apt-get install fakeroot bzip2 kernel-package libncurses-dev



Were you planning to compile as root ? That's evil, don't do it. Compile as a normal user : for that you have to put yourself in the src group. My username is mastah, so I type :

sudo adduser mastah src


Then you must logout, and log back in.


  • Getting the source

First you must download the latest kernel. Browse the Linux Kernel FTP server and download the latest kernel, or simply issue that command while you are in your download directory :

wget ftp://ftp.kernel.org/pub/linux/kernel/v2.6/linux-2.6.20.tar.bz2
wget ftp://ftp.kernel.org/pub/linux/kernel/v2.6/patch-2.6.20.1.bz2



  • Configuration & Compilation

The work begins :

cd /usr/src
tar jxf /path/to/linux-2.6.20.tar.bz2


At this point prepare to apply the latest patch (or the one of your choice). Remember that patches are generally mutually exclusive : you can only apply one patch to the vanilla (generic) kernel. Some notorious kernel patches include : the realtime patches by Ingo Molnar, and of course the CK patches by Con Kolivas.

In this example we are to use the official 2.6.20.1 patch :

mv linux-2.6.20/ linux-2.6.20.1
cd linux-2.6.20.1/


Note that if we were to apply the CK patch, replacing 2.6.20.1 with 2.6.20-ck1 would do the trick.

Then test that the patch will apply cleanly :

bzcat /path/to/patch-2.6.20.1.bz2 | patch -p1 --dry-run


If everything outputs ok, then you can apply the patch for real :

bzcat /path/to/patch-2.6.20.1.bz2 | patch -p1


Use the configuration of your choice (I generally use Debian's kernel one) :

cp /boot/config-$(uname -r) .config


Then the fun starts.

To enter the kernel configuration utility, type :

make menuconfig



Some important points regarding kernel configuration, presented as examples :

Processor type and features ---> Processor family ---> (X) Core 2/newer Xeon
...use your specific CPU here.


Processor type and features ---> Timer Frequency ---> (X) 1000 HZ
...for a desktop machine, a high timer frequency is often ideal, as it will confer low latency. For a laptop, choosing a lower frequency will save battery power.


Processor type and features ---> Preemption Model ---> (X) Preemptible Kernel (Low-Latency Desktop)
...for a desktop machine, you can choose either Volontary Kernel Preemption or Preemptible Kernel. A fully preemptible kernel will have lower throughput, but quicker response time.


Kernel Hacking --->
...this refers to kernel debugging and other various options mainly useful to kernel developers : if you don't plan to do some kernel hacking, you can turn off all those features, as they make the kernel bigger (thus slower).


Device Drivers ---> Graphics support ---> < > Support for frame buffer devices
...you want boot time to be as short as possible, so why care about a boot splash image or fancy fonts in console ? So unless you need this, you can turn it off.


Device Drivers ---> Virtualization  --->  Kernel-based Virtual Machine (KVM) support
...a new feature as of Linux kernel 2.6.20. Be sure to also include INTEL or AMD specific module if you are to use KVM. Very useful feature for virtualization machines like QEMU.

The rest is only optional : setting IDE drivers, SATA drivers (if applicable) and main filesystem to Y (building them statically) will allow you to build a kernel without a RAM disk (initrd), thus speeding the boot process. You have to know your specific chipsets names (output of lspci) by which they are referred by the kernel team. Check the kernel documentation for more (in the /path/to/kernel/Documentation directory).

Finally, don't forget to SAVE.

Now clean the source tree :

fakeroot make-kpkg clean


Then build the kernel package :

fakeroot make-kpkg --append-to-version "<-suffix>" --revision "<revision#>" --us --uc --initrd kernel_image kernel_headers



  • Installation

Install the newly built kernel and its headers :

cd ..
sudo dpkg -i linux-image-2.6.20.1<-suffix>_<revision#>_i386.deb
sudo dpkg -i linux-headers-2.6.20.1<-suffix>_<revision#>_i386.deb


Replace <-suffix> with whatever you want (like -mykernel), and change <revision#> with a number (like the date, 23022007).

Using the --initrd (RAM disk, to preloads some required modules) parameter gives a working kernel with less efforts (you can build your kernel in an entirely modular way, so you don't have to tweak everything).

Everything is ready, you can reboot...

Make sure you boot into your new kernel!

20 comments:

Costas said...

Hi there,

If I compile a custom kernel, so I lose the capability of using my distros apt-get utility?
If so, what can be done to remedy this?

Cheers

Hugues said...

Not sure I understand what you mean exactly.

"lose the capability of using my distros apt-get utility"?

I posted a little more up-to-date how-to here (my nickname is "antidrugue" on linuxforums.org) :
http://www.linuxforums.org/forum/331741-post2.html

Or you can check the "Debian Reference, Chapter 7" :
http://www.debian.org/doc/manuals/reference/ch-kernel.en.html

"apt-get" will work just fine on a custom compiled kernel. Same way it works on a stock Debian kernel.

Thanks for the feed back.

Costa said...

Hi there,

Cheers for replying.

I guess I am not 100% clear on how packages work on linux.
Once you have your source ready and you decide to compile it in a deb package
will that deb ONLY work on the specific kernel it was built on or is it possible to use it on other kernels?

I assume that you can use deb packages compiled versus a 2.6 kernel on any incarnation of the 2.6 kernel, be it 2.6.5 or 2.6.17(latest). Is that so?
Also, I assume 2.6 deb packages will probably not work on a 2.4 system and vice
versa. Again, is that so?

If this is how things are, I can now comprehend how packages downloaded via
apt-get, probably compiled on a kernel with a different SUBLEVEL (the 3rd number of the kernel ID, e.g. 2.6.17)to the one on my system work.

I thought that a package on the breezy badger repositories will only work
with the shipped kernel and that's that if you decide to change your kernel.

Sorry for my long-winded reply and thanks for all your help.

Costa

Anonymous said...

an other really good howto for all debian user ;)

Carcass

Anonymous said...

Really good!

Thanks

Anonymous said...

Nice walkthrough...believe me I've seen a few of late.

I'm using sarge-amd64 stable and I think I'm running into a devfs-udev-LVM issue. I have udev installed for the 2.6.18.2 kernel but on boot up I get a message along the lines of "cannot open device /dev/mapper/VG1-LV1" (Volume group VG1 and Logical Volume LV1 which has my "/" filesystem on it.) It then says something about devfs not recognised and goes into kernel panic. I've read elsewhere that mkinitrd is used by make-kpkg but it cannot be used with udev. yaird and makeinitrd-tools are not available on the stable source list so what to do? Am I forced to either upgrade to Etch or compile a kernel with no initrd? Have you come across this sort of problem before? Sorry to hassle you with this involved query but I'm at my wits end. :-)

Thanks in advance
Ryan

Hugues said...

A few things here.

Sarge ships with udev 0.56, but compiling the 2.6.18 kernel requires udev 0.71 (see Documentation/Changes). So I guess it is possible things could go wrong because of that. You can either upgrade to Etch (Etch is RC 1 now, and schedule for December 4 : it is perfectly stable), or use the backported version of udev for Sarge (from backports.org).

I believe initramfs-tools is available for Sarge at backports.org too (Etch use initramfs by default now). You could perhaps generate an initrd by using initramfs instead of initrd, and see if it works for you :
mkinitramfs -o /boot/initrd-2.6.18.2-...

Of course you can simply not use initrd at all. You will simply need to compile as Y (not as module) the hard drive controller, the root filesystem, and LVM.

If you need any more help, feel free to start a thread at linuxforums.org and I'm sure we'll find a solution there (my nickname is "antidrugue" on that forum).

Good luck, and thanks for the feedback !

Hugues

Anonymous said...

aahhh! That totally explains my dfficulty with udev. I didn't realise etch was so close to release. I'll give it a go.

Cheers,

Ryan

El Cerrajero said...

Nice howto but it is not a good idea to compile a vanilla kernel source but a Debian kernel source.

The kernel team does not release a new kernel whenever a security bug has been fixed and reserves new versions for technological advancements.

On the other hand, Debian maintainers have crafted security fixes plus a selected bunch of patches that ain't going to be in your compiling way.

Anonymous said...

That's very helpful!

Thanks

Anonymous said...

what about xconfig, the qt prerequisites and setting QTDIR?

Hugues said...

Hi anonymous,

If I remember correctly, for make xconfig to work you need to install the package libqt3-mt-dev.

The other possible interface (gnome style) is make gconfig, which depends on libglade2-dev.

Personally I don't use them, as I find make menuconfig to work much better.

Thanks for the feedback,

Hugues

Anonymous said...

Ok, I know I am doing SOMETHING very wrong, but I don't know what. I tried yesterday to build and install a 2.6.20 kernel with CK patches. Everything seemed to go well (configured the kernel, made an image, made modules, made an initrd and installed them all). However, when I tried to boot into the new kernel it hung - not a kernel panic, it just got stuck when it tried to load the filesystem and stopped dead.

If anyone had any ideas, I would love help.

Thanks!

Hugues said...

Starting with 2.6.19 the IDE/SATA configuration of the kernel changed drastically.

So you might want to verify that you included the proper SATA (or IDE) drivers for your disk controller (in make menuconfig).

Perhaps that is the problem.

If not, feel free to start on a thread on LinuxForums.org, and I will help you there if I am around.

Thanks for reading,

Hugues

Anonymous said...

what about initramfs? didnt that depricate initrd?

ShinnRa said...

those two params: --us --uc force not to sign package and changelog. So my question: is there any option to force sign package ?

graysky said...

Great guide, thanks! If you have a dual or quad core CPU, you can make the build process much faster by doing the following before the "build the kernel package" step:

For dual core chips:
# export CONCURRENCY_LEVEL=3

For quad core chips:
# export CONCURRENCY_LEVEL=5

The number in the end means 1 + "the number of cores."

graysky said...

Scratch that... use a number that is the SAME as your cores, i.e. dual =2 and quad =4.

For more, see:

http://forums.debian.net/viewtopic.php?t=33960&highlight=

garingz said...

How to compile my kernel that define rdtsc/rdtscll function?

TommyTomTom said...

You commented that "You want to compile your very own linux kernel ? No ? Well you should" and I just wonder "Why should I?"

I just bought but not received yet a Intel Core 2 Extreme Quad Core QX9300 laptop and I will be putting 64bit debian or ubuntu on it and I wonder if you get more power/speed by compiling your own kernel???