What is kjournald and why it's using 99% of IO? What about noatime?

I have noticed this process in iotop using a lot of CPU/IO time and I started to wonder what is it doing. It turns out this is a journaling process of ext3 partitions: http://serverfault.com/questions/236836/kjournald-reasons-for-high-usage Very often it’s related to not using noatime mount option. Generally this mount attribute is used to lower the IO load, but it prevents kernel from setting last access time to a file (which is rarely needed):

UUID=9a95d8ab-78c7-53de-b8bc-a8a2340d6250 /mnt ext3 defaults,noatime 0 2

This is a example of a /etc/fstab line (more here: http://tldp.org/LDP/solrhe/Securing-Optimizing-Linux-RH-Edition-v1.3/chap6sec73.html ) I recommend setting noatime everywhere you can, especially in high-read workloads – for more details take a look at this question: http://serverfault.com/questions/47466/drawbacks-of-mounting-a-filesystem-with-noatime Take a look at following snippet and see that “Access” doesn’t change when accessing file:

    wlangiewicz@hd0:~$ touch abcd
    wlangiewicz@hd0:~$ stat abcd
      File: `abcd'
      Size: 0           Blocks: 0          IO Block: 4096   regular empty file
    Device: 802h/2050d  Inode: 27705348    Links: 1
    Access: (0644/-rw-r--r--)  Uid: ( 1000/wlangiewicz)   Gid: ( 1000/wlangiewicz)
    Access: 2013-02-18 19:56:00.000000000 +0000
    Modify: 2013-02-18 19:56:00.000000000 +0000
    Change: 2013-02-18 19:56:00.000000000 +0000
    wlangiewicz@hd0:~$ cat abcd
    wlangiewicz@hd0:~$ stat abcd
      File: `abcd'
      Size: 0           Blocks: 0          IO Block: 4096   regular empty file
    Device: 802h/2050d  Inode: 27705348    Links: 1
    Access: (0644/-rw-r--r--)  Uid: ( 1000/wlangiewicz)   Gid: ( 1000/wlangiewicz)
    Access: 2013-02-18 19:56:00.000000000 +0000
    Modify: 2013-02-18 19:56:00.000000000 +0000
    Change: 2013-02-18 19:56:00.000000000 +0000
    wlangiewicz@hd0:~$ echo a > abcd
    wlangiewicz@hd0:~$ stat abcd
      File: `abcd'
      Size: 2           Blocks: 8          IO Block: 4096   regular file
    Device: 802h/2050d  Inode: 27705348    Links: 1
    Access: (0644/-rw-r--r--)  Uid: ( 1000/wlangiewicz)   Gid: ( 1000/wlangiewicz)
    Access: 2013-02-18 19:56:00.000000000 +0000
    Modify: 2013-02-18 19:56:28.000000000 +0000
    Change: 2013-02-18 19:56:28.000000000 +0000
    wlangiewicz@hd0:~$ cat abcd
    a
    wlangiewicz@hd0:~$ stat abcd
      File: `abcd'
      Size: 2           Blocks: 8          IO Block: 4096   regular file
    Device: 802h/2050d  Inode: 27705348    Links: 1
    Access: (0644/-rw-r--r--)  Uid: ( 1000/wlangiewicz)   Gid: ( 1000/wlangiewicz)
    Access: 2013-02-18 19:56:00.000000000 +0000
    Modify: 2013-02-18 19:56:28.000000000 +0000
    Change: 2013-02-18 19:56:28.000000000 +0000