Files
rockchip-kernel/include/linux
Jens Axboe fe0f07d08e direct-io: only inc/dec inode->i_dio_count for file systems
do_blockdev_direct_IO() increments and decrements the inode
->i_dio_count for each IO operation. It does this to protect against
truncate of a file. Block devices don't need this sort of protection.

For a capable multiqueue setup, this atomic int is the only shared
state between applications accessing the device for O_DIRECT, and it
presents a scaling wall for that. In my testing, as much as 30% of
system time is spent incrementing and decrementing this value. A mixed
read/write workload improved from ~2.5M IOPS to ~9.6M IOPS, with
better latencies too. Before:

clat percentiles (usec):
 |  1.00th=[   33],  5.00th=[   34], 10.00th=[   34], 20.00th=[   34],
 | 30.00th=[   34], 40.00th=[   34], 50.00th=[   35], 60.00th=[   35],
 | 70.00th=[   35], 80.00th=[   35], 90.00th=[   37], 95.00th=[   80],
 | 99.00th=[   98], 99.50th=[  151], 99.90th=[  155], 99.95th=[  155],
 | 99.99th=[  165]

After:

clat percentiles (usec):
 |  1.00th=[   95],  5.00th=[  108], 10.00th=[  129], 20.00th=[  149],
 | 30.00th=[  155], 40.00th=[  161], 50.00th=[  167], 60.00th=[  171],
 | 70.00th=[  177], 80.00th=[  185], 90.00th=[  201], 95.00th=[  270],
 | 99.00th=[  390], 99.50th=[  398], 99.90th=[  418], 99.95th=[  422],
 | 99.99th=[  438]

In other setups, Robert Elliott reported seeing good performance
improvements:

https://lkml.org/lkml/2015/4/3/557

The more applications accessing the device, the worse it gets.

Add a new direct-io flags, DIO_SKIP_DIO_COUNT, which tells
do_blockdev_direct_IO() that it need not worry about incrementing
or decrementing the inode i_dio_count for this caller.

Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Theodore Ts'o <tytso@mit.edu>
Cc: Elliott, Robert (Server Storage) <elliott@hp.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Jens Axboe <axboe@fb.com>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2015-04-24 15:45:28 -04:00
..
2015-02-19 13:31:40 +03:00
2014-11-24 17:24:08 -05:00
2015-04-02 16:25:04 -04:00
2015-01-29 10:49:28 +02:00
2015-03-25 20:28:11 -04:00
2015-03-29 13:26:54 -07:00
2015-03-11 15:56:59 -07:00
2014-12-18 19:08:10 -08:00
2015-02-12 18:54:15 -08:00
2014-12-19 22:55:06 +01:00
2015-01-21 19:21:30 +01:00
2014-11-10 09:27:30 -07:00
2015-03-02 16:43:41 -05:00
2014-12-31 13:06:50 -05:00
2014-10-09 11:35:48 +03:00
2014-10-08 16:01:41 -04:00
2015-01-25 23:17:28 -05:00
2015-01-27 11:09:13 +01:00
2014-09-29 15:37:01 -04:00
2015-01-15 10:34:54 +01:00
2015-01-15 10:34:54 +01:00
2015-02-13 21:21:36 -08:00
2015-02-17 14:34:51 -08:00
2014-12-18 09:39:51 +01:00
2015-01-04 23:11:43 -05:00
2014-10-09 22:25:58 -04:00
2015-02-11 17:06:02 -08:00
2014-10-24 00:14:36 +02:00
2015-03-11 17:56:28 -04:00
2014-11-04 13:29:38 +00:00
2015-01-25 23:17:28 -05:00
2015-02-17 14:34:52 -08:00
2015-01-06 11:01:13 -08:00
2015-02-16 17:56:03 -08:00
2015-02-13 21:21:41 -08:00
2015-01-22 15:10:56 +01:00
2015-03-11 21:55:28 -04:00
2015-04-11 15:53:35 -04:00
2015-01-03 14:32:57 -05:00
2015-02-13 21:21:35 -08:00
2015-04-11 22:29:44 -04:00
2014-11-28 16:08:16 +01:00
2014-12-09 12:05:24 +02:00
2014-12-15 23:49:28 +02:00