Alexey Dobriyan f22ef333c3
cpumask: make cpumask_next() out-of-line ...
Every for_each_XXX_cpu() invocation calls cpumask_next() which is an
inline function:
static inline unsigned int cpumask_next(int n, const struct cpumask *srcp)
{
/* -1 is a legal arg here. */
if (n != -1)
cpumask_check(n);
return find_next_bit(cpumask_bits(srcp), nr_cpumask_bits, n + 1);
}
However!
find_next_bit() is regular out-of-line function which means "nr_cpu_ids"
load and increment happen at the caller resulting in a lot of bloat
x86_64 defconfig:
add/remove: 3/0 grow/shrink: 8/373 up/down: 155/-5668 (-5513)
x86_64 allyesconfig-ish:
add/remove: 3/1 grow/shrink: 57/634 up/down: 3515/-28177 (-24662) !!!
Some archs redefine find_next_bit() but it is OK:
m68k inline but SMP is not supported
arm out-of-line
unicore32 out-of-line
Function call will happen anyway, so move load and increment into callee.
Link: http://lkml.kernel.org/r/20170824230010.GA1593@avx2
Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com >
Signed-off-by: Andrew Morton <akpm@linux-foundation.org >
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org >
2017-09-08 18:26:51 -07:00
2017-07-09 18:48:37 -07:00
2017-08-27 15:47:43 -07:00
2017-09-08 18:26:48 -07:00
2017-08-01 16:46:45 +02:00
2017-08-30 14:01:47 +02:00
2017-08-01 16:46:44 +02:00
2017-08-28 16:40:18 +05:30
2017-07-17 17:26:14 +02:00
2017-08-23 09:16:35 +02:00
2017-09-05 08:45:36 +01:00
2017-09-07 13:51:13 -07:00
2017-08-31 15:31:42 +01:00
2017-07-13 15:58:02 -04:00
2017-09-07 13:51:13 -07:00
2017-09-06 14:45:08 -07:00
2017-09-06 14:45:08 -07:00
2017-09-07 12:24:50 -07:00
2017-08-02 10:26:42 +02:00
2017-08-28 17:11:28 +02:00
2017-09-04 12:54:42 +02:00
2017-06-19 19:09:19 +02:00
2017-07-27 13:43:22 +01:00
2017-08-30 15:17:45 -07:00
2017-08-31 09:15:21 +02:00
2017-09-07 13:55:16 -07:00
2017-07-27 00:05:22 -07:00
2017-08-09 18:52:07 +01:00
2017-08-15 12:50:48 +01:00
2017-09-06 20:49:49 -07:00
2017-09-05 09:11:45 -07:00
2017-07-06 11:38:59 -07:00
2017-07-13 13:56:24 -07:00
2017-08-31 18:08:47 +02:00
2017-08-07 14:28:51 +02:00
2017-09-05 12:50:00 -07:00
2017-08-04 13:34:14 -07:00
2017-07-17 17:32:05 +02:00
2017-08-29 08:33:24 -07:00
2017-07-04 22:35:16 +01:00
2017-08-10 12:29:02 +02:00
2017-07-12 16:26:05 -07:00
2017-08-01 12:03:09 -07:00
2017-09-07 11:59:42 -07:00
2017-09-08 18:26:49 -07:00
2017-09-08 18:26:48 -07:00
2017-07-17 17:26:14 +02:00
2017-08-23 12:49:55 -06:00
2017-07-29 09:00:03 -06:00
2017-08-08 14:58:03 -04:00
2017-08-18 08:36:58 -06:00
2017-09-07 11:59:42 -07:00
2017-07-29 09:00:03 -06:00
2017-07-06 16:24:33 -07:00
2017-08-16 15:34:13 -07:00
2017-08-23 22:38:08 -07:00
2017-07-17 13:37:56 -07:00
2017-08-28 11:13:21 -07:00
2017-09-07 21:11:05 -07:00
2017-07-10 10:51:53 -07:00
2017-07-10 16:32:34 -07:00
2017-07-10 16:32:34 -07:00
2017-07-03 16:56:28 -06:00
2017-08-04 09:27:41 +08:00
2017-06-30 12:00:51 -07:00
2017-08-18 08:24:21 -07:00
2017-09-07 11:59:42 -07:00
2017-07-14 12:37:41 -07:00
2017-07-03 16:14:51 -07:00
2017-08-31 17:32:38 -04:00
2017-07-06 16:24:29 -07:00
2017-09-04 12:21:28 -07:00
2017-09-04 12:21:28 -07:00
2017-08-29 15:14:38 +02:00
2017-08-28 17:35:43 +02:00
2017-06-22 15:43:47 +01:00
2017-09-04 00:05:22 +02:00
2017-09-07 10:15:40 -07:00
2017-09-04 00:06:02 +02:00
2017-09-08 18:26:51 -07:00
2017-08-10 12:28:57 +02:00
2017-07-12 16:26:00 -07:00
2017-07-19 08:55:18 -07:00
2017-09-06 17:27:24 -07:00
2017-07-19 08:55:18 -07:00
2017-07-17 16:25:11 +02:00
2017-08-28 10:22:27 +09:00
2017-08-28 09:58:27 -04:00
2017-08-28 18:02:46 +02:00
2017-08-24 13:23:03 -07:00
2017-06-19 11:03:51 -04:00
2017-07-27 08:15:43 +10:00
2017-08-26 09:19:13 +02:00
2017-09-06 21:55:10 +05:30
2017-08-29 15:34:44 -06:00
2017-08-29 15:34:44 -06:00
2017-08-29 15:34:46 -06:00
2017-07-17 13:42:48 +02:00
2017-08-28 16:55:47 +02:00
2017-09-07 09:42:35 -07:00
2017-06-18 10:08:55 -06:00
2017-07-29 09:01:02 -04:00
2017-08-18 16:30:17 -07:00
2017-06-20 12:18:27 +02:00
2017-07-12 16:26:01 -07:00
2017-07-10 16:32:35 -07:00
2017-08-16 09:27:55 +09:00
2017-08-24 11:59:36 -07:00
2017-07-03 18:44:02 -04:00
2017-08-28 16:24:22 +02:00
2017-06-30 12:00:51 -07:00
2017-09-08 18:26:50 -07:00
2017-09-06 17:27:26 -07:00
2017-07-09 09:31:22 -07:00
2017-06-23 19:59:08 -04:00
2017-07-07 20:09:10 -04:00
2017-07-20 09:27:29 -04:00
2017-08-01 14:36:35 +02:00
2017-07-22 00:04:51 +02:00
2017-08-31 09:47:22 -06:00
2017-08-09 13:09:28 -06:00
2017-07-12 16:26:03 -07:00
2017-06-30 13:49:53 -07:00
2017-09-05 11:54:41 -07:00
2017-09-08 18:26:46 -07:00
2017-06-30 04:14:14 -04:00
2017-09-08 18:26:45 -07:00
2017-08-15 23:20:30 +10:00
2017-09-07 09:25:15 -07:00
2017-07-31 17:33:53 +02:00
2017-08-30 14:36:44 -07:00
2017-06-30 09:47:24 +03:00
2017-06-16 11:48:40 -04:00
2017-08-07 11:39:22 -07:00
2017-06-21 14:37:12 -04:00
2017-06-20 07:25:17 -03:00
2017-09-01 18:38:09 -07:00
2017-07-01 07:39:08 -07:00
2017-09-08 18:26:49 -07:00
2017-07-10 16:32:34 -07:00
2017-08-10 16:28:53 +02:00
2017-09-08 18:26:49 -07:00
2017-09-08 18:26:49 -07:00
2017-07-18 11:38:02 +02:00
2017-07-10 10:51:53 -07:00
2017-08-15 13:58:48 +02:00
2017-09-08 18:26:46 -07:00
2017-06-30 12:00:51 -07:00
2017-07-19 08:55:18 -07:00
2017-08-07 11:39:22 -07:00
2017-08-16 16:40:02 +02:00
2017-08-18 11:22:27 +02:00
2017-07-04 12:46:16 +02:00
2017-09-05 12:50:00 -07:00
2017-08-29 15:14:38 +02:00
2017-07-17 09:22:34 -07:00
2017-08-10 12:28:59 +02:00
2017-07-26 13:08:54 +02:00
2017-09-08 18:26:47 -07:00
2017-07-29 09:00:03 -06:00
2017-07-18 11:38:04 +02:00
2017-06-30 12:00:51 -07:00
2017-07-03 21:13:25 -07:00
2017-07-10 16:32:31 -07:00
2017-07-06 16:24:34 -07:00
2017-09-08 18:26:51 -07:00
2017-08-14 13:33:39 -07:00
2017-07-08 11:05:34 +02:00
2017-08-02 16:34:47 -07:00
2017-06-26 10:39:40 +02:00
2017-08-02 22:41:02 +02:00
2017-06-20 12:18:27 +02:00
2017-08-29 21:10:40 +02:00
2017-08-29 08:36:58 -07:00
2017-07-03 16:54:58 -07:00
2017-07-10 16:32:33 -07:00
2017-07-19 15:33:50 -07:00
2017-08-29 15:14:38 +02:00
2017-08-01 12:03:10 -07:00
2017-06-22 10:29:53 -04:00
2017-08-28 20:51:22 +02:00
2017-09-05 14:42:52 -07:00
2017-07-18 11:38:00 +02:00
2017-08-18 15:32:01 -07:00
2017-09-06 17:27:27 -07:00
2017-09-08 18:26:46 -07:00
2017-07-06 16:24:34 -07:00
2017-09-08 18:26:46 -07:00
2017-09-08 18:26:46 -07:00
2017-09-08 18:26:46 -07:00
2017-06-16 11:48:37 -04:00
2017-08-17 10:30:49 +02:00
2017-09-08 18:26:45 -07:00
2017-09-08 18:26:49 -07:00
2017-06-20 15:42:32 -04:00
2017-08-31 16:13:00 -07:00
2017-09-08 18:26:47 -07:00
2017-08-28 16:19:44 +02:00
2017-07-19 08:55:18 -07:00
2017-07-02 15:37:23 -07:00
2017-06-30 12:00:51 -07:00
2017-06-30 12:00:51 -07:00
2017-07-30 08:23:27 -07:00
2017-06-29 13:50:38 -07:00
2017-08-15 20:23:23 -07:00
2017-07-24 13:52:59 -07:00
2017-09-06 14:45:08 -07:00
2017-08-28 17:44:00 +02:00
2017-08-28 17:44:00 +02:00
2017-07-01 07:39:08 -07:00
2017-07-13 16:00:15 -04:00
2017-07-13 16:00:12 -04:00
2017-07-21 11:51:19 -04:00
2017-07-13 17:12:07 -04:00
2017-08-01 16:28:55 -04:00
2017-08-18 12:35:02 +02:00
2017-07-06 16:24:32 -07:00
2017-07-06 16:24:35 -07:00
2017-07-06 11:30:07 -04:00
2017-08-10 11:06:38 +02:00
2017-07-25 18:05:25 +02:00
2017-08-30 14:46:32 +02:00
2017-08-28 17:33:23 +02:00
2017-07-10 14:27:39 +02:00
2017-08-21 12:19:57 -05:00
2017-06-22 11:14:48 -05:00
2017-07-22 00:04:50 +02:00
2017-06-19 22:06:28 -04:00
2017-08-18 15:32:01 -07:00
2017-07-10 16:32:31 -07:00
2017-09-06 17:27:28 -07:00
2017-09-06 17:27:26 -07:00
2017-09-06 17:27:27 -07:00
2017-06-30 12:00:51 -07:00
2017-08-11 10:35:07 -07:00
2017-09-03 17:49:17 -07:00
2017-06-20 15:42:32 -04:00
2017-07-26 17:41:06 -04:00
2017-09-04 13:56:37 -07:00
2017-08-06 20:55:29 -07:00
2017-08-06 20:55:29 -07:00
2017-06-30 12:00:51 -07:00
2017-08-21 12:47:31 -07:00
2017-07-03 08:31:43 +02:00
2017-07-24 23:02:02 +02:00
2017-06-24 01:41:55 +02:00
2017-06-28 01:52:52 +02:00
2017-07-24 23:53:45 +02:00
2017-06-20 12:18:27 +02:00
2017-06-25 21:58:46 -04:00
2017-09-08 18:26:47 -07:00
2017-06-30 12:00:51 -07:00
2017-06-28 22:49:11 +10:00
2017-07-22 00:04:51 +02:00
2017-08-01 15:22:55 -07:00
2017-08-16 16:28:47 -07:00
2017-07-03 15:12:52 -07:00
2017-08-21 14:06:46 +02:00
2017-08-17 22:00:59 +02:00
2017-08-30 14:36:44 -07:00
2017-07-15 12:44:02 -07:00
2017-06-29 10:48:57 +01:00
2017-09-08 18:26:48 -07:00
2017-09-08 18:26:48 -07:00
2017-09-07 21:11:05 -07:00
2017-07-24 16:03:22 -07:00
2017-08-17 10:40:26 +02:00
2017-07-03 16:20:28 +01:00
2017-08-14 13:00:49 -04:00
2017-09-08 18:26:49 -07:00
2017-09-08 18:26:49 -07:00
2017-07-12 23:11:23 +02:00
2017-09-08 18:26:49 -07:00
2017-08-10 12:28:55 +02:00
2017-08-10 12:28:55 +02:00
2017-09-08 18:26:49 -07:00
2017-08-03 09:45:47 -07:00
2017-08-01 12:03:10 -07:00
2017-08-07 14:16:22 -07:00
2017-07-19 08:55:18 -07:00
2017-08-28 20:51:20 +02:00
2017-08-28 20:51:20 +02:00
2017-07-06 16:24:30 -07:00
2017-08-06 20:55:29 -07:00
2017-09-06 17:27:28 -07:00
2017-09-06 17:27:28 -07:00
2017-09-06 17:27:24 -07:00
2017-07-06 20:57:13 -07:00
2017-08-16 16:28:47 -07:00
2017-09-01 20:22:03 -07:00
2017-07-12 16:26:03 -07:00
2017-09-06 17:27:24 -07:00
2017-08-29 15:14:38 +02:00
2017-08-03 21:37:29 -07:00
2017-08-17 08:08:58 -07:00
2017-09-04 11:52:29 -07:00
2017-07-24 16:04:08 -07:00
2017-08-17 08:10:04 -07:00
2017-09-08 18:26:48 -07:00
2017-08-18 01:54:22 +02:00
2017-08-17 07:26:07 -07:00
2017-07-06 16:24:31 -07:00
2017-09-08 18:26:46 -07:00
2017-09-08 18:26:46 -07:00
2017-07-18 11:38:03 +02:00
2017-07-31 10:55:24 -03:00
2017-09-06 14:45:08 -07:00
2017-07-19 08:55:18 -07:00
2017-08-17 16:29:19 -07:00
2017-07-03 16:56:25 -06:00
2017-08-30 11:20:09 -07:00
2017-08-15 18:34:46 +01:00
2017-08-31 11:09:45 -04:00
2017-06-20 22:13:59 -07:00
2017-08-08 17:51:34 -07:00
2017-08-29 13:29:29 +02:00
2017-08-28 20:51:23 +02:00
2017-07-30 07:52:19 -07:00
2017-08-28 16:15:42 +02:00
2017-08-31 17:32:39 -04:00
2017-07-10 07:40:49 -04:00
2017-09-08 18:26:51 -07:00
2017-06-30 12:00:51 -07:00
2017-06-30 12:00:51 -07:00
2017-07-24 17:50:37 +02:00
2017-06-22 16:15:45 -07:00
2017-07-26 13:30:23 -06:00
2017-07-17 09:52:58 -07:00
2017-09-06 17:27:29 -07:00
2017-09-08 18:26:47 -07:00
2017-09-08 18:26:48 -07:00
2017-07-05 09:54:14 +02:00
2017-08-31 18:50:14 +02:00
2017-06-20 12:19:14 +02:00
2017-08-18 15:32:01 -07:00
2017-09-05 06:34:17 -07:00
f22ef333c3