wait: introduce wait_event_exclusive_cmd

It's just a variant of wait_event_cmd(), with exclusive flag being set.

For cases like RAID5, which puts many processes to sleep until 1/4
resources are free, a wake_up wakes up all processes to run, but
there is one process being able to get the resource as it's protected
by a spin lock. That ends up introducing heavy lock contentions, and
hurts performance badly.

Here introduce wait_event_exclusive_cmd to relieve the lock contention
naturally by letting wake_up just wake up one process.

Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
v2: its assumed that wait*() and __wait*() have the same arguments - peterz

Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>

diff --git a/include/linux/wait.h b/include/linux/wait.h
index 2db8334..db78c72 100644 (file)
--- a/include/linux/wait.h
+++ b/include/linux/wait.h
@@ -358,6 +358,19 @@ do {                                                                       \
        __ret;                                                          \
 })
 
+#define __wait_event_exclusive_cmd(wq, condition, cmd1, cmd2)          \
+       (void)___wait_event(wq, condition, TASK_UNINTERRUPTIBLE, 1, 0,  \
+                           cmd1; schedule(); cmd2)
+/*
+ * Just like wait_event_cmd(), except it sets exclusive flag
+ */
+#define wait_event_exclusive_cmd(wq, condition, cmd1, cmd2)            \
+do {                                                                   \
+       if (condition)                                                  \
+               break;                                                  \
+       __wait_event_exclusive_cmd(wq, condition, cmd1, cmd2);          \
+} while (0)
+
 #define __wait_event_cmd(wq, condition, cmd1, cmd2)                    \
        (void)___wait_event(wq, condition, TASK_UNINTERRUPTIBLE, 0, 0,  \
                            cmd1; schedule(); cmd2)