Waiman Long 405963b6a5 locking/qrwlock: Don't contend with readers when setting _QW_WAITING ... The current cmpxchg() loop in setting the _QW_WAITING flag for writers in queue_write_lock_slowpath() will contend with incoming readers causing possibly extra cmpxchg() operations that are wasteful. This patch changes the code to do a byte cmpxchg() to eliminate contention with new readers. A multithreaded microbenchmark running 5M read_lock/write_lock loop on a 8-socket 80-core Westmere-EX machine running 4.0 based kernel with the qspinlock patch have the following execution times (in ms) with and without the patch: With R:W ratio = 5:1 Threads w/o patch with patch % change ------- --------- ---------- -------- 2 990 895 -9.6% 3 2136 1912 -10.5% 4 3166 2830 -10.6% 5 3953 3629 -8.2% 6 4628 4405 -4.8% 7 5344 5197 -2.8% 8 6065 6004 -1.0% 9 6826 6811 -0.2% 10 7599 7599 0.0% 15 9757 9766 +0.1% 20 13767 13817 +0.4% With small number of contending threads, this patch can improve locking performance by up to 10%. With more contending threads, however, the gain diminishes. Signed-off-by: Waiman Long <Waiman.Long@hp.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Douglas Hatch <doug.hatch@hp.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Scott J Norton <scott.norton@hp.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/1433863153-30722-3-git-send-email-Waiman.Long@hp.com Signed-off-by: Ingo Molnar <mingo@kernel.org>		2015-06-19 09:45:38 +02:00
..
lglock.c
lockdep.c	lockdep: Do not break user-visible string	2015-06-03 10:07:09 +02:00
lockdep_internals.h
lockdep_proc.c
lockdep_states.h
locktorture.c
Makefile	locking/qrwlock: Rename QUEUE_RWLOCK to QUEUED_RWLOCKS	2015-05-12 09:46:00 +02:00
mcs_spinlock.h	locking/qspinlock: Introduce a simple generic 4-byte queued spinlock	2015-05-08 12:36:25 +02:00
mutex-debug.c
mutex-debug.h
mutex.c
mutex.h
osq_lock.c
percpu-rwsem.c
qrwlock.c	locking/qrwlock: Don't contend with readers when setting _QW_WAITING	2015-06-19 09:45:38 +02:00
qspinlock.c	locking/pvqspinlock: Implement simple paravirt support for the qspinlock	2015-05-08 12:37:05 +02:00
qspinlock_paravirt.h	locking/arch: Rename set_mb() to smp_store_mb()	2015-05-19 08:32:00 +02:00
rtmutex-debug.c
rtmutex-debug.h
rtmutex-tester.c
rtmutex.c	rtmutex: Warn if trylock is called from hard/softirq context	2015-05-13 22:49:12 +02:00
rtmutex.h
rtmutex_common.h
rwsem-spinlock.c
rwsem-xadd.c	locking/rwsem: Reduce spinlock contention in wakeup after up_read()/up_write()	2015-05-08 12:27:59 +02:00
rwsem.c
rwsem.h
semaphore.c
spinlock.c
spinlock_debug.c