Song Liu a10787e6d5 bpf: Enable task local storage for tracing programs ... To access per-task data, BPF programs usually creates a hash table with pid as the key. This is not ideal because: 1. The user need to estimate the proper size of the hash table, which may be inaccurate; 2. Big hash tables are slow; 3. To clean up the data properly during task terminations, the user need to write extra logic. Task local storage overcomes these issues and offers a better option for these per-task data. Task local storage is only available to BPF_LSM. Now enable it for tracing programs. Unlike LSM programs, tracing programs can be called in IRQ contexts. Helpers that access task local storage are updated to use raw_spin_lock_irqsave() instead of raw_spin_lock_bh(). Tracing programs can attach to functions on the task free path, e.g. exit_creds(). To avoid allocating task local storage after bpf_task_storage_free(). bpf_task_storage_get() is updated to not allocate new storage when the task is not refcounted (task->usage == 0). Signed-off-by: Song Liu <songliubraving@fb.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Acked-by: KP Singh <kpsingh@kernel.org> Acked-by: Martin KaFai Lau <kafai@fb.com> Link: https://lore.kernel.org/bpf/20210225234319.336131-2-songliubraving@fb.com		2021-02-26 11:51:47 -08:00
..
bpf	bpf: Enable task local storage for tracing programs	2021-02-26 11:51:47 -08:00
cgroup	cgroup: fix psi monitor for root cgroup	2021-01-19 11:37:05 -05:00
configs	staging: ION: remove some references to CONFIG_ION	2021-01-06 17:39:38 +01:00
debug	smp: Cleanup smp_call_function*()	2020-11-24 16:47:49 +01:00
dma	dma-mapping: benchmark: use u8 for reserved field in uAPI structure	2021-02-05 12:48:46 +01:00
entry	entry: Explicitly flush pending rcuog wakeup before last rescheduling point	2021-02-17 14:12:43 +01:00
events	The performance event updates for v5.12 are:	2021-02-21 12:49:32 -08:00
gcov	init/gcov: allow CONFIG_CONSTRUCTORS on UML to fix module gcov	2021-02-05 11:03:47 -08:00
irq	Updates for the irq subsystem:	2021-02-21 11:53:06 -08:00
kcsan	kcsan: Rewrite kcsan_prandom_u32_max() without prandom_u32_state()	2021-01-04 14:39:07 -08:00
livepatch	livepatch: Use the default ftrace_ops instead of REGS when ARGS is available	2020-11-13 12:15:28 -05:00
locking	Scheduler updates for v5.12:	2021-02-21 12:35:04 -08:00
power	Merge branches 'powercap' and 'pm-misc'	2021-02-15 18:50:01 +01:00
printk	Merge branch 'printk-rework' into for-linus	2021-01-25 14:29:35 +01:00
rcu	Scheduler updates for v5.12:	2021-02-21 12:35:04 -08:00
sched	Scheduler updates for v5.12:	2021-02-21 12:35:04 -08:00
time	These are the latest RCU updates for v5.12:	2021-02-21 12:04:41 -08:00
trace	bpf: Enable task local storage for tracing programs	2021-02-26 11:51:47 -08:00
.gitignore
acct.c	kernel/acct.c: use #elif instead of #end and #elif	2020-12-15 22:46:15 -08:00
async.c
audit.c	audit: replace atomic_add_return()	2020-12-02 22:52:16 -05:00
audit.h
audit_fsnotify.c	fsnotify: generalize handle_inode_event()	2020-12-03 14:58:35 +01:00
audit_tree.c	fsnotify: generalize handle_inode_event()	2020-12-03 14:58:35 +01:00
audit_watch.c	fsnotify: generalize handle_inode_event()	2020-12-03 14:58:35 +01:00
auditfilter.c
auditsc.c	audit/stable-5.11 PR 20201214	2020-12-16 10:54:03 -08:00
backtracetest.c
bounds.c
capability.c
compat.c
configs.c
context_tracking.c
cpu.c	cpu/hotplug: Add lockdep_is_cpus_held()	2021-01-06 16:24:59 -08:00
cpu_pm.c
crash_core.c	kdump: append uts_namespace.name offset to VMCOREINFO	2020-12-15 22:46:18 -08:00
crash_dump.c
cred.c
delayacct.c
dma.c
exec_domain.c
exit.c	kernel/io_uring: cancel io_uring before task works	2020-12-30 19:36:54 -07:00
extable.c
fail_function.c	fault-injection: handle EI_ETYPE_TRUE	2020-12-15 22:46:19 -08:00
fork.c	bpf: Enable task local storage for tracing programs	2021-02-26 11:51:47 -08:00
freezer.c
futex.c	Merge branch 'linus' into locking/core, to pick up upstream fixes	2021-02-12 12:54:58 +01:00
gen_kheaders.sh
groups.c
hung_task.c	kernel/hung_task.c: make type annotations consistent	2020-11-02 12:14:19 -08:00
iomem.c
irq_work.c	irq_work: Optimize irq_work_single()	2020-11-24 16:47:49 +01:00
jump_label.c	jump_label: Fix usage in module __init	2020-12-18 16:53:12 +01:00
kallsyms.c	treewide: Convert macro and uses of __section(foo) to __section("foo")	2020-10-25 14:51:49 -07:00
kcmp.c	Merge branch 'exec-update-lock-for-v5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace	2020-12-15 19:36:48 -08:00
Kconfig.freezer
Kconfig.hz
Kconfig.locks
Kconfig.preempt	preempt: Introduce CONFIG_PREEMPT_DYNAMIC	2021-02-17 14:12:24 +01:00
kcov.c	kernel: make kcov_common_handle consider the current context	2020-11-02 18:00:20 -08:00
kexec.c
kexec_core.c	Merge branch 'work.elf-compat' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs	2021-02-21 09:29:23 -08:00
kexec_elf.c
kexec_file.c	crypto: sha - split sha.h into sha1.h and sha2.h	2020-11-20 14:45:33 +11:00
kexec_internal.h
kheaders.c
kmod.c
kprobes.c	kretprobe: Avoid re-registration of the same kretprobe earlier	2021-01-29 17:29:16 -05:00
ksysfs.c
kthread.c	- Correct the marking of kthreads which are supposed to run on a specific,	2021-01-24 10:09:20 -08:00
latencytop.c
Makefile	kcov: don't instrument with UBSAN	2020-12-15 22:46:19 -08:00
module-internal.h
module.c	Modules updates for v5.11	2020-12-17 13:01:31 -08:00
module_signature.c
module_signing.c
notifier.c
nsproxy.c	fixes-v5.11	2020-12-14 16:40:27 -08:00
padata.c
panic.c	panic: don't dump stack twice on warn	2020-11-14 11:26:04 -08:00
params.c	Modules updates for v5.11	2020-12-17 13:01:31 -08:00
pid.c	Merge branch 'exec-update-lock-for-v5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace	2020-12-15 19:36:48 -08:00
pid_namespace.c	fixes-v5.11	2020-12-14 16:40:27 -08:00
profile.c
ptrace.c	Merge branch 'akpm' (patches from Andrew)	2020-12-15 12:53:37 -08:00
range.c
reboot.c	reboot: hide from sysfs not applicable settings	2020-12-15 22:46:19 -08:00
regset.c
relay.c	relay: allow the use of const callback structs	2020-12-15 22:46:18 -08:00
resource.c	kernel/resource.c: fix kernel-doc markups	2020-12-15 22:46:18 -08:00
resource_kunit.c	resource: provide meaningful MODULE_LICENSE() in test suite	2020-11-25 18:52:35 +01:00
rseq.c
scftorture.c	scftorture: Add debug output for wrong-CPU warning	2021-01-04 13:53:41 -08:00
scs.c	scs: switch to vmapped shadow stacks	2020-12-01 10:30:28 +00:00
seccomp.c	seccomp updates for v5.11-rc1	2020-12-16 11:30:10 -08:00
signal.c	for-linus-2021-01-24	2021-01-24 09:35:28 -08:00
smp.c	smp: Process pending softirqs in flush_smp_call_function_from_idle()	2021-02-17 14:12:42 +01:00
smpboot.c	kthread: Extract KTHREAD_IS_PER_CPU	2021-01-22 15:09:42 +01:00
smpboot.h
softirq.c	Misc fixes/updates:	2020-12-27 09:06:10 -08:00
stackleak.c
stacktrace.c
static_call.c	static_call: Allow module use without exposing static_call_key	2021-02-17 14:12:42 +01:00
stop_machine.c	Merge branch 'linus' into sched/core, to resolve semantic conflict	2020-11-27 11:10:50 +01:00
sys.c	fs: Remove dcookies support	2021-01-29 10:06:46 +05:30
sys_ni.c	epoll: wire up syscall epoll_pwait2	2020-12-19 11:18:38 -08:00
sysctl-test.c
sysctl.c	rcu: Panic after fixed number of stalls	2020-11-19 19:37:16 -08:00
task_work.c	task_work: remove legacy TWA_SIGNAL path	2020-12-12 09:17:38 -07:00
taskstats.c	treewide: rename nla_strlcpy to nla_strscpy.	2020-11-16 08:08:54 -08:00
test_kprobes.c
torture.c	torture: Maintain torture-specific set of CPUs-online books	2021-01-06 17:17:22 -08:00
tracepoint.c	tracepoints: Migrate to use SYSCALL_WORK flag	2020-11-16 21:53:15 +01:00
tsacct.c
ucount.c
uid16.c
uid16.h
umh.c
up.c
user-return-notifier.c
user.c
user_namespace.c	fixes-v5.11	2020-12-14 16:40:27 -08:00
usermode_driver.c
utsname.c
utsname_sysctl.c
watch_queue.c
watchdog.c	kernel/watchdog: fix watchdog_allowed_mask not used warning	2020-11-14 11:26:03 -08:00
watchdog_hld.c
workqueue.c	workqueue: Restrict affinity change to rescuer	2021-01-22 15:09:43 +01:00
workqueue_internal.h