Merge 5.10.28 into android12-5.10
Changes in 5.10.28 arm64: mm: correct the inside linear map range during hotplug check bpf: Fix fexit trampoline. virtiofs: Fail dax mount if device does not support it ext4: shrink race window in ext4_should_retry_alloc() ext4: fix bh ref count on error paths fs: nfsd: fix kconfig dependency warning for NFSD_V4 rpc: fix NULL dereference on kmalloc failure iomap: Fix negative assignment to unsigned sis->pages in iomap_swapfile_activate ASoC: rt1015: fix i2c communication error ASoC: rt5640: Fix dac- and adc- vol-tlv values being off by a factor of 10 ASoC: rt5651: Fix dac- and adc- vol-tlv values being off by a factor of 10 ASoC: sgtl5000: set DAP_AVC_CTRL register to correct default value on probe ASoC: es8316: Simplify adc_pga_gain_tlv table ASoC: soc-core: Prevent warning if no DMI table is present ASoC: cs42l42: Fix Bitclock polarity inversion ASoC: cs42l42: Fix channel width support ASoC: cs42l42: Fix mixer volume control ASoC: cs42l42: Always wait at least 3ms after reset NFSD: fix error handling in NFSv4.0 callbacks kernel: freezer should treat PF_IO_WORKER like PF_KTHREAD for freezing vhost: Fix vhost_vq_reset() io_uring: fix ->flags races by linked timeouts scsi: st: Fix a use after free in st_open() scsi: qla2xxx: Fix broken #endif placement staging: comedi: cb_pcidas: fix request_irq() warn staging: comedi: cb_pcidas64: fix request_irq() warn ASoC: rt5659: Update MCLK rate in set_sysclk() ASoC: rt711: add snd_soc_component remove callback thermal/core: Add NULL pointer check before using cooling device stats locking/ww_mutex: Simplify use_ww_ctx & ww_ctx handling locking/ww_mutex: Fix acquire/release imbalance in ww_acquire_init()/ww_acquire_fini() nvmet-tcp: fix kmap leak when data digest in use io_uring: imply MSG_NOSIGNAL for send[msg]()/recv[msg]() calls static_call: Align static_call_is_init() patching condition ext4: do not iput inode under running transaction in ext4_rename() io_uring: call req_set_fail_links() on short send[msg]()/recv[msg]() with MSG_WAITALL net: mvpp2: fix interrupt mask/unmask skip condition flow_dissector: fix TTL and TOS dissection on IPv4 fragments can: dev: move driver related infrastructure into separate subdir net: introduce CAN specific pointer in the struct net_device can: tcan4x5x: fix max register value brcmfmac: clear EAP/association status bits on linkdown events ath11k: add ieee80211_unregister_hw to avoid kernel crash caused by NULL pointer rtw88: coex: 8821c: correct antenna switch function netdevsim: dev: Initialize FIB module after debugfs iwlwifi: pcie: don't disable interrupts for reg_lock ath10k: hold RCU lock when calling ieee80211_find_sta_by_ifaddr() net: ethernet: aquantia: Handle error cleanup of start on open appletalk: Fix skb allocation size in loopback case net: ipa: remove two unused register definitions net: ipa: fix register write command validation net: wan/lmc: unregister device when no matching device is found net: 9p: advance iov on empty read bpf: Remove MTU check in __bpf_skb_max_len ACPI: tables: x86: Reserve memory occupied by ACPI tables ACPI: processor: Fix CPU0 wakeup in acpi_idle_play_dead() ALSA: usb-audio: Apply sample rate quirk to Logitech Connect ALSA: hda: Re-add dropped snd_poewr_change_state() calls ALSA: hda: Add missing sanity checks in PM prepare/complete callbacks ALSA: hda/realtek: fix a determine_headset_type issue for a Dell AIO ALSA: hda/realtek: call alc_update_headset_mode() in hp_automute_hook ALSA: hda/realtek: fix mute/micmute LEDs for HP 640 G8 xtensa: fix uaccess-related livelock in do_page_fault xtensa: move coprocessor_flush to the .text section KVM: SVM: load control fields from VMCB12 before checking them KVM: SVM: ensure that EFER.SVME is set when running nested guest or on nested vmexit PM: runtime: Fix race getting/putting suppliers at probe PM: runtime: Fix ordering in pm_runtime_get_suppliers() tracing: Fix stack trace event size s390/vdso: copy tod_steering_delta value to vdso_data page s390/vdso: fix tod_steering_delta type mm: fix race by making init_zero_pfn() early_initcall drm/amdkfd: dqm fence memory corruption drm/amdgpu: fix offset calculation in amdgpu_vm_bo_clear_mappings() drm/amdgpu: check alignment on CPU page for bo map reiserfs: update reiserfs_xattrs_initialized() condition drm/imx: fix memory leak when fails to init drm/tegra: dc: Restore coupling of display controllers drm/tegra: sor: Grab runtime PM reference across reset vfio/nvlink: Add missing SPAPR_TCE_IOMMU depends pinctrl: rockchip: fix restore error in resume extcon: Add stubs for extcon_register_notifier_all() functions extcon: Fix error handling in extcon_dev_register firmware: stratix10-svc: reset COMMAND_RECONFIG_FLAG_PARTIAL to 0 usb: dwc3: pci: Enable dis_uX_susphy_quirk for Intel Merrifield video: hyperv_fb: Fix a double free in hvfb_probe firewire: nosy: Fix a use-after-free bug in nosy_ioctl() usbip: vhci_hcd fix shift out-of-bounds in vhci_hub_control() USB: quirks: ignore remote wake-up on Fibocom L850-GL LTE modem usb: musb: Fix suspend with devices connected for a64 usb: xhci-mtk: fix broken streams issue on 0.96 xHCI cdc-acm: fix BREAK rx code path adding necessary calls USB: cdc-acm: untangle a circular dependency between callback and softint USB: cdc-acm: downgrade message to debug USB: cdc-acm: fix double free on probe failure USB: cdc-acm: fix use-after-free after probe failure usb: gadget: udc: amd5536udc_pci fix null-ptr-dereference usb: dwc2: Fix HPRT0.PrtSusp bit setting for HiKey 960 board. usb: dwc2: Prevent core suspend when port connection flag is 0 usb: dwc3: qcom: skip interconnect init for ACPI probe usb: dwc3: gadget: Clear DEP flags after stop transfers in ep disable soc: qcom-geni-se: Cleanup the code to remove proxy votes staging: rtl8192e: Fix incorrect source in memcpy() staging: rtl8192e: Change state information from u16 to u8 driver core: clear deferred probe reason on probe retry drivers: video: fbcon: fix NULL dereference in fbcon_cursor() riscv: evaluate put_user() arg before enabling user access Revert "kernel: freezer should treat PF_IO_WORKER like PF_KTHREAD for freezing" bpf: Use NOP_ATOMIC5 instead of emit_nops(&prog, 5) for BPF_TRAMP_F_CALL_ORIG Linux 5.10.28 Signed-off-by: Greg Kroah-Hartman <gregkh@google.com> Change-Id: Ifdbbeda8de3ee22a7aa3f5d3b10becf0aba1a124
This commit is contained in:
@@ -431,7 +431,7 @@ static int bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
|
||||
|
||||
tprogs[BPF_TRAMP_FENTRY].progs[0] = prog;
|
||||
tprogs[BPF_TRAMP_FENTRY].nr_progs = 1;
|
||||
err = arch_prepare_bpf_trampoline(image,
|
||||
err = arch_prepare_bpf_trampoline(NULL, image,
|
||||
st_map->image + PAGE_SIZE,
|
||||
&st_ops->func_models[i], 0,
|
||||
tprogs, NULL);
|
||||
|
||||
@@ -827,7 +827,7 @@ static int __init bpf_jit_charge_init(void)
|
||||
}
|
||||
pure_initcall(bpf_jit_charge_init);
|
||||
|
||||
static int bpf_jit_charge_modmem(u32 pages)
|
||||
int bpf_jit_charge_modmem(u32 pages)
|
||||
{
|
||||
if (atomic_long_add_return(pages, &bpf_jit_current) >
|
||||
(bpf_jit_limit >> PAGE_SHIFT)) {
|
||||
@@ -840,7 +840,7 @@ static int bpf_jit_charge_modmem(u32 pages)
|
||||
return 0;
|
||||
}
|
||||
|
||||
static void bpf_jit_uncharge_modmem(u32 pages)
|
||||
void bpf_jit_uncharge_modmem(u32 pages)
|
||||
{
|
||||
atomic_long_sub(pages, &bpf_jit_current);
|
||||
}
|
||||
|
||||
@@ -59,19 +59,10 @@ void bpf_image_ksym_del(struct bpf_ksym *ksym)
|
||||
PAGE_SIZE, true, ksym->name);
|
||||
}
|
||||
|
||||
static void bpf_trampoline_ksym_add(struct bpf_trampoline *tr)
|
||||
{
|
||||
struct bpf_ksym *ksym = &tr->ksym;
|
||||
|
||||
snprintf(ksym->name, KSYM_NAME_LEN, "bpf_trampoline_%llu", tr->key);
|
||||
bpf_image_ksym_add(tr->image, ksym);
|
||||
}
|
||||
|
||||
static struct bpf_trampoline *bpf_trampoline_lookup(u64 key)
|
||||
{
|
||||
struct bpf_trampoline *tr;
|
||||
struct hlist_head *head;
|
||||
void *image;
|
||||
int i;
|
||||
|
||||
mutex_lock(&trampoline_mutex);
|
||||
@@ -86,14 +77,6 @@ static struct bpf_trampoline *bpf_trampoline_lookup(u64 key)
|
||||
if (!tr)
|
||||
goto out;
|
||||
|
||||
/* is_root was checked earlier. No need for bpf_jit_charge_modmem() */
|
||||
image = bpf_jit_alloc_exec_page();
|
||||
if (!image) {
|
||||
kfree(tr);
|
||||
tr = NULL;
|
||||
goto out;
|
||||
}
|
||||
|
||||
tr->key = key;
|
||||
INIT_HLIST_NODE(&tr->hlist);
|
||||
hlist_add_head(&tr->hlist, head);
|
||||
@@ -101,9 +84,6 @@ static struct bpf_trampoline *bpf_trampoline_lookup(u64 key)
|
||||
mutex_init(&tr->mutex);
|
||||
for (i = 0; i < BPF_TRAMP_MAX; i++)
|
||||
INIT_HLIST_HEAD(&tr->progs_hlist[i]);
|
||||
tr->image = image;
|
||||
INIT_LIST_HEAD_RCU(&tr->ksym.lnode);
|
||||
bpf_trampoline_ksym_add(tr);
|
||||
out:
|
||||
mutex_unlock(&trampoline_mutex);
|
||||
return tr;
|
||||
@@ -187,10 +167,143 @@ bpf_trampoline_get_progs(const struct bpf_trampoline *tr, int *total)
|
||||
return tprogs;
|
||||
}
|
||||
|
||||
static void __bpf_tramp_image_put_deferred(struct work_struct *work)
|
||||
{
|
||||
struct bpf_tramp_image *im;
|
||||
|
||||
im = container_of(work, struct bpf_tramp_image, work);
|
||||
bpf_image_ksym_del(&im->ksym);
|
||||
trace_android_vh_set_memory_nx((unsigned long)im->image, 1);
|
||||
bpf_jit_free_exec(im->image);
|
||||
bpf_jit_uncharge_modmem(1);
|
||||
percpu_ref_exit(&im->pcref);
|
||||
kfree_rcu(im, rcu);
|
||||
}
|
||||
|
||||
/* callback, fexit step 3 or fentry step 2 */
|
||||
static void __bpf_tramp_image_put_rcu(struct rcu_head *rcu)
|
||||
{
|
||||
struct bpf_tramp_image *im;
|
||||
|
||||
im = container_of(rcu, struct bpf_tramp_image, rcu);
|
||||
INIT_WORK(&im->work, __bpf_tramp_image_put_deferred);
|
||||
schedule_work(&im->work);
|
||||
}
|
||||
|
||||
/* callback, fexit step 2. Called after percpu_ref_kill confirms. */
|
||||
static void __bpf_tramp_image_release(struct percpu_ref *pcref)
|
||||
{
|
||||
struct bpf_tramp_image *im;
|
||||
|
||||
im = container_of(pcref, struct bpf_tramp_image, pcref);
|
||||
call_rcu_tasks(&im->rcu, __bpf_tramp_image_put_rcu);
|
||||
}
|
||||
|
||||
/* callback, fexit or fentry step 1 */
|
||||
static void __bpf_tramp_image_put_rcu_tasks(struct rcu_head *rcu)
|
||||
{
|
||||
struct bpf_tramp_image *im;
|
||||
|
||||
im = container_of(rcu, struct bpf_tramp_image, rcu);
|
||||
if (im->ip_after_call)
|
||||
/* the case of fmod_ret/fexit trampoline and CONFIG_PREEMPTION=y */
|
||||
percpu_ref_kill(&im->pcref);
|
||||
else
|
||||
/* the case of fentry trampoline */
|
||||
call_rcu_tasks(&im->rcu, __bpf_tramp_image_put_rcu);
|
||||
}
|
||||
|
||||
static void bpf_tramp_image_put(struct bpf_tramp_image *im)
|
||||
{
|
||||
/* The trampoline image that calls original function is using:
|
||||
* rcu_read_lock_trace to protect sleepable bpf progs
|
||||
* rcu_read_lock to protect normal bpf progs
|
||||
* percpu_ref to protect trampoline itself
|
||||
* rcu tasks to protect trampoline asm not covered by percpu_ref
|
||||
* (which are few asm insns before __bpf_tramp_enter and
|
||||
* after __bpf_tramp_exit)
|
||||
*
|
||||
* The trampoline is unreachable before bpf_tramp_image_put().
|
||||
*
|
||||
* First, patch the trampoline to avoid calling into fexit progs.
|
||||
* The progs will be freed even if the original function is still
|
||||
* executing or sleeping.
|
||||
* In case of CONFIG_PREEMPT=y use call_rcu_tasks() to wait on
|
||||
* first few asm instructions to execute and call into
|
||||
* __bpf_tramp_enter->percpu_ref_get.
|
||||
* Then use percpu_ref_kill to wait for the trampoline and the original
|
||||
* function to finish.
|
||||
* Then use call_rcu_tasks() to make sure few asm insns in
|
||||
* the trampoline epilogue are done as well.
|
||||
*
|
||||
* In !PREEMPT case the task that got interrupted in the first asm
|
||||
* insns won't go through an RCU quiescent state which the
|
||||
* percpu_ref_kill will be waiting for. Hence the first
|
||||
* call_rcu_tasks() is not necessary.
|
||||
*/
|
||||
if (im->ip_after_call) {
|
||||
int err = bpf_arch_text_poke(im->ip_after_call, BPF_MOD_JUMP,
|
||||
NULL, im->ip_epilogue);
|
||||
WARN_ON(err);
|
||||
if (IS_ENABLED(CONFIG_PREEMPTION))
|
||||
call_rcu_tasks(&im->rcu, __bpf_tramp_image_put_rcu_tasks);
|
||||
else
|
||||
percpu_ref_kill(&im->pcref);
|
||||
return;
|
||||
}
|
||||
|
||||
/* The trampoline without fexit and fmod_ret progs doesn't call original
|
||||
* function and doesn't use percpu_ref.
|
||||
* Use call_rcu_tasks_trace() to wait for sleepable progs to finish.
|
||||
* Then use call_rcu_tasks() to wait for the rest of trampoline asm
|
||||
* and normal progs.
|
||||
*/
|
||||
call_rcu_tasks_trace(&im->rcu, __bpf_tramp_image_put_rcu_tasks);
|
||||
}
|
||||
|
||||
static struct bpf_tramp_image *bpf_tramp_image_alloc(u64 key, u32 idx)
|
||||
{
|
||||
struct bpf_tramp_image *im;
|
||||
struct bpf_ksym *ksym;
|
||||
void *image;
|
||||
int err = -ENOMEM;
|
||||
|
||||
im = kzalloc(sizeof(*im), GFP_KERNEL);
|
||||
if (!im)
|
||||
goto out;
|
||||
|
||||
err = bpf_jit_charge_modmem(1);
|
||||
if (err)
|
||||
goto out_free_im;
|
||||
|
||||
err = -ENOMEM;
|
||||
im->image = image = bpf_jit_alloc_exec_page();
|
||||
if (!image)
|
||||
goto out_uncharge;
|
||||
|
||||
err = percpu_ref_init(&im->pcref, __bpf_tramp_image_release, 0, GFP_KERNEL);
|
||||
if (err)
|
||||
goto out_free_image;
|
||||
|
||||
ksym = &im->ksym;
|
||||
INIT_LIST_HEAD_RCU(&ksym->lnode);
|
||||
snprintf(ksym->name, KSYM_NAME_LEN, "bpf_trampoline_%llu_%u", key, idx);
|
||||
bpf_image_ksym_add(image, ksym);
|
||||
return im;
|
||||
|
||||
out_free_image:
|
||||
bpf_jit_free_exec(im->image);
|
||||
out_uncharge:
|
||||
bpf_jit_uncharge_modmem(1);
|
||||
out_free_im:
|
||||
kfree(im);
|
||||
out:
|
||||
return ERR_PTR(err);
|
||||
}
|
||||
|
||||
static int bpf_trampoline_update(struct bpf_trampoline *tr)
|
||||
{
|
||||
void *old_image = tr->image + ((tr->selector + 1) & 1) * PAGE_SIZE/2;
|
||||
void *new_image = tr->image + (tr->selector & 1) * PAGE_SIZE/2;
|
||||
struct bpf_tramp_image *im;
|
||||
struct bpf_tramp_progs *tprogs;
|
||||
u32 flags = BPF_TRAMP_F_RESTORE_REGS;
|
||||
int err, total;
|
||||
@@ -200,41 +313,42 @@ static int bpf_trampoline_update(struct bpf_trampoline *tr)
|
||||
return PTR_ERR(tprogs);
|
||||
|
||||
if (total == 0) {
|
||||
err = unregister_fentry(tr, old_image);
|
||||
err = unregister_fentry(tr, tr->cur_image->image);
|
||||
bpf_tramp_image_put(tr->cur_image);
|
||||
tr->cur_image = NULL;
|
||||
tr->selector = 0;
|
||||
goto out;
|
||||
}
|
||||
|
||||
im = bpf_tramp_image_alloc(tr->key, tr->selector);
|
||||
if (IS_ERR(im)) {
|
||||
err = PTR_ERR(im);
|
||||
goto out;
|
||||
}
|
||||
|
||||
if (tprogs[BPF_TRAMP_FEXIT].nr_progs ||
|
||||
tprogs[BPF_TRAMP_MODIFY_RETURN].nr_progs)
|
||||
flags = BPF_TRAMP_F_CALL_ORIG | BPF_TRAMP_F_SKIP_FRAME;
|
||||
|
||||
/* Though the second half of trampoline page is unused a task could be
|
||||
* preempted in the middle of the first half of trampoline and two
|
||||
* updates to trampoline would change the code from underneath the
|
||||
* preempted task. Hence wait for tasks to voluntarily schedule or go
|
||||
* to userspace.
|
||||
* The same trampoline can hold both sleepable and non-sleepable progs.
|
||||
* synchronize_rcu_tasks_trace() is needed to make sure all sleepable
|
||||
* programs finish executing.
|
||||
* Wait for these two grace periods together.
|
||||
*/
|
||||
synchronize_rcu_mult(call_rcu_tasks, call_rcu_tasks_trace);
|
||||
|
||||
err = arch_prepare_bpf_trampoline(new_image, new_image + PAGE_SIZE / 2,
|
||||
err = arch_prepare_bpf_trampoline(im, im->image, im->image + PAGE_SIZE,
|
||||
&tr->func.model, flags, tprogs,
|
||||
tr->func.addr);
|
||||
if (err < 0)
|
||||
goto out;
|
||||
|
||||
if (tr->selector)
|
||||
WARN_ON(tr->cur_image && tr->selector == 0);
|
||||
WARN_ON(!tr->cur_image && tr->selector);
|
||||
if (tr->cur_image)
|
||||
/* progs already running at this address */
|
||||
err = modify_fentry(tr, old_image, new_image);
|
||||
err = modify_fentry(tr, tr->cur_image->image, im->image);
|
||||
else
|
||||
/* first time registering */
|
||||
err = register_fentry(tr, new_image);
|
||||
err = register_fentry(tr, im->image);
|
||||
if (err)
|
||||
goto out;
|
||||
if (tr->cur_image)
|
||||
bpf_tramp_image_put(tr->cur_image);
|
||||
tr->cur_image = im;
|
||||
tr->selector++;
|
||||
out:
|
||||
kfree(tprogs);
|
||||
@@ -366,18 +480,12 @@ void bpf_trampoline_put(struct bpf_trampoline *tr)
|
||||
goto out;
|
||||
if (WARN_ON_ONCE(!hlist_empty(&tr->progs_hlist[BPF_TRAMP_FEXIT])))
|
||||
goto out;
|
||||
bpf_image_ksym_del(&tr->ksym);
|
||||
/* This code will be executed when all bpf progs (both sleepable and
|
||||
* non-sleepable) went through
|
||||
* bpf_prog_put()->call_rcu[_tasks_trace]()->bpf_prog_free_deferred().
|
||||
* Hence no need for another synchronize_rcu_tasks_trace() here,
|
||||
* but synchronize_rcu_tasks() is still needed, since trampoline
|
||||
* may not have had any sleepable programs and we need to wait
|
||||
* for tasks to get out of trampoline code before freeing it.
|
||||
/* This code will be executed even when the last bpf_tramp_image
|
||||
* is alive. All progs are detached from the trampoline and the
|
||||
* trampoline image is patched with jmp into epilogue to skip
|
||||
* fexit progs. The fentry-only trampoline will be freed via
|
||||
* multiple rcu callbacks.
|
||||
*/
|
||||
synchronize_rcu_tasks();
|
||||
trace_android_vh_set_memory_nx((unsigned long)tr->image, 1);
|
||||
bpf_jit_free_exec(tr->image);
|
||||
hlist_del(&tr->hlist);
|
||||
kfree(tr);
|
||||
out:
|
||||
@@ -436,8 +544,18 @@ void notrace __bpf_prog_exit_sleepable(void)
|
||||
rcu_read_unlock_trace();
|
||||
}
|
||||
|
||||
void notrace __bpf_tramp_enter(struct bpf_tramp_image *tr)
|
||||
{
|
||||
percpu_ref_get(&tr->pcref);
|
||||
}
|
||||
|
||||
void notrace __bpf_tramp_exit(struct bpf_tramp_image *tr)
|
||||
{
|
||||
percpu_ref_put(&tr->pcref);
|
||||
}
|
||||
|
||||
int __weak
|
||||
arch_prepare_bpf_trampoline(void *image, void *image_end,
|
||||
arch_prepare_bpf_trampoline(struct bpf_tramp_image *tr, void *image, void *image_end,
|
||||
const struct btf_func_model *m, u32 flags,
|
||||
struct bpf_tramp_progs *tprogs,
|
||||
void *orig_call)
|
||||
|
||||
Reference in New Issue
Block a user