Merge tag 'vfs-7.4-rc1.sync_close' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull file closing updates from Christian Brauner:
"This makes closing files synchronous for close_range(), exec and exit.
Since fput() was switched to task work every final __fput() of an
exiting task is queued on task work and run from exit_task_work().
exec and close_range() do the same. For a task with a large descriptor
table that is a cmpxchg() and a TIF_NOTIFY_RESUME test-and-set per
file plus a list entry for any later task_work_cancel() to search
under pi_lock, and then a second pass over every struct file that
isn't cache hot anymore by then. Oleg and Neil have pointed this out
over the years.
close_files(), close_range() and close-on-exec now flush and put each
file inline the way close(2) does. Kernel threads aren't affected.
They share init_files which init_task pins forever so their
exit_files() never drops the last reference, and their fput() keeps
punting to a workqueue. The one place that could put a table from a
[58 lines not shown]
Merge tag 'vfs-7.4-rc1.shared.super' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull superblock updates from Christian Brauner:
"This renames vfs_get_super() to get_tree_super() and exports it so a
filesystem that needs its own superblock matching policy can reuse it
instead of open-coding sget_fc() plus fill_super().
erofs uses it to share a superblock between file-backed mounts"
* tag 'vfs-7.4-rc1.shared.super' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs:
fs: rename vfs_get_super() to get_tree_super() and export it
Merge tag 'vfs-7.4-rc1.shared.net-next.pidfd' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull pidfd updates from Christian Brauner:
"This adds thread specific pidfds to af_unix sockets.
SO_PASSPIDFD and SO_PEERPIDFD hand out a pidfd for the thread-group
leader. A coredump server on the coredump socket can't get a handle on
the thread that took the signal and is writing the dump, and anything
doing per-thread authentication has the same problem. The sending and
the connecting thread are recorded in addition to the thread-group
leader and sk_peer_pid becomes an array indexed by pid type.
SO_PASSPIDFD_THREAD works like SO_PASSPIDFD but delivers an
SCM_PIDFD_THREAD cmsg for the sending thread. Both options are
independent so a receiver can ask for both. On SOCK_STREAM sockets
recvmsg() only stops merging data at a thread boundary when the
receiver asked for a thread pidfd. SO_PEERPIDFD_THREAD returns a pidfd
for the thread that called connect(), listen() or socketpair(). pidfs
records the coredump on the dumping thread's struct pid too so its
[14 lines not shown]
Merge tag 'vfs-7.4-rc1.netfs' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull netfs updates from Christian Brauner:
"This contains the netfs and cachefiles changes for this cycle from
David.
Cachefiles learns to cache the most common class of files properly,
files that are created or truncated, written once sequentially and
closed. So far it relied on the backing filesystem's idea of what is
stored and asked with SEEK_DATA and SEEK_HOLE before every read it
prepared, two seeks per read, and the backing filesystem is free to
insert or remove runs of zeros which produces false positives and
false negatives. The object type in the state xattr is now actually
set to CACHEFILES_CONTENT_{SINGLE,ALL,BACKFS_MAP} instead of zero. For
files written contiguously the type is ALL, the backing file's i_size
tracks how much is cached and the object size lives in the xattr, and
reads skip the seeks entirely. Netfslib queries the cache occupancy
through a new ->query_occupancy() method that returns up to two
extents at once, ->prepare_read() only limits the size of a cache read
[34 lines not shown]
Merge tag 'vfs-7.4-rc1.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull mount updates from Christian Brauner:
"This adds OPEN_TREE_DROP_MNTNS_MOUNTS.
crun started using OPEN_TREE_NAMESPACE to set up the container's mount
namespace and open_tree(OPEN_TREE_CLONE) for the sources of its bind
mounts. That fails on any host with a mount namespace pinned below
such a source. The recursive clone picks the pinned namespace up and
move_mount() refuses to attach the tree with ELOOP. Only a younger
mount namespace may ever be bound into an older one or mount namespace
references could form a loop. A namespace created with
OPEN_TREE_NAMESPACE is younger than everything else on the system so
from inside it every pinned namespace is older. By the time this fails
setns() has run and the host tree is out of reach, so the runtime has
to predict the situation beforehand and lose the optimization. snapd
pins mount namespaces under /run/snapd/ns so on Ubuntu that is any
container bind mounting the host root.
[15 lines not shown]
Merge tag 'vfs-7.4-rc1.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull misc vfs updates from Christian Brauner:
"Fixes:
- aio: a keyed poll completion on an epoll instance could recurse
into the same epoll waitqueue lock when the result eventfd is
watched by that very instance. An unprivileged process could make a
CPU spin on the recursive lock. eventfd-backed poll completions are
always deferred now.
- fsnotify_inoderemove() is only called once the inode is really
dead, no nlink and no aliases, and open_by_handle_at() can't add an
alias to a dead inode anymore. The race could crash fsnotify or
lose inode mark notifications.
- devtmpfs: the conversion of the public instance to ->get_tree()
broke reconfiguring the shared superblock's options on later mounts
which made the boot time defaults permanent. A reconfigure context
[97 lines not shown]
x86/uaccess: Remove __get_user() and __put_user()
Nothing that builds on x86 uses __get_user() or __put_user() any more.
They were the "unchecked" user accessors: no range check, on the
assumption that the caller had already done an access_ok(). That
stopped buying anything once every user access had to be enabled and
disabled on its own, and on x86 __get_user() had become slower than
get_user(), since it does an lfence where get_user() checks the address
with a couple of plain instructions. Several of the last users were
also relying on them for things they were never meant to do: accessing
kernel addresses, or staying quiet in atomic context.
Remove the macros and the out-of-line __{get,put}_user_nocheck_N helpers
behind them, so that new users cannot appear.
Other architectures still provide their own versions. UML gets its
accessors from asm-generic and is not affected. arm64's __get_user()
and __put_user() have always been the same as get_user() and put_user(),
so there is no hurry there.
[2 lines not shown]
x86/sev: Use explicit user-or-kernel accessors for #VC memory operands
vc_read_mem() and vc_write_mem() access the memory operand of an
emulated MOVS or INS/OUTS. That address may be a kernel address or a
user one, and the access must be a single MOV rather than a string
instruction, since the operand can itself be MMIO and take a nested #VC.
The code used __get_user() and __put_user() for this, on the grounds
that they do no pointer checks. That is a side effect of how x86
happens to implement them, not something they are for, and they are
going away.
Replace them with two local helpers that say what this is: a "user or
kernel" access with fault handling. They enable user access by hand
rather than through user_access_begin(), whose range check would reject
a kernel address, and use unsafe_get_user() and unsafe_put_user() for
the single MOV and its exception table entry. The read side uses
__uaccess_begin_nospec(), keeping the lfence that __get_user() used to
provide.
[12 lines not shown]
macintosh: ans-lcd: use get_user() for reading user bytes
anslcd_write() checks the user buffer with access_ok() and then reads it
one byte at a time with __get_user(), writing each byte to the LCD in
between. The ANSLCD_SENDCTRL ioctl does the same with the ioctl
argument, but with no access_ok() at all, so on architectures where
__get_user() does not check the address it will read from any address it
is given.
Use get_user() in both places. The ignored return values stay as they
were: a fault still reads as zero, which ends the SENDCTRL loop.
This is part of getting rid of __get_user() and __put_user() entirely.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
char: nwflash: use get_user() in write_block()
write_block() reads the user buffer one byte at a time with __get_user()
while programming and then verifying the flash, relying on the
access_ok() done in flash_write(). Each __get_user() enables and
disables user access on its own anyway, so the separation saves nothing.
Use get_user().
This is part of getting rid of __get_user() and __put_user() entirely.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
drm/i915/gem: Use get_user()/put_user() where no locks are held
execbuffer prefaults the relocation array and reads timeline fence
values one element at a time with __get_user(), after checking each
array with access_ok() up front. Each of those accesses enables and
disables user access on its own, so the up-front check saves nothing.
Both run with no object locks held: the prefault between dropping and
retaking the ww locks, the fence parsing before the ww context exists.
Use get_user(). The access_ok() checks stay, since they also decide how
a bad array fails.
The gem mman selftests use the unchecked variants on addresses they got
from their own mmaps, with no check at all. Convert those too.
This is part of getting rid of __get_user() and __put_user() entirely.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
drm/i915/gem: Don't fault on presumed_offset write-back
The execbuffer relocation fast path writes each updated presumed_offset
back to the user's relocation array with __put_user(), ignoring any
failure. It does that while holding the objects' dma_resv locks, and
with page faults enabled unless the relocation cache happens to have an
atomic mapping active. So if that write faults, for example on a
copy-on-write page after a fork(), the fault handler takes mmap_lock
under dma_resv. That is the opposite of the order the i915 fault
handler uses, which takes the object lock under mmap_lock.
The comment above it claimed this could not happen, because "we did
previously check if the relocations were writable (access_ok)".
access_ok() is purely a range check. The comment dates from when this
was access_ok(VERIFY_WRITE), and that only ever made a difference on the
original i386, whose write-protect bit did not stop supervisor writes.
So nothing has made sure the array was writable for decades.
Do the write with page faults disabled, and if it fails, fall back to
[16 lines not shown]
xen: use get_user()/put_user() in gntdev and privcmd
gntdev_copy() updates per-segment status words with __get_user() and
__put_user(). privcmd writes back mmap batch errors and the resource
frame count with __put_user(). The pointers were validated earlier,
away from the accesses: by a copy_from_user() of the enclosing
structure, or by an access_ok() on the whole array before a callback
walks it. Each access enables and disables user access on its own
anyway, so the separation saves nothing.
Use get_user() and put_user().
This is part of getting rid of __get_user() and __put_user() entirely.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
vhost: vringh: use get_user()/put_user() for IOTLB 16-bit accesses
getu16_iotlb() and putu16_iotlb() use __get_user() and __put_user() when
the IOTLB maps to user virtual addresses. Nothing nearby checks the
translated address: that relies on the mapping having been validated
against the user's VMAs when it was set up.
Use get_user() and put_user(), which do the same single, untorn access
and also check the address. The only atomic-context use, the vdpa
simulators' notification check, now disables page faults around it, so
the might_fault() in the checked accessors stays quiet there.
This is part of getting rid of __get_user() and __put_user() entirely.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
vdpa_sim: disable page faults around vringh_need_notify_iotlb()
When vdpa_sim runs with use_va (the default), its rings live in user
memory, and vringh reads the ring's flags and event index with
__get_user() through the user address. The net and block simulators
call vringh_need_notify_iotlb() with bottom halves disabled, so those
reads happen in atomic context.
That only works today because __get_user() has no might_fault() check: a
fault is not handled in atomic context and simply comes back as -EFAULT.
Make that explicit with pagefault_disable() around the call, so vringh
can use the normal user accessors without complaint. Only the
notification check needs this: the rest of the ring processing runs in
the same kthread without bottom halves disabled, and can fault normally.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
ubi: use put_user() when filling the erase counter array
ubi_get_ec_info() checks the user's erase_counters array with
access_ok() and then fills it one entry at a time with __put_user(),
taking the wear-leveling lock for each entry in between. Each of those
accesses enables and disables user access on its own, so the up-front
check saves nothing.
Use put_user(). The access_ok() stays, so a bad array still fails
before anything is written.
This is part of getting rid of __get_user() and __put_user() entirely.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
PNP: isapnp: use put_user() in the proc read loop
isapnp_proc_bus_read() checks the user buffer with access_ok() and then
copies the configuration registers one byte at a time with __put_user(),
reading each register in between. Each of those accesses enables and
disables user access on its own, so the up-front check saves nothing.
Use put_user(). The ignored return value and the access_ok() stay as
they were.
This is part of getting rid of __get_user() and __put_user() entirely.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
PCI/proc: Use get_user()/put_user() for config space reads and writes
proc_bus_pci_read() and proc_bus_pci_write() check the user buffer with
access_ok() and then copy config space one byte, word or dword at a time
with __put_user() and __get_user(). Each of those accesses enables and
disables user access on its own, so the up-front check saves nothing.
Use put_user() and get_user(). This deliberately changes nothing else:
the return values were ignored before and still are, and the access_ok()
stays, since it is the only error these functions report for a bad
buffer.
This is part of getting rid of __get_user() and __put_user() entirely.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
drivers/char/mem: use get_user()/put_user() for /dev/port byte copies
read_port() and write_port() check the whole user buffer with
access_ok() and then move it one byte at a time with __put_user() and
__get_user(), doing an inb() or outb() for each. Each of those accesses
enables and disables user access on its own, so the up-front check saves
nothing.
Use put_user() and get_user(). The access_ok() stays, because it also
makes a partly invalid buffer fail up front, rather than after some
ports have already been read or written.
This is part of getting rid of __get_user() and __put_user() entirely.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
x86/xen: Fix a stale __get_user() reference in a comment
The comment in set_aliased_prot() says the target address is faulted in
with __get_user(), but the code has used copy_from_kernel_nofault() for
a long time. Make the comment say so.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
x86/fpu: Use get_user()/put_user() for the signal frame fields
The FPU signal frame code reads and writes a handful of individual
fields (the xstate magic words, the legacy status word, the xfeatures
header and PKRU) with __get_user() and __put_user(). These rely on an
access_ok() of the whole frame done earlier by the caller. Each of
those accesses enables and disables user access on its own, so the
up-front check saves nothing. On x86, __get_user() is even more
expensive than get_user(): it does an lfence, where get_user() checks
the address with a couple of plain instructions.
Use get_user() and put_user(). The accesses that run with page faults
disabled stay that way; get_user() and put_user() are fine there.
This is part of getting rid of __get_user() and __put_user() entirely.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
x86/ptrace: Use get_user() in the general register set writers
genregs_set() and genregs32_set() read the user buffer one word at a
time with __get_user(), calling putreg() for each. The access_ok() that
makes this safe is in the generic copy_regset_from_user(), on the other
side of an indirect call through the regset. Each of those accesses
enables and disables user access on its own, so the up-front check saves
nothing. On x86, __get_user() is even more expensive than get_user():
it does an lfence, where get_user() checks the address with a couple of
plain instructions.
Use get_user(), so each read checks its own address.
This is part of getting rid of __get_user() and __put_user() entirely.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
perf/x86: Use scoped user access for the user callchain frames
The user callchain walkers read each two-word stack frame with
__get_user(), after checking the frame with valid_user_frame(), which is
just __access_ok(). They run with page faults disabled, since they can
be called from NMI context.
Read each frame inside a scoped_user_read_access() instead, which does
the range check itself and needs only a single stac/clac pair for both
loads. That leaves valid_user_frame() unused, so remove it.
This is part of getting rid of __get_user() and __put_user() entirely.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
x86/stacktrace: Use scoped user access in copy_stack_frame()
copy_stack_frame() reads the two words of a user stack frame with
__get_user(), after an __access_ok() on the frame and inside
pagefault_disable(), since the user stack walk can run in atomic
context.
Do the two reads in a scoped_user_read_access() instead. It nests
inside the pagefault_disable() section, does the range check itself, and
needs only a single stac/clac pair for both loads.
This is part of getting rid of __get_user() and __put_user() entirely.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
x86/signal: Use scoped user access in the sigreturn system calls
All four sigreturn variants (32-bit sigreturn and rt_sigreturn, native
64-bit and x32 rt_sigreturn) do an access_ok() on the whole signal frame
only to read the signal mask with __get_user(). The 64-bit and x32
variants also read uc_flags that way. Everything else in the frame is
read by helpers that do their own checking.
Read those fields inside a scoped_user_read_access() instead, which does
the range check once and puts a single stac/clac pair around plain
loads. The scoped accessor shadows 'frame' with a checked copy only
inside its block, so the badframe path still reports the original
pointer.
This is part of getting rid of __get_user() and __put_user() entirely.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
x86/entry: Use get_user() for the fast 32-bit syscall EBP fetch
__do_fast_syscall_32() fetches the user's EBP from the stack slot the
vDSO stashed it in. On 64-bit kernels it used __get_user(), as a
micro-optimization: the address is explicitly 32 bits, so it cannot be
out of range.
That optimization has turned into a pessimization. On x86-64,
get_user() checks the address with a compare and a conditional move,
while __get_user() does an lfence instead. Use get_user() on both
32-bit and 64-bit, which also removes the IS_ENABLED(CONFIG_X86_64)
special case.
This is part of getting rid of __get_user() and __put_user() entirely.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
net: use put_user() for msghdr results written back to user space
____sys_recvmsg() writes msg_flags and msg_controllen back with
__put_user(), and the compat sendmmsg()/recvmmsg() loops write msg_len
the same way. None of these has an access_ok() of its own: they rely on
the earlier copy_from_user() of the message header having checked the
range. For the compat msg_len that is not even true, since msg_len
lives after the compat_msghdr that was copied. It is only safe because
compat pointers are 32-bit. The native mmsg paths already use
put_user().
Use put_user() everywhere. On x86 that adds a couple of instructions to
check the address, on a path that has to enable and disable user access
for each write anyway.
This is part of getting rid of __get_user() and __put_user() entirely.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
bpf: Use get_user() for kprobe_multi symbols and uprobe_multi arrays
copy_user_syms() and bpf_uprobe_multi_link_attach() check whole user
arrays with access_ok() up front, then read them one element at a time
with __get_user(), with other work in between: allocations and
strncpy_from_user() in copy_user_syms(), validation in the uprobe_multi
loop. Each of those accesses enables and disables user access on its
own, so the up-front check saves nothing. On x86, __get_user() is even
more expensive than get_user(): it does an lfence, where get_user()
checks the address with a couple of plain instructions.
Use get_user(). The access_ok() checks stay, because they also make a
bad array fail with -EFAULT before anything else is looked up or
allocated. Update the comment that said they were there for
__get_user().
This is part of getting rid of __get_user() and __put_user() entirely.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
syscall_user_dispatch: Use get_user() for the selector
syscall_user_dispatch() reads the selector byte with __get_user(),
relying on an access_ok() done once at prctl time when the selector
address was set. So the pointer being dereferenced was validated in a
different system call, and the separation saves nothing anyway:
__get_user() still enables and disables user access, and on x86 it does
an lfence that get_user() avoids.
Use get_user(), and drop the comment explaining why the unchecked access
was safe.
The selector may be a tagged address. The prctl-time check is done on
untagged_addr(selector), but it is the tagged pointer that gets stored,
and __get_user() never checked it. get_user() does, and on x86 with LAM
a tagged pointer fails its range check. So untag it at the point of use
as well. The prctl-time check stays, so a bad address is still reported
when it is set.
[3 lines not shown]
uprobes: Use get_user() in is_trap_at_addr()
is_trap_at_addr() reads the instruction at a user address with
__get_user(), inside pagefault_disable() since the caller holds
mmap_lock. There is no access_ok() in front of it, and nothing here
needs the unchecked version: with page faults disabled, get_user()'s
might_fault() check returns early, so that is no reason to avoid it
either.
Use get_user(), which also checks that the address is a user address,
and on x86 avoids the lfence that __get_user() does.
This is part of getting rid of __get_user() and __put_user() entirely.
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
