[libc++] Use public os_sync API instead of private __ulock on newer Apple platforms (#202519)
The atomic wait and wake implementation on Apple platforms currently
relies on `__ulock_wait` and `__ulock_wake`, which are private kernel
APIs. This is a problem for anyone shipping apps through the App Store
since Apple flags private symbol usage during review.
Starting with macOS 14.4 and iOS 17.4, Apple ships public replacements
through `os_sync_wait_on_address` and `os_sync_wake_by_address_any/all`
in `<os/os_sync_wait_on_address.h>`. These cover the same functionality
and are documented, stable, and safe for App Store submissions.
This patch makes use of the public APIs instead of the private ones
whenever the underlying OS permits it.
This takes over #182947.
Fixes #182908
Fixes #146142
Co-authored-by: Bbn08 <atrancendentbeing at gmail.com>
[libc] Include linux headers to get ioctl macros (#204555)
Linux has many existing ioctls and keeps adding them, so a
hand-maintained list would always be out of date. Additionally, some
ioctls have architecture specific numbers (some in a very subtle way --
by having the number depend on the size of a structure).
asm/ioctls.h and linux/sockios.h are pretty clean, and are already
included by glibc, so we can just do the same to get the latest
definitions.
[AMDGPU] Mark all instructions in WWM region as convergent
Mark instructions between ENTER_STRICT_WWM and EXIT_STRICT_WWM as
convergent, so they don't get moved out of the whole wave mode region
(see the licm-wwm.mir test). This doesn't automagically fix all our
woes, since things can still be moved out of the region before we even
run si-wqm, but there are rumours about moving WWM formation earlier
anyway.
This is not a substitute for proper WWM support - in particular, this
would inhibit most optimizations inside WWM regions with complex control
flow. Right now most WWM is relatively limited in size and complexity,
so I think this is acceptable until we get a more principled solution.
I haven't thought too much about whether or not we need this for WQM as
well.
Assisted by: Claude Sonnet
commit-id:9204c7e2
[AMDGPU][doc] Refactor Barrier Execution Model
Remove everything that has to do with named barriers and put it in a series of model extensions specific to /sbarrier/named-barriers.
I had to change a few things to make it fit, in summary:
Base Model:
* Stylistic changes that make it easier to refer to specific rules. Each rule is in a rubric instead of a bullet point.
* (-) No longer defines `barrier-mutually-exclusive`
* (-) No longer defines barrier `join` and any associated rule.
New named barrier extensions
* Define "named barrier" as a sub-type of barrier objects. This makes barrier-mutually-exclusive redundant.
* Define barrier join as an op that can exclusively be done on `named barrier objects`.
* Define rules relating to join and its ordering with other barrier operations
Following these changes, the target tables changed a bit as well.
[2 lines not shown]
[libomp] Parse OMP_DEFAULT_DEVICE with new device trait parser
... but do not yet expose the new functionalities to the user. This is a
backward compatible update that is going to be followed by the step to
the OpenMP 6.0 semantics as defined in 4.3.8.
AMDGPU: Add subtarget feature for controllable xnack modes
This replaces the previously removed xnack-any-only feature,
with the inversion xnack-on-off-modes. All pre-gfx12.5 xnack
targets support the controllable mode. Ignore explicitly
set xnack settings the same way as is done for xnack requests
on other unsupported targets.
[AMDGPU] Mark all instructions in WWM region as convergent
Mark instructions between ENTER_STRICT_WWM and EXIT_STRICT_WWM as
convergent, so they don't get moved out of the whole wave mode region
(see the licm-wwm.mir test). This doesn't automagically fix all our
woes, since things can still be moved out of the region before we even
run si-wqm, but there are rumours about moving WWM formation earlier
anyway.
This is not a substitute for proper WWM support - in particular, this
would inhibit most optimizations inside WWM regions with complex control
flow. Right now most WWM is relatively limited in size and complexity,
so I think this is acceptable until we get a more principled solution.
I haven't thought too much about whether or not we need this for WQM as
well.
Assisted by: Claude Sonnet
commit-id:9204c7e2
Repurpose MIFlag::NoConvergent
The NoConvergent MIFlag allows us to mark specific instances of
convergent (as indicated by their MCID) MachineInstrs as not convergent.
Sometimes it's useful to do the opposite as well - mark certain
instances of instructions that are not normally convergent as
convergent (for instance inside WWM regions on AMDGPU).
This patch renames the NoConvergent flag to OverrideConvergence. This
can be set to communicate that if the opcode is usually convergent, then
this particular instance of it isn't, and the other way around. When
changing the opcode of an instruction, we first check if the new opcode
has the same "convergence" as the old one - if it does, then we preserve
the flag, otherwise we clear it since we can get the correct convergence
from the opcode now.
Assisted by: Claude Sonnet
commit-id:93c99000
[X86] Fix stale kill flag when folding VPMOV*2M + KMOV to VMOVMSK (#204342)
tryCompressVPMOVPattern folds VPMOV*2M + KMOV into a single VMOVMSK in
place: it changes the KMOV's opcode and repoints its source operand from
the mask k-register to the XMM source via MachineOperand::setReg().
setReg() only changes the register number and keeps the operand's other
flags, so the kill flag computed for the mask ("killed $k0") is reused
for the XMM source. When the source is still live this marks it killed,
which the machine verifier reports as a use of an undefined register.
We should instead use the kill flag from the VPMOV's source operand. The
forward scan already guarantees the source is not redefined between the
VPMOV and the KMOV, so the VPMOV's flag is correct at the relocated
read.
Found via @jlebar's X86 LLVM bug-hunt / FuzzX effort:
https://github.com/SemiAnalysisAI/FuzzX/tree/master/x86/bugs/043-compress-evex-vpmov-srcvec-clobber-kmov
cc @jlebar
[X86][GlobalISel] Explicitly legalize G_INVOKE_REGION_START (#203503)
Removing dependency on the legacy ruleset similarly to #197374
The missing testing coverage was found during LegacyLegalizerInfo
removal in #197308
[LifetimeSafety] Count escape facts when classifying persistent origins (#204485)
computePersistentOrigins marks an origin persistent (kept across CFG
blocks)
only if it appears in more than one block, but it omitted
OriginEscapesFact.
A global is not seeded at function entry, so a global assigned a stack
address
on a conditional or loop path had its origin appear only in the storing
block;
misclassified as block-local, its loan was dropped at the join before
the
escape check, a silently missed dangling-global
(stack-use-after-return).
Count the escaped origin as a cross-block appearance.
Assisted-by: Claude Opus 4.8
Co-authored-by: Gabor Horvath <gaborh at apple.com>
[Flang][Driver] Support for -fsplit-lto-unit option in flang driver (#202858)
When mixing Fortran objects from Flang with C/C++ objects compiled by
Clang during a combined LTO build, it is necessary to ensure that all
files use the same setting for split-lto-unit. This requires the support
for -fsplit-lto-unit option in the flang driver. This support is added
as part of this commit.
Co-authored-by: Shivarama Rao <shivarama.rao at amd.com>
AMDGPU: Add subtarget feature for controllable xnack modes
This replaces the previously removed xnack-any-only feature,
with the inversion xnack-on-off-modes. All pre-gfx12.5 xnack
targets support the controllable mode. Ignore explicitly
set xnack settings the same way as is done for xnack requests
on other unsupported targets.
AMDGPU: Remove xnack-any-only subtarget feature and handling (#204514)
This reverts commit f4caa0a172d96597c375e6b6b2192c289723a6b9.
This feature was added to gfx12-5-generic only, which does not make
sense given that both gxf1250 and gfx1251 have the same unconditional
xnack handling. It also does not make sense to diagnose trying to use
a specific xnack mode on the generic target only, and only from the
backend.
The current feature management is a confusing mess, given that we have
2 parallel feature systems. AMDGPUTargetParser has a table containing
a bitmask of features, which already contained FEATURE_XNACK_ALWAYS
for gfx1250/gfx1251, but not gfx12-5-generic. Add this handling there
so the sanitizer detection is consistent on the generic target.
These 2 feature tables probably should be unified in some way. We also
probably should have a subtarget feature for the xnack handling, but it
should be inverted. xnack-any-only is an antifeature, in that it removes
functionality from the base target. It would be better to invert this,
so all of the older targets support configurable xnack modes.
[AMDGPU] Capping max number of registers to function's occupancy budget for indirect calls (#199765)
Depends on https://github.com/llvm/llvm-project/pull/199746
Changed the SetMaxReg Lambda function to cap at
ST.getMaxNumVectorRegs(F) for VGPRs and AGPRs and ST.getMaxNumSGPRs(F)
for SGPR. ST.getMaxNumVectorRegs(F) and ST.getMaxNumSGPRs(F) returns the
budget for the given function F and is mainly determined by the
occupancy of the function.
This cap is needed since the module's max could overestimate register
usage because it includes functions that earlier in the compiler was
determined inaccessible by that indirect call. ST.getMaxNumVectorRegs(F)
and ST.getMaxNumSGPRs(F) were calculated according to an more accurate
call graph.
This fixes an issue of overinflated number of VGPR/AGPR for kernels that
have indirect function calls. This inflation has led to some kernels
with indirect calls to go over the limit for VGPR/AGPR and crash.
[5 lines not shown]
[CommandLine] Make cl::boolOrDefault a scoped enum
Prevents implicit conversion to bool/int, where BOU_FALSE wrongly
evaluated as true. All uses qualified as cl::boolOrDefault::BOU_*.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply at anthropic.com>
[NFC][AMDGPU] Introduce SIInstrFlags predicates for TSFlags access (#201512)
Add inline predicate templates to SIDefines.h for each TSFlags bit
(isSALU, isVOP3, isFLAT, isMaybeAtomic, hasFPClamp, ...) plus compound
predicates (isAtomic, isSegmentSpecificFLAT, isImage, isVMEM).
Route all SIInstrInfo.h thin wrapper bodies through the new predicates
instead of reading TSFlags bits directly.
---------
Co-authored-by: Claude Sonnet 4.6 <noreply at anthropic.com>
