[LoopIdiom] Recognize the ctlz/cttz idiom when the input is known non-zero (#222373)
LoopIdiomRecognize converts a shift-until-zero loop to `ctlz`/`cttz`.
When the loop uses the count outside the loop, it needs a zero check of
the input before the loop. Without the check, the inputs 0 and 1 give
the same count. The recognizer stops when the check is not present.
The check is not necessary when the input is known non-zero. InstCombine
can remove the check for such inputs. Then the recognizer does not
convert the loop. This happens in `sbc_calc_scalefactors` from ffmpeg
with #222334: the input is an or reduction that starts at 32768, so it
is never zero, and the loop stays a loop.
This patch makes the recognizer accept a known non-zero input in place
of the zero check. The conversion is exact in this case.
`ctlz-loop.ll` in PhaseOrdering had a FIXME for a loop over `abs(n)`.
The recognizer now converts that loop too.
[2 lines not shown]
[AArch64][PAC] Reset `killed` operand flags in outlined functions
Presently, MachineOutliner does not take `killed` operand flags into
account when merging instruction sequences. While it sounds perfectly
reasonable not to inhibit merging of the instruction sequences that
only differ in `killed` flags (for N flags there is technically 2^N
valid ways to drop some subset of them), copying these flags from
an arbitrarily chosen representative instruction may result in
incorrect codegen of PAuth-related pseudo instructions on AArch64.
To keep `killed` flags conservatively correct as if `OUTLINED_FUNCTION`s
are virtually re-inserted at every call site, this patch takes the
simplest approach of resetting every `killed` flag inside the
outlined functions.
ig4: Add Lunar Lake-M I2C controllers 4 and 5
Commit 851dffef532a added the Lunar Lake-M I2C controllers 0 through 3
(0xa878-0xa87b), which sit on PCI device 0x15. The platform exposes two
further controllers at 0xa850 and 0xa851 on PCI device 0x19, reported by
Intel as I2C #4 and #5. This mirrors the layout already handled for
Arrow Lake-U, where both the 0x777x and 0x775x ranges are listed.
On an HP OmniBook X Flip 16-as0xxx (Core Ultra 9 288V) the firmware
enables only four of the six controllers, and both HID devices sit on
the two that were missing: an ELAN2514 touchscreen on controller 4 and
a SYNA3503 touchpad on controller 5. Neither attaches without this
change, so the machine has no working pointing device.
Like the other four, these use the Tiger Lake revision of the I2C IP;
Linux treats 0xa850/0xa851 identically to 0xa878-0xa87b in
intel-lpss-pci.c.
Tested on: HP OmniBook X Flip 16-as0xxx (Intel Core Ultra 9 288V)
[4 lines not shown]
[libc] Add wcscasecmp and wcsncasecmp implementations. (#222154)
Add a helper to wctype_utils.h for three-way comparison of individual
`wchar_t`s. As is the case in existing code, `wchar_t` is presumed to be
UTF-32.
Assisted-by: Automated tooling, human reviewed.
[ConstantTime] Use ARITH_FENCE as the combine barrier in CT_SELECT expansion
Replace the CopyToReg/CopyFromReg vreg barrier in ExpandCTSELECT with
ISD::ARITH_FENCE. The legalizer no longer creates vregs or picks register
classes via getRegClassFor, and scalable vectors are now covered too.
Codegen tests regenerated; output stays branchless.
[ConstantTime] Move the CT_SELECT expansion into a helper
The CT_SELECT case in ExpandNode was around 180 lines. Move it to
SelectionDAGLegalize::ExpandCTSELECT and name the operands, so the case is
three lines. Pure code motion: emitted assembly is unchanged for every RUN
line of the existing ct.select codegen tests.
Also use a temporary for the promoted SELECT so the call fits on one line
instead of wrapping after the assignment.
Co-Authored-By: Claude Opus 5 <noreply at anthropic.com>
[ConstantTime] Allow dead llvm.ct.select to be removed
An unused llvm.ct.select outlived the InstructionSimplify fold, which kept
both value arguments live and contradicted the LangRef text saying the unused
argument becomes dead code. Add the intrinsic to the wouldInstructionBeTriviallyDead
whitelist next to llvm.allow.runtime.check and llvm.allow.ubsan.check, which
carry IntrInaccessibleMemOnly for the same reason: to pin the call in place,
not to model a real memory access.
Keep IntrInaccessibleMemOnly rather than switching to IntrNoMem. As a pure
value the call is sunk into a conditionally-executed block by InstCombine
(gated on mayWriteToMemory) and split into one copy per branch arm by GVN PRE.
Also correct the undef condition semantics, trim the declaration list, and
drop claims from the definition comment that the memory effect does not
actually provide.
Co-Authored-By: Claude Opus 5 <noreply at anthropic.com>
[ConstantTime] Address reviewer feedback on llvm.ct.select core
- Fix vector-split build error: call the existing SplitRes_Select for
CT_SELECT instead of the nonexistent SplitRes_CT_SELECT.
- Fix a VP_MERGE result-split regression: the CT_SELECT switch reformat had
moved ISD::VP_MERGE onto the SplitRes_SELECT_CC line, which reads operand 4
(out of bounds for VP_MERGE's 4 operands) and builds a SELECT_CC without
splitting the EVL, crashing or miscompiling any vp.merge whose result
vector must be split. Restore it to the SplitRes_Select group.
- Accept byte types (bN and vectors of them) in the Verifier and document
them in LangRef; add byte-typed X86 codegen and Verifier test coverage.
- Drop the redundant `VT0 == MVT::i1` guard in visitCT_SELECT; the inner
condition-type checks already cover correctness after promotion.
- LangRef: reword the constant-fold rule to the enforceable "constant
operand" form, restate fast-math flags via the general FP-call rule (only
nnan/ninf are poison-generating), and switch undef/poison to match select,
dropping the noundef return attribute so poison propagates (both arms always
evaluate, so poison in either yields poison).
- Revert an unrelated whitespace change in LegalizeTypes.h.
[2 lines not shown]
[ConstantTime] Address reviewer feedback for llvm.ct.select core
Model llvm.ct.select as IntrInaccessibleMemOnly so the call stays pinned
without pessimizing alias analysis. Make CT_SELECT flagless (drop the
getCTSelect flags parameter, dead flag propagation, and
setFlags-after-getNode) and assert its condition is scalar. Blend the FP
memory fallback at the widest legal integer width with ext-load and
trunc-store tails instead of illegal i8 chunks. Restore the LangRef
section dropped in the rebase onto the Markdown docs migration and note
RISC-V Zkt/Zvkt. Apply review style fixes and regenerate affected tests.
[ConstantTime] Fix CT_SELECT expansion to preserve constant-time guarantees
Create CT_SELECT nodes for scalar types regardless of target support, so
they survive DAGCombiner (visitCT_SELECT is conservative). Expand to
AND/OR/XOR during operation legalization after SETCC is lowered, preventing
the sext(setcc)->select fold chain that converts constant-time patterns
into data-dependent conditional moves (e.g. movn/movz on MIPS).
The mask uses SUB(0, AND(Cond, 1)) instead of SIGN_EXTEND because type
legalization already promoted i1 to the SetCC result type, making
SIGN_EXTEND a no-op for same-width types.
[ConstantTime][LLVM] Add llvm.ct.select intrinsic with generic SelectionDAG lowering
[LLVM][CodeGen] Improve CTSELECT fallback lowering and target support modeling (#179395)
This pull request refactors and improves the **fallback handling** of
constant-time select (CTSELECT) in LLVM’s code generation
infrastructure. The changes clarify semantics, simplify
target-capability checks, and improve the correctness and
maintainability of fallback lowering, without changing the intended
constant-time guarantees.
- **CTSELECT semantics**
- Clarified documentation for the `CTSELECT` node to explicitly describe
its operands and its role as the lowering target for the constant-time
select intrinsic.
- **TargetLowering cleanup**
- Removed CTSELECT-specific entries from `SelectSupportKind`.
- Introduced a dedicated `isCtSelectSupported(EVT)` hook to cleanly
[23 lines not shown]
[TOSA] Add pass to sanitize gather/scatter indices (#221200)
Add new TosaGatherScatterHardeningPass TOSA pass to clamp indices before
they are fed to a tosa.gather or tosa.scatter unless a suitable clamp
sequence already exists. This enables avoiding out-of-bounds accesses on
untrusted indices in tosa.gather and tosa.scatter lowering.
The pass fails if the dimension indexed by the indices tensor of any of
the tosa.gather or tosa.scatter is dynamic. The pass supports EXT-INT64
by using the same element type in the clamp min/max sequence as the
indices tensor. In case the indexed dimension is bigger than can be
represented in the element type bitwidth the maximum value in that
bitwidth is used.
Assisted-by: codex
Signed-off-by: Thomas Preud'homme <thomas.preudhomme at arm.com>
