[InstCombine] Fold select of pow into select of the differing operand
When both arms of a select are calls to `llvm.pow` with one use that
differ in exactly one operand, sink the select into that operand:
$$
\mathrm{select}(c,\ x^{y},\ x^{z}) \rightarrow x^{\mathrm{select}(c,\ y,\ z)}
$$
$$
\mathrm{select}(c,\ x^{z},\ y^{z}) \rightarrow \mathrm{select}(c,\ x,\ y)^{z}
$$
This removes one `pow` call. FMF are intersected and debug locations
are merged. The transform is skipped when a differing operand is a
constant, since that may enable a cheaper lowering (e.g.
$x^2 \rightarrow x \cdot x$).
[mlir] `HasParallelRegion` means region may be parallel (#231372)
Clarify the documentation of `HasParallelRegion`. The presence of the
trait means that the op may have parallel semantics. This trait could be
turned into an op interface in the future. Also add the trait to some
operations where it was missing.
Assisted-by: Codex
Reland [AMDGPU] Handle gfx950 v_pk_{add,mul,fma}_f32 invalid op_sel (#232016)
gfx950 v_pk_add/mul/fma_f32 for certain op_sel combinations:
- Case 1 (v_pk_add/mul_f32): src0/src1 both VGPR with op_sel[1:0] ==
2'b10. Adjusted in GCNHazardRecognizer::fixPkF32InvalidOpSel by
commuting src0/src1 in place.
- Case 2/3 (v_pk_fma_f32): op_sel[2:1] == 2'b01, 2'b10, or 2'b11. Fixed
via SIInstrInfo::decomposePkFMAF32, which splits the packed FMA into two
independent scalar v_fma_f32s; wired in from both
SITargetLowering::AdjustInstrPostInstrSelection (SelectionDAG) and
AMDGPUInstructionSelector::select's G_FMA case (GlobalISel), both gated
on SIInstrInfo::isLegalPkFMAF32OpSel. A safety net that crashes the
compiler is added in GCNHazardRecognizer::fixPkF32InvalidOpSel.
Also reject the invalid op_sel encodings directly in the assembler
(AMDGPUAsmParser::validateOpSel) so hand-written/inline asm can't encode
them.
[2 lines not shown]
[flang] Classify MOVE_ALLOC with noncoarray FROM as SIMPLE (#224923)
F2023 16.1 p5 classifies `MOVE_ALLOC` as simple when its `FROM` argument
is not a coarray, and no other standard intrinsic subroutine is pure or
simple. Mark a `MOVE_ALLOC` call `SIMPLE` and `PURE` only when `FROM` is
not a coarray. A call with a coarray `FROM` in a pure subprogram or `DO
CONCURRENT` is now also reported as a reference to an impure procedure.
Update `F202X.md` and add a test.
Part of #221457.
[libc] Use a syscall wrapper for `sched_setparam` (#231893)
Follow up of https://github.com/llvm/llvm-project/pull/231865.
This PR move `sched_setparam` behind the centralized syscall wrappers,
following `libc/docs/dev/syscall_wrapper_refactor.md`.
[Clang][counted_by] Use the innermost struct that contains the count
__builtin_dynamic_object_size and __builtin_counted_by_ref find the
count by walking the base expression to a struct that contains it, then
searching that struct's fields for the count. For a '.' member access,
StructAccessBase preferred the *outermost* such struct. When the struct
with the 'counted_by' field is embedded more than once, the search from
the outer struct finds the first copy's count, whichever copy is
accessed:
struct inner { int count; int *buf __counted_by(count); };
struct outer { struct inner a, b; };
__builtin_dynamic_object_size((*o).b.buf, 1) // used o->a.count
*__builtin_counted_by_ref((*o).b.buf) = n; // wrote o->a.count
Stop at the innermost member expression whose struct contains the count,
as is already done for '->' and array subscripts. Sema requires the
count to be in the same struct as the counted field, so the innermost
[3 lines not shown]
[mlir][memref] Fix the allocation interface model for realloc (#231608)
Use `DefaultReallocationInterface` as the concrete model for
`memref.realloc` so it gets the default `HoistingKind::None`.
The current parameter selects `DefaultAllocationInterface` and
incorrectly enables `Loop | Block` hoisting.
Assisted-by: Codex
[Clang][counted_by][NFC] Simplify getFieldOffsetInBits
getFieldOffsetInBits() kept its own running field index and didn't
advance it inside unions, which only gave the right answer because every
union member is at offset zero. Look each field's offset up by its own
index instead, which needs no union special case. The function only
used CodeGenFunction to get the ASTContext, so take that directly.
Assisted-by: Claude Opus 5.5
[Clang][counted_by] Scale the bound when a counted_by pointer is cast
-fsanitize=array-bounds compared the index of a subscript through a cast
'counted_by' pointer against the count as is, although the index counts
elements of the cast-to type and the count counts elements of the
field's pointee type:
struct s { int count; int *buf __counted_by(count); };
((char *)p->buf)[i] // checked 'i < count', not 'i < count * 4':
// valid accesses to the last 3/4 of the buffer
// trapped
((long *)p->buf)[i] // checked 'i < count', not 'i < count / 2':
// out-of-bounds accesses went undetected
__builtin_dynamic_object_size of the same expression already computed
the size in bytes, so the two disagreed.
When the accessed element size differs from the counted one, convert
[5 lines not shown]
[Clang][counted_by] Find the count in the struct that contains the field
For -fsanitize=array-bounds, the count was located relative to the
'counted_by' field within the field's *outermost lexical* record. In C a
struct defined inside another struct is still a type of its own, so that
record isn't necessarily the object the field is part of. When the inner
struct only appears in the outer one through a pointer, the field can't
be found in the outer struct at all, and the bounds check was silently
dropped:
struct holder {
struct inner {
int count;
int *buf __counted_by(count);
} *ptr;
};
int f(struct inner *q, int i) { return q->buf[i]; } // no check
[6 lines not shown]
[Clang][counted_by] Load the count through its l-value
Both __builtin_dynamic_object_size and -fsanitize=array-bounds loaded the
'counted_by' count as a raw integer of the count's type, with 'int'
alignment, at a whole-byte offset. That's wrong in several ways:
- Bit-field counts are miscompiled. __bdos loaded the whole storage
unit without shifting or masking. The bounds check truncated the
count's bit offset to bytes and loaded an 'int' from there, a
misaligned load that could overlap the flexible array member:
struct s {
unsigned flags : 3;
unsigned count : 13;
int array[] __counted_by(count);
};
- Counts that are less than 4-byte aligned (e.g. a 'short' at offset
2, or any count in a packed struct) were loaded with 'align 4', which
[27 lines not shown]
[Clang][Sanitizers] Check '++', '--' and _Complex subscripts as accesses
-fsanitize=array-bounds allows an index one past the end of an array
when only the address of the element is formed ('&a[i]'), but not when
the element is read or written. Whether a subscript is an access was
only determined by EmitCheckedLValue and by the scalar and aggregate
loads, and only for a bare subscript. Everywhere else the subscript was
emitted through EmitLValue, which treats it as forming an address, so
these accesses one past the end went undetected:
int a[4];
a[i]++; --a[i]; // also on '_Atomic' and 'counted_by' arrays
(a[i]) = 1; (a[i])++; // parenthesized subscripts
_Complex double c[4];
c[i]++; c[i] += 1.0; c[i] = 1.0; x = c[i]; x = __real__ c[i];
Add EmitAccessedLValue, which emits an array subscript (looking through
parentheses) as an access when the bounds check is enabled, and use it
for the operands of '++' and '--', for '_Complex' loads, stores and
[5 lines not shown]
[Clang][counted_by] Size '&p->field' using the struct that's accessed
__builtin_dynamic_object_size(&p->field, 0) on a struct with a
'counted_by' flexible array member is computed as
sizeof(struct) - offsetof(struct, field) + count * sizeof(element)
but 'struct' was taken to be the outermost lexical record of the field's
parent, not the struct of the object being accessed. That's only the
same thing when the field is a direct member, or is in a sub-struct that
happens to be declared inside the struct with the flexible array member.
Otherwise:
- For a field of a separately declared sub-struct ('&p->hdr.len'), the
flexible array member wasn't found and the size fell back to
unknown.
- For a struct with a flexible array member that's declared inside
another struct but used on its own, the other struct's layout was
[21 lines not shown]
[Clang][counted_by] Don't re-evaluate the base of a counted_by pointer access for -fsanitize=array-bounds
When indexing a 'counted_by' pointer field with -fsanitize=array-bounds,
EmitCountedByBoundsChecking was handed an invalid address and re-emitted
the field's l-value to locate the count. That re-evaluated the struct
base, duplicating its side-effects:
struct p { int count; int *buf __counted_by(count); };
struct p *q;
int f(int i) { return q++->buf[i]; } // 'q' incremented twice
int g(int i) { return get()->buf[i]; } // 'get' called twice
i.e. enabling the sanitizer changed the program's behavior.
When the subscript base is exactly the field load, emit the field's
l-value once, load the pointer from it (as the default case of
EmitPointerWithAlignment does, through EmitLoadOfLValue so that a
'__ptrauth'-qualified field is still authenticated), and pass the
field's address to the bounds check. When there's an intervening cast (e.g.
[6 lines not shown]
[Clang][counted_by][NFC] Add missing tests for counted_by and __builtin_counted_by_ref
Add Sema coverage for __builtin_counted_by_ref:
- a bit-field counter on a 'counted_by' pointer (not just a FAM),
- the '_or_null' and 'sized_by' variants, which return 'void *',
- qualified, enum, and typedef counter types, which determine the
returned pointee type,
- nested, anonymous, by-value, and subscripted bases.
Add CodeGen coverage for the base-expression shapes reaching a counted_by
flexible array member, for both __builtin_dynamic_object_size and
-fsanitize=array-bounds: call and side-effecting bases (which __bdos
must not re-evaluate), pointer chains, conditional bases, negative
constant indices, enum counters, and mixed index widths.
Assisted-by: Claude Opus 5.5
[mlir][scf] Fix front peeling skipped due to float rounding in trip count (#231100)
fixes #231062 which affects loops with constant bounds where the `step`
and `upper bound - lower bound` are very close and by "close" the `upper
bound - lower bound` exceeds step by less than about half a float32 ulp
at that magnitude.
eg:
`ub - lb`: 20000001, `step`: 20000000
`ub - lb`: 1099511693312, `step`: 1099511627776
removing the float conversation and hence the ceil operation and
comparing `*ubInt - *lbInt <= *stepInt` with repro as a test.
[mlir][scf] Fix front-peel unsigned loops with at most one iteration (#231542)
For unsigned loops with constant bounds, read the bounds as `APInt` and
skip peeling when `ub <= lb` or `ub - lb <= step` (compared unsigned),
bail out if failed.
from #231100 and fixes #231541
[mlir][bufferization] Fix bug when moving dealloc operation (#229296)
This commit fixes bugs that were identified in #227975.
The core reason is that the existing code does not perform checks on
wether it is safe to move `dealloc`, and would blindly do a
`deallocOp->moveAfter(lastUser)`.
We have to check that where we move it to is:
- After all the operand definitions that the `dealloc` uses
- Before any of ops that consume the results produced by the `dealloc`
This is what the `findValidInsertionPoint` method does.
AI disclosure: assistance from copilot to navigate code base, and plan
out the code. Was still written by me.
[CIR] Support statements after co_return (#231046)
Emit code for statements after a co_return in a new block, which may be
reachable through a label. Unlike classic codegen, this patch still
keeps unreachable code alive through flattening to stay closer to the
source code, though this could still be changed.
---------
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
