[AArch64] Lower rmw sub to FNEG+LDFADD when LSFE present (#220568)
This patch enables `atomicrmw sub` nodes to be lowered into
FNEG+LD(B)FADD instructions when LSFE is present and we are not in
strictfp environment.
Additionally, this patch fixes a bug which allowed atomicrmw with fp128
to be considered legal when LSFE present.
[CIR] Use the modern enum case classes
The `I32EnumAttrCase` family carries an `Attr` half, and an `IntegerAttr`
predicate with it, that a CIR enum has no use for now that the enums derive
from `EnumInfo`. Upstream says of those forms that they "are not needed when
using the newer `EnumCase` form".
Rename all 198 of them to `I32EnumCase`, `I32BitEnumCaseNone`,
`I32BitEnumCaseBit` and `BitEnumCaseGroup`. The group class drops its width
prefix because the modern spelling takes the width from its cases.
NFC, mechanical.
[CIR] Drop the redundant suffix from the inline kind mnemonic
inline_kind was the one CIR enum attribute mnemonic still repeating what its
C++ enum class name says. The attribute now spells
`#cir.inline<always_inline>`. The operation argument keeps the name
inline_kind, since that is the accessor name, so the printed form reads
`inline_kind = #cir.inline<always_inline>`.
The enum's summary also becomes "inline kind" rather than the camelCase
"inlineKind", which is what generated docs show now that CIR_InlineKindAttr
no longer overrides it.
25 CHECK lines change across four test files. Nine are in an
aarch64-registered-target test, unsupported in an X86-only build, but the
substitution matches the two CIR tests that do run.
[CIR] Derive lowering attr names from cppClassName, not the def name
CIRLoweringEmitter built its CXX_ABI_ALWAYS_LEGAL_ATTRS entries with
GetOpCppClassName, which splits the TableGen def name at the first
underscore. That works only while every def is named CIR_<CppClassName>Attr.
When one is not, the emitter writes an `isa<>` for a class that does not
exist, and the failure lands as a compile error in generated code.
Attributes carry the authoritative name in cppClassName, which
GenerateAttrToValueVisitor was already reading. Factor that out as
GetAttrCppClassRef and use it for both attribute paths. GetOpCppClassName
stays for operations.
NFC, and checkable. No CIR attribute overrides cppClassName, so the generated
CIRLowering.inc is byte-identical.
[CIR] Drop dead ceremony around the CIR enum attributes
Five things that no longer earn their place in the CIR enum attribute
machinery.
CIR_CleanupKindAttr carried three. Its cppClassName restated the default
AttrDef already derives. Its skipDefaultBuilders plus hand-written
AttrBuilder existed only to default $value to CleanupKind::All, which no
caller relies on, so the generated builders stayed suppressed for nothing.
And its summary and description restated the name, overriding the enum's own
"cleanup kind" that EnumAttr would otherwise inherit. The isNormal, isEH and
isNormalAndEH helpers stay.
CIR_TLSModelAttr's summary restated its name the same way, so only that goes.
CIR_DefaultValuedEnumParameter has never had a user.
NFC.
[CIR] Migrate the FPClassTest bit enum and unquote its flags
cir.is_fp_class printed its flags inconsistently. Single-bit values came out
bare, as in `fcSNan`, while group values and combinations came out quoted, as
in `"fcInf"` and `"fcSNan|fcNegInf"`. That comes from I32BitEnumAttr setting
printBitEnumQuoted, which EnumAttr.td keeps only for backwards compatibility.
Clearing the bit and using the `enum` directive selects the separator-aware
parser and printer, so every value now spells unquoted:
cir.is_fp_class %x, fcSNan|fcNegInf : (!cir.float) -> !cir.bool
The enum also drops its specialized IntegerAttr for a CIR_EnumAttr wrapper,
giving it the standalone spelling `#cir.fp_class<fcSNan|fcNegInf>`. This
changes operation syntax, so it updates 37 CHECK lines.
[CIR] Migrate GlobalLinkageKind, CallingConv and SideEffect off IntegerAttr
GlobalLinkageKind, CallingConv and SideEffect generated IntegerAttr
subclasses with no dialect spelling of their own. Each now sets
genSpecializedAttr = 0 and gains a CIR_EnumAttr wrapper, and cir.global wraps
$linkage in `enum()`. GlobalLinkageKind spells `#cir.linkage<internal>`,
dropping both the `global_` prefix and the `_kind` suffix.
cir.func and cir.call print all three by hand, but they stream
stringifyGlobalLinkageKind(getLinkage()) and friends, which take the enum
rather than the attribute, so those sites are unchanged.
Operation syntax is unchanged.
[CIR] Move the CIR enums off the legacy EnumAttrInfo hierarchy
MLIR has two enum hierarchies. `EnumAttrInfo` doubles as an `IntegerAttr`
constraint, so every CIR enum had to clear `genSpecializedAttr` to say it did
not want one. `EnumInfo` describes a C++ enum and nothing more.
Derive the CIR bases from `I32Enum`, `I64Enum` and `I32BitEnum`, and widen
`CIR_EnumAttr` to the `EnumInfo` that upstream `EnumAttr` already takes. The
flag no longer exists to clear. `FPClassTestEnum` gets unquoted printing from
`BitEnumBase` rather than overriding `printBitEnumQuoted`, and
`CIR_KnownFuncKind` drops a `parameterPrinter` the generated `operator<<`
now covers, still spelling `#cir.func_identity<"std::find">`.
AMDGPU wraps an `I32Enum` in an `EnumAttr` with this same bracketed format.
Parsing moves to the generated `FieldParser`, whose diagnostic names the
accepted spellings, so two `expected-error` lines change. Generated attribute
code drops 16 KB as 28 inlined parsers collapse into it.
[CIR] Migrate AssumeBundleKind, AtomicFetchKind and AsmFlavor off IntegerAttr
AssumeBundleKind, AtomicFetchKind and AsmFlavor generated IntegerAttr
subclasses with no dialect spelling of their own. Each now sets
genSpecializedAttr = 0 and gains a CIR_EnumAttr wrapper.
Unlike the other CIR operation enums, these three are reached through
hand-written parsers and printers, so they needed checking individually.
cir.atomic.fetch references $binop declaratively and gains an `enum()`
wrapper. The other two need no change, since printAssumeBundle is already
typed on cir::AssumeBundleKindAttr and InlineAsmOp::print streams the enum
rather than the attribute.
Operation syntax is unchanged.
[CIR] Migrate MemOrder and SyncScopeKind off IntegerAttr
MemOrder and SyncScopeKind, the enums the atomic operations share, generated
IntegerAttr subclasses with no dialect spelling of their own.
Both now set genSpecializedAttr = 0 and gain CIR_EnumAttr wrappers, spelling
`#cir.mem_order<seq_cst>` and `#cir.sync_scope<system>`, and the atomic
operations wrap their arguments in `enum()` to keep the bare keyword.
`enum()` works as an optional-group anchor, so the `syncscope` and `atomic`
groups on cir.load and cir.store are unaffected. Operation syntax is
unchanged.
[CIR] Delete the unused cir::VisibilityAttr
CIR_VisibilityAttr had no users. `cir.global` and `cir.func` carry visibility
as `EnumProp<CIR_VisibilityKind>`, a property rather than an attribute, so
nothing ever built or printed the attribute.
Its only consumer was CIRGenModule::getGlobalVisibilityAttrFromDecl, itself
never called, and that was the only caller of
getGlobalVisibilityKindFromClangVisibility, so all three go together. The
similar getCIRVisibilityKind does have a caller and stays, as does
CIR_VisibilityKind, which the property is built from.
This also removes one of the two attributes overriding their assembly format
to a bare `$value`.
[CIR] Migrate seven operation enums off IntegerAttr
CastKind, DynamicCastKind, CmpOpKind, ComplexRangeKind, InitCatchKind,
CaseOpKind and AwaitKind generated IntegerAttr subclasses. The operations
printed them symbolically, but in an attribute dictionary `cir.cast bitcast`
was stored as `kind = 1 : i32`.
Each enum now sets genSpecializedAttr = 0 and gains a CIR_EnumAttr wrapper,
and the operations wrap the argument in `enum()` to keep the bare keyword,
giving spellings like `#cir.cast<bitcast>`. Mnemonics drop the suffix the C++
class name carries. DynamicCastKind spells out `dynamic_cast`, since
`dyn_cast` is taken by the operation and by `#cir.dyn_cast_info`.
CUDADeviceVarKind gets no wrapper, being only a raw parameter of
CIR_CUDAVarRegistrationInfoAttr.
Operation syntax is unchanged, and enum-attrs.cir covers the new spellings.
[CIR] Drop lang_address_space's custom parenthesized attribute format (#220880)
CIR_LangAddressSpaceAttr overrode its assembly format to
`(` custom<AddressSpaceValue>($value) `)`. The parentheses defeated the
dialect's `#cir.mnemonic<...>` syntax, so the attribute printed as
`#cir<lang_address_space(offload_global)>`. It now uses the bracketed
CIR_EnumAttr default and spells
`#cir.lang_address_space<offload_global>`.
The `lang_address_space(x)` spelling inside `!cir.ptr` and `cir.global`
is
unaffected, since that goes through MemorySpaceAttrInterface in
CIRTypes.cpp.
The attribute-level pair in CIRAttrs.cpp was only reachable from the
deleted
format, so it goes away.
[CIR][NFC] Share getSuccessorRegions across region-branch ops
Six of the ten CIR ops implementing RegionBranchOpInterface reported the same
successors: any of their regions may be entered from the parent operation, and
every region exit goes back to it. Add a CIR_EnterAnyRegionBranchOpBase class
that appends that definition to the one inherited from CIR_RegionBranchOpBase,
retarget the six ops onto it and delete their hand-written definitions.
The shared definition walks getRegions() rather than naming region accessors.
For all six ops the entry regions were exactly the declared regions in
declaration order, so it reports the same successors in the same order. The doc
comments of the deleted ScopeOp and TernaryOp definitions go with them, instead
of being left behind on the neighbouring builders.
IfOp, GlobalOp, TryOp and AwaitOp stay on the base class, since their successors
depend on the operation: IfOp falls back to the parent when the else region is
empty, GlobalOp skips its optional ctor and dtor regions, TryOp iterates
variadic handler regions, and AwaitOp routes ready to resume and suspend.
[CIR][NFC] Share getSuccessorInputs across region-branch ops
The ten CIR ops implementing RegionBranchOpInterface each hand-wrote
getSuccessorInputs, and all ten bodies were equivalent: regions take no
inputs, and returning to the parent yields the parent's results. Three did
not look equivalent but are: CleanupScopeOp and CoroBodyOp returned an empty
ValueRange unconditionally and declare no results, and AwaitOp returned
region block arguments but carries NoRegionArguments, so those ranges are
always empty.
Add a CIR_RegionBranchOpBase ODS class that declares the method and generates
the single shared body through extraClassDefinition, mirroring the existing
CIR_LoopOpBase, and retarget all ten ops onto it.
The generated CIROps.h.inc is unchanged and CIROps.cpp.inc gains exactly the
ten definitions removed from CIRDialect.cpp.
[CIR] Add RegionBranchOpInterface unit tests and fix cir.await successors (#220507)
Adds unit tests for the five CIR ops that implement
`RegionBranchOpInterface`
with no coverage today, and fixes a `cir.await` bug the new tests found:
`verifyControlFlowInterfaceConsistency` fails on `cir.await` today,
because the
op and its own terminator disagree. `cir.condition` terminates the ready
region
and reports `{resume, suspend}` when the parent is an await, while
`AwaitOp::getSuccessorRegions` listed all three regions as entry
successors and
reported the parent op as the successor of every region exit.
The corrected edges match what `cir.condition` already reports:
- from the parent, the only entry successor is `ready`
- from `ready`, the successors are `resume` and `suspend`
- from `suspend` or `resume`, the successor is the parent operation
[mlir][arith] Fold scaling_extf and scaling_truncf with constant operands (#215123)
Add constant folders for the two MXFP scaling casts, mirroring the
expansion
in `ExpandOps.cpp`:
```
scaling_extf(in, scale) -> mulf(extf(in), extf(scale))
scaling_truncf(in, scale) -> truncf(in / extf(scale))
```
Note the asymmetry the expansion already has: `scaling_extf` widens the
scale
to the result type, `scaling_truncf` to the type of `in`.
These were the only two ops in the dialect with neither a folder nor a
canonicalizer, so `-canonicalize` left them alone even with both
operands
constant, while `-arith-expand -canonicalize` folded them away.
[32 lines not shown]
[CIR][SPIR-V] Set spir_kernel calling convention for AMDGCN-flavored HIP kernels (#214246)
CIR emitted no calling convention for HIP `__global__` kernels on the
`spirv64-amd-amdhsa` target, unlike generic SPIR-V, which already gets
`spirv_kernel` CC
Check the S3 plugin under mypy strict
Nothing checked the plugin's annotations, and it showed. The grant
helpers took Iterable of GrantRow or Mapping of str to Any, a union
that accepts whatever the Mapping accepts, so the TypedDict added
nothing. It now takes GrantRow alone. The rows a datastore query hands
over are Any, which is assignable to that, so the TypedDict is what
stops the Any at the boundary rather than a cast the callers would
have to write.
The plugin joins the mypy workflow's list and passes strict. What that
took is small. Two list variables in the listen text needed a type.
Values read off middleware.call are Any and were returned as bool or
int or None, so they land in an annotated name first. The port
validator helper is untyped and only forwards to port.validate_port,
so the plugin calls that directly. The delegates read the config as
the entry it is rather than dumping it, and the listen delegate dumps
each listener itself. audited_bucket_names queries through
middleware.call, where the result is typed as the list it is, instead
[5 lines not shown]
