[libc++][filesystem] Fix truncation in vformat_string dynamic fallback (#203531)
The `vformat_string` fallback path (used when the message exceeds the
stack buffer) had two bugs that caused the last character to be silently
dropped:
1. **Off-by-one in `resize_and_overwrite`**: `vsnprintf` returns the
number of bytes needed *excluding* the null terminator. The fallback
called `resize_and_overwrite(size, ...)`, which passed `n == size`
to `vsnprintf`. Since `vsnprintf` needs `n+1` bytes to write `n`
characters plus the null terminator, it truncated the output by one
character. Fixed by keeping `resize_and_overwrite(size, ...)` and
passing `n + 1` to `vsnprintf`: `std::string` always has room for a
null terminator past its size, so there is no need to request
`size + 1` characters from the string.
2. **`ap` was never the problem**: Because the first `vsnprintf` call
uses `apcopy` (not `ap`), `ap` remains valid for use in the
fallback. The fallback correctly uses `ap` directly.
[13 lines not shown]
[SDAG] Add MLOAD/MSTORE mask scalarization support to LegalizeVectorTypes (#230202)
This PR adds support for the scalarization of mask operands for
MLOAD/MSTORE, and mirrors the logic in `LegalizeIntegerTypes`.
NOTE: for MSTORE on AArch64, only the mask can reach this function as
stores with illegal data types are expanded before SDAG (see #228122)
Fixes e.g. the crash:
```llvm
; llc -mtriple=aarch64 -mattr=+sve crash.ll
define <1 x double> @masked_load_v1f64(ptr %ap, ptr %bp) {
%a = load <1 x double>, ptr %ap
%b = load <1 x double>, ptr %bp
%mask = fcmp oeq <1 x double> %a, %b
%load = call <1 x double> @llvm.masked.load.v1f64(ptr %ap, i32 8, <1 x i1> %mask, <1 x double> zeroinitializer)
ret <1 x double> %load
}
```
[libc][docs] Finish MyST migration for selected libc docs (#233690)
This migrates the remaining llvm-libc doc files to markdown.
The main concern I have is that there are still 6 eval-rst blocks
wrapping grid-tables, and native markdown tables have no way to
represent the colspan and rowspan cells in those files. In the end, I
decided to send this off as is, since it should have no visible
difference, and we can eliminate eval-rst later.
Migration notes from the agent:
- Restored underscore anchor labels such as `libc_gpu_rpc` because
rst2myst turned them into hyphenated names that other pages still
reference.
- Replaced Markdown links to Sphinx labels with `{ref}` roles because
plain links did not resolve to the labelled pages.
- Converted the flat grid tables in `headers/math/index.md` and
`headers/complex.md` to pipe tables using the `{{ check }}` substitution
[35 lines not shown]
[libc][docs] Rename selected rst docs to md (#233687)
Tracking issue: #201242
See the [migration guide] for more information.
[migration guide]:
https://llvm.org/docs/SphinxQuickstartTemplate.html#markdown-migration-guidelines
This is the initial straight rename commit. It will probably break the
docs build, but it has to be a separate PR for blame preservation
purposes.
[llvm][CodeView] Don't visit anything in destructor of field list deserializer (#231287)
When a `visitKnownMember` callback errors in a field list deserializer
pipeline, we never call `visitMemberEnd` afterward - we immediately
stop.
If the `FieldListDeserializer` would call
`TypeRecordMapping::visitTypeEnd`, it would assert with [`!MemberKind &&
"Still in a member
mapping!"`](https://github.com/llvm/llvm-project/blob/0f8f2d14865a5b83a7a8ca57f63eaca64be7bbe5/llvm/lib/DebugInfo/CodeView/TypeRecordMapping.cpp#L244),
because `MemberKind` was never reset after the error.
This PR removes the call in the destructor of `FieldListDeserializer`.
It was a no-op in the success case anyway, because the reader wouldn't
be advanced. And even if it would, we only ever read from an `ArrayRef`
of the field list's data, so there's no point in caring about the
reader's position.
[AtomicExpand][NVPTX] Add single-lane atomic RMW coverage
Record current expansion and code generation before changing single-lane RMW legalization. Check the full pipeline separately from direct instruction selection while their output differs.
Revert "[libc] Remove full header dependencies from proxy header targets" (#234668)
Reverts llvm/llvm-project#234150 due to breakage on the
[libc-x86_64-debian-gcc-fullbuild-dbg](https://lab.llvm.org/buildbot/#/builders/131)
bot. The issue is that GCC's immintrin.h includes <stdlib.h> before
we've built our own version, which means GCC ends up parsing the system
header, and tripping over inconsistent header state.
I don't see an obvious fix, so reverting this for the time being.
[TailCallElim] Place byval copy-back after trailing instructions
When eliminating a self-call with byval arguments, the next iteration's
value is copied into the function's byval argument at the call's
position. But instructions between the call and the return are left in
place, and the original semantics have them read the caller's own byval
value, since the callee only ever saw a private copy. A trailing load of
the byval argument therefore read the new value.
Insert the copy-back right before the new back-edge branch, after those
instructions. They are side-effect free, so they cannot observe or
clobber the temporary being copied.
Fixes #234334
LLM-aided
[libc][math] Optimize exp and share double-precision evaluation with exp2.
Refactor double-precision exp to share most of the computations with exp2,
and adding support for `LIBC_MATH_HAS_SMALL_TABLES` and
`LIBC_MATH_HAS_SKIP_ACCURATE_PASS`.
Assisted by: Gemini is used for performance and code size analysis and refactoring.
TableGen: Look up sub-register index compositions in an array
compose() looked up each composition in a std::map. It is called for
every super-register class set in extendSuperRegClasses, and by
addComposite. Sub-register indices are numbered densely, so also keep
each index's compositions in an array indexed by enum value. For
AMDGPU's 400 indices this is about 1 MB.
AMDGPU tablegen runs that build the register bank:
Gigainstructions Gigacycles CPU seconds
Before 746 474 148
After 721 (-3.4%) 446 (-6.1%) 139 (-5.8%)
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
Return conflicting compositions from addComposite immediately
Check for an existing composition up front: return a conflicting one
right away, instead of after updating B's ranges from a composition that
isn't recorded. This no longer needs the try_emplace result. The
generated code for all in-tree targets is unchanged, and the instruction
count is the same.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
TableGen: Skip classes without matching super-register classes in RegisterBankEmitter
visitRegisterBankClasses checks every sub-register index for every pair
of register classes. Skip the per-index loop if RC is not in any of the
class's super-register class sets.
AMDGPU tablegen run for AMDGPUGenRegisterBank.inc:
Gigacycles CPU seconds
Before 51.7 16.0
After 48.4 (-6.4%) 14.9 (-6.9%)
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
TableGen: Insert super-register class sets in one step
extendSuperRegClasses added each super-register class to the composed
index's set with a separate addSuperRegClass call, which looks up the
set in the DenseMap every time. Look up the set once and insert the
whole range.
AMDGPU tablegen runs that build the register bank:
Gigainstructions Gigacycles CPU seconds
Before 797 489 153
After 755 (-5.2%) 477 (-2.4%) 149 (-2.5%)
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
TableGen: Return early from addComposite for known compositions
computeComposites calls addComposite for each pair of sub-registers of
every register with a distinct sub-register structure, so most calls
repeat a composition that is already known. Each call still did three
std::map lookups per hardware mode to update the sub-register ranges.
If the composition already exists and its result has an offset in every
mode, the rest of the function has no effect, so return early. Cache
whether the result has an offset in every mode, since offsets are never
cleared once set.
AMDGPU tablegen runs that build the register bank:
Gigainstructions Gigacycles CPU seconds
Before 916 540 169
After 797 (-13.0%) 489 (-9.5%) 153 (-9.4%)
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
TableGen: Look up sub-registers by index in a sorted array
inferMatchingSuperRegClass looks up a sub-register by index for every
member of every register class, and each lookup walks the register's
std::map of sub-registers. Keep a copy of the map in an array sorted by
index, and binary search it instead. The instruction count is about the
same, but there are far fewer cache misses.
AMDGPU tablegen runs that build the register bank:
Gigacycles CPU seconds Peak RSS (RegisterBank)
Before 650 201 837 MB
After 543 (-16.5%) 169 (-16.1%) 858 MB (+2.5%)
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
Free the sub-register map once the sorted array is built
The map is not modified after computeSuperRegs, so keep only the array,
with the sub-register index pointers so it can also be iterated, and
search it by the index's enum value.
AMDGPU tablegen runs that build the register bank:
Gigacycles CPU seconds Peak RSS (RegisterBank)
Before 541 169 857 MB
After 540 (-0.1%) 169 (+0.1%) 792 MB (-7.6%)
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
SelectionDAG: Build FP immediates from integer bits in the legalizer
Teach the generic legalizer to legalize FP immediates as a bitcast of
an opaque integer constant. Migrate LoongArch to use the new helper
instead of doing it in custom lowering.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
CodeGen: Return how to materialize FP immediates from targets (#233766)
Replace the isFPImmLegal target hook with getFPImmAction, which
returns whether an FP immediate is legal or how it should be
legalized. This will allow adding a way for the legalizer to build an
immediate from its integer bits, instead of each target doing so in
custom lowering or selection.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[X86] Simplify sret forwarding check
Find the caller's single sret argument and look for it among the call's
sret arguments with range-based loops, as suggested in review.
LLM-aided
[X86] Detect forwarded sret pointers in IR for sibling calls
When the caller has an sret return register, isEligibleForSiblingCallOpt
only tail called if the callee was passed the caller's sret pointer.
That was proven by matching a CopyToReg of the argument into the sret
vreg in the SelectionDAG. SelectionDAG is built per basic block, and that
CopyToReg is only emitted in the entry block, so the same forwarding
behind a branch was missed and the call was emitted as call + ret with a
frame.
Prove forwarding in IR instead: the call has an sret argument that is the
caller's own sret Argument. This does not depend on which block holds the
call. Drop the DAG pattern match; it only covered cases where the sret
pointer is synthesized without an IR argument, which are rare.
Fixes #234665.
LLM-aided
[flang] Fix fixed line length overflow diagnostic (#233353)
Fixes #232487
Report the large-columns diagnostic when a positive decimal value for
`-ffixed-line-length` exceeds the `int64_t` parsing range of
`getAsInteger`.
Add a regression test for the overflowing value.
