BSD Commit Log Search

[AsmParser][SystemZ][z/OS] Add in support to allow use of additional comment strings.

- Currently, MCAsmInfo provides a CommentString attribute, that various targets can set, so that the AsmLexer can appropriately lex a string as a comment based on the set value of the attribute.
- However, AsmLexer also supports a few additional comment syntaxes, in addition to what's specified as a CommentString attribute. This includes regular C-style block comments (/* ... */), regular C-style line comments (// .... ) and #. While I'm not sure as to why this behaviour exists, I am assuming it does to maintain backward compatibility with GNU AS (see https://sourceware.org/binutils/docs/as/Comments.html#Comments for reference)
For example:
Consider a target which sets the CommentString attribute to '*'.
The following strings are all lexed as comments.

```
"# abc" -> comment
"// abc" -> comment
"/* abc */ -> comment
"* abc" -> comment
```

- In HLASM however, only "*" is accepted as a comment string, and nothing else.
- To achieve this, an additional attribute (`AllowAdditionalComments`) has been added to MCAsmInfo. If this attribute is set to false, then only the string specified by the CommentString attribute is used as a possible comment string to be lexed by the AsmLexer. The regular C-style block comments, line comments and "#" are disabled. As a final note, "#" will still be treated as a comment, if the CommentString attribute is set to "#".

Depends on https://reviews.llvm.org/D99277

    [4 lines not shown]

[AsmParser] Recognize more escaped characters between single quotes

The GNU AS manual states the following about single-character constants enclosed within single quotes:

>  Some backslash escapes apply to characters, \b, \f, \n, \r, \t, and \" with the same meaning as for strings, plus \' for a single quote.

Add two more characters to the switch handling this case to match GAS behaviour, plus a test to make sure nothing regresses.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D99609

[M68k] Add support for Motorola literal syntax to AsmParser

These look like $00A0cf for hex and  %001010101 for binary. They are used in Motorola assembly syntax.

Differential Revision: https://reviews.llvm.org/D98519

[AsmParser][SystemZ][z/OS] Add in support to accept "#" as part of an Identifier token

- This patch adds in support to accept the "#" character as part of an Identifier.
- This support is needed especially for the HLASM dialect since "#" is treated as part of the valid "Alphabet" range
- The way this is done is by making use of the previous precedent set by the `AllowAtInIdentifier` field in `MCAsmLexer.h`. A new field called `AllowHashInIdentifier` is introduced.
- The static function `IsIdentifierChar` is also updated to accept the `#` character if the `AllowHashInIdentifier` field is set to true.
Note: The field introduced in `MCAsmLexer.h` could very well be moved to `MCAsmInfo.h`. I'm not opposed to it. I decided to put it in `MCAsmLexer` since there seems to be some sort of precedent already with `AllowAtInIdentifier`.

Reviewed By: abhina.sreeskantharajan, nickdesaulniers, MaskRay

Differential Revision: https://reviews.llvm.org/D99277

[AsmParser][SystemZ][z/OS] Re-introduce HLASM comment syntax

- https://reviews.llvm.org/rGb605cfb336989705f391d255b7628062d3dfe9c3 was reverted due to sanitizer bugs in the introduced unit-test (specifically in the Address sanitizer https://lab.llvm.org/buildbot/#/builders/5/builds/5697)
- This patch attempts to rectify that, as well as re-factor parts of the test
- The issue was previously, within the `setupCallToAsmParser` function in the unit-test, `SrcMgr` was declared as a local variable. `SrcMgr` owns a unique pointer. Since the variable goes out of scope at the end of the function, the unique pointer is released.
- This patch, moves the declaration of the `SrcMgr` variable to a class field, since the scope will remain until the class's destructor is invoked (which in this case is at the end of the unit test)
- Furthermore, this patch also moves the `MCContext Ctx` declaration from a local variable instance inside a function, to a unique pointer class field. This ensures the instantiation of the MCContext remains until the tear down of the test.

Reviewed By: abhina.sreeskantharajan

Differential Revision: https://reviews.llvm.org/D99004

Revert "[AsmParser][SystemZ][z/OS] Reland "Introduce HLASM Comment Syntax""

This reverts commit b605cfb336989705f391d255b7628062d3dfe9c3.

Differential Revision: https://reviews.llvm.org/D98744

[AsmParser][SystemZ][z/OS] Reland "Introduce HLASM Comment Syntax"

- Previously, https://reviews.llvm.org/D97703 was [[ https://reviews.llvm.org/D98543 | reverted ]] as it broke when building the unit tests when shared libs on.
- This patch reverts the "revert" and makes two minor changes
- The first is it also links in the MCParser lib when building the unittest. This should resolve the issue when building with with shared libs on and off
- The second renames the name of the unit test from `SystemZAsmLexer` to `SystemZAsmLexerTests` since the convention for unittest binaries is to suffix the name of the unit test with "Tests"

Reviewed By: Kai

Differential Revision: https://reviews.llvm.org/D98666

Revert "[AsmParser][SystemZ][z/OS] Introducing HLASM Comment Syntax"

This reverts commit bcdd40f802a5dfd7b3ac11304e6099bfcdd25b1e.
See https://reviews.llvm.org/D98543.

[AsmParser][SystemZ][z/OS] Introducing HLASM Comment Syntax

- This patch adds in support for the ordinary HLASM comment syntax asm
  statements (Reference - Chapter 7, Comment Statements, Ordinary Comment
  Statements)
- In brief, the ordinary comment syntax if used, must begin with the "*"
  character
- To achieve this, this patch makes use of the CommentString attribute
  provided in the base MCAsmInfo class
- In the SystemZMCAsmInfo class, the CommentString attribute was set to
  "*" based on the assembler dialect
- Furthermore, a new attribute RestrictCommentString, is provided to only
  treat a string as a comment if it appears at the start of the asm
  statement. Example: "jo *-4" is valid in HLASM (jump back 4 bytes from
  current point - similar to jo -4 in gnu asm) and we don't want "*-4" to
  be treated as a comment.
- RFC for HLASM Parser support implementation: https://lists.llvm.org/pipermail/llvm-dev/2021-January/147686.html

Reviewed By: scott.linder, Kai

    [2 lines not shown]

[MC] Fix ICE with non-newline terminated input

There is an explicit option for the lexer to support this, but we crash
when `-preserve-comments` is enabled because it checks for
`getTok().getString().empty()` to detect the case. This doesn't
work currently because the lexer reports this case as a string of length
1, containing a null byte.

Change the lexer to instead report this case via an empty string, as the
null terminator isn't logically a part of the textual input, and the
check for `.empty()` seems natural and obvious in the calling code.

Reviewed By: niravd

Differential Revision: https://reviews.llvm.org/D92681

[ms] [llvm-ml] Lex MASM strings, including escaping

Allow single-quoted strings and double-quoted character values, as well as doubled-quote escaping.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D89731

[ms] [llvm-ml] Add MASM hex float support

Implement MASM's syntax for specifying floats in raw hexadecimal bytes.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D87401

[ms] [llvm-ml] Add support for .radix directive, and accept all radix specifiers

Add support for .radix directive, and radix specifiers [yY] (binary), [oOqQ] (octal), and [tT] (decimal).

Also, when lexing MASM integers, require radix specifier; MASM requires that all literals without a radix specifier be treated as in the default radix. (e.g., 0100 = 100)

Relanding D87400, now with fewer ms-inline-asm tests broken!

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D88337

Revert "[ms] [llvm-ml] Add support for .radix directive, and accept all radix specifiers"

This reverts commit 5dd1b6d612655c9006ba97a8b6487ded80719b48.

Fix include location (accidentally committed a local variation)

[ms] [llvm-ml] Add support for .radix directive, and accept all radix specifiers

Add support for .radix directive, and radix specifiers [yY] (binary), [oOqQ] (octal), and [tT] (decimal).

Also, when lexing MASM integers, require radix specifier; MASM requires that all literals without a radix specifier be treated as in the default radix. (e.g., 0100 = 100)

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D87400

[DPWBS-1377] Merge community master into htc/master

The following TriCore files needed to be touched due to changes in the
upstream community:

TriCoreAsmParser.cpp:   8d5bf0422bc3c8cc7017602375122f7bfea70dab
TriCoreAsmPrinter.cpp:  6d2d589b06fcb31220bdf5aa09b7a6d5b34ef252

The following tests needed to be updated as well:

[Clang]
Preprocessor/init.c:    91cdbd521a38495c66e30636943563ca70d3c022

[LLVM]
copy-pseudo-register-invalid.mir:                   4ad76852584480b646d1ce360202e18591ea8938
legalize-bswap-invalid.mir:                         4ad76852584480b646d1ce360202e18591ea8938
legalize-invalid-inttoptr.mir:                      4ad76852584480b646d1ce360202e18591ea8938
legalize-store-invalid-memsize.mir:                 4ad76852584480b646d1ce360202e18591ea8938
prologue-epilogue-invalid-stack-size.mir:           4ad76852584480b646d1ce360202e18591ea8938

    [9 lines not shown]

[ms] [llvm-ml] Add a draft MASM parser

Summary:
Many directives are unavailable, and support for others may be limited.

This first draft has preliminary support for:
    - conditional directives (including errors),
    - data allocation (unsigned types up to 8 bytes, and ALIGN),
    - equates/variables (numeric and text),
    - and procedure directives (without parameters),
as well as COMMENT, ECHO, INCLUDE, INCLUDELIB, PUBLIC, and EXTERN. Text variables (aka text macros) are expanded in-place wherever the identifier occurs.

We deliberately ignore all ml.exe processor directives.

Prominent features not yet supported:
    - structs
    - macros (both procedures and functions)
    - procedures (with specified parameters)
    - substitution & expansion operators

    [12 lines not shown]

Merge community 'master' into HighTec htc/master

[MC] Fix floating-point literal lexing.

This patch has three related fixes to improve float literal lexing:

1. Make AsmLexer::LexDigit handle floats without a decimal point more
   consistently.
2. Make AsmLexer::LexFloatLiteral print an error for floats which are
   apparently missing an "e".
3. Make APFloat::convertFromString use binutils-compatible exponent
   parsing.

Together, this fixes some cases where a float would be incorrectly
rejected, fixes some cases where the compiler would crash, and improves
diagnostics in some cases.

Patch by Brandon Jones.

Differential Revision: https://reviews.llvm.org/D57321

llvm-svn: 357214

[NFC] Fix typos: preceeding -> preceding

llvm-svn: 354715

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

[WebAssembly] replaced .param/.result by .functype

Summary:
This makes it easier/cleaner to generate a single signature from
this directive. Also:
- Adds the symbol name, such that we don't depend on the location
  of this directive anymore.
- Actually constructs the signature in the assembler, and make the
  assembler own it.
- Refactor the use of MVT vs ValType in the streamer and assembler
  to require less conversions overall.
- Changed 700 or so tests to use it.

Reviewers: sbc100, dschuff

Subscribers: jgravelle-google, eraman, aheejin, sunfish, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D54652

llvm-svn: 347228

[MC] Separate masm integer literal lexer support from inline asm

Summary:
This renames the IsParsingMSInlineAsm member variable of AsmLexer to
LexMasmIntegers and moves it up to MCAsmLexer. This is the only behavior
controlled by that variable. I added a public setter, so that it can be
set from outside or from the llvm-mc command line. We may need to
arrange things so that users can get this behavior from clang, but
that's future work.

I also put additional hex literal lexing functionality under this flag
to fix PR32973. It appears that this hex literal parsing wasn't intended
to be enabled in non-masm-style blocks.

Now, masm integers (0b1101 and 0ABCh) work in __asm blocks from clang,
but 0b label references work when using .intel_syntax in standalone .s
files.

However, 0b label references will *not* work from __asm blocks in clang.

    [17 lines not shown]

[MC] Fix regression tests on Windows when git “core.autocrlf” is set to true.

Differential Revision: https://reviews.llvm.org/D39737

This is the second attempt to commit this. The test was broken on Linux in the first attempt.

llvm-svn: 318560

Revert "[MC] Fix regression tests on Windows when git “core.autocrlf” is set to true."

This reverts commit r318528.

MC/AsmParser/preserve-comments-crlf.s fails on linux.

llvm-svn: 318533

[MC] Fix regression tests on Windows when git “core.autocrlf” is set to true.

Differential Revision: https://reviews.llvm.org/D39737

llvm-svn: 318528

[MC] Lex CRLF as one token

This will prevent doubling of line endings when parsing assembly and
emitting assembly.

Otherwise we'd parse the directive, consume the end of statement, hit
the next end of statement, and emit a fresh newline.

llvm-svn: 315943

[MC] - Don't assert when non-english characters are used.

I found that llvm-mc does not like non-english characters even in comments,
which it tries to tokenize.

Problem happens because of functions like isdigit(), isalnum() which takes
int argument and expects it is not negative.
But at the same time MCParser uses char* to store input buffer poiner, char has signed value,
so it is possible to pass negative value to one of functions from above and
that triggers an assert. 
Testcase for demonstration is provided.

To fix the issue helper functions were introduced in StringExtras.h

Differential revision: https://reviews.llvm.org/D38461

llvm-svn: 314883

Sort the remaining #include lines in include/... and lib/....

I did this a long time ago with a janky python script, but now
clang-format has built-in support for this. I fed clang-format every
line with a #include and let it re-sort things according to the precise
LLVM rules for include ordering baked into clang-format these days.

I've reverted a number of files where the results of sorting includes
isn't healthy. Either places where we have legacy code relying on
particular include ordering (where possible, I'll fix these separately)
or where we have particular formatting around #include lines that
I didn't want to disturb in this patch.

This patch is *entirely* mechanical. If you get merge conflicts or
anything, just ignore the changes in this patch and run clang-format
over your #include lines in the files.

Sorry for any noise here, but it is important to keep these things
stable. I was seeing an increasing number of patches with irrelevant

    [5 lines not shown]