-
Notifications
You must be signed in to change notification settings - Fork 508
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MOD-6933 MOD-6750 MOD-6749: Lexer consumes some punctuation characters #4604
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #4604 +/- ##
==========================================
+ Coverage 85.80% 85.96% +0.16%
==========================================
Files 188 190 +2
Lines 33107 34517 +1410
==========================================
+ Hits 28407 29674 +1267
- Misses 4700 4843 +143 ☔ View full report in Codecov by Sentry. |
…arch into nafraf_parser-v3-punct
This reverts commit 0dae78b.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍🏼 👏🏼 Nice! Few comments
Successfully created backport PR for |
Description
Lexer consumes some punctuation characters. This generates wrong queries, and causes errors parsing numeric operators and fixing numeric range syntax:
#4516
#4505
A new parser (v3) was created to support
DIALECT 5
with the new behaviorWhich issues this PR fixes
Main objects this PR modified
To create
src/query_parser/v3/
we usedsrc/query_parser/v2/
as base, adding the following changes:1.1 In
lexer.rl
punctuation characters are handle asPUNCTUATION
token to be processed by the parser.From the ragel-guide, the punct characters are:
But from the previous list, there are some characters which have an special meaning in the parser and are handled as an specific token.
!
,#
,$
,&
,'
,,
,.
,/
,<
,=
,>
,?
,@
,\
,^
,_
,`"
%
(
)
*
-
+
:
;
[
]
{
|
}
~
1.2 In
lexer.rl
the format ofNUMBER
was modified to support float numbers without leading zero.1.3. In
parser.y
we created a new rule for unary operators:~
,-
, to prevent the use of an operator between the colon after the field name and the parenthesis which specify the query pattern.i.e. The following queries are invalid:
1.4
parser.y
: To support numbers with multiple signs+
, a new rule was added:num(A) ::= PLUS num(B)
Mark if applicable
In addition
This PR also merges to the master branch the following PRs:
#4505 (MOD-6750)
#4516 (MOD-6749)