Lexer/parser: Pine line-wrapping rule + dangling Dedents between call arguments#50
Open
FellowTraveler wants to merge 2 commits into
Open
Lexer/parser: Pine line-wrapping rule + dangling Dedents between call arguments#50FellowTraveler wants to merge 2 commits into
FellowTraveler wants to merge 2 commits into
Conversation
A named argument whose value wraps onto an indented continuation line (e.g. a multiline ternary) leaves a closing Dedent in the token stream after the comma. arguments() only skipped Newline+Indent there, so the next argument failed to parse. Consume such Dedents after a comma, guarded so the argument list's own closing Dedent (immediately followed by RParen) is preserved. Test: multiline-ternary named argument followed by another argument (fails with IndentationError-free parse error before this fix).
… of 4)
Pine's line-wrapping spec: a statement may wrap onto following lines as
long as the continuation's indentation is NOT a multiple of 4 spaces —
4-space multiples are reserved for delimiting local blocks. The lexer
treated every deeper line start as Indent, so the canonical wrapped
style failed to parse:
can_trade = a < 2
and b
and c
-> 'Unexpected token: And'
Fix: at line start, when the indentation is not a multiple of 4, join
the line to the previous logical line — drop the Newline that ended it
and leave the indent stack untouched. Existing testdata is unaffected
(it contains no non-multiple-of-4 indentation).
Tests: lexer token-stream tests (a wrapped continuation emits no
Indent/Dedent and no Newline before the continuation's operator; a
4-space block still emits Indent/Dedent) and parser tests for a wrapped
and-chain and a wrapped ternary inside an if-block.
ferranbt
requested changes
Jul 9, 2026
| } | ||
|
|
||
| #[test] | ||
| fn test_line_wrapped_bool_expression() { |
Owner
There was a problem hiding this comment.
Could you move these two tests to the testdata folder?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is the lexer/parser slice of #49, split out (as offered there) so each piece can be reviewed on its own. The code is unchanged from #49 — the commits are cherry-picked as-is. The three split PRs are independent: each compiles and passes the full workspace suite standalone (
cargo test,just fmt-check,just clippyall clean on this branch), and any merge order is conflict-free.Both fixes were found while running real published Pine strategies through pinecone bar-by-bar.
1.
fix(parser): consume dangling Dedents between call argumentsA named argument whose value wraps onto an indented continuation line (the common
table.cell(..., bgcolor = cond ? a :\n<indent>b, ...)idiom) leaves a closingDedentin the token stream after the comma.arguments()only skippedNewline+Indentthere, so the next argument failed to parse. The fix consumes such Dedents after a comma, guarded so the argument list's own closing Dedent (immediately followed byRParen) is preserved.Test: multiline-ternary named argument followed by another argument — fails to parse before the fix.
2.
feat(lexer): Pine's line-wrapping rulePer the Pine Script reference (Line wrapping), a statement may wrap onto following lines as long as the continuation's indentation is not a multiple of 4 spaces — multiples of 4 are reserved for delimiting local blocks. The lexer treated every deeper line start as
Indent, so this canonical published-script style failed withUnexpected token: And:The fix: at line start, when the indentation is not a multiple of 4, join the line to the previous logical line (drop the
Newlinethat ended it; leave the indent stack untouched). The existing testdata corpus contains no non-multiple-of-4 indentation, so its behavior is unchanged — verified by the full suite.Tests: lexer token-stream tests (a wrapped continuation emits no
Indent/Dedentand noNewlinebefore the continuation's operator; a 4-space block still emitsIndent/Dedent), plus parser tests for a wrappedand-chain and a wrapped ternary inside anifblock.Note: the two commits are ordered — the lexer commit's parser tests build on the Dedent fix — so they travel together in this PR.
Validation
Beyond the tests in this PR, the combined change set (this PR plus its two siblings from #49) was validated externally: we run full Pine strategies through pinecone one bar at a time and diff the emitted entries/exits against TradingView's own strategy-tester CSV export for the same symbol/timeframe. With all of them in place, two real strategies match TV 100% — 56/56 signals on a 4H strategy and 21/21 on a daily strategy, including exit-reason order comments — over their full comparison windows.