Compare commits

..
Author SHA1 Message Date
Charles-P. Clermont c5bb4c8eff Add strict2 let closures for Liquid composition 2026-09-24 10:10:28 -04:00
Tobi Lütke 807d45a6b3 Surface blank-body inline errors in strict2
Keep the historical blank-body inline-error suppression for lax and strict parse modes, but stop suppressing the rendered error text when the template was parsed in strict2. Raised-error rendering continues to raise in all modes.

Store the resolved template error mode on the render context while a template renders so BlockBody can distinguish strict2 from compatibility modes when deciding whether a blank tag should hide inline error text.

Add integration coverage for lax/strict suppression, strict2 non-suppression across blank body forms, nonblank bodies, and raised-error behavior.
2026-07-05 18:56:24 +00:00
Guilherme CarreiroandGitHub 7a5e45fc47 Bump to 5.13.0 (#2103) 2026-06-29 17:41:55 +02:00
Ian Ker-SeymerandIan Ker-Seymer 9efca9f718 Use liquid-spec branch for self specs 2026-06-11 23:20:43 -04:00
Ian Ker-SeymerandIan Ker-Seymer 529800f46b Let environment self shadow SelfDrop 2026-06-11 23:20:43 -04:00
Guilherme CarreiroandGitHub 7b368dffb8 Fix SelfDrop equality (#2091) 2026-06-05 10:47:37 +02:00
Guilherme CarreiroandGitHub 742ac3dbf5 Prevent SelfDrop context mutation across render boundaries (#2082) 2026-05-20 09:34:37 +02:00
Ian Ker-SeymerandIan Ker-Seymer 1954a2655c Update liquid-spec adapters 2026-04-28 10:11:38 -04:00
Alok SwamyandGitHub 6d81b1b68c Merge pull request #2077 from Shopify/remove-strict2-from-error-message
Remove "strict2" from bare bracket error message
2026-04-24 16:21:51 -04:00
Alok SwamyandClaude Opus 4.7 dfddd8f390 Remove "strict2" from bare bracket error message
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
2026-04-24 16:18:05 -04:00
Alok SwamyandGitHub 95ce7e7fa1 Merge pull request #2067 from Shopify/strict2-increment-decrement
Add strict2_parse to increment and decrement tags
2026-04-24 16:11:49 -04:00
Alok SwamyandGitHub 197d755e0c Merge pull request #2065 from Shopify/strict2-assign-capture
Add strict2_parse to assign and capture tags
2026-04-24 16:10:59 -04:00
Alok SwamyandGitHub d0c5444db1 Merge pull request #2060 from Shopify/bare-bracket-self-keyword
Reject bare-bracket syntax in strict2 and introduce `self` keyword
2026-04-24 16:06:02 -04:00
Alok SwamyandClaude Opus 4.6 c99036046e Add strict2_parse to increment and decrement tags
Both tags previously accepted any string as a variable name via
`markup.strip`. Now they use `parse_with_selected_parser` and validate
the variable name with `p.consume(:id)` in strict2 mode.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 15:32:53 -04:00
Alok SwamyandClaude Opus 4.6 0d5c15a03e Add strict2_parse to assign and capture tags
Both tags previously used only regex (VariableSignature) to validate
variable names, which allowed invalid identifiers like (a(b(c) and
[x.y] in all parse modes.

- assign: strict2_parse uses Parser to validate the LHS as a valid
  identifier before delegating RHS to Variable
- capture: strict2_parse uses Parser to validate the variable name
  as a valid identifier
- Both tags now include ParserSwitching and dispatch through
  strict_parse_with_error_mode_fallback
- Lax mode is unchanged — invalid names are still accepted

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-25 12:07:23 -04:00
Alok SwamyandClaude Opus 4.6 346166b600 Add for_loop? to Include tag for AST-based keyword detection
Store @is_for_loop during parsing so consumers can determine the
with/for keyword from the AST instead of re-parsing raw markup.
Matches the existing pattern in the Render tag.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-24 14:52:55 -04:00
Alok SwamyandClaude Opus 4.6 532b439063 Reject bare-bracket syntax in strict2 and introduce self keyword
Add bare-bracket rejection to Parser#expression in strict2 mode, so that
`['var']` is disallowed and `self['var']` is the required syntax.

- Add `Expression::SELF` constant ('self')
- Add `Parser#reject_bare_brackets` option, checked in `expression`
- Add `ParseContext#reject_bare_brackets?` and `force_reject_bare_brackets`
- Add `VariableLookupDrop` for `self['var']` scope-chain lookups
- Add `Variable#==` for rewriter state comparison
- Update `Context#find_variable` to return `VariableLookupDrop` for `self`

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 12:13:59 -04:00
69 changed files with 2216 additions and 1925 deletions
+1 -1
View File
@@ -1 +1 @@
3.4.1
4.0.2
+2 -1
View File
@@ -32,6 +32,7 @@ group :test do
end
group :spec do
gem 'liquid-spec', github: 'Shopify/liquid-spec', branch: 'main'
# Using feature branch until https://github.com/Shopify/liquid-spec/pull/144 is merged
gem 'liquid-spec', github: 'Shopify/liquid-spec', branch: 'self-drop-env-lookup-specs'
gem 'activesupport', require: false
end
+12
View File
@@ -1,5 +1,17 @@
# Liquid Change Log
## 5.13.0
* Add TruffleRuby in CI [Benoit Daloze]
* Skip slow test raising many exceptions on non-CRuby [Benoit Daloze]
* Reject bare-bracket syntax in strict2 and introduce `self` keyword by [Alok Swamy]
* Add strict2_parse to assign and capture tags by [Alok Swamy]
* Add strict2_parse to increment and decrement tags by [Alok Swamy]
* Update liquid-spec adapters for `missing_features` [Ian Ker-Seymer]
* Prevent `SelfDrop` context mutation across render boundaries [Guilherme Carreiro]
* Fix `SelfDrop` equality [Guilherme Carreiro]
* Let environment `self` shadow `SelfDrop` [Ian Ker-Seymer]
## 5.11.0
* Revert the Inline Snippets tag (#2001), treat its inclusion in the latest Liquid release as a bug, and allow for feedback on RFC#1916 to better support Liquid developers [Guilherme Carreiro]
* Rename the `:rigid` error mode to `:strict2` and display a warning when users attempt to use the `:rigid` mode [Guilherme Carreiro]
+44
View File
@@ -118,6 +118,50 @@ This is useful for doing things like enabling strict mode only in the theme edit
It is recommended that you enable `:strict` or `:warn` mode on new apps to stop invalid templates from being created.
It is also recommended that you use it in the template editors of existing apps to give editors better error messages.
### Strict2 bindings and closures
In `error_mode: :strict2`, `let` binds a value or defines a callable Liquid body:
```liquid
{% let count = 1 %}
{% let title(step) %}
{% let count = count | plus: step %}
{{ count }}
{% endlet %}
{{ title(2) }}
```
Each call has fresh locals and writable parameters. Captures read the latest parent
bindings. `set name = expression` updates the nearest local binding, or creates a
shadow at the invocation root when the name is captured or absent. Initializers run
before installing the new binding. Loops have fresh `let` bindings each iteration
and can shadow outer names; conditional branches share their surrounding scope.
Duplicate `let` declarations in one scope
and `assign` inside closure bodies are parse errors. Calls accept positional
arguments and invoke only Liquid closures.
Custom `Liquid::Block` subclasses can use the same body rules:
```ruby
def parse(tokens)
@closure_body = parse_closure_body(tokens)
end
def render_to_output_buffer(context, output)
body = @closure_body.bind(context)
exports = body.invoke(context, output: output)
exports['title']&.render_to_output_buffer(context, output, ['Hello'])
output
end
```
`invoke` renders into the supplied buffer and returns direct-child body-form `let`
bindings by name. Value bindings and declarations nested in control flow are not
exported. Each invocation has fresh state; bind the body at render time rather than
caching a closure on the parsed tag. For parameterized bodies, pass
`parameters: ['name']` to `parse_closure_body`, then pass an argument array as the
second argument to `invoke`. The syntax remains visible to `ParseTreeVisitor`.
### Undefined variables and filters
By default, the renderer doesn't raise or in any other way notify you if some variables or filters are missing, i.e. not passed to the `render` method.
+4 -2
View File
@@ -151,6 +151,8 @@ end
desc('run liquid-spec suite across all adapters')
task :spec do
adapters = Dir['./spec/*.rb'].join(',')
sh "bundle exec liquid-spec matrix --adapters=#{adapters} --reference=ruby_liquid"
Dir['./spec/*.rb'].sort.each do |adapter|
puts "=== Running #{adapter} ==="
sh 'bundle', 'exec', 'liquid-spec', 'run', adapter, '--no-max-failures'
end
end
-30
View File
@@ -1,30 +0,0 @@
# Autoresearch Ideas
## Dead Ends (tried and failed)
- **Tag name interning** (skip+byte dispatch): saves 878 allocs but verification loop overhead kills speed
- **String dedup (-@)** for filter names: no alloc savings, creates temp strings anyway
- **Split-based tokenizer**: 2.5x faster C-level split but can't handle {{ followed by %} nesting
- **Streaming tokenizer**: needs own StringScanner (+alloc), per-shift overhead worse than eager array
- **Merge simple_lookup? into initialize**: logic overhead offsets saved index call
- **Cursor for filter scanning**: cursor.reset overhead worse than inline byte loops
- **Direct strainer call**: YJIT already inlines context.invoke_single well
- **TruthyCondition subclass**: YJIT polymorphism at evaluate call site hurts more than 115 saved allocs
- **Index loop for filters**: YJIT optimizes each+destructure MUCH better than manual filter[0]/filter[1]
## Key Insights
- YJIT monomorphism > allocation reduction at this scale
- C-level StringScanner.scan/skip > Ruby-level byte loops (already applied)
- String#split is 2.5x faster than manual tokenization, but Liquid's grammar is too complex for regex
- 74% of total CPU time is GC — alloc reduction is the highest-leverage optimization
- But YJIT-deoptimization from polymorphism costs more than the GC savings
## Remaining Ideas
- **Tokenizer: use String#index + byteslice instead of StringScanner**: avoid the StringScanner overhead entirely for the simple case of finding {%/{{ delimiters
- **Pre-freeze all Condition operator lambdas**: reduce alloc in Condition initialization
- **Avoid `@blocks = []` in If with single-element optimization**: use `@block` ivar for single condition, only create array for elsif
- **Reduce ForloopDrop allocation**: reuse ForloopDrop objects across iterations or use a lighter-weight object
- **VariableLookup: single-segment optimization**: for "product.title" (1 lookup), use an ivar instead of 1-element Array
-109
View File
@@ -1,109 +0,0 @@
# Autoresearch: Liquid Parse+Render Performance
## Objective
Optimize the Shopify Liquid template engine's parse and render performance.
The workload is the ThemeRunner benchmark which parses and renders real Shopify
theme templates (dropify, ripen, tribble, vogue) with realistic data from
`performance/shopify/database.rb`. We measure parse time, render time, and
object allocations. The optimization target is combined parse+render time (µs).
## How to Run
Run `./auto/autoresearch.sh` — it runs unit tests, liquid-spec conformance,
then the performance benchmark, outputting metrics in parseable format.
## Metrics
- **Primary (optimization target)**: `combined_µs` (µs, lower is better) — sum of parse + render time
- **Secondary (tradeoff monitoring)**:
- `parse_µs` — time to parse all theme templates (Liquid::Template#parse)
- `render_µs` — time to render all pre-compiled templates
- `allocations` — total object allocations for one parse+render cycle
Parse dominates (~70-75% of combined). Allocations correlate with GC pressure.
## Files in Scope
- `lib/liquid/*.rb` — core Liquid library (parser, lexer, context, expression, etc.)
- `lib/liquid/tags/*.rb` — tag implementations (for, if, assign, etc.)
- `performance/bench_quick.rb` — benchmark script
## Off Limits
- `test/` — tests must continue to pass unchanged
- `performance/tests/` — benchmark templates, do not modify
- `performance/shopify/` — benchmark data/filters, do not modify
## Constraints
- All unit tests must pass (`bundle exec rake base_test`)
- liquid-spec failures must not increase beyond 2 (pre-existing UTF-8 edge cases)
- No new gem dependencies
- Semantic correctness must be preserved — templates must render identical output
- **Security**: Liquid runs untrusted user code. See Strategic Direction for details.
## Strategic Direction
The long-term goal is to converge toward a **single-pass, forward-only parsing
architecture** using one shared StringScanner instance. The current system has
multiple redundant passes: Tokenizer → BlockBody → Lexer → Parser → Expression
→ VariableLookup, each re-scanning portions of the source. A unified scanner
approach would:
1. **One StringScanner** flows through the entire parse — no intermediate token
arrays, no re-lexing filter chains, no string reconstruction in Parser#expression.
2. **Emit a lightweight IL or normalized AST** during the single forward pass,
decoupling strictness checking from the hot parse path. The LiquidIL project
(`~/src/tries/2026-01-05-liquid-il`) demonstrated this: a recursive-descent
parser emitting IL directly achieved significant speedups.
3. **Minimal backtracking** — the scanner advances forward, byte-checking as it
goes. liquid-c (`~/src/tries/2026-01-16-Shopify-liquid-c`) showed that a
C-level cursor-based tokenizer eliminates most allocation overhead.
Current fast-path optimizations (byte-level tag/variable/for/if parsing) are
steps toward this goal. Each one replaces a regex+MatchData pattern with
forward-only byte scanning. The remaining Lexer→Parser path for filter args
is the next target for elimination.
**Security note**: Liquid executes untrusted user templates. All parsing must
use explicit byte-range checks. Never use eval, send on user input, dynamic
method dispatch, const_get, or any pattern that lets template authors escape
the sandbox.
## Baseline
- **Commit**: 4ea835a (original, before any optimizations)
- **combined_µs**: 7,374
- **parse_µs**: 5,928
- **render_µs**: 1,446
- **allocations**: 62,620
## Progress Log
- 3329b09: Replace FullToken regex with manual byte parsing → combined 7,262 (-1.5%)
- 97e6893: Replace VariableParser regex with manual byte scanner → combined 6,945 (-5.8%), allocs 58,009
- 2b78e4b: getbyte instead of string indexing in whitespace_handler/create_variable → allocs 51,477
- d291e63: Lexer equal? for frozen arrays, \s+ whitespace skip → combined ~6,331
- d79b9fa: Avoid strip alloc in Expression.parse, byteslice for strings → allocs 49,151
- fa41224: Short-circuit parse_number with first-byte check → allocs 48,240
- c1113ad: Fast-path String in render_obj_to_output → combined ~6,071
- 25f9224: Fast-path simple variable parsing (skip Lexer/Parser) → combined ~5,860, allocs 45,202
- 3939d74: Replace SIMPLE_VARIABLE regex with byte scanner → combined ~5,717, allocs 42,763
- fe7a2f5: Fast-path simple if conditions → combined ~5,444, allocs 41,490
- cfa0dfe: Replace For tag Syntax regex with manual byte parser → combined ~4,974, allocs 39,847
- 8a92a4e: Unified fast-path Variable: parse name directly, only lex filter chain → combined ~5,060, allocs 40,520
- 58d2514: parse_tag_token returns [tag_name, markup, newlines] → combined ~4,815, allocs 37,355
- db43492: Hoist write score check out of render loop → render ~1,345
- 17daac9: Extend fast-path to quoted string literal variables → all 1,197 variables fast-pathed
- 9fd7cec: Split filter parsing: no-arg filters scanned directly, Lexer only for args → combined ~4,595, allocs 35,159
- e5933fc: Avoid array alloc in parse_tag_token via class ivars → allocs 34,281
- 2e207e6: Replace WhitespaceOrNothing regex with byte-level blank_string? → combined ~4,800
- 526af22: invoke_single fast path for no-arg filter invocation → allocs 32,621
- 76ae8f1: find_variable top-scope fast path → combined ~4,740
- 4cda1a5: slice_collection: skip copy for full Array → allocs 32,004
- 79840b1: Replace SIMPLE_CONDITION regex with manual byte parser → combined ~4,663, allocs 31,465
- 69430e9: Replace INTEGER_REGEX/FLOAT_REGEX with byte-level parse_number → allocs 31,129
- 405e3dc: Frozen EMPTY_ARRAY/EMPTY_HASH for Context @filters/@disabled_tags → allocs 31,009
- b90d7f0: Avoid unnecessary array wrapping for Context environments → allocs 30,709
- 3799d4c: Lazy seen={} hash in Utils.to_s/inspect → allocs 30,169
- 0b07487: Fast-path VariableLookup: skip scan_variable for simple identifiers → allocs 29,711
- 9de1527: Introduce Cursor class for centralized byte-level scanning
- dd4a100: Remove dead parse_tag_token/SIMPLE_CONDITION (now in Cursor)
- cdc3438: For tag: migrate lax_parse to Cursor with zero-alloc scanning → allocs 29,620
## Current Best
- **combined_µs**: ~3,400 (-54% from original 7,374 baseline)
- **parse_µs**: ~2,300
- **render_µs**: ~1,100
- **allocations**: 24,882 (-60% from original 62,620 baseline)
-48
View File
@@ -1,48 +0,0 @@
#!/usr/bin/env bash
# Autoresearch benchmark runner for Liquid performance optimization
# Runs: unit tests → performance benchmark (3 runs, takes best)
# Outputs METRIC lines for the agent to parse
# Exit code 0 = all good, non-zero = broken
set -euo pipefail
cd "$(dirname "$0")/.."
# ── Step 1: Unit tests (fast gate) ──────────────────────────────────
echo "=== Unit Tests ==="
TEST_OUT=$(bundle exec rake base_test 2>&1)
TEST_RESULT=$(echo "$TEST_OUT" | tail -1)
if echo "$TEST_OUT" | grep -q 'failures\|errors' && ! echo "$TEST_RESULT" | grep -q '0 failures, 0 errors'; then
echo "$TEST_OUT" | grep -E 'Failure|Error|failures|errors' | head -20
echo "FATAL: unit tests failed"
exit 1
fi
echo "$TEST_RESULT"
# ── Step 2: Performance benchmark (3 runs, take best) ──────────────
echo ""
echo "=== Performance Benchmark (3 runs) ==="
BEST_COMBINED=999999
BEST_PARSE=0
BEST_RENDER=0
BEST_ALLOC=0
for i in 1 2 3; do
OUT=$(bundle exec ruby performance/bench_quick.rb 2>&1)
P=$(echo "$OUT" | grep '^parse_us=' | cut -d= -f2)
R=$(echo "$OUT" | grep '^render_us=' | cut -d= -f2)
C=$(echo "$OUT" | grep '^combined_us=' | cut -d= -f2)
A=$(echo "$OUT" | grep '^allocations=' | cut -d= -f2)
echo " run $i: combined=${C}µs (parse=${P} render=${R}) allocs=${A}"
if [ "$C" -lt "$BEST_COMBINED" ]; then
BEST_COMBINED=$C
BEST_PARSE=$P
BEST_RENDER=$R
BEST_ALLOC=$A
fi
done
echo ""
echo "METRIC combined_us=$BEST_COMBINED"
echo "METRIC parse_us=$BEST_PARSE"
echo "METRIC render_us=$BEST_RENDER"
echo "METRIC allocations=$BEST_ALLOC"
-40
View File
@@ -1,40 +0,0 @@
#!/usr/bin/env bash
# Auto-research benchmark script for Liquid
# Runs: unit tests → liquid-spec → performance benchmark
# Outputs machine-readable metrics on success
# Exit code 0 = all good, non-zero = broken
set -euo pipefail
cd "$(dirname "$0")/.."
# ── Step 1: Unit tests (fast gate) ──────────────────────────────────
echo "=== Unit Tests ==="
if ! bundle exec rake base_test 2>&1; then
echo "FATAL: unit tests failed"
exit 1
fi
# ── Step 2: liquid-spec (correctness gate) ──────────────────────────
echo ""
echo "=== Liquid Spec ==="
SPEC_OUTPUT=$(bundle exec liquid-spec run spec/ruby_liquid.rb 2>&1 || true)
echo "$SPEC_OUTPUT" | tail -3
# Extract failure count from "Total: N passed, N failed, N errors" line
# Allow known pre-existing failures (≤2)
TOTAL_LINE=$(echo "$SPEC_OUTPUT" | grep "^Total:" || echo "Total: 0 passed, 0 failed, 0 errors")
FAILURES=$(echo "$TOTAL_LINE" | sed -n 's/.*\([0-9][0-9]*\) failed.*/\1/p')
ERRORS=$(echo "$TOTAL_LINE" | sed -n 's/.*\([0-9][0-9]*\) error.*/\1/p')
FAILURES=${FAILURES:-0}
ERRORS=${ERRORS:-0}
TOTAL_BAD=$((FAILURES + ERRORS))
if [ "$TOTAL_BAD" -gt 2 ]; then
echo "FATAL: liquid-spec has $FAILURES failures and $ERRORS errors (threshold: 2)"
exit 1
fi
# ── Step 3: Performance benchmark ──────────────────────────────────
echo ""
echo "=== Performance Benchmark ==="
bundle exec ruby performance/bench_quick.rb 2>&1
-30
View File
@@ -1,30 +0,0 @@
{"type":"config","name":"Liquid parse+render performance (tenderlove-inspired)","metricName":"combined_µs","metricUnit":"µs","bestDirection":"lower"}
{"run":1,"commit":"c09e722","metric":3818,"metrics":{"parse_µs":2722,"render_µs":1096,"allocations":24881},"status":"keep","description":"Baseline: 3,818µs combined, 24,881 allocs","timestamp":1773348490227}
{"run":2,"commit":"c09e722","metric":4063,"metrics":{"parse_µs":2901,"render_µs":1162,"allocations":24003},"status":"discard","description":"Tag name interning via skip+byte dispatch: saves 878 allocs but verification loop slower than scan","timestamp":1773348738557,"segment":0}
{"run":3,"commit":"c09e722","metric":3881,"metrics":{"parse_µs":2720,"render_µs":1161,"allocations":24881},"status":"discard","description":"String dedup (-@) for filter names: no alloc savings, no speed benefit","timestamp":1773348781481,"segment":0}
{"run":4,"commit":"c09e722","metric":3970,"metrics":{"parse_µs":2829,"render_µs":1141,"allocations":24881},"status":"discard","description":"Streaming tokenizer: needs own StringScanner (+1 alloc), per-shift overhead worse than saved array","timestamp":1773348883093,"segment":0}
{"run":5,"commit":"c09e722","metric":0,"metrics":{"parse_µs":0,"render_µs":0,"allocations":0},"status":"crash","description":"REVERTED: split-based tokenizer — regex can't handle unclosed tags inside raw blocks","timestamp":1773349089230,"segment":0}
{"run":6,"commit":"c09e722","metric":0,"metrics":{"parse_µs":0,"render_µs":0,"allocations":0},"status":"crash","description":"REVERTED: split regex tokenizer v2 — can't handle {{ followed by %} (variable-becomes-tag nesting)","timestamp":1773349248313,"segment":0}
{"run":7,"commit":"c09e722","metric":3861,"metrics":{"parse_µs":2744,"render_µs":1117,"allocations":24881},"status":"discard","description":"Merge simple_lookup? dot position into initialize — logic overhead offsets saved index call","timestamp":1773349376707,"segment":0}
{"run":8,"commit":"c09e722","metric":4048,"metrics":{"parse_µs":2929,"render_µs":1119,"allocations":24881},"status":"discard","description":"Use Cursor regex for filter name scanning — cursor.reset + method dispatch overhead worse than inline bytes","timestamp":1773349447172,"segment":0}
{"run":9,"commit":"c09e722","metric":3872,"metrics":{"parse_µs":2744,"render_µs":1128,"allocations":24881},"status":"discard","description":"Direct strainer call in Variable#render — YJIT already inlines context.invoke_single well","timestamp":1773349497593,"segment":0}
{"run":10,"commit":"c09e722","metric":3839,"metrics":{"parse_µs":2732,"render_µs":1107,"allocations":24879},"status":"discard","description":"Array#[] fast path for slice_collection with limit/offset — only 2 alloc savings, not meaningful","timestamp":1773349555348,"segment":0}
{"run":11,"commit":"c09e722","metric":3889,"metrics":{"parse_µs":2770,"render_µs":1119,"allocations":24766},"status":"discard","description":"TruthyCondition for simple if checks: -115 allocs but YJIT polymorphism at evaluate call site hurts speed","timestamp":1773349649377,"segment":0}
{"run":12,"commit":"c09e722","metric":4150,"metrics":{"parse_µs":2769,"render_µs":1381,"allocations":24881},"status":"discard","description":"Index loop for filters: YJIT optimizes each+destructure better than manual indexing","timestamp":1773349699285,"segment":0}
{"run":13,"commit":"b7ae55f","metric":3556,"metrics":{"parse_µs":2388,"render_µs":1168,"allocations":24882},"status":"keep","description":"Replace StringScanner tokenizer with String#byteindex — 12% faster parse, no regex overhead for delimiter finding","timestamp":1773349875890,"segment":0}
{"run":14,"commit":"e25f2f1","metric":3464,"metrics":{"parse_µs":2335,"render_µs":1129,"allocations":24882},"status":"keep","description":"Confirmation run: byteindex tokenizer consistently 3,400-3,600µs","timestamp":1773349889465,"segment":0}
{"run":15,"commit":"b37fa98","metric":3490,"metrics":{"parse_µs":2331,"render_µs":1159,"allocations":24882},"status":"keep","description":"Clean up tokenizer: remove unused StringScanner setup and regex constants","timestamp":1773349928672,"segment":0}
{"run":16,"commit":"b37fa98","metric":3638,"metrics":{"parse_µs":2460,"render_µs":1178,"allocations":24882},"status":"discard","description":"Single-char byteindex for %} search: Ruby loop overhead worse for nearby targets","timestamp":1773349985509,"segment":0}
{"run":17,"commit":"b37fa98","metric":3553,"metrics":{"parse_µs":2431,"render_µs":1122,"allocations":25256},"status":"discard","description":"Regex simple_variable_markup: MatchData creates 374 extra allocs, offsetting speed gain","timestamp":1773350066627,"segment":0}
{"run":18,"commit":"b37fa98","metric":3629,"metrics":{"parse_µs":2455,"render_µs":1174,"allocations":25002},"status":"discard","description":"String.new(capacity: 4096) for output buffer: allocates more objects, not fewer","timestamp":1773350101852,"segment":0}
{"run":19,"commit":"f6baeae","metric":3350,"metrics":{"parse_µs":2212,"render_µs":1138,"allocations":24882},"status":"keep","description":"parse_tag_token without StringScanner: pure byte ops avoid reset(token) overhead, -12% combined","timestamp":1773350230252,"segment":0}
{"run":20,"commit":"f6baead","metric":0,"metrics":{"parse_µs":0,"render_µs":0,"allocations":0},"status":"crash","description":"REVERTED: regex ultra-fast path for Variable — name pattern too broad, matches invalid trailing dots","timestamp":1773350472859,"segment":0}
{"run":21,"commit":"ae9a2e2","metric":3314,"metrics":{"parse_µs":2203,"render_µs":1111,"allocations":24882},"status":"keep","description":"Clean confirmation run: 3,314µs (-55% from main), stable","timestamp":1773350544354,"segment":0}
{"run":22,"commit":"ae9a2e2","metric":3497,"metrics":{"parse_µs":2336,"render_µs":1161,"allocations":24882},"status":"discard","description":"Regex fast path for no-filter variables: include? + match? overhead exceeds byte scan savings","timestamp":1773350641375,"segment":0}
{"run":23,"commit":"ca327b0","metric":3445,"metrics":{"parse_µs":2284,"render_µs":1161,"allocations":24647},"status":"keep","description":"Condition#evaluate: skip loop block for simple conditions (no child_relation) — saves 235 allocs","timestamp":1773350691752,"segment":0}
{"run":24,"commit":"99454a9","metric":3489,"metrics":{"parse_µs":2353,"render_µs":1136,"allocations":24647},"status":"keep","description":"Replace simple_lookup? byte scan with match? regex — 8x faster per call, cleaner code","timestamp":1773350837721,"segment":0}
{"run":25,"commit":"99454a9","metric":3797,"metrics":{"parse_µs":2636,"render_µs":1161,"allocations":29627},"status":"discard","description":"Regex name extraction in try_fast_parse: MatchData creates 5K extra allocs, much worse","timestamp":1773351048938,"segment":0}
{"run":26,"commit":"db348e0","metric":3459,"metrics":{"parse_µs":2318,"render_µs":1141,"allocations":24647},"status":"keep","description":"Inline to_liquid_value in If render — avoids one method dispatch per condition evaluation","timestamp":1773351080001,"segment":0}
{"run":27,"commit":"b195d09","metric":3496,"metrics":{"parse_µs":2356,"render_µs":1140,"allocations":24530},"status":"keep","description":"Replace @blocks.each with while loop in If render — avoids block proc allocation per render","timestamp":1773351101134,"segment":0}
{"run":28,"commit":"b195d09","metric":3648,"metrics":{"parse_µs":2457,"render_µs":1191,"allocations":24530},"status":"discard","description":"While loop in For render: YJIT optimizes each well for hot loops with many iterations","timestamp":1773351142275,"segment":0}
{"run":29,"commit":"b195d09","metric":3966,"metrics":{"parse_µs":2641,"render_µs":1325,"allocations":24060},"status":"discard","description":"While loop for environment search: -470 allocs but YJIT deopt makes render 16% slower","timestamp":1773351193863,"segment":0}
+4 -2
View File
@@ -52,14 +52,15 @@ end
require "liquid/version"
require "liquid/deprecations"
require "liquid/const"
require 'liquid/byte_tables'
require 'liquid/cursor'
require 'liquid/standardfilters'
require 'liquid/file_system'
require 'liquid/parser_switching'
require 'liquid/tag'
require 'liquid/block'
require 'liquid/parse_tree_visitor'
require 'liquid/closure_body'
require 'liquid/closure'
require 'liquid/call_expression'
require 'liquid/interrupts'
require 'liquid/tags'
require "liquid/environment"
@@ -67,6 +68,7 @@ require 'liquid/lexer'
require 'liquid/parser'
require 'liquid/i18n'
require 'liquid/drop'
require 'liquid/self_drop'
require 'liquid/tablerowloop_drop'
require 'liquid/forloop_drop'
require 'liquid/extensions'
+15 -8
View File
@@ -10,10 +10,13 @@ module Liquid
end
def parse(tokens)
@body = new_body
while parse_body(@body, tokens)
end
@body.freeze
parse_block_body(tokens)
end
# Parse an external block body with closure validation and lexical declarations.
def parse_closure_body(tokens, parameters: Const::EMPTY_ARRAY)
parse_context.with_closure(parameters) { parse_block_body(tokens) }
ClosureBody.new(@body, parameters)
end
# For backwards compatibility
@@ -60,15 +63,19 @@ module Liquid
@tag_name
end
# Cache block delimiters per tag name to avoid repeated string allocation
BLOCK_DELIMITER_CACHE = Hash.new { |h, k| h[k] = "end#{k}".freeze }
def block_delimiter
@block_delimiter ||= BLOCK_DELIMITER_CACHE[block_name]
@block_delimiter ||= "end#{block_name}"
end
private
def parse_block_body(tokens)
@body = new_body
while parse_body(@body, tokens)
end
@body.freeze
end
# @api public
def new_body
parse_context.new_block_body
+70 -108
View File
@@ -1,5 +1,7 @@
# frozen_string_literal: true
require 'English'
module Liquid
class BlockBody
LiquidTagToken = /\A\s*(#{TagName})\s*(.*?)\z/o
@@ -36,7 +38,7 @@ module Liquid
private def parse_for_liquid_tag(tokenizer, parse_context)
while (token = tokenizer.shift)
unless token.empty? || BlockBody.blank_string?(token)
unless token.empty? || token.match?(WhitespaceOrNothing)
unless token =~ LiquidTagToken
# line isn't empty but didn't match tag syntax, yield and let the
# caller raise a syntax error
@@ -51,7 +53,8 @@ module Liquid
end
unless (tag = parse_context.environment.tag_for_name(tag_name))
# end parsing if we reach an unknown tag; let the caller determine how to proceed
# end parsing if we reach an unknown tag and let the caller decide
# determine how to proceed
return yield tag_name, markup
end
new_tag = tag.parse(tag_name, markup, tokenizer, parse_context)
@@ -96,7 +99,9 @@ module Liquid
context.handle_error(exc, line_number)
else
error_message = context.handle_error(exc, line_number)
unless blank_tag # conditional for backwards compatibility
error_mode = context.registers.static[:template_error_mode]
suppress_error_text = blank_tag && error_mode != :strict2 && error_mode != :rigid
unless suppress_error_text # blank-tag suppression is kept for backwards compatibility outside strict2
output << error_message
end
end
@@ -121,38 +126,48 @@ module Liquid
end
end
def self.blank_string?(str)
str.match?(WhitespaceOrNothing)
end
private def parse_for_document(tokenizer, parse_context, &block)
while (token = tokenizer.shift)
next if token.empty?
first_byte = token.getbyte(0)
if first_byte == Cursor::LCURLY
second_byte = token.getbyte(1)
if second_byte == Cursor::PCT
# handle_tag_token returns:
# nil — tag parsed normally, continue (update line number)
# :next — 'liquid' inline tag; skip line number update
# :unknown — end tag or unknown tag; yield to caller and return
# :invalid — malformed tag token; delegate to handle_invalid_tag_token
result = handle_tag_token(token, parse_context, tokenizer)
next unless result # nil: normal
next if result == :next # :next: 'liquid'
return yield(@_unknown_tag_name, parse_context.cursor.tag_markup) if result == :unknown
return handle_invalid_tag_token(token, parse_context, &block) # :invalid
elsif second_byte == Cursor::LCURLY
whitespace_handler(token, parse_context)
@nodelist << create_variable(token, parse_context)
@blank = false
else
# Fallback: text token starting with '{'
append_text_token(token, parse_context)
case
when token.start_with?(TAGSTART)
whitespace_handler(token, parse_context)
unless token =~ FullToken
return handle_invalid_tag_token(token, parse_context, &block)
end
tag_name = Regexp.last_match(2)
markup = Regexp.last_match(4)
if parse_context.line_number
# newlines inside the tag should increase the line number,
# particularly important for multiline {% liquid %} tags
parse_context.line_number += Regexp.last_match(1).count("\n") + Regexp.last_match(3).count("\n")
end
if tag_name == 'liquid'
parse_liquid_tag(markup, parse_context)
next
end
unless (tag = parse_context.environment.tag_for_name(tag_name))
# end parsing if we reach an unknown tag and let the caller decide
# determine how to proceed
return yield tag_name, markup
end
new_tag = tag.parse(tag_name, markup, tokenizer, parse_context)
@blank &&= new_tag.blank?
@nodelist << new_tag
when token.start_with?(VARSTART)
whitespace_handler(token, parse_context)
@nodelist << create_variable(token, parse_context)
@blank = false
else
append_text_token(token, parse_context)
if parse_context.trim_whitespace
token.lstrip!
end
parse_context.trim_whitespace = false
@nodelist << token
@blank &&= token.match?(WhitespaceOrNothing)
end
parse_context.line_number = tokenizer.line_number
end
@@ -160,54 +175,8 @@ module Liquid
yield nil, nil
end
# Handles a {%...%} tag token. Does not receive the outer block — callers handle
# yield/block passing themselves, keeping the Proc off the hot path.
# Returns:
# nil — tag parsed, caller continues the loop
# :next — 'liquid' inline tag; caller skips line number update
# :unknown — unknown/end tag; @_unknown_tag_name holds the tag name;
# markup is in parse_context.cursor.tag_markup
# :invalid — malformed token; caller delegates to handle_invalid_tag_token
private def handle_tag_token(token, parse_context, tokenizer)
whitespace_handler(token, parse_context)
cursor = parse_context.cursor
tag_name = cursor.parse_tag_token(token)
return :invalid unless tag_name
markup = cursor.tag_markup
if parse_context.line_number
newlines = cursor.tag_newlines
parse_context.line_number += newlines if newlines > 0
end
if tag_name == 'liquid'
parse_liquid_tag(markup, parse_context)
return :next
end
tag = parse_context.environment.tag_for_name(tag_name)
unless tag
# end parsing if we reach an unknown tag; let the caller determine how to proceed
@_unknown_tag_name = tag_name
return :unknown
end
new_tag = tag.parse(tag_name, markup, tokenizer, parse_context)
@blank &&= new_tag.blank?
@nodelist << new_tag
nil
end
def append_text_token(token, parse_context)
token.lstrip! if parse_context.trim_whitespace
parse_context.trim_whitespace = false
@nodelist << token
@blank &&= BlockBody.blank_string?(token)
end
private :append_text_token
private def whitespace_handler(token, parse_context)
if token.getbyte(2) == Cursor::DASH
def whitespace_handler(token, parse_context)
if token[2] == WhitespaceControl
previous_token = @nodelist.last
if previous_token.is_a?(String)
first_byte = previous_token.getbyte(0)
@@ -217,7 +186,7 @@ module Liquid
end
end
end
parse_context.trim_whitespace = (token.getbyte(token.bytesize - 3) == Cursor::DASH)
parse_context.trim_whitespace = (token[-3] == WhitespaceControl)
end
def blank?
@@ -249,35 +218,24 @@ module Liquid
end
def render_to_output_buffer(context, output)
freeze
freeze unless frozen?
resource_limits = context.resource_limits
resource_limits.increment_render_score(@nodelist.length)
context.resource_limits.increment_render_score(@nodelist.length)
# Hot render loop — split on check_write so the common case (no resource
# limits) pays zero branch cost per node.
idx = 0
if resource_limits.render_length_limit || resource_limits.last_capture_length
while (node = @nodelist[idx])
if node.instance_of?(String)
output << node
else
render_node(context, output, node)
break if context.interrupt?
end
idx += 1
resource_limits.increment_write_score(output)
end
else
while (node = @nodelist[idx])
if node.instance_of?(String)
output << node
else
render_node(context, output, node)
break if context.interrupt?
end
idx += 1
while (node = @nodelist[idx])
if node.instance_of?(String)
output << node
else
render_node(context, output, node)
# If we get an Interrupt that means the block must stop processing. An
# Interrupt is any command that stops block execution such as {% break %}
# or {% continue %}. These tags may also occur through Block or Include tags.
break if context.interrupt? # might have happened in a for-block
end
idx += 1
context.resource_limits.increment_write_score(output)
end
output
@@ -285,15 +243,19 @@ module Liquid
private
# Indirection allows subclasses to intercept per-node rendering.
def render_node(context, output, node)
BlockBody.render_node(context, output, node)
end
def create_variable(token, parse_context)
len = token.bytesize
if len >= 4 && token.getbyte(len - 1) == Cursor::RCURLY && token.getbyte(len - 2) == Cursor::RCURLY
markup = parse_context.cursor.parse_variable_token(token)
if token.end_with?("}}")
i = 2
i = 3 if token[i] == "-"
parse_end = token.length - 3
parse_end -= 1 if token[parse_end] == "-"
markup_end = parse_end - i + 1
markup = markup_end <= 0 ? "" : token.slice(i, markup_end)
return Variable.new(markup, parse_context)
end
-40
View File
@@ -1,40 +0,0 @@
# frozen_string_literal: true
module Liquid
# Pre-computed 256-entry boolean lookup tables for byte classification.
# Built once at load time; used as TABLE[byte] — a single array index
# instead of 3-5 comparison operators per check.
#
# Performance: neutral to slightly faster vs. chained comparisons.
# Readability: replaces expressions like
# (b >= 97 && b <= 122) || (b >= 65 && b <= 90) || b == 95
# with the intent-revealing
# ByteTables::IDENT_START[b]
module ByteTables
# [a-zA-Z_] — valid first byte of an identifier
IDENT_START = Array.new(256, false).tap do |t|
(97..122).each { |b| t[b] = true } # a-z
(65..90).each { |b| t[b] = true } # A-Z
t[95] = true # _
end.freeze
# [a-zA-Z0-9_-] — valid continuation byte of an identifier
IDENT_CONT = Array.new(256, false).tap do |t|
(97..122).each { |b| t[b] = true } # a-z
(65..90).each { |b| t[b] = true } # A-Z
(48..57).each { |b| t[b] = true } # 0-9
t[95] = true # _
t[45] = true # -
end.freeze
# [0-9] — ASCII digit
DIGIT = Array.new(256, false).tap do |t|
(48..57).each { |b| t[b] = true }
end.freeze
# [ \t\n\v\f\r] — ASCII whitespace (mirrors Ruby's \s)
WHITESPACE = Array.new(256, false).tap do |t|
[32, 9, 10, 11, 12, 13].each { |b| t[b] = true } # space, tab, \n, \v, \f, \r
end.freeze
end
end
+33
View File
@@ -0,0 +1,33 @@
# frozen_string_literal: true
module Liquid
class CallExpression
attr_reader :target, :arguments
def initialize(target, arguments)
@target = target.freeze
@arguments = arguments.freeze
freeze
end
def evaluate(context)
context.resource_limits.with_capture do
render_to_output_buffer(context, +'')
end
end
def render_to_output_buffer(context, output)
closure = context.evaluate(@target)
raise Liquid::ArgumentError, "Expected a Liquid closure" unless closure.is_a?(Closure)
arguments = @arguments.map { |argument| context.evaluate(argument) }
closure.render_to_output_buffer(context, output, arguments)
end
class ParseTreeVisitor < Liquid::ParseTreeVisitor
def children
[@node.target] + @node.arguments
end
end
end
end
+96
View File
@@ -0,0 +1,96 @@
# frozen_string_literal: true
module Liquid
# A render-time value. Captures retain binding hashes, not copies of their values.
class Closure
def initialize(body, parameters, context)
@body = body
@parameters = parameters
@scopes = context.scopes.dup
@environments = context.environments.dup
@static_environments = context.static_environments
end
# Render into the supplied buffer and return direct-child body-form let bindings.
def invoke(context, arguments = Const::EMPTY_ARRAY, output: +'')
invocation = render_invocation(context, arguments, output)
@body.exports(invocation)
end
def render_to_output_buffer(context, output, arguments = Const::EMPTY_ARRAY)
render_invocation(context, arguments, output)
output
end
def to_liquid
self
end
def to_s
''
end
private
def render_invocation(context, arguments, output)
unless arguments.length == @parameters.length
raise Liquid::ArgumentError, "Expected #{@parameters.length} closure arguments, got #{arguments.length}"
end
invocation = context.new_closure_subcontext(@scopes, @environments, @static_environments)
@parameters.each_with_index { |name, index| invocation[name] = arguments[index] }
@body.render_to_output_buffer(invocation, output)
invocation
end
# @api private
class CapturedBindings
def initialize(scopes, environments, context)
@scopes = scopes
@environments = environments
@context = context
@locals = {}
end
def key?(name)
local_key?(name) || @environments.any? { |environment| environment.key?(name) }
end
def local_key?(name)
@locals.key?(name) || @scopes.any? { |scope| scope.key?(name) } ||
@environments.any? { |environment| environment.is_a?(CapturedBindings) && environment.local_key?(name) }
end
def []=(name, value)
@locals[name] = value
end
def [](name)
find_variable(name, @context, raise_on_not_found: false)
end
def find_variable(name, context, raise_on_not_found: true)
return context.lookup_and_evaluate(@locals, name, raise_on_not_found: raise_on_not_found) if @locals.key?(name)
scope = @scopes.find { |candidate| candidate.key?(name) }
return context.lookup_and_evaluate(scope, name, raise_on_not_found: raise_on_not_found) if scope
find_in_environments(name, context, raise_on_not_found: raise_on_not_found)
end
private
def find_in_environments(name, context, raise_on_not_found:)
@environments.each do |environment|
if environment.is_a?(CapturedBindings)
return environment.find_variable(name, context, raise_on_not_found: raise_on_not_found)
end
value = context.lookup_and_evaluate(environment, name, raise_on_not_found: raise_on_not_found)
return value if !value.nil? || context.strict_variables && raise_on_not_found
end
nil
end
end
end
end
+31
View File
@@ -0,0 +1,31 @@
# frozen_string_literal: true
module Liquid
# Parsed syntax shared by let and closure-aware custom blocks.
class ClosureBody
attr_reader :body
def initialize(body, parameters = Const::EMPTY_ARRAY)
@body = body
@parameters = parameters.dup.freeze
@exports = body.nodelist.select { |node| node.is_a?(Let) && node.body_form? }.map(&:to).freeze
freeze
end
def bind(context)
Closure.new(self, @parameters, context)
end
def render_to_output_buffer(context, output)
@body.render_to_output_buffer(context, output)
end
def exports(context)
context.scopes.last.slice(*@exports)
end
def nodelist
@body.nodelist
end
end
end
+6 -7
View File
@@ -65,21 +65,20 @@ module Liquid
end
def evaluate(context = deprecated_default_context)
result = interpret_condition(@left, @right, @operator, context)
# Fast path: no child conditions (most common)
return result unless @child_relation
condition = self
while condition.child_relation
result = nil
loop do
result = interpret_condition(condition.left, condition.right, condition.operator, context)
case condition.child_relation
when :or
break if Liquid::Utils.to_liquid_value(result)
when :and
break unless Liquid::Utils.to_liquid_value(result)
else
break
end
condition = condition.child_condition
result = interpret_condition(condition.left, condition.right, condition.operator, context)
end
result
end
+87 -66
View File
@@ -24,15 +24,10 @@ module Liquid
def initialize(environments = {}, outer_scope = {}, registers = {}, rethrow_errors = false, resource_limits = nil, static_environments = {}, environment = Environment.default)
@environment = environment
@environments = environments.is_a?(Array) ? environments : [environments]
@environments = [environments]
@environments.flatten!
@static_environments = if static_environments.is_a?(Array)
static_environments.frozen? ? static_environments : static_environments.freeze
elsif static_environments.empty?
Const::EMPTY_ARRAY
else
[static_environments].freeze
end
@static_environments = [static_environments].flatten(1).freeze
@scopes = [outer_scope || {}]
@registers = registers.is_a?(Registers) ? registers : Registers.new(registers)
@errors = []
@@ -40,13 +35,14 @@ module Liquid
@strict_variables = false
@resource_limits = resource_limits || ResourceLimits.new(environment.default_resource_limits)
@base_scope_depth = 0
@interrupts = Const::EMPTY_ARRAY
@filters = Const::EMPTY_ARRAY
@interrupts = []
@filters = []
@global_filter = nil
@disabled_tags = Const::EMPTY_HASH
@disabled_tags = {}
# Lazy-init StringScanner — only needed if Context#[] is called during render
@string_scanner = nil
# Instead of constructing new StringScanner objects for each Expression parse,
# we recycle the same one.
@string_scanner = StringScanner.new("")
@registers.static[:cached_partials] ||= {}
@registers.static[:file_system] ||= environment.file_system
@@ -77,7 +73,7 @@ module Liquid
# Note that this does not register the filters with the main Template object. see <tt>Template.register_filter</tt>
# for that
def add_filters(filters)
filters = Array(filters).flatten.compact
filters = [filters].flatten.compact
@filters += filters
@strainer = nil
end
@@ -88,12 +84,11 @@ module Liquid
# are there any not handled interrupts?
def interrupt?
!@interrupts.equal?(Const::EMPTY_ARRAY) && @interrupts.any?
!@interrupts.empty?
end
# push an interrupt to the stack. this interrupt is considered not handled.
def push_interrupt(e)
@interrupts = [] if @interrupts.frozen?
@interrupts.push(e)
end
@@ -114,20 +109,6 @@ module Liquid
strainer.invoke(method, *args).to_liquid
end
# Arity-specialized filter delegation — generated to match StrainerTemplate's specializations.
# The pattern (avoid *args splat) is the same for each arity; generating makes it explicit.
{
invoke_single: ['input'],
invoke_two: ['input', 'arg1'],
}.each do |method_name, params|
all_params = (["method"] + params).join(", ")
module_eval(<<~RUBY, __FILE__, __LINE__ + 1)
def #{method_name}(#{all_params})
strainer.#{method_name}(#{all_params}).to_liquid
end
RUBY
end
# Push new local scope on the stack. use <tt>Context#stack</tt> instead
def push(new_scope = {})
@scopes.unshift(new_scope)
@@ -181,6 +162,40 @@ module Liquid
end
end
# @api private
def new_closure_subcontext(scopes, environments, static_environments)
new_isolated_subcontext.tap do |invocation|
captures = Closure::CapturedBindings.new(scopes, environments + static_environments, invocation)
invocation.captured_bindings = captures
invocation.environments = [captures]
invocation.static_environments = static_environments
invocation.strict_variables = strict_variables
invocation.strict_filters = strict_filters
invocation.global_filter = global_filter
invocation.template_name = template_name
invocation.partial = partial
invocation.base_scope_depth = base_scope_depth + @scopes.length
end
end
# @api private
def with_binding_scope
previous_root = @binding_root
stack do
@binding_root = @scopes.first
yield
end
ensure
@binding_root = previous_root
end
# @api private
def set_local(name, value)
root = @binding_root || @scopes.last
scope = @scopes.find { |owned| owned.key?(name) || owned.equal?(root) }
scope[name] = value
end
def clear_instance_assigns
@scopes[0] = {}
end
@@ -199,11 +214,20 @@ module Liquid
# Example:
# products == empty #=> products.empty?
def [](expression)
evaluate(Expression.parse(expression, @string_scanner ||= StringScanner.new("")))
evaluate(Expression.parse(expression, @string_scanner))
end
def key?(key)
!find_variable(key, raise_on_not_found: false).nil?
find_variable(key, raise_on_not_found: false) != nil
end
# Checks whether a variable is defined in any scope, including nil-valued keys.
# Unlike #key?, this uses Hash#key? so that variables explicitly set to nil
# are still considered defined.
def variable_defined?(key)
@scopes.any? { |s| s.key?(key) } ||
@environments.any? { |e| e.key?(key) } ||
@static_environments.any? { |e| e.key?(key) }
end
def evaluate(object)
@@ -212,38 +236,31 @@ module Liquid
# Fetches an object starting at the local scope and then moving up the hierachy
def find_variable(key, raise_on_not_found: true)
# Fast path: check top scope first (most common in for loops)
scope = @scopes[0]
if scope.key?(key)
variable = lookup_and_evaluate(scope, key, raise_on_not_found: raise_on_not_found)
elsif @scopes.length == 1
# Only one scope and key not found — go straight to environments
variable = try_variable_find_in_environments(key, raise_on_not_found: raise_on_not_found)
else
# Multiple scopes — search through all of them
scope = @scopes.find { |s| s.key?(key) }
# This was changed from find() to find_index() because this is a very hot
# path and find_index() is optimized in MRI to reduce object allocation
index = @scopes.find_index { |s| s.key?(key) }
variable = if scope
lookup_and_evaluate(scope, key, raise_on_not_found: raise_on_not_found)
else
try_variable_find_in_environments(key, raise_on_not_found: raise_on_not_found)
end
fallback_to_self_drop = key == Expression::SELF && index.nil? && !@captured_bindings&.local_key?(key)
variable = if index
lookup_and_evaluate(@scopes[index], key, raise_on_not_found: raise_on_not_found)
else
try_variable_find_in_environments(
key,
raise_on_not_found: raise_on_not_found && !fallback_to_self_drop,
)
end
# `self` resolves to a SelfDrop (enabling `self['var']` lookups),
# but only after the normal environment lookup doesn't find a value.
return @self_drop ||= SelfDrop.new(self) if fallback_to_self_drop && variable.nil?
# update variable's context before invoking #to_liquid
# Fast path: primitive types don't need context= or to_liquid conversion
case variable
when String, Integer, Float, NilClass, TrueClass, FalseClass, Array, Hash, Time
return variable
end
variable.context = self if variable.respond_to?(:context=)
liquid_variable = variable.to_liquid
if variable != liquid_variable
liquid_variable.context = self if liquid_variable.respond_to?(:context=)
end
liquid_variable.context = self if variable != liquid_variable && liquid_variable.respond_to?(:context=)
liquid_variable
end
@@ -263,7 +280,6 @@ module Liquid
end
def with_disabled_tags(tag_names)
@disabled_tags = {} if @disabled_tags.frozen?
tag_names.each do |name|
@disabled_tags[name] = @disabled_tags.fetch(name, 0) + 1
end
@@ -280,23 +296,28 @@ module Liquid
protected
attr_writer :base_scope_depth, :warnings, :errors, :strainer, :filters, :disabled_tags
attr_writer :base_scope_depth, :warnings, :errors, :strainer, :filters, :disabled_tags, :environments, :static_environments, :captured_bindings
private
attr_reader :base_scope_depth
def try_variable_find_in_environments(key, raise_on_not_found:)
found = find_in_envs(@environments, key, raise_on_not_found: raise_on_not_found)
return found unless found.nil? && !(@strict_variables && raise_on_not_found)
if @captured_bindings
return @captured_bindings.find_variable(key, self, raise_on_not_found: raise_on_not_found)
end
find_in_envs(@static_environments, key, raise_on_not_found: raise_on_not_found)
end
def find_in_envs(envs, key, raise_on_not_found:)
envs.each do |environment|
@environments.each do |environment|
found_variable = lookup_and_evaluate(environment, key, raise_on_not_found: raise_on_not_found)
return found_variable if !found_variable.nil? || (@strict_variables && raise_on_not_found)
if !found_variable.nil? || @strict_variables && raise_on_not_found
return found_variable
end
end
@static_environments.each do |environment|
found_variable = lookup_and_evaluate(environment, key, raise_on_not_found: raise_on_not_found)
if !found_variable.nil? || @strict_variables && raise_on_not_found
return found_variable
end
end
nil
end
-317
View File
@@ -1,317 +0,0 @@
# frozen_string_literal: true
require "strscan"
module Liquid
# Single-pass forward-only scanner for Liquid parsing.
# Wraps StringScanner with higher-level methods for common Liquid constructs.
# One Cursor per template parse — threaded through all parsing code.
class Cursor
# Byte constants
SPACE = 32
TAB = 9
NL = 10
CR = 13
FF = 12
DASH = 45 # '-'
DOT = 46 # '.'
COLON = 58 # ':'
PIPE = 124 # '|'
QUOTE_S = 39 # "'"
QUOTE_D = 34 # '"'
LBRACK = 91 # '['
RBRACK = 93 # ']'
LPAREN = 40 # '('
RPAREN = 41 # ')'
QMARK = 63 # '?'
HASH = 35 # '#'
USCORE = 95 # '_'
COMMA = 44
ZERO = 48
NINE = 57
PCT = 37 # '%'
LCURLY = 123 # '{'
RCURLY = 125 # '}'
attr_reader :ss
def initialize(source)
@source = source
@ss = StringScanner.new(source)
end
# ── Position ────────────────────────────────────────────────────
def pos = @ss.pos
def pos=(n)
@ss.pos = n
end
def eos? = @ss.eos?
def peek_byte = @ss.peek_byte
def scan_byte = @ss.scan_byte
# Reset scanner to a new string (for reuse on sub-markup)
def reset(source)
@source = source
@ss.string = source
end
# Extract a slice from the source (deferred allocation)
def slice(start, len)
@source.byteslice(start, len)
end
# ── Whitespace ──────────────────────────────────────────────────
# Skip spaces/tabs/newlines/cr
def skip_ws
while (b = @ss.peek_byte)
case b
when SPACE, TAB, CR, FF, NL then @ss.scan_byte
else break
end
end
end
# Check if remaining bytes are all whitespace (or EOS).
# exist?(/\S/) returns nil when no non-whitespace remains, without advancing position.
def rest_blank?
!@ss.exist?(/\S/)
end
# Regex for identifier: [a-zA-Z_][\w-]*\??
ID_REGEX = /[a-zA-Z_][\w-]*\??/
# ── Identifiers ─────────────────────────────────────────────────
# Skip an identifier without allocating a string. Returns length skipped, or 0.
def skip_id
@ss.skip(ID_REGEX) || 0
end
# Check if next id matches expected string, consume if so. No allocation.
def expect_id(expected)
start = @ss.pos
len = @ss.skip(ID_REGEX)
if len == expected.bytesize
# Compare bytes directly without allocating a string
i = 0
while i < len
unless @source.getbyte(start + i) == expected.getbyte(i)
@ss.pos = start
return false
end
i += 1
end
return true
end
@ss.pos = start if len
false
end
# Scan a single identifier: [a-zA-Z_][\w-]*\??
# Returns the string or nil if not at an identifier
def scan_id
@ss.scan(ID_REGEX)
end
# Scan a tag name: '#' or \w+
def scan_tag_name
if @ss.peek_byte == HASH
@ss.scan_byte
"#"
else
scan_id
end
end
# Regex for numbers: -?\d+(\.\d+)?
FLOAT_REGEX = /-?\d+\.\d+/
INT_REGEX = /-?\d+/
# ── Numbers ─────────────────────────────────────────────────────
# Try to scan an integer or float. Returns the number or nil.
def scan_number
if (s = @ss.scan(FLOAT_REGEX))
s.to_f
elsif (s = @ss.scan(INT_REGEX))
s.to_i
end
end
# Regex for quoted string content (without quotes)
SINGLE_QUOTED_CONTENT = /'([^']*)'/
DOUBLE_QUOTED_CONTENT = /"([^"]*)"/
# ── Strings ─────────────────────────────────────────────────────
# Scan a quoted string ('...' or "..."). Returns the content without quotes, or nil.
def scan_quoted_string
if @ss.scan(SINGLE_QUOTED_CONTENT) || @ss.scan(DOUBLE_QUOTED_CONTENT)
@ss[1]
end
end
# Regex for quoted strings (single or double quoted, including quotes)
QUOTED_STRING_RAW = /"[^"]*"|'[^']*'/
# Scan a quoted string including quotes. Returns the full "..." or '...' string, or nil.
def scan_quoted_string_raw
@ss.scan(QUOTED_STRING_RAW)
end
# Regex for dotted identifier: name(.name)*
DOTTED_ID_REGEX = /[a-zA-Z_][\w-]*\??(?:\.[a-zA-Z_][\w-]*\??)*/
# ── Expressions ─────────────────────────────────────────────────
# Scan a simple variable lookup: name(.name)* — no brackets, no filters
# Returns the string or nil
def scan_dotted_id
@ss.scan(DOTTED_ID_REGEX)
end
# Skip a fragment without allocating. Returns length skipped, or 0.
def skip_fragment
@ss.skip(QUOTED_STRING_RAW) || @ss.skip(UNQUOTED_FRAGMENT) || 0
end
# Regex for unquoted fragment: non-whitespace/comma/pipe sequence
UNQUOTED_FRAGMENT = /[^\s,|]+/
# Scan a "QuotedFragment" — a quoted string or non-whitespace/comma/pipe run
def scan_fragment
@ss.scan(QUOTED_STRING_RAW) || @ss.scan(UNQUOTED_FRAGMENT)
end
# ── Comparison operators ────────────────────────────────────────
# Identity map used for frozen string interning: StringScanner#scan returns a
# new unfrozen String on every call. Indexing into this hash returns the frozen
# literal stored here, avoiding a separate allocation and enabling faster
# equality checks downstream (frozen strings can be compared by identity).
COMPARISON_OPS = {
'==' => '==',
'!=' => '!=',
'<>' => '<>',
'<=' => '<=',
'>=' => '>=',
'<' => '<',
'>' => '>',
'contains' => 'contains',
}.freeze
# Scan a comparison operator. Returns frozen string or nil.
# Regex for comparison operators
COMPARISON_OP_REGEX = /==|!=|<>|<=|>=|<|>|contains(?!\w)/
def scan_comparison_op
if (op = @ss.scan(COMPARISON_OP_REGEX))
COMPARISON_OPS[op]
end
end
# ── Tag parsing helpers ─────────────────────────────────────────
# Results from last parse_tag_token call (avoids array allocation)
attr_reader :tag_markup, :tag_newlines
# Parse the interior of a tag token: "{%[-] tag_name markup [-]%}"
# Pure byte operations — avoids StringScanner reset overhead.
# Returns tag_name string or nil. Sets tag_markup and tag_newlines.
def parse_tag_token(token)
len = token.bytesize
pos = 2 # skip "{%"
pos += 1 if token.getbyte(pos) == DASH # skip '-'
nl = 0
# Skip whitespace, count newlines
while pos < len
b = token.getbyte(pos)
case b
when SPACE, TAB, CR, FF then pos += 1
when NL then pos += 1
nl += 1
else break
end
end
# Scan tag name: '#' or [a-zA-Z_][\w-]*
name_start = pos
b = token.getbyte(pos)
if b == HASH
pos += 1
elsif b && ByteTables::IDENT_START[b]
pos += 1
while pos < len
b = token.getbyte(pos)
break unless ByteTables::IDENT_CONT[b]
pos += 1
end
pos += 1 if pos < len && token.getbyte(pos) == QMARK
else
return
end
tag_name = token.byteslice(name_start, pos - name_start)
# Skip whitespace after tag name, count newlines
while pos < len
b = token.getbyte(pos)
case b
when SPACE, TAB, CR, FF then pos += 1
when NL then pos += 1
nl += 1
else break
end
end
# markup is everything up to optional '-' before '%}'
markup_end = len - 2
markup_end -= 1 if markup_end > pos && token.getbyte(markup_end - 1) == DASH
@tag_markup = pos >= markup_end ? "" : token.byteslice(pos, markup_end - pos)
@tag_newlines = nl
tag_name
end
# Parse variable token interior: extract markup from "{{[-] ... [-]}}"
def parse_variable_token(token)
len = token.bytesize
return if len < 4
i = 2
i = 3 if token.getbyte(i) == DASH
parse_end = len - 3
parse_end -= 1 if token.getbyte(parse_end) == DASH
markup_len = parse_end - i + 1
markup_len <= 0 ? "" : token.byteslice(i, markup_len)
end
# ── Simple condition parser ─────────────────────────────────────
# Results from last parse_simple_condition call
attr_reader :cond_left, :cond_op, :cond_right
# Parse "expr [op expr]" from current position to end.
# Returns true on success, nil on failure. Sets cond_left, cond_op, cond_right.
def parse_simple_condition
skip_ws
@cond_left = scan_fragment
return unless @cond_left
skip_ws
if eos?
@cond_op = nil
@cond_right = nil
return true
end
@cond_op = scan_comparison_op
return unless @cond_op
skip_ws
@cond_right = scan_fragment
return unless @cond_right
skip_ws
return unless eos? # trailing junk
true
end
end
end
+6 -2
View File
@@ -20,7 +20,9 @@ module Liquid
end
def parse(tokenizer, parse_context)
while parse_body(tokenizer)
@local_bindings = parse_context.with_document_scope do
while parse_body(tokenizer)
end
end
@body.freeze
rescue SyntaxError => e
@@ -38,7 +40,9 @@ module Liquid
end
def render_to_output_buffer(context, output)
@body.render_to_output_buffer(context, output)
return @body.render_to_output_buffer(context, output) unless @local_bindings
context.with_binding_scope { @body.render_to_output_buffer(context, output) }
end
def render(context)
+53 -77
View File
@@ -2,6 +2,8 @@
module Liquid
class Expression
SELF = 'self'
LITERALS = {
nil => nil,
'nil' => nil,
@@ -16,9 +18,16 @@ module Liquid
'-' => VariableLookup.parse("-", nil).freeze,
}.freeze
DOT = ".".ord
ZERO = "0".ord
NINE = "9".ord
DASH = "-".ord
# Use an atomic group (?>...) to avoid pathological backtracing from
# malicious input as described in https://github.com/Shopify/liquid/issues/1357
RANGES_REGEX = /\A\(\s*(?>(\S+)\s*\.\.)\s*(\S+)\s*\)\z/
INTEGER_REGEX = /\A(-?\d+)\z/
FLOAT_REGEX = /\A(-?\d+)\.\d+\z/
class << self
def safe_parse(parser, ss = StringScanner.new(""), cache = nil)
@@ -27,18 +36,13 @@ module Liquid
def parse(markup, ss = StringScanner.new(""), cache = nil)
return unless markup
return markup unless markup.is_a?(String)
# Only strip if there's leading/trailing whitespace (avoids allocation)
first_byte = markup.getbyte(0)
if first_byte && ByteTables::WHITESPACE[first_byte]
markup = markup.strip
elsif first_byte
markup = markup.strip if ByteTables::WHITESPACE[markup.getbyte(markup.bytesize - 1)]
end
markup = markup.strip # markup can be a frozen string
if (markup.start_with?('"') && markup.end_with?('"')) ||
(markup.start_with?("'") && markup.end_with?("'"))
return markup.byteslice(1, markup.bytesize - 2)
return markup[1..-2]
elsif LITERALS.key?(markup)
return LITERALS[markup]
end
@@ -70,85 +74,57 @@ module Liquid
end
end
def parse_number(markup, _ss = nil)
len = markup.bytesize
return if len == 0
# Quick reject: first byte must be digit or dash
pos = 0
first = markup.getbyte(pos)
if first == Cursor::DASH
pos += 1
return if pos >= len
b = markup.getbyte(pos)
return unless ByteTables::DIGIT[b]
pos += 1
elsif ByteTables::DIGIT[first]
pos += 1
else
return
end
# Scan digits
while pos < len
b = markup.getbyte(pos)
break unless ByteTables::DIGIT[b]
pos += 1
end
# If we consumed everything, it's a simple integer
if pos == len
def parse_number(markup, ss)
# check if the markup is simple integer or float
case markup
when INTEGER_REGEX
return Integer(markup, 10)
when FLOAT_REGEX
return markup.to_f
end
# Check for dot (float)
if markup.getbyte(pos) == Cursor::DOT
dot_pos = pos
pos += 1
# Must have at least one digit after dot
digit_after_dot = pos
while pos < len
b = markup.getbyte(pos)
break unless ByteTables::DIGIT[b]
ss.string = markup
# the first byte must be a digit or a dash
byte = ss.scan_byte
pos += 1
end
return false if byte != DASH && (byte < ZERO || byte > NINE)
if pos > digit_after_dot && pos == len
# Simple float like "123.456"
return markup.to_f
elsif pos > digit_after_dot
# Float followed by more content: "1.2.3.4" — scan to find where the
# numeric portion ends (stop at next dot or non-digit).
return scan_float_with_trailing(markup, pos, len)
else
# dot at end: "123."
return markup.byteslice(0, dot_pos).to_f
if byte == DASH
peek_byte = ss.peek_byte
# if it starts with a dash, the next byte must be a digit
return false if peek_byte.nil? || !(peek_byte >= ZERO && peek_byte <= NINE)
end
# The markup could be a float with multiple dots
first_dot_pos = nil
num_end_pos = nil
while (byte = ss.scan_byte)
return false if byte != DOT && (byte < ZERO || byte > NINE)
# we found our number and now we are just scanning the rest of the string
next if num_end_pos
if byte == DOT
if first_dot_pos.nil?
first_dot_pos = ss.pos
else
# we found another dot, so we know that the number ends here
num_end_pos = ss.pos - 1
end
end
end
# Not a number (has non-digit, non-dot characters)
nil
end
num_end_pos = markup.length if ss.eos?
private
# Scans forward from `pos` through digits, returning the float up to the
# next dot or the end of string. Returns nil when a non-digit, non-dot
# byte is found (not a valid number). Used by parse_number for inputs
# like "1.2.3.4" where the float literal ends at the second dot.
def scan_float_with_trailing(markup, pos, len)
while pos < len
b = markup.getbyte(pos)
return markup.byteslice(0, pos).to_f if b == Cursor::DOT
return unless ByteTables::DIGIT[b]
pos += 1
if num_end_pos
# number ends with a number "123.123"
markup.byteslice(0, num_end_pos).to_f
else
# number ends with a dot "123."
markup.byteslice(0, first_dot_pos).to_f
end
markup.byteslice(0, pos).to_f
end
end
end
+6 -4
View File
@@ -2,6 +2,7 @@
module Liquid
class Lexer
ASSIGN = [:assign, "="].freeze
CLOSE_ROUND = [:close_round, ")"].freeze
CLOSE_SQUARE = [:close_square, "]"].freeze
COLON = [:colon, ":"].freeze
@@ -29,7 +30,6 @@ module Liquid
RUBY_WHITESPACE = [" ", "\t", "\r", "\n", "\f"].freeze
SINGLE_STRING_LITERAL = /'[^\']*'/
WHITESPACE_OR_NOTHING = /\s*/
WHITESPACE = /\s+/
SINGLE_COMPARISON_TOKENS = [].tap do |table|
table["<".ord] = COMPARISON_LESS_THAN
@@ -105,7 +105,7 @@ module Liquid
output = []
until ss.eos?
ss.skip(WHITESPACE)
ss.skip(WHITESPACE_OR_NOTHING)
break if ss.eos?
@@ -115,10 +115,10 @@ module Liquid
if (special = SPECIAL_TABLE[peeked])
ss.scan_byte
# Special case for ".."
if special.equal?(DOT) && ss.peek_byte == DOT_ORD
if special == DOT && ss.peek_byte == DOT_ORD
ss.scan_byte
output << DOTDOT
elsif special.equal?(DASH)
elsif special == DASH
# Special case for negative numbers
if (peeked_byte = ss.peek_byte) && NUMBER_TABLE[peeked_byte]
ss.pos -= 1
@@ -134,6 +134,8 @@ module Liquid
if (peeked_byte = ss.peek_byte) && (found = sub_table[peeked_byte])
output << found
ss.scan_byte
elsif peeked == "=".ord && ss.peek_byte != "<".ord && ss.peek_byte != ">".ord && ss.peek_byte != "!".ord
output << ASSIGN
else
raise_syntax_error(start_pos, ss)
end
+56 -6
View File
@@ -3,7 +3,7 @@
module Liquid
class ParseContext
attr_accessor :locale, :line_number, :trim_whitespace, :depth
attr_reader :partial, :warnings, :error_mode, :environment, :expression_cache, :string_scanner, :cursor
attr_reader :partial, :warnings, :error_mode, :environment
def initialize(options = Const::EMPTY_HASH)
@environment = options.fetch(:environment, Environment.default)
@@ -24,8 +24,6 @@ module Liquid
{}
end
@cursor = Cursor.new("")
self.depth = 0
self.partial = false
end
@@ -40,7 +38,11 @@ module Liquid
def new_parser(input)
@string_scanner.string = input
Parser.new(@string_scanner)
Parser.new(
@string_scanner,
reject_bare_brackets: @error_mode == :strict2 || @error_mode == :rigid,
allow_calls: @error_mode == :strict2,
)
end
def new_tokenizer(source, start_line_number: nil, for_liquid_tag: false)
@@ -62,8 +64,8 @@ module Liquid
# compatibility while raising awareness about strict2 parser standards,
# the safe flag supports API users make a deliberate decision.
#
# In strict2 mode, markup MUST come from a string returned by the parser
# (e.g., parser.expression). We're not calling the parser here to
# In strict2 mode, markup MUST come from parser.expression, which returns
# markup or a parsed callable node. We're not calling the parser here to
# prevent redundant parser overhead.
raise Liquid::InternalError, "unsafe parse_expression cannot be used in strict2 mode"
end
@@ -71,6 +73,54 @@ module Liquid
Expression.parse(markup, @string_scanner, @expression_cache)
end
def with_document_scope(&block)
previous_usage = @uses_local_bindings
@uses_local_bindings = false
with_lexical_scope(&block)
@uses_local_bindings
ensure
@uses_local_bindings = previous_usage
end
def with_lexical_scope(parameters = Const::EMPTY_ARRAY)
previous_scope = @lexical_scope
@lexical_scope = parameters.to_h { |name| [name, true] }
yield
ensure
@lexical_scope = previous_scope
end
def local_declarations?
@lexical_scope && !@lexical_scope.empty?
end
def declare_local(name)
@lexical_scope ||= {}
raise SyntaxError, "Duplicate let declaration '#{name}'" if @lexical_scope.key?(name)
@lexical_scope[name] = true
end
def with_closure(parameters = Const::EMPTY_ARRAY, &block)
require_closures!
previous_closure = @in_closure
@in_closure = true
with_lexical_scope(parameters, &block)
ensure
@in_closure = previous_closure
end
def in_closure?
@in_closure
end
def require_closures!
@uses_local_bindings = true
return if @error_mode == :strict2
raise SyntaxError, "Closures require strict2 mode"
end
def partial=(value)
@partial = value
@options = value ? partial_options : @template_options
+78 -4
View File
@@ -2,10 +2,12 @@
module Liquid
class Parser
def initialize(input)
def initialize(input, reject_bare_brackets: false, allow_calls: false)
ss = input.is_a?(StringScanner) ? input : StringScanner.new(input)
@tokens = Lexer.tokenize(ss)
@p = 0 # pointer to current location
@reject_bare_brackets = reject_bare_brackets
@allow_calls = allow_calls
end
def jump(point)
@@ -50,9 +52,14 @@ module Liquid
token = @tokens[@p]
case token[0]
when :id
return callable_or_lookup if @allow_calls
str = consume
str << variable_lookups
when :open_square
if @reject_bare_brackets
raise SyntaxError, "Bare bracket access is not allowed. Use #{Expression::SELF}['...'] instead"
end
str = consume.dup
str << expression
str << consume(:close_square)
@@ -65,12 +72,41 @@ module Liquid
consume(:dotdot)
last = expression
consume(:close_round)
return RangeLookup.new(Expression.parse(first), Expression.parse(last)) unless first.is_a?(String) && last.is_a?(String)
"(#{first}..#{last})"
else
raise SyntaxError, "#{token} is not a valid expression"
end
end
def expression_with_markup
start = @p
# Parsing lookups appends to consumed identifier strings.
markup_tokens = @tokens[start...-1].map { |token| token[1].dup }
value = expression
markup = value.is_a?(String) ? value : markup_tokens.take(@p - start).join
[value, markup]
end
def remaining_markup
@tokens[@p...-1].map { |token| token[1] }.join(' ')
end
def parameters
parameters = []
return parameters unless consume?(:open_round)
unless look(:close_round)
parameters << consume(:id)
parameters << consume(:id) while consume?(:comma) && !look(:close_round)
end
consume(:close_round)
raise SyntaxError, "Duplicate closure parameter" unless parameters.uniq.length == parameters.length
parameters
end
def argument
str = +""
# might be a keyword argument (identifier: expression)
@@ -83,9 +119,6 @@ module Liquid
end
def variable_lookups
# Fast path: no lookups at all (most common case for simple identifiers)
return "" unless look(:dot) || look(:open_square)
str = +""
loop do
if look(:open_square)
@@ -101,5 +134,46 @@ module Liquid
end
str
end
private
def callable_or_lookup
target = callable_lookups(consume(:id))
return target unless consume?(:open_round)
arguments = []
unless look(:close_round)
arguments << Expression.parse(expression)
arguments << Expression.parse(expression) while consume?(:comma) && !look(:close_round)
end
consume(:close_round)
target = VariableLookup.parse(target) if target.is_a?(String)
CallExpression.new(target, arguments)
end
def callable_lookups(target)
loop do
if consume?(:dot)
target = append_lookup(target, consume(:id), command: true)
next
end
break unless consume?(:open_square)
key = expression
consume(:close_square)
target = append_lookup(target, key)
end
target
end
def append_lookup(target, key, command: false)
if target.is_a?(String) && key.is_a?(String)
return target << (command ? ".#{key}" : "[#{key}]")
end
target = VariableLookup.parse(target) if target.is_a?(String)
target.add_lookup(command ? key : Expression.parse(key), command: command)
target
end
end
end
+6 -6
View File
@@ -6,15 +6,15 @@ module Liquid
def initialize(registers = {})
@static = registers.is_a?(Registers) ? registers.static : registers
@changes = nil
@changes = {}
end
def []=(key, value)
(@changes ||= {})[key] = value
@changes[key] = value
end
def [](key)
if @changes&.key?(key)
if @changes.key?(key)
@changes[key]
else
@static[key]
@@ -22,13 +22,13 @@ module Liquid
end
def delete(key)
@changes&.delete(key)
@changes.delete(key)
end
UNDEFINED = Object.new
def fetch(key, default = UNDEFINED, &block)
if @changes&.key?(key)
if @changes.key?(key)
@changes.fetch(key)
elsif default != UNDEFINED
if block_given?
@@ -42,7 +42,7 @@ module Liquid
end
def key?(key)
@changes&.key?(key) || @static.key?(key)
@changes.key?(key) || @static.key?(key)
end
end
-1
View File
@@ -9,7 +9,6 @@ module Liquid
:cumulative_assign_score_limit
attr_reader :render_score,
:assign_score,
:last_capture_length,
:cumulative_render_score,
:cumulative_assign_score
+54
View File
@@ -0,0 +1,54 @@
# frozen_string_literal: true
module Liquid
# @liquid_public_docs
# @liquid_type object
# @liquid_name self
# @liquid_summary
# Provides access to variables through the current scope chain.
# @liquid_description
# The `self` object resolves variables through the normal lookup hierarchy
# (local > file > global) without exposing filters, interrupts, errors,
# or other context internals. It's used when bare bracket notation
# (`['variable']`) needs to be replaced with an explicit variable lookup.
#
# If `self` is explicitly assigned as a local variable (e.g. `{% assign self = 'value' %}`),
# then the local value takes precedence over the `self` object.
# @liquid_access global
class SelfDrop < Drop
def initialize(self_context)
super()
@self_context = self_context
end
def [](key)
@self_context.find_variable(key)
rescue UndefinedVariable
nil
end
def key?(key)
@self_context.variable_defined?(key)
end
def to_liquid
self
end
def ==(other)
other.is_a?(SelfDrop) && other.self_context.equal?(@self_context)
end
alias_method :eql?, :==
def hash
@self_context.object_id.hash
end
protected
attr_reader :self_context
undef context=
end
end
+10 -63
View File
@@ -275,71 +275,18 @@ module Liquid
words = Utils.to_integer(words)
words = 1 if words <= 0
return input if words + 1 > MAX_I32
# Scan words tracking byte positions; build the normalized (single-space)
# result string only when truncation is actually needed.
len = input.bytesize
pos = 0
word_count = 0
# Flat array of [start, end, start, end, ...] for up to `words` words.
# Avoids allocating a result string in the common no-truncation case.
positions = []
# Skip leading whitespace
while pos < len
break unless ByteTables::WHITESPACE[input.getbyte(pos)]
pos += 1
wordlist = begin
input.split(" ", words + 1)
rescue RangeError
# integer too big for String#split, but we can semantically assume no truncation is needed
return input if words + 1 > MAX_I32
raise # unexpected error
end
return input if wordlist.length <= words
while pos < len
word_start = pos
word_count += 1
# Scan to end of word
while pos < len
break if ByteTables::WHITESPACE[input.getbyte(pos)]
pos += 1
end
if word_count <= words
positions.push(word_start, pos) # [start, end, start, end, ...]
else
# Truncation confirmed — build normalized result from stored positions
result = +input.byteslice(positions[0], positions[1] - positions[0])
i = 2
while i < positions.length
result << " " << input.byteslice(positions[i], positions[i + 1] - positions[i])
i += 2
end
return result << Utils.to_s(truncate_string)
end
# Skip whitespace between words
while pos < len
break unless ByteTables::WHITESPACE[input.getbyte(pos)]
pos += 1
end
end
# Fewer words than requested — no truncation needed, return original unchanged.
return input if word_count < words
# Exactly `words` words. Ruby's split(" ", words+1) would produce a words+1-th
# empty element when input has trailing whitespace, triggering the truncation path.
# Match that behaviour: if the input ends with whitespace, normalize and append
# truncate_string even though no word was cut.
if len > 0 && ByteTables::WHITESPACE[input.getbyte(len - 1)]
result = +input.byteslice(positions[0], positions[1] - positions[0])
i = 2
while i < positions.length
result << " " << input.byteslice(positions[i], positions[i + 1] - positions[i])
i += 2
end
return result << Utils.to_s(truncate_string)
end
input
wordlist.pop
truncate_string = Utils.to_s(truncate_string)
wordlist.join(" ").concat(truncate_string)
end
# @liquid_public_docs
-26
View File
@@ -58,31 +58,5 @@ module Liquid
rescue ::ArgumentError => e
raise Liquid::ArgumentError, e.message, e.backtrace
end
# Arity-specialized filter invocation.
# Avoids *args splat allocation for the common 0-arg and 1-arg cases.
# `invoke` (general case) still uses *args for 2+ extra arguments.
{
invoke_single: ['input'],
invoke_two: ['input', 'arg1'],
}.each do |method_name, params|
all_params = (["method"] + params).join(", ")
send_params = params.join(", ")
# __LINE__ + 1 is a parse-time constant; both generated methods will report
# the same file:line in backtraces. The method name in the trace distinguishes them.
module_eval(<<~RUBY, __FILE__, __LINE__ + 1)
def #{method_name}(#{all_params})
if self.class.invokable?(method)
send(method, #{send_params})
elsif @context.strict_filters
raise Liquid::UndefinedFilter, "undefined filter \#{method}"
else
input
end
rescue ::ArgumentError => e
raise Liquid::ArgumentError, e.message, e.backtrace
end
RUBY
end
end
end
+4
View File
@@ -7,6 +7,8 @@ require_relative "tags/break"
require_relative "tags/inline_comment"
require_relative "tags/for"
require_relative "tags/assign"
require_relative "tags/let"
require_relative "tags/set"
require_relative "tags/ifchanged"
require_relative "tags/case"
require_relative "tags/include"
@@ -37,6 +39,8 @@ module Liquid
'case' => Case,
'ifchanged' => Ifchanged,
'assign' => Assign,
'let' => Let,
'set' => SetTag,
'for' => For,
'#' => InlineComment,
'break' => Break,
+30 -4
View File
@@ -18,6 +18,8 @@ module Liquid
# @liquid_syntax_keyword variable_name The name of the variable being created.
# @liquid_syntax_keyword value The value you want to assign to the variable.
class Assign < Tag
include ParserSwitching
Syntax = /(#{VariableSignature}+)\s*=\s*(.*)\s*/om
# @api private
@@ -29,6 +31,12 @@ module Liquid
def initialize(tag_name, markup, parse_context)
super
raise SyntaxError, "assign is not allowed in closure bodies" if parse_context.in_closure?
parse_with_selected_parser(markup)
end
def lax_parse(markup)
if markup =~ Syntax
@to = Regexp.last_match(1)
@from = Variable.new(Regexp.last_match(2), parse_context)
@@ -37,10 +45,29 @@ module Liquid
end
end
def strict_parse(markup)
lax_parse(markup)
end
def strict2_parse(markup)
unless markup =~ Syntax
self.class.raise_syntax_error(parse_context)
end
lhs = Regexp.last_match(1).strip
rhs = Regexp.last_match(2)
p = @parse_context.new_parser(lhs)
@to = p.consume(:id)
p.consume(:end_of_string)
@from = Variable.new(rhs, parse_context)
end
def render_to_output_buffer(context, output)
val = @from.render(context)
context.scopes.last[@to] = val
context.resource_limits.increment_assign_score(assign_score_of(val))
context.resource_limits.increment_assign_score(self.class.assign_score_of(val))
output
end
@@ -48,9 +75,8 @@ module Liquid
true
end
private
def assign_score_of(val)
# @api private
def self.assign_score_of(val)
if val.instance_of?(String)
val.bytesize
elsif val.instance_of?(Array)
+18
View File
@@ -20,10 +20,18 @@ module Liquid
# @liquid_syntax_keyword variable The name of the variable being created.
# @liquid_syntax_keyword value The value you want to assign to the variable.
class Capture < Block
include ParserSwitching
Syntax = /(#{VariableSignature}+)/o
attr_reader :to
def initialize(tag_name, markup, options)
super
parse_with_selected_parser(markup)
end
def lax_parse(markup)
if markup =~ Syntax
@to = Regexp.last_match(1)
else
@@ -31,6 +39,16 @@ module Liquid
end
end
def strict_parse(markup)
lax_parse(markup)
end
def strict2_parse(markup)
p = @parse_context.new_parser(markup.strip)
@to = p.consume(:id)
p.consume(:end_of_string)
end
def render_to_output_buffer(context, output)
context.resource_limits.with_capture do
capture_output = render(context)
+16
View File
@@ -23,13 +23,29 @@ module Liquid
# {% decrement variable_name %}
# @liquid_syntax_keyword variable_name The name of the variable being decremented.
class Decrement < Tag
include ParserSwitching
attr_reader :variable_name
def initialize(tag_name, markup, options)
super
parse_with_selected_parser(markup)
end
def lax_parse(markup)
@variable_name = markup.strip
end
def strict_parse(markup)
lax_parse(markup)
end
def strict2_parse(markup)
p = @parse_context.new_parser(markup.strip)
@variable_name = p.consume(:id)
p.consume(:end_of_string)
end
def render_to_output_buffer(context, output)
counter_environment = context.environments.first
value = counter_environment[@variable_name] || 0
+52 -58
View File
@@ -25,6 +25,8 @@ module Liquid
# @liquid_optional_param range [untyped] A custom numeric range to iterate over.
# @liquid_optional_param reversed [untyped] Iterate in reverse order.
class For < Block
Syntax = /\A(#{VariableSegment}+)\s+in\s+(#{QuotedFragment}+)\s*(reversed)?/o
attr_reader :collection_name, :variable_name, :limit, :from
def initialize(tag_name, markup, options)
@@ -36,15 +38,10 @@ module Liquid
end
def parse(tokens)
if parse_body(@for_block, tokens)
parse_body(@else_block, tokens)
parse_context.with_lexical_scope do
parse_loop(tokens)
@has_local_declarations = parse_context.local_declarations?
end
if blank?
@else_block&.remove_blank_strings
@for_block.remove_blank_strings
end
@else_block&.freeze
@for_block.freeze
end
def nodelist
@@ -70,52 +67,18 @@ module Liquid
protected
# Fast byte-level parser for "var in collection [reversed] [limit:N] [offset:N]"
def lax_parse(markup)
c = @parse_context.cursor
c.reset(markup)
c.skip_ws
# Parse variable name
var_start = c.pos
var_len = c.skip_id
raise SyntaxError, options[:locale].t("errors.syntax.for") if var_len == 0
@variable_name = c.slice(var_start, var_len)
# Expect "in"
c.skip_ws
raise SyntaxError, options[:locale].t("errors.syntax.for") unless c.expect_id("in")
c.skip_ws
# Parse collection name
col_start = c.pos
if c.peek_byte == Cursor::LPAREN
# Parenthesized range: (1..10)
depth = 1
c.scan_byte
while !c.eos? && depth > 0
b = c.scan_byte
depth += 1 if b == Cursor::LPAREN
depth -= 1 if b == Cursor::RPAREN
end
else
c.skip_fragment
end
collection_name = c.slice(col_start, c.pos - col_start)
@name = "#{@variable_name}-#{collection_name}"
@collection_name = parse_expression(collection_name)
c.skip_ws
@reversed = c.expect_id("reversed")
c.skip_ws
# Parse limit:/offset: if present.
# Cursor doesn't handle key:value attributes — delegate to regex for limit:/offset:.
if !c.eos? && (rest = c.slice(c.pos, markup.bytesize - c.pos)).include?(':')
rest.scan(TagAttributes) do |key, value|
if markup =~ Syntax
@variable_name = Regexp.last_match(1)
collection_name = Regexp.last_match(2)
@reversed = !!Regexp.last_match(3)
@name = "#{@variable_name}-#{collection_name}"
@collection_name = parse_expression(collection_name)
markup.scan(TagAttributes) do |key, value|
set_attribute(key, value)
end
else
raise SyntaxError, options[:locale].t("errors.syntax.for")
end
end
@@ -124,10 +87,10 @@ module Liquid
@variable_name = p.consume(:id)
raise SyntaxError, options[:locale].t("errors.syntax.for_invalid_in") unless p.id?('in')
collection_name = p.expression
collection_name, collection_markup = p.expression_with_markup
@collection_name = parse_expression(collection_name, safe: true)
@name = "#{@variable_name}-#{collection_name}"
@name = "#{@variable_name}-#{collection_markup}"
@reversed = p.id?('reversed')
while p.look(:comma) || p.look(:id)
@@ -143,7 +106,21 @@ module Liquid
private
alias_method :strict2_parse, :strict_parse
def parse_loop(tokens)
if parse_body(@for_block, tokens)
parse_body(@else_block, tokens)
end
if blank?
@else_block&.remove_blank_strings
@for_block.remove_blank_strings
end
@else_block&.freeze
@for_block.freeze
end
def strict2_parse(markup)
strict_parse(markup)
end
def collection_segment(context)
offsets = context.registers[:for] ||= {}
@@ -152,14 +129,22 @@ module Liquid
offsets[@name].to_i
else
from_value = context.evaluate(@from)
from_value.nil? ? 0 : Utils.to_integer(from_value)
if from_value.nil?
0
else
Utils.to_integer(from_value)
end
end
collection = context.evaluate(@collection_name)
collection = collection.to_a if collection.is_a?(Range)
limit_value = context.evaluate(@limit)
to = limit_value && (Utils.to_integer(limit_value) + from)
to = if limit_value.nil?
nil
else
Utils.to_integer(limit_value) + from
end
segment = Utils.slice_collection(collection, from, to)
segment.reverse! if @reversed
@@ -183,7 +168,7 @@ module Liquid
segment.each do |item|
context[@variable_name] = item
@for_block.render_to_output_buffer(context, output)
render_iteration(context, output)
loop_vars.send(:increment!)
# Handle any interrupts if they exist.
@@ -200,6 +185,12 @@ module Liquid
output
end
def render_iteration(context, output)
return @for_block.render_to_output_buffer(context, output) unless @has_local_declarations
context.stack { @for_block.render_to_output_buffer(context, output) }
end
def set_attribute(key, expr, safe: false)
case key
when 'offset'
@@ -214,7 +205,10 @@ module Liquid
end
def render_else(context, output)
@else_block ? @else_block.render_to_output_buffer(context, output) : output
return output unless @else_block
return @else_block.render_to_output_buffer(context, output) unless @has_local_declarations
context.stack { @else_block.render_to_output_buffer(context, output) }
end
class ParseTreeVisitor < Liquid::ParseTreeVisitor
+4 -28
View File
@@ -51,17 +51,14 @@ module Liquid
end
def render_to_output_buffer(context, output)
idx = 0
blocks = @blocks
while idx < blocks.length
block = blocks[idx]
result = block.evaluate(context)
result = result.to_liquid_value if result.respond_to?(:to_liquid_value)
@blocks.each do |block|
result = Liquid::Utils.to_liquid_value(
block.evaluate(context),
)
if result
return block.attachment.render_to_output_buffer(context, output)
end
idx += 1
end
output
@@ -89,27 +86,6 @@ module Liquid
end
def lax_parse(markup)
# Fastest path: simple identifier truthiness like "product.available" or "forloop.first"
if (simple = Variable.simple_variable_markup(markup))
return Condition.new(parse_expression(simple))
end
# Fast path: simple condition without and/or — use Cursor.
# The include? pre-checks are both a correctness guard (parse_simple_condition
# only handles a single comparison) and a perf gate (avoids cursor allocation
# for the compound-condition case that will always fall through to lax_parse).
if !markup.include?(' and ') && !markup.include?(' or ')
cursor = @parse_context.cursor
cursor.reset(markup)
if cursor.parse_simple_condition
return Condition.new(
parse_expression(cursor.cond_left),
cursor.cond_op,
cursor.cond_right ? parse_expression(cursor.cond_right) : nil,
)
end
end
expressions = markup.scan(ExpressionsAndOperators)
raise SyntaxError, options[:locale].t("errors.syntax.if") unless expressions.pop =~ Syntax
+13 -4
View File
@@ -20,7 +20,8 @@ module Liquid
class Include < Tag
prepend Tag::Disableable
SYNTAX = /(#{QuotedFragment}+)(\s+(?:with|for)\s+(#{QuotedFragment}+))?(\s+(?:as)\s+(#{VariableSegment}+))?/o
FOR = 'for'
SYNTAX = /(#{QuotedFragment}+)(\s+(with|#{FOR})\s+(#{QuotedFragment}+))?(\s+(?:as)\s+(#{VariableSegment}+))?/o
Syntax = SYNTAX
attr_reader :template_name_expr, :variable_name_expr, :attributes
@@ -84,12 +85,18 @@ module Liquid
alias_method :parse_context, :options
private :parse_context
def for_loop?
@is_for_loop
end
def strict2_parse(markup)
p = @parse_context.new_parser(markup)
@template_name_expr = safe_parse_expression(p)
@variable_name_expr = safe_parse_expression(p) if p.id?("for") || p.id?("with")
with_or_for = p.id?("for") || p.id?("with")
@variable_name_expr = safe_parse_expression(p) if with_or_for
@alias_name = p.consume(:id) if p.id?("as")
@is_for_loop = (with_or_for == FOR)
p.consume?(:comma)
@@ -111,11 +118,13 @@ module Liquid
def lax_parse(markup)
if markup =~ SYNTAX
template_name = Regexp.last_match(1)
variable_name = Regexp.last_match(3)
with_or_for = Regexp.last_match(3)
variable_name = Regexp.last_match(4)
@alias_name = Regexp.last_match(5)
@alias_name = Regexp.last_match(6)
@variable_name_expr = variable_name ? parse_expression(variable_name) : nil
@template_name_expr = parse_expression(template_name)
@is_for_loop = (with_or_for == FOR)
@attributes = {}
markup.scan(TagAttributes) do |key, value|
+16
View File
@@ -23,13 +23,29 @@ module Liquid
# {% increment variable_name %}
# @liquid_syntax_keyword variable_name The name of the variable being incremented.
class Increment < Tag
include ParserSwitching
attr_reader :variable_name
def initialize(tag_name, markup, options)
super
parse_with_selected_parser(markup)
end
def lax_parse(markup)
@variable_name = markup.strip
end
def strict_parse(markup)
lax_parse(markup)
end
def strict2_parse(markup)
p = @parse_context.new_parser(markup.strip)
@variable_name = p.consume(:id)
p.consume(:end_of_string)
end
def render_to_output_buffer(context, output)
counter_environment = context.environments.first
value = counter_environment[@variable_name] || 0
+54
View File
@@ -0,0 +1,54 @@
# frozen_string_literal: true
module Liquid
class Let < Block
attr_reader :to, :from, :parameters, :closure_body
def initialize(tag_name, markup, parse_context)
super
parse_context.require_closures!
parser = parse_context.new_parser(markup)
@to = parser.consume(:id)
if parser.consume?(:assign)
raise SyntaxError, "Expected let initializer" if parser.look(:end_of_string)
@from = Variable.new(parser, parse_context)
else
@parameters = parser.parameters.freeze
parser.consume(:end_of_string)
end
parse_context.declare_local(@to)
end
def parse(tokens)
return unless body_form?
@closure_body = parse_closure_body(tokens, parameters: @parameters)
end
def body_form?
@from.nil?
end
def nodelist
body_form? ? super : Const::EMPTY_ARRAY
end
def render_to_output_buffer(context, output)
value = body_form? ? @closure_body.bind(context) : @from.render(context)
context.resource_limits.increment_assign_score(Assign.assign_score_of(value))
context[@to] = value
output
end
def blank?
true
end
class ParseTreeVisitor < Liquid::ParseTreeVisitor
def children
@node.body_form? ? [@node.closure_body] : [@node.from]
end
end
end
end
+39
View File
@@ -0,0 +1,39 @@
# frozen_string_literal: true
module Liquid
class SetTag < Tag
attr_reader :to, :from
def initialize(tag_name, markup, parse_context)
super
parse_context.require_closures!
parser = parse_context.new_parser(markup)
@to = parser.consume(:id)
parser.consume(:assign)
raise SyntaxError, "Expected set initializer" if parser.look(:end_of_string)
@from = Variable.new(parser, parse_context)
end
def render_to_output_buffer(context, output)
value = if context.variable_defined?(@to)
@from.render(context)
else
context.stack(@to => nil) { @from.render(context) }
end
context.resource_limits.increment_assign_score(Assign.assign_score_of(value))
context.set_local(@to, value)
output
end
def blank?
true
end
class ParseTreeVisitor < Liquid::ParseTreeVisitor
def children
[@node.from]
end
end
end
end
+12 -1
View File
@@ -34,6 +34,13 @@ module Liquid
parse_with_selected_parser(markup)
end
def parse(tokens)
parse_context.with_lexical_scope do
super
@has_local_declarations = parse_context.local_declarations?
end
end
def strict2_parse(markup)
p = @parse_context.new_parser(markup)
@@ -99,7 +106,11 @@ module Liquid
context[@variable_name] = item
output << "<td class=\"col#{tablerowloop.col}\">"
super
if @has_local_declarations
context.stack { super }
else
super
end
output << '</td>'
# Handle any interrupts if they exist.
+13 -2
View File
@@ -151,8 +151,10 @@ module Liquid
c
when Liquid::Drop
drop = args.shift
drop.context = Context.new([drop, assigns], instance_assigns, registers, @rethrow_errors, @resource_limits, {}, @environment)
drop = args.shift
c = Context.new([drop, assigns], instance_assigns, registers, @rethrow_errors, @resource_limits, {}, @environment)
drop.context = c if drop.respond_to?(:context=)
c
when Hash
Context.new([args.shift, assigns], instance_assigns, registers, @rethrow_errors, @resource_limits, {}, @environment)
when nil
@@ -187,12 +189,20 @@ module Liquid
context.template_name ||= name
previous_error_mode = context.registers.static[:template_error_mode]
context.registers.static[:template_error_mode] = @error_mode
begin
# render the nodelist.
@root.render_to_output_buffer(context, output || +'')
rescue Liquid::MemoryError => e
context.handle_error(e)
ensure
if previous_error_mode
context.registers.static[:template_error_mode] = previous_error_mode
else
context.registers.static.delete(:template_error_mode)
end
@errors = context.errors
end
end
@@ -224,6 +234,7 @@ module Liquid
end
@warnings = parse_context.warnings
@error_mode = parse_context.error_mode
parse_context
end
+103 -94
View File
@@ -1,23 +1,37 @@
# frozen_string_literal: true
require "strscan"
module Liquid
class Tokenizer
attr_reader :line_number, :for_liquid_tag
TAG_END = /%\}/
TAG_OR_VARIABLE_START = /\{[\{\%]/
NEWLINE = /\n/
OPEN_CURLEY = "{".ord
CLOSE_CURLEY = "}".ord
PERCENTAGE = "%".ord
def initialize(
source:,
string_scanner: nil,
string_scanner:,
line_numbers: false,
line_number: nil,
for_liquid_tag: false
)
@line_number = line_number || (line_numbers ? 1 : nil)
@for_liquid_tag = for_liquid_tag
@source = source.to_s
@source = source.to_s.to_str
@offset = 0
@tokens = []
tokenize
if @source
@ss = string_scanner
@ss.string = @source
tokenize
end
end
def shift
@@ -40,113 +54,108 @@ module Liquid
if @for_liquid_tag
@tokens = @source.split("\n")
else
tokenize_fast
@tokens << shift_normal until @ss.eos?
end
@source = nil
@ss = nil
end
# Fast tokenizer using String#byteindex instead of StringScanner regex.
# String#byteindex is ~40% faster for finding { delimiters.
def tokenize_fast
src = @source
unless src.valid_encoding?
raise SyntaxError, "Invalid byte sequence in #{src.encoding}"
def shift_normal
token = next_token
return unless token
token
end
def next_token
# possible states: :text, :tag, :variable
byte_a = @ss.peek_byte
if byte_a == OPEN_CURLEY
@ss.scan_byte
byte_b = @ss.peek_byte
if byte_b == PERCENTAGE
@ss.scan_byte
return next_tag_token
elsif byte_b == OPEN_CURLEY
@ss.scan_byte
return next_variable_token
end
@ss.pos -= 1
end
len = src.bytesize
pos = 0
next_text_token
end
while pos < len
# Find next { which could start a tag or variable
idx = src.byteindex('{', pos)
def next_text_token
start = @ss.pos
unless idx
# No more tags/variables — rest is text
@tokens << src.byteslice(pos, len - pos) if pos < len
break
end
unless @ss.skip_until(TAG_OR_VARIABLE_START)
token = @ss.rest
@ss.terminate
return token
end
next_byte = idx + 1 < len ? src.getbyte(idx + 1) : nil
if next_byte == Cursor::PCT # {%
# Emit text before tag
@tokens << src.byteslice(pos, idx - pos) if idx > pos
# Find %} to close the tag
close = src.byteindex('%}', idx + 2)
if close
@tokens << src.byteslice(idx, close + 2 - idx)
pos = close + 2
else
# Emit malformed token to propagate a missing-terminator error in the parser
@tokens << "{%"
pos = idx + 2
end
elsif next_byte == Cursor::LCURLY # {{
# Emit text before variable, then scan for the closing }}.
@tokens << src.byteslice(pos, idx - pos) if idx > pos
pos = scan_variable_token(src, idx, len)
else
# Lone '{' — not the start of a tag or variable.
# Find the next '{{' or '{%' to know where this text token ends.
# Using two byteindex calls avoids a nested loop and is always O(n).
tag_start = src.byteindex('{%', idx + 1)
var_start = src.byteindex('{{', idx + 1)
next_token = [tag_start, var_start].compact.min
if next_token
@tokens << src.byteslice(pos, next_token - pos)
pos = next_token
else
@tokens << src.byteslice(pos, len - pos)
pos = len
end
end
pos = @ss.pos -= 2
@source.byteslice(start, pos - start)
rescue ::ArgumentError => e
if e.message == "invalid byte sequence in #{@ss.string.encoding}"
raise SyntaxError, "Invalid byte sequence in #{@ss.string.encoding}"
else
raise
end
end
# Scans a {{ ... }} variable token starting at `idx` in `src`.
# Emits the token to @tokens and returns the new position after the token.
# Handles }}, single }, and embedded {% ... %} (nested tag inside variable).
private def scan_variable_token(src, idx, len)
# Byte-by-byte scan: find } or {, then inspect the next byte.
scan_pos = idx + 2
while scan_pos < len
b = src.getbyte(scan_pos)
if b == Cursor::RCURLY # }
if scan_pos + 1 >= len
# } at end of string — emit token up to here
@tokens << src.byteslice(idx, scan_pos + 1 - idx)
return scan_pos + 1
end
b2 = src.getbyte(scan_pos + 1)
if b2 == Cursor::RCURLY
# Found }} — close variable
@tokens << src.byteslice(idx, scan_pos + 2 - idx)
return scan_pos + 2
else
# } followed by non-} — emit token up to here (matches original: @ss.pos -= 1)
@tokens << src.byteslice(idx, scan_pos + 1 - idx)
return scan_pos + 1
end
elsif b == Cursor::LCURLY && scan_pos + 1 < len && src.getbyte(scan_pos + 1) == Cursor::PCT
# Found {% inside {{ — scan to %} and emit as one token
close = src.byteindex('%}', scan_pos + 2)
if close
@tokens << src.byteslice(idx, close + 2 - idx)
return close + 2
else
@tokens << src.byteslice(idx, len - idx)
return len
end
else
scan_pos += 1
def next_variable_token
start = @ss.pos - 2
byte_a = byte_b = @ss.scan_byte
while byte_b
byte_a = @ss.scan_byte while byte_a && byte_a != CLOSE_CURLEY && byte_a != OPEN_CURLEY
break unless byte_a
if @ss.eos?
return byte_a == CLOSE_CURLEY ? @source.byteslice(start, @ss.pos - start) : "{{"
end
byte_b = @ss.scan_byte
if byte_a == CLOSE_CURLEY
if byte_b == CLOSE_CURLEY
return @source.byteslice(start, @ss.pos - start)
elsif byte_b != CLOSE_CURLEY
@ss.pos -= 1
return @source.byteslice(start, @ss.pos - start)
end
elsif byte_a == OPEN_CURLEY && byte_b == PERCENTAGE
return next_tag_token_with_start(start)
end
byte_a = byte_b
end
# Reached end without finding }} — malformed
@tokens << "{{"
idx + 2
"{{"
end
def next_tag_token
start = @ss.pos - 2
if (len = @ss.skip_until(TAG_END))
@source.byteslice(start, len + 2)
else
"{%"
end
end
def next_tag_token_with_start(start)
@ss.skip_until(TAG_END)
@source.byteslice(start, @ss.pos - start)
end
end
end
+16 -19
View File
@@ -8,9 +8,6 @@ module Liquid
def self.slice_collection(collection, from, to)
if (from != 0 || !to.nil?) && collection.respond_to?(:load_slice)
collection.load_slice(from, to)
elsif from == 0 && to.nil? && collection.is_a?(Array)
# Fast path: no offset/limit on an Array — return as-is (avoid copy)
collection
else
slice_collection_using_each(collection, from, to)
end
@@ -18,17 +15,23 @@ module Liquid
def self.slice_collection_using_each(collection, from, to)
segments = []
index = 0
# String is Enumerable but #each is not defined; handle it as a single-element collection
# Maintains Ruby 1.8.7 String#each behaviour on 1.9
if collection.is_a?(String)
return collection.empty? ? [] : [collection]
end
return [] unless collection.respond_to?(:each)
index = 0
collection.each do |item|
break if to && to <= index
segments << item if from <= index
if to && to <= index
break
end
if from <= index
segments << item
end
index += 1
end
@@ -90,14 +93,8 @@ module Liquid
obj
end
# Cached string representations for common small integers (0-999)
# Avoids repeated Integer#to_s allocations during rendering
SMALL_INT_STRINGS = Array.new(1000) { |i| i.to_s.freeze }.freeze
def self.to_s(obj, seen = nil)
def self.to_s(obj, seen = {})
case obj
when Integer
obj >= 0 && obj < 1000 ? SMALL_INT_STRINGS[obj] : obj.to_s
when BigDecimal
obj.to_s("F")
when Hash
@@ -105,30 +102,30 @@ module Liquid
# custom implementation. Otherwise we use Liquid's default
# implementation.
if obj.class.instance_method(:to_s) == HASH_TO_S_METHOD
hash_inspect(obj, seen || {})
hash_inspect(obj, seen)
else
obj.to_s
end
when Array
array_inspect(obj, seen || {})
array_inspect(obj, seen)
else
obj.to_s
end
end
def self.inspect(obj, seen = nil)
def self.inspect(obj, seen = {})
case obj
when Hash
# If the custom hash implementation overrides `#inspect`, use their
# custom implementation. Otherwise we use Liquid's default
# implementation.
if obj.class.instance_method(:inspect) == HASH_INSPECT_METHOD
hash_inspect(obj, seen || {})
hash_inspect(obj, seen)
else
obj.inspect
end
when Array
array_inspect(obj, seen || {})
array_inspect(obj, seen)
else
obj.inspect
end
+26 -331
View File
@@ -12,26 +12,6 @@ module Liquid
# {{ user | link }}
#
class Variable
# Checks if markup is a simple "name.lookup.chain" with no filters/brackets/quotes.
# Returns the trimmed markup string, or nil if not simple.
def self.simple_variable_markup(markup)
return if markup.empty?
return unless markup.match?(SIMPLE_VARIABLE_RE)
# Avoid allocation when there's no surrounding whitespace (the common case)
first = markup.getbyte(0)
last = markup.getbyte(markup.bytesize - 1)
needs_strip = first == Cursor::SPACE || first == Cursor::TAB || first == Cursor::NL || first == Cursor::CR ||
last == Cursor::SPACE || last == Cursor::TAB || last == Cursor::NL || last == Cursor::CR
needs_strip ? markup.strip : markup
end
# Cache for [filtername, EMPTY_ARRAY] tuples — avoids repeated array creation
NO_ARG_FILTER_CACHE = Hash.new { |h, k| h[k] = [k, Const::EMPTY_ARRAY].freeze }
# Regex for a simple variable lookup with optional surrounding whitespace.
# Shares the identifier grammar with VariableLookup::SIMPLE_LOOKUP_RE.
SIMPLE_VARIABLE_RE = /\A\s*[\w-]+\??(?:\.[\w-]+\??)*\s*\z/
FilterMarkupRegex = /#{FilterSeparator}\s*(.*)/om
FilterParser = /(?:\s+|#{QuotedFragment}|#{ArgumentSeparator})+/o
FilterArgsRegex = /(?:#{FilterArgumentSeparator}|#{ArgumentSeparator})\s*((?:\w+\s*\:\s*)?#{QuotedFragment})/o
@@ -45,295 +25,29 @@ module Liquid
include ParserSwitching
def initialize(markup, parse_context)
@markup = markup
@markup = markup.is_a?(Parser) ? markup.remaining_markup : markup
@name = nil
@parse_context = parse_context
@line_number = parse_context.line_number
# Fast path: try to parse without going through Lexer → Parser
# Skip for strict2/rigid modes which require different parsing
# Fast path only for lax/warn modes — strict modes need full error checking
error_mode = parse_context.error_mode
if error_mode == :strict2 || error_mode == :rigid || error_mode == :strict || !try_fast_parse(markup, parse_context)
strict_parse_with_error_mode_fallback(markup)
end
end
private def try_fast_parse(markup, parse_context)
pos = fast_scan_name(markup)
return false unless pos
# fast_resolve_name calls VariableLookup.parse_simple / Expression::LITERALS — the
# only sites that can raise SyntaxError on malformed input. The byte scanners return
# false instead of raising.
begin
fast_resolve_name(markup, parse_context)
rescue SyntaxError
return false
end
# End of markup — no filters
if pos >= markup.bytesize
@filters = Const::EMPTY_ARRAY
return true
end
# Must be followed by a pipe filter separator
return false unless markup.getbyte(pos) == Cursor::PIPE
fast_scan_filters(markup, pos, parse_context)
end
# Scan the variable name (quoted string or identifier chain) at the start of markup.
# Returns the position after the name + trailing whitespace, or false on failure.
# Sets @_fast_name_start and @_fast_name_end for fast_resolve_name.
private def fast_scan_name(markup)
len = markup.bytesize
return false if len == 0
# Skip leading whitespace
pos = 0
while pos < len
b = markup.getbyte(pos)
break unless b == Cursor::SPACE || b == Cursor::TAB || b == Cursor::NL || b == Cursor::CR
pos += 1
end
return false if pos >= len
b = markup.getbyte(pos)
if b == Cursor::QUOTE_S || b == Cursor::QUOTE_D
# Quoted string literal: scan to matching close quote
quote = b
@_fast_name_start = pos
pos += 1
pos += 1 while pos < len && markup.getbyte(pos) != quote
pos += 1 if pos < len # skip closing quote
@_fast_name_end = pos
elsif ByteTables::IDENT_START[b]
# Identifier chain: [a-zA-Z_][a-zA-Z0-9_-]*(.[a-zA-Z_][a-zA-Z0-9_-]*)*
@_fast_name_start = pos
pos += 1
while pos < len
b = markup.getbyte(pos)
if ByteTables::IDENT_CONT[b]
pos += 1
elsif b == Cursor::DOT
pos += 1
return false if pos >= len
b = markup.getbyte(pos)
return false unless ByteTables::IDENT_START[b]
pos += 1
else
break
end
end
@_fast_name_end = pos
else
return false
end
# Skip whitespace after name
while pos < len
b = markup.getbyte(pos)
break unless b == Cursor::SPACE || b == Cursor::TAB || b == Cursor::NL || b == Cursor::CR
pos += 1
end
pos
end
# Resolve the scanned name bytes to a Liquid expression object.
# Reads @_fast_name_start / @_fast_name_end set by fast_scan_name.
# Sets @name. May raise SyntaxError (rescued in try_fast_parse).
private def fast_resolve_name(markup, parse_context)
name_start = @_fast_name_start
name_end = @_fast_name_end
len = markup.bytesize
# Avoid byteslice when the name spans the whole markup (no surrounding whitespace/filters)
expr_markup = name_start == 0 && name_end == len ? markup : markup.byteslice(name_start, name_end - name_start)
cache = parse_context.expression_cache
ss = parse_context.string_scanner
first_byte = expr_markup.getbyte(0)
@name = if first_byte == Cursor::QUOTE_S || first_byte == Cursor::QUOTE_D
# String literal — strip enclosing quotes
expr_markup.byteslice(1, expr_markup.bytesize - 2)
elsif Expression::LITERALS.key?(expr_markup)
Expression::LITERALS[expr_markup]
elsif cache
cache[expr_markup] || (cache[expr_markup] = VariableLookup.parse_simple(expr_markup, ss, cache).freeze)
else
VariableLookup.parse_simple(expr_markup, ss || StringScanner.new(""), nil).freeze
end
end
# Scan the filter chain starting at `pos` (the first '|').
# Returns true on success (sets @filters), false to fall back to the Lexer.
# Rescues SyntaxError from Expression.parse inside fast_scan_filter_args.
private def fast_scan_filters(markup, pos, parse_context)
len = markup.bytesize
@filters = []
filter_pos = pos
while filter_pos < len && markup.getbyte(filter_pos) == Cursor::PIPE
filter_pos += 1
# Skip spaces after pipe (tabs/newlines handled in the between-filters skip below)
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) == Cursor::SPACE
# Scan filter name: must start with [a-zA-Z_]
fname_start = filter_pos
b = filter_pos < len ? markup.getbyte(filter_pos) : nil
break unless b && ByteTables::IDENT_START[b]
filter_pos += 1
while filter_pos < len
b = markup.getbyte(filter_pos)
break unless ByteTables::IDENT_CONT[b]
filter_pos += 1
end
filtername = markup.byteslice(fname_start, filter_pos - fname_start)
# Skip whitespace after filter name
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) == Cursor::SPACE
if filter_pos < len && markup.getbyte(filter_pos) == Cursor::COLON
# Has arguments — fast-scan positional args; fall to Lexer on keyword args
filter_pos += 1 # skip ':'
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) == Cursor::SPACE
result = fast_scan_filter_args(markup, filter_pos, parse_context)
return fall_to_lexer_filters(markup, pos, fname_start, len, parse_context) if result == :fall_to_lexer
filter_args, filter_pos = result
@filters << [filtername, filter_args]
else
# No-arg filter — reuse the cached [name, EMPTY_ARRAY] tuple
@filters << NO_ARG_FILTER_CACHE[filtername]
end
# Skip whitespace (including tabs and newlines) between filters
filter_pos += 1 while filter_pos < len && (
markup.getbyte(filter_pos) == Cursor::SPACE ||
markup.getbyte(filter_pos) == Cursor::TAB ||
markup.getbyte(filter_pos) == Cursor::NL ||
markup.getbyte(filter_pos) == Cursor::CR
)
end
# Trailing bytes that aren't a pipe mean something the fast path doesn't handle
return false if filter_pos < len
@filters = Const::EMPTY_ARRAY if @filters.empty?
true
rescue SyntaxError
# Expression.parse (called inside fast_scan_filter_args for identifier args) can
# raise SyntaxError on malformed input. Fall back to full Lexer parse.
@name = nil
@filters = nil
false
end
# Called when fast_scan_filter_args encounters keyword args or an unrecognised
# token. Hands the remaining filter chain (from the pipe before fname_start)
# to the full Lexer-based parser, merges results into @filters, and returns true.
private def fall_to_lexer_filters(markup, pos, fname_start, len, parse_context)
# Walk back from fname_start to find the pipe that opened this filter.
# Equivalent to: markup.rindex('|', fname_start), bounded by pos.
rest_start = fname_start
rest_start -= 1 while rest_start > pos && markup.getbyte(rest_start) != Cursor::PIPE
rest_markup = markup.byteslice(rest_start, len - rest_start)
p = parse_context.new_parser(rest_markup)
while p.consume?(:pipe)
fn = p.consume(:id)
fa = p.consume?(:colon) ? parse_filterargs(p) : Const::EMPTY_ARRAY
@filters << lax_parse_filter_expressions(fn, fa)
end
p.consume(:end_of_string)
@filters = Const::EMPTY_ARRAY if @filters.empty?
true
end
# Scan positional filter arguments starting at `filter_pos`.
# Returns [filter_args_array, new_filter_pos] on success, or :fall_to_lexer when
# keyword args or unrecognised tokens are encountered.
private def fast_scan_filter_args(markup, filter_pos, parse_context)
len = markup.bytesize
filter_args = []
loop do
arg_start = filter_pos
b = filter_pos < len ? markup.getbyte(filter_pos) : nil
if b == Cursor::QUOTE_S || b == Cursor::QUOTE_D
# Quoted string argument
quote = b
filter_pos += 1
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) != quote
filter_pos += 1 if filter_pos < len # skip closing quote
filter_args << markup.byteslice(arg_start + 1, filter_pos - arg_start - 2)
elsif b && (ByteTables::DIGIT[b] ||
(b == Cursor::DASH && filter_pos + 1 < len && ByteTables::DIGIT[markup.getbyte(filter_pos + 1)]))
# Numeric argument (integer or float, optionally negative)
filter_pos += 1 if b == Cursor::DASH
filter_pos += 1 while filter_pos < len && ByteTables::DIGIT[markup.getbyte(filter_pos)]
if filter_pos < len && markup.getbyte(filter_pos) == Cursor::DOT # float
filter_pos += 1
filter_pos += 1 while filter_pos < len && ByteTables::DIGIT[markup.getbyte(filter_pos)]
end
num_str = markup.byteslice(arg_start, filter_pos - arg_start)
filter_args << (num_str.include?('.') ? num_str.to_f : num_str.to_i)
elsif b && ByteTables::IDENT_START[b]
# Identifier argument — may be a variable lookup or keyword arg
id_start = filter_pos
filter_pos += 1
while filter_pos < len
b2 = markup.getbyte(filter_pos)
break unless ByteTables::IDENT_CONT[b2] || b2 == Cursor::DOT
filter_pos += 1
end
filter_pos += 1 if filter_pos < len && markup.getbyte(filter_pos) == Cursor::QMARK
# Peek past whitespace: if followed by ':', this is a keyword arg → fall to Lexer
kw_check = filter_pos
kw_check += 1 while kw_check < len && markup.getbyte(kw_check) == Cursor::SPACE
return :fall_to_lexer if kw_check < len && markup.getbyte(kw_check) == Cursor::COLON
id_markup = markup.byteslice(id_start, filter_pos - id_start)
filter_args << Expression.parse(id_markup, parse_context.string_scanner, parse_context.expression_cache)
else
return :fall_to_lexer
end
# Skip whitespace after argument
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) == Cursor::SPACE
# Comma: more arguments follow; anything else: done with this filter's args
if filter_pos < len && markup.getbyte(filter_pos) == Cursor::COMMA
filter_pos += 1
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) == Cursor::SPACE
else
break
end
end
[filter_args, filter_pos]
strict_parse_with_error_mode_fallback(markup)
end
def raw
@markup
end
def ==(other)
self.class == other.class && name == other.name && filters == other.filters
end
def markup_context(markup)
markup = @markup if markup.is_a?(Parser)
"in \"{{#{markup}}}\""
end
def lax_parse(markup)
@filters = Const::EMPTY_ARRAY
@filters = []
return unless markup =~ MarkupWithQuotedFragment
name_markup = Regexp.last_match(1)
@@ -345,21 +59,19 @@ module Liquid
next unless f =~ /\w+/
filtername = Regexp.last_match(0)
filterargs = f.scan(FilterArgsRegex).flatten
@filters = [] if @filters.frozen?
@filters << lax_parse_filter_expressions(filtername, filterargs)
end
end
end
def strict_parse(markup)
@filters = Const::EMPTY_ARRAY
@filters = []
p = @parse_context.new_parser(markup)
return if p.look(:end_of_string)
@name = parse_context.safe_parse_expression(p)
while p.consume?(:pipe)
@filters = [] if @filters.frozen?
filtername = p.consume(:id)
filterargs = p.consume?(:colon) ? parse_filterargs(p) : Const::EMPTY_ARRAY
@filters << lax_parse_filter_expressions(filtername, filterargs)
@@ -368,16 +80,13 @@ module Liquid
end
def strict2_parse(markup)
@filters = Const::EMPTY_ARRAY
p = @parse_context.new_parser(markup)
@filters = []
p = markup.is_a?(Parser) ? markup : @parse_context.new_parser(markup)
return if p.look(:end_of_string)
@name = parse_context.safe_parse_expression(p)
while p.consume?(:pipe)
@filters = [] if @filters.frozen?
@filters << strict2_parse_filter_expressions(p)
end
@filters << strict2_parse_filter_expressions(p) while p.consume?(:pipe)
p.consume(:end_of_string)
end
@@ -393,37 +102,28 @@ module Liquid
obj = context.evaluate(@name)
@filters.each do |filter_name, filter_args, filter_kwargs|
if filter_args.empty? && !filter_kwargs
obj = context.invoke_single(filter_name, obj)
elsif !filter_kwargs && filter_args.length == 1
# Single positional arg — most common after no-arg
obj = context.invoke_two(filter_name, obj, context.evaluate(filter_args[0]))
else
filter_args = evaluate_filter_expressions(context, filter_args, filter_kwargs)
obj = context.invoke(filter_name, obj, *filter_args)
end
filter_args = evaluate_filter_expressions(context, filter_args, filter_kwargs)
obj = context.invoke(filter_name, obj, *filter_args)
end
context.apply_global_filter(obj)
end
def render_to_output_buffer(context, output)
# Fast path: no filters and no global filter
obj = if @filters.empty? && context.global_filter.nil?
context.evaluate(@name)
else
render(context)
if @name.is_a?(CallExpression) && @filters.empty? && context.global_filter.nil?
return @name.render_to_output_buffer(context, output)
end
obj = render(context)
render_obj_to_output(obj, output)
output
end
def render_obj_to_output(obj, output)
if obj.instance_of?(String)
output << obj
elsif obj.nil?
case obj
when NilClass
# Do nothing
elsif obj.instance_of?(Array)
when Array
obj.each do |o|
render_obj_to_output(o, output)
end
@@ -437,7 +137,7 @@ module Liquid
end
def disabled_tags
Const::EMPTY_ARRAY
[]
end
private
@@ -446,8 +146,7 @@ module Liquid
filter_args = []
keyword_args = nil
unparsed_args.each do |a|
# Fast check: keyword args must contain ':'
if a.include?(':') && (matches = a.match(JustTagAttributes))
if (matches = a.match(JustTagAttributes))
keyword_args ||= {}
keyword_args[matches[1]] = parse_context.parse_expression(matches[2])
else
@@ -500,24 +199,20 @@ module Liquid
end
def evaluate_filter_expressions(context, filter_args, filter_kwargs)
parsed_args = filter_args.map { |expr| context.evaluate(expr) }
if filter_kwargs
parsed_args = filter_args.map { |expr| context.evaluate(expr) }
parsed_kwargs = {}
filter_kwargs.each do |key, expr|
parsed_kwargs[key] = context.evaluate(expr)
end
parsed_args << parsed_kwargs
parsed_args
elsif filter_args.empty?
Const::EMPTY_ARRAY
else
filter_args.map { |expr| context.evaluate(expr) }
end
parsed_args
end
class ParseTreeVisitor < Liquid::ParseTreeVisitor
def children
[@node.name] + @node.filters.flatten
[@node.name] + @node.filters.flatten.flat_map { |value| value.is_a?(Hash) ? value.values : value }
end
end
end
+24 -140
View File
@@ -10,108 +10,11 @@ module Liquid
new(markup, string_scanner, cache)
end
# Fast parse that skips simple_lookup? check — caller guarantees simple identifier chain
def self.parse_simple(markup, string_scanner = nil, cache = nil)
new(markup, string_scanner, cache, true)
end
# Fast manual scanner replacing markup.scan(VariableParser)
# VariableParser = /\[(?>[^\[\]]+|\g<0>)*\]|[\w-]+\??/
# Splits "product.variants[0].title" into ["product", "variants", "[0]", "title"]
def self.scan_variable(markup)
result = []
pos = 0
len = markup.bytesize
while pos < len
byte = markup.getbyte(pos)
if byte == 91 # '['
# Scan balanced brackets
depth = 1
start = pos
pos += 1
while pos < len && depth > 0
b = markup.getbyte(pos)
depth += 1 if b == 91
depth -= 1 if b == 93
pos += 1
end
if depth == 0
result << markup.byteslice(start, pos - start)
else
# Unbalanced bracket - skip '[' and continue
pos = start + 1
end
elsif byte == 46 # '.'
pos += 1
elsif ByteTables::IDENT_CONT[byte] # [\w-]
start = pos
pos += 1
while pos < len
b = markup.getbyte(pos)
break unless ByteTables::IDENT_CONT[b]
pos += 1
end
# Check trailing '?'
if pos < len && markup.getbyte(pos) == 63
pos += 1
end
result << markup.byteslice(start, pos - start)
else
pos += 1
end
end
result
end
# Check if markup is a simple identifier chain: [\w-]+\??(.[\w-]+\??)*
# Uses C-level match? — 8x faster than Ruby byte scanning
SIMPLE_LOOKUP_RE = /\A[\w-]+\??(?:\.[\w-]+\??)*\z/
def self.simple_lookup?(markup)
markup.bytesize > 0 && markup.match?(SIMPLE_LOOKUP_RE)
end
def initialize(markup, string_scanner = StringScanner.new(""), cache = nil, simple = false)
# Fast path: simple identifier chain without brackets
if simple || self.class.simple_lookup?(markup)
dot_pos = markup.index('.')
if dot_pos.nil?
@name = markup
@lookups = Const::EMPTY_ARRAY
@command_flags = 0
return
end
@name = markup.byteslice(0, dot_pos)
# Build lookups array from remaining dot-separated segments
lookups = []
@command_flags = 0
pos = dot_pos + 1
len = markup.bytesize
while pos < len
seg_start = pos
while pos < len
b = markup.getbyte(pos)
break if b == 46 # '.'
pos += 1
end
seg = markup.byteslice(seg_start, pos - seg_start)
if COMMAND_METHODS.include?(seg)
@command_flags |= 1 << lookups.length
end
lookups << seg
pos += 1 # skip dot
end
@lookups = lookups
return
end
lookups = self.class.scan_variable(markup)
def initialize(markup, string_scanner = StringScanner.new(""), cache = nil)
lookups = markup.scan(VariableParser)
name = lookups.shift
if name&.start_with?('[') && name.end_with?(']')
if name&.start_with?('[') && name&.end_with?(']')
name = Expression.parse(
name[1..-2],
string_scanner,
@@ -123,8 +26,9 @@ module Liquid
@lookups = lookups
@command_flags = 0
@lookups.each_with_index do |lookup, i|
if lookup&.start_with?('[') && lookup.end_with?(']')
@lookups.each_index do |i|
lookup = lookups[i]
if lookup&.start_with?('[') && lookup&.end_with?(']')
lookups[i] = Expression.parse(
lookup[1..-2],
string_scanner,
@@ -136,6 +40,12 @@ module Liquid
end
end
# @api private
def add_lookup(key, command: false)
@command_flags |= 1 << @lookups.length if command && COMMAND_METHODS.include?(key)
@lookups << key
end
def lookup_command?(lookup_index)
@command_flags & (1 << lookup_index) != 0
end
@@ -145,34 +55,26 @@ module Liquid
object = context.find_variable(name)
@lookups.each_index do |i|
lookup = @lookups[i]
key = lookup.instance_of?(String) ? lookup : context.evaluate(lookup)
key = context.evaluate(@lookups[i])
# Cast "key" to its liquid value to enable it to act as a primitive value
# Fast path: strings and integers (most common key types) don't need conversion
unless key.instance_of?(String) || key.instance_of?(Integer)
key = Liquid::Utils.to_liquid_value(key)
end
key = Liquid::Utils.to_liquid_value(key)
# If object is a hash- or array-like object we look for the
# presence of the key and if its available we return it
if accessible?(object, key)
if object.respond_to?(:[]) &&
((object.respond_to?(:key?) && object.key?(key)) ||
(object.respond_to?(:fetch) && key.is_a?(Integer)))
# if its a proc we will replace the entry with the proc
object = context.lookup_and_evaluate(object, key)
# Skip to_liquid for common primitive types (they return self)
unless object.instance_of?(String) || object.instance_of?(Integer) || object.instance_of?(Float) ||
object.instance_of?(Array) || object.instance_of?(Hash) || object.nil?
object = liquidize(object, context)
end
res = context.lookup_and_evaluate(object, key)
object = res.to_liquid
# Some special cases. If the part wasn't in square brackets and
# no key with the same name was found we interpret following calls
# as commands and call them on the current object
elsif lookup_command?(i) && object.respond_to?(key)
object = object.send(key)
unless object.instance_of?(String) || object.instance_of?(Integer) || object.instance_of?(Array) || object.nil?
object = liquidize(object, context)
end
object = object.send(key).to_liquid
# Handle string first/last like ActiveSupport does (returns first/last character)
# ActiveSupport returns "" for empty strings, not nil
@@ -186,6 +88,9 @@ module Liquid
return nil unless context.strict_variables
raise Liquid::UndefinedVariable, "undefined variable #{key}"
end
# If we are dealing with a drop here we have to
object.context = context if object.respond_to?(:context=)
end
object
@@ -195,27 +100,6 @@ module Liquid
self.class == other.class && state == other.state
end
private
# Returns true if +object+ has +key+ accessible via [] lookup.
def accessible?(object, key)
if object.instance_of?(Hash)
object.key?(key)
else
object.respond_to?(:[]) &&
((object.respond_to?(:key?) && object.key?(key)) ||
(object.respond_to?(:fetch) && key.is_a?(Integer)))
end
end
# Calls to_liquid on +object+ and wires up the context reference if needed.
# Skipped for primitive types that return self from to_liquid.
def liquidize(object, context)
object = object.to_liquid
object.context = context if object.respond_to?(:context=)
object
end
protected
def state
+1 -1
View File
@@ -2,5 +2,5 @@
# frozen_string_literal: true
module Liquid
VERSION = "5.12.0"
VERSION = "5.13.0"
end
-62
View File
@@ -1,62 +0,0 @@
# frozen_string_literal: true
# Quick benchmark for autoresearch: measures parse µs, render µs, and object allocations
# Outputs machine-readable metrics to stdout
require_relative 'theme_runner'
RubyVM::YJIT.enable if defined?(RubyVM::YJIT)
runner = ThemeRunner.new
# Warmup — enough iterations for YJIT to fully optimize hot paths
20.times { runner.compile }
20.times { runner.render }
GC.start
GC.compact if GC.respond_to?(:compact)
# Measure parse
parse_times = []
10.times do
GC.disable
t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
runner.compile
t1 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
GC.enable
GC.start
parse_times << (t1 - t0) * 1_000_000 # µs
end
# Measure render
render_times = []
10.times do
GC.disable
t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
runner.render
t1 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
GC.enable
GC.start
render_times << (t1 - t0) * 1_000_000 # µs
end
# Measure object allocations for one parse+render cycle
require 'objspace'
GC.start
GC.disable
before = ObjectSpace.count_objects.values_at(:TOTAL).first - ObjectSpace.count_objects.values_at(:FREE).first
runner.compile
runner.render
after = ObjectSpace.count_objects.values_at(:TOTAL).first - ObjectSpace.count_objects.values_at(:FREE).first
GC.enable
allocations = after - before
parse_us = parse_times.min.round(0)
render_us = render_times.min.round(0)
combined_us = parse_us + render_us
puts "RESULTS"
puts "parse_us=#{parse_us}"
puts "render_us=#{render_us}"
puts "combined_us=#{combined_us}"
puts "allocations=#{allocations}"
+16 -3
View File
@@ -6,14 +6,24 @@
$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
require 'liquid'
require_relative 'support/liquid_spec_adapter_helper'
LiquidSpec.configure do |config|
# Run core Liquid specs
config.features = [:core]
config.missing_features = [
:activesupport,
:lax_parsing,
:shopify_filters,
:shopify_includes,
:shopify_blank,
:shopify_error_handling,
:shopify_error_format,
:shopify_string_access,
]
end
# Compile a template string into a Liquid::Template
LiquidSpec.compile do |ctx, source, options|
options[:error_mode] ||= :strict
ctx[:template] = Liquid::Template.parse(source, **options)
end
@@ -28,9 +38,12 @@ LiquidSpec.render do |ctx, assigns, options|
static_environments: assigns,
registers: registers,
rethrow_errors: options[:strict_errors],
resource_limits: LiquidSpecAdapterHelper.resource_limits(options),
)
context.exception_renderer = options[:exception_renderer] if options[:exception_renderer]
ctx[:template].render(context)
LiquidSpecAdapterHelper.with_frozen_time do
ctx[:template].render(context)
end
end
+16 -4
View File
@@ -6,15 +6,24 @@
$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
require 'liquid'
require_relative 'support/liquid_spec_adapter_helper'
LiquidSpec.configure do |config|
config.features = [:core, :lax_parsing]
config.missing_features = [
:activesupport,
:shopify_filters,
:shopify_includes,
:shopify_blank,
:shopify_error_handling,
:shopify_error_format,
:shopify_string_access,
]
end
# Compile a template string into a Liquid::Template
LiquidSpec.compile do |ctx, source, options|
# Force lax mode
options = options.merge(error_mode: :lax)
# Default to lax mode while still honoring specs that explicitly set error_mode.
options = { error_mode: :lax }.merge(options)
ctx[:template] = Liquid::Template.parse(source, **options)
end
@@ -26,9 +35,12 @@ LiquidSpec.render do |ctx, assigns, options|
static_environments: assigns,
registers: registers,
rethrow_errors: options[:strict_errors],
resource_limits: LiquidSpecAdapterHelper.resource_limits(options),
)
context.exception_renderer = options[:exception_renderer] if options[:exception_renderer]
ctx[:template].render(context)
LiquidSpecAdapterHelper.with_frozen_time do
ctx[:template].render(context)
end
end
+15 -3
View File
@@ -7,14 +7,23 @@
$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
require 'active_support/all'
require 'liquid'
require_relative 'support/liquid_spec_adapter_helper'
LiquidSpec.configure do |config|
# Run core Liquid specs plus ActiveSupport SafeBuffer tests
config.features = [:core, :activesupport]
config.missing_features = [
:lax_parsing,
:shopify_filters,
:shopify_includes,
:shopify_blank,
:shopify_error_handling,
:shopify_error_format,
:shopify_string_access,
]
end
# Compile a template string into a Liquid::Template
LiquidSpec.compile do |ctx, source, options|
options[:error_mode] ||= :strict
ctx[:template] = Liquid::Template.parse(source, **options)
end
@@ -29,9 +38,12 @@ LiquidSpec.render do |ctx, assigns, options|
static_environments: assigns,
registers: registers,
rethrow_errors: options[:strict_errors],
resource_limits: LiquidSpecAdapterHelper.resource_limits(options),
)
context.exception_renderer = options[:exception_renderer] if options[:exception_renderer]
ctx[:template].render(context)
LiquidSpecAdapterHelper.with_frozen_time do
ctx[:template].render(context)
end
end
+14 -2
View File
@@ -13,9 +13,18 @@ end
require 'active_support/all'
require 'liquid'
require_relative 'support/liquid_spec_adapter_helper'
LiquidSpec.configure do |config|
config.features = [:core, :activesupport]
config.missing_features = [
:lax_parsing,
:shopify_filters,
:shopify_includes,
:shopify_blank,
:shopify_error_handling,
:shopify_error_format,
:shopify_string_access,
]
end
# Compile a template string into a Liquid::Template
@@ -33,9 +42,12 @@ LiquidSpec.render do |ctx, assigns, options|
static_environments: assigns,
registers: registers,
rethrow_errors: options[:strict_errors],
resource_limits: LiquidSpecAdapterHelper.resource_limits(options),
)
context.exception_renderer = options[:exception_renderer] if options[:exception_renderer]
ctx[:template].render(context)
LiquidSpecAdapterHelper.with_frozen_time do
ctx[:template].render(context)
end
end
@@ -0,0 +1,24 @@
# frozen_string_literal: true
module LiquidSpecAdapterHelper
extend self
def resource_limits(render_options)
return unless render_options[:resource_limits]
Liquid::ResourceLimits.new({}).tap do |limits|
render_options[:resource_limits].each do |key, value|
limits.public_send(:"#{key}=", value)
end
end
end
def with_frozen_time(&block)
original_tz = ENV['TZ']
ENV['TZ'] = 'UTC'
Liquid::Spec::TimeFreezer.freeze(Liquid::Spec::AdapterRunner::TEST_TIME, &block)
ensure
ENV['TZ'] = original_tz
end
end
+40
View File
@@ -97,6 +97,46 @@ class AssignTest < Minitest::Test
assert_equal(12, assign_score_of('int' => 123, 'str' => 'abcd'))
end
def test_assign_with_valid_identifier_in_strict2
assert_template_result("hello", "{% assign my_var = 'hello' %}{{ my_var }}", error_mode: :strict2)
end
def test_assign_with_hyphen_in_strict2
assert_template_result("hello", "{% assign my-var = 'hello' %}{{ my-var }}", error_mode: :strict2)
end
def test_assign_rejects_parentheses_in_variable_name_in_strict2
assert_raises(Liquid::SyntaxError) do
Liquid::Template.parse("{% assign (a(b(c) = 1234 %}", error_mode: :strict2)
end
end
def test_assign_rejects_brackets_in_variable_name_in_strict2
assert_raises(Liquid::SyntaxError) do
Liquid::Template.parse("{% assign [x.y] = 'hello' %}", error_mode: :strict2)
end
end
def test_assign_rejects_dot_in_variable_name_in_strict2
assert_raises(Liquid::SyntaxError) do
Liquid::Template.parse("{% assign a.b = 'hello' %}", error_mode: :strict2)
end
end
def test_assign_rejects_numeric_variable_name_in_strict2
assert_raises(Liquid::SyntaxError) do
Liquid::Template.parse("{% assign 1abc = 'hello' %}", error_mode: :strict2)
end
end
def test_assign_allows_invalid_names_in_lax
assert_template_result("1234", "{% assign (a(b(c) = 1234 %}{{ self['(a(b(c)'] }}", error_mode: :lax)
end
def test_assign_with_filter_in_strict2
assert_template_result("HELLO", "{% assign my_var = 'hello' | upcase %}{{ my_var }}", error_mode: :strict2)
end
private
class ObjectWrapperDrop < Liquid::Drop
@@ -0,0 +1,85 @@
# frozen_string_literal: true
require 'test_helper'
class BlankBodyErrorHandlingTest < Minitest::Test
COMPARISON_ERROR = 'Liquid error (line 1): comparison of Integer with String failed'
INVALID_INTEGER_ERROR = 'Liquid error (line 1): invalid integer'
def render_inline(source, error_mode:, assigns: {})
Liquid::Template.parse(source, line_numbers: true, error_mode: error_mode).render(assigns, render_errors: true)
end
def assert_render_raises(source, error_mode:, assigns: {}, message: nil)
error = assert_raises(Liquid::ArgumentError) do
Liquid::Template.parse(source, line_numbers: true, error_mode: error_mode).render!(assigns)
end
assert_includes(error.message, message) if message
end
def test_blank_if_body_suppresses_inline_error_text_in_lax_and_strict
[:lax, :strict].each do |mode|
assert_equal('', render_inline('{% if 5 > "x" %}{% endif %}', error_mode: mode))
end
end
def test_blank_unless_body_suppresses_inline_error_text_in_lax_and_strict
[:lax, :strict].each do |mode|
assert_equal('', render_inline('{% unless 5 > "x" %} {% endunless %}', error_mode: mode))
end
end
def test_blank_for_body_suppresses_inline_error_text_in_lax_and_strict
[:lax, :strict].each do |mode|
assert_equal('', render_inline('{% for i in (1..3) offset: xs %}{% endfor %}', error_mode: mode, assigns: { 'xs' => 'bad' }))
end
end
def test_strict2_blank_if_body_shows_inline_error_text
assert_equal(COMPARISON_ERROR, render_inline('{% if 5 > "x" %}{% endif %}', error_mode: :strict2))
end
def test_strict2_whitespace_if_body_shows_inline_error_text
assert_equal(COMPARISON_ERROR, render_inline('{% if 5 > "x" %} {% endif %}', error_mode: :strict2))
end
def test_strict2_assign_if_body_shows_inline_error_text
assert_equal(COMPARISON_ERROR, render_inline('{% if 5 > "x" %}{% assign a = 1 %}{% endif %}', error_mode: :strict2))
end
def test_strict2_comment_if_body_shows_inline_error_text
assert_equal(COMPARISON_ERROR, render_inline('{% if 5 > "x" %}{% comment %}c{% endcomment %}{% endif %}', error_mode: :strict2))
end
def test_strict2_capture_if_body_shows_inline_error_text
assert_equal(COMPARISON_ERROR, render_inline('{% if 5 > "x" %}{% capture c %}text{% endcapture %}{% endif %}', error_mode: :strict2))
end
def test_strict2_blank_unless_body_shows_inline_error_text
assert_equal(COMPARISON_ERROR, render_inline('{% unless 5 > "x" %} {% endunless %}', error_mode: :strict2))
end
def test_strict2_blank_for_body_shows_inline_error_text
assert_equal(INVALID_INTEGER_ERROR, render_inline('{% for i in (1..3) offset: xs %}{% endfor %}', error_mode: :strict2, assigns: { 'xs' => 'bad' }))
end
def test_nonblank_bodies_show_inline_error_text_in_all_modes
[:lax, :strict, :strict2].each do |mode|
assert_equal(COMPARISON_ERROR, render_inline('{% if 5 > "x" %}{% echo 1 %}{% endif %}', error_mode: mode))
assert_equal(COMPARISON_ERROR, render_inline('{% if 5 > "x" %}{{ "" }}{% endif %}', error_mode: mode))
assert_equal(COMPARISON_ERROR, render_inline('{% if 5 > "x" %}{% else %}E{% endif %}', error_mode: mode))
end
end
def test_raised_errors_are_not_swallowed_by_blank_if_body
[:lax, :strict, :strict2].each do |mode|
assert_render_raises('{% if 5 > "x" %}{% endif %}', error_mode: mode, message: 'comparison of Integer with String failed')
end
end
def test_raised_errors_are_not_swallowed_by_blank_for_body
[:lax, :strict, :strict2].each do |mode|
assert_render_raises('{% for i in (1..3) offset: xs %}{% endfor %}', error_mode: mode, assigns: { 'xs' => 'bad' }, message: 'invalid integer')
end
end
end
+36 -1
View File
@@ -6,7 +6,11 @@ class CaptureTest < Minitest::Test
include Liquid
def test_captures_block_content_in_variable
assert_template_result("test string", "{% capture 'var' %}test string{% endcapture %}{{var}}", {})
assert_template_result("test string", "{% capture var %}test string{% endcapture %}{{var}}", {})
end
def test_captures_block_content_in_quoted_variable_in_lax
assert_template_result("test string", "{% capture 'var' %}test string{% endcapture %}{{var}}", {}, error_mode: :lax)
end
def test_capture_with_hyphen_in_variable_name
@@ -49,4 +53,35 @@ class CaptureTest < Minitest::Test
t.render!
assert_equal(9, t.resource_limits.assign_score)
end
def test_capture_with_valid_identifier_in_strict2
assert_template_result("hello", "{% capture my_var %}hello{% endcapture %}{{ my_var }}", error_mode: :strict2)
end
def test_capture_with_hyphen_in_strict2
assert_template_result("hello", "{% capture my-var %}hello{% endcapture %}{{ my-var }}", error_mode: :strict2)
end
def test_capture_rejects_parentheses_in_variable_name_in_strict2
assert_raises(Liquid::SyntaxError) do
Liquid::Template.parse("{% capture (x[y %}hello{% endcapture %}", error_mode: :strict2)
end
end
def test_capture_rejects_dot_in_variable_name_in_strict2
assert_raises(Liquid::SyntaxError) do
Liquid::Template.parse("{% capture a.b %}hello{% endcapture %}", error_mode: :strict2)
end
end
def test_capture_rejects_numeric_variable_name_in_strict2
assert_raises(Liquid::SyntaxError) do
Liquid::Template.parse("{% capture 1abc %}hello{% endcapture %}", error_mode: :strict2)
end
end
def test_capture_allows_invalid_names_in_lax
t = Liquid::Template.parse("{% capture (x[y %}hello{% endcapture %}", error_mode: :lax)
assert_equal("(x[y", t.root.nodelist.first.to)
end
end
+2 -2
View File
@@ -296,8 +296,8 @@ class ContextTest < Minitest::Test
end
def test_access_variable_with_hash_notation
assert_template_result('baz', '{{ ["foo"] }}', { "foo" => "baz" })
assert_template_result('baz', '{{ [bar] }}', { 'foo' => 'baz', 'bar' => 'foo' })
assert_template_result('baz', '{{ foo }}', { "foo" => "baz" })
assert_template_result('baz', '{{ self[bar] }}', { 'foo' => 'baz', 'bar' => 'foo' })
end
def test_access_hashes_with_hash_access_variables
+5 -2
View File
@@ -268,8 +268,11 @@ class ErrorHandlingTest < Minitest::Test
output = Liquid::Template.parse("{% assign x = 0 %}{% if 1 < '2' %}not blank{% assign x = 3 %}{% endif %}{{ x }}").render
assert_equal("Liquid error: comparison of Integer with String failed0", output)
output = Liquid::Template.parse("{% assign x = 0 %}{% if 1 < '2' %}{% assign x = 3 %}{% endif %}{{ x }}").render
assert_equal("0", output)
source = "{% assign x = 0 %}{% if 1 < '2' %}{% assign x = 3 %}{% endif %}{{ x }}"
[:lax, :warn, :strict].each do |mode|
assert_equal("0", Liquid::Template.parse(source, error_mode: mode).render)
end
assert_equal("Liquid error: comparison of Integer with String failed0", Liquid::Template.parse(source, error_mode: :strict2).render)
end
def test_syntax_error_is_raised_with_template_name
+547
View File
@@ -0,0 +1,547 @@
# frozen_string_literal: true
require 'test_helper'
class LetClosureTest < Minitest::Test
def test_value_binding_and_filtered_initializer
assert_render('HELLO:3', "{% let title = product.title | upcase %}{{ title }}:{% let n = 1 | plus: 2 %}{{ n }}", 'product' => { 'title' => 'hello' })
end
def test_initializer_reads_parent_before_installing_shadow
assert_render('231', '{% let n = 1 %}{% let title(step) %}{% let n = n | plus: step %}{{ n }}{% endlet %}{{ title(1) }}{{ title(2) }}{{ n }}')
end
def test_empty_and_parameterless_bodies
assert_render('ok', '{% let empty %}{% endlet %}{% let title() %}ok{% endlet %}{{ empty() }}{{ title() }}')
end
def test_parameters_are_writable_and_fresh
assert_render('23', '{% let title(n,) %}{% set n = n | plus: 1 %}{{ n }}{% endlet %}{{ title(1,) }}{{ title(2) }}')
end
def test_live_captures
assert_render('12', '{% let n = 1 %}{% let title %}{{ n }}{% endlet %}{{ title() }}{% set n = 2 %}{{ title() }}')
end
def test_captured_environments_fall_through_nil_values
context = Liquid::Context.build(environments: [{ 'n' => nil }, { 'n' => 'fallback' }])
source = '{{ n }}/{% let show %}{{ n }}{% endlet %}{{ show() }}'
assert_equal('fallback/fallback', parse(source).render!(context))
end
def test_captured_environments_keep_false_values
context = Liquid::Context.build(environments: [{ 'n' => false }, { 'n' => 'fallback' }])
source = '{{ n }}/{% let show %}{{ n }}{% endlet %}{{ show() }}'
assert_equal('false/false', parse(source).render!(context))
end
def test_captured_environments_use_hash_defaults
context = Liquid::Context.build(environments: Hash.new('fallback'))
source = '{{ n }}/{% let show %}{{ n }}{% endlet %}{{ show() }}'
assert_equal('fallback/fallback', parse(source).render!(context))
end
def test_captured_environments_fall_through_drops
context = Liquid::Context.build(environments: [Liquid::Drop.new, { 'n' => 'fallback' }])
source = '{{ n }}/{% let outer %}{% let inner %}{{ n }}{% endlet %}{{ inner() }}{% endlet %}{{ outer() }}'
assert_equal('fallback/fallback', parse(source).render!(context))
end
def test_captured_static_environments_preserve_fallthrough_and_defaults
context = Liquid::Context.build(environments: [], static_environments: [{ 'n' => nil }, Hash.new('fallback')])
source = '{{ n }}/{% let show %}{{ n }}{% endlet %}{{ show() }}'
assert_equal('fallback/fallback', parse(source).render!(context))
end
def test_captured_local_nil_shadows_environment_values
context = Liquid::Context.build(environments: { 'n' => 'dynamic' }, static_environments: { 'n' => 'static' })
source = '{% let n = nil %}{{ n }}/{% let outer %}{% let inner %}{{ n }}{% endlet %}{{ inner() }}{% endlet %}{{ outer() }}'
assert_equal('/', parse(source).render!(context))
end
def test_captured_local_self_nil_does_not_become_a_self_drop
source = '{% let self = nil %}{% let show %}{% if self == nil %}nil{% endif %}{% endlet %}{{ show() }}'
assert_render('nil', source)
end
def test_captured_environments_remain_live_and_read_only
values = { 'n' => 1 }
source = '{% let show %}{{ n }}{% set n = n | plus: 10 %}{{ n }}{% endlet %}{{ show() }}{% increment n %}{{ show() }}'
assert_render('1111212', source, values)
assert_equal(2, values['n'])
end
def test_strict_captured_environments_stop_at_nil
context = Liquid::Context.build(environments: [{ 'n' => nil }, { 'n' => 'fallback' }])
source = '{{ n }}/{% let show %}{{ n }}{% endlet %}{{ show() }}'
assert_equal('/', parse(source).render!(context, strict_variables: true))
end
def test_strict_captured_environments_reject_missing_keys_before_fallback
source = '{% let outer %}{% let inner %}{{ n }}{% endlet %}{{ inner() }}{% endlet %}{{ outer() }}'
context = Liquid::Context.build(environments: [{}, { 'n' => 'fallback' }])
assert_raises(Liquid::UndefinedVariable) { parse(source).render!(context, strict_variables: true) }
end
def test_strict_captured_environments_reject_hash_defaults
context = Liquid::Context.build(environments: Hash.new('fallback'))
source = '{% let show %}{{ n }}{% endlet %}{{ show() }}'
assert_raises(Liquid::UndefinedVariable) { parse(source).render!(context, strict_variables: true) }
end
def test_strict_captured_self_lookup_can_fall_back_to_self_drop
source = '{% let show %}{{ self["n"] }}{% endlet %}{{ show() }}'
assert_equal('value', parse(source).render!({ 'n' => 'value' }, strict_variables: true))
end
def test_render_in_closures_inherits_only_static_globals
file_system = StubFileSystem.new('snippet' => '{{ g }}/{{ dynamic }}/{{ local }};')
environment = Liquid::Environment.build(file_system: file_system)
context = Liquid::Context.build(environment: environment, environments: { 'dynamic' => 'D' }, static_environments: { 'g' => 'G' })
source = '{% let local = "L" %}{% render "snippet" %}{% let show %}{% render "snippet" %}{% render "snippet", dynamic: dynamic, local: local %}{% endlet %}{{ show() }}'
assert_equal('G//;G//;G/D/L;', parse(source, environment: environment).render!(context))
end
def test_render_in_closures_uses_globals_despite_local_shadows
file_system = StubFileSystem.new('snippet' => '{{ g }}')
environment = Liquid::Environment.build(file_system: file_system)
context = Liquid::Context.build(environment: environment, static_environments: { 'g' => 'G' })
source = '{% render "snippet" %}/{% let g = nil %}{% let outer %}{% let inner %}{{ g }}{% render "snippet" %}{% endlet %}{{ inner() }}{% endlet %}{{ outer() }}'
assert_equal('G/G', parse(source, environment: environment).render!(context))
end
def test_capture_shadows_are_fresh_and_read_only
assert_render('221', '{% let n = 1 %}{% let title %}{% set n = n | plus: 1 %}{{ n }}{% endlet %}{{ title() }}{{ title() }}{{ n }}')
end
def test_set_creates_missing_binding_at_invocation_root
source = '{% let title %}{% for i in (1..2) %}{% set n = n | plus: i %}{% endfor %}{{ n }}{% endlet %}{{ title() }}{{ title() }}'
assert_equal('33', parse(source).render!({}, strict_variables: true))
end
def test_set_searches_owned_control_flow_scopes
source = '{% let title(n) %}{% for i in (1..2) %}{% set n = n | plus: i %}{% set i = i | plus: 10 %}{{ i }},{% endfor %}{{ n }}{% endlet %}{{ title(1) }}'
assert_render('11,12,4', source)
end
def test_set_shadows_capture_at_root_from_inside_loop
source = '{% let n = 1 %}{% let title %}{% for i in (1..2) %}{% set n = n | plus: i %}{% endfor %}{{ n }}{% endlet %}{{ title() }}{{ n }}'
assert_render('41', source)
end
def test_let_shadows_in_loop_and_can_execute_repeatedly
source = '{% let n = 10 %}{% for i in (1..3) %}{% let n = i %}{% set n = n | plus: 1 %}{{ n }}{% endfor %}{{ n }}'
assert_render('23410', source)
end
def test_each_iteration_initializer_reads_outer_binding
source = '{% let n = 10 %}{% for i in (1..2) %}{% let n = n | plus: 1 %}{{ n }},{% endfor %}{{ n }}'
assert_render('11,11,10', source)
end
def test_retained_callbacks_capture_iteration_local_bindings
source = '{% let n = 10 %}{% let first = nil %}{% let second = nil %}{% for i in (1..2) %}{% let n = n | plus: i %}{% let title %}{{ n }}{% endlet %}{% if i == 1 %}{% set first = title %}{% else %}{% set second = title %}{% endif %}{% endfor %}{{ first() }}/{{ second() }}:{{ n }}'
assert_render('11/12:10', source)
end
def test_set_traverses_fresh_iteration_frames_and_persists_at_root
source = '{% let captured = 5 %}{% let title(n) %}{% for i in (1..2) %}{% let step = i %}{% set n = n | plus: step %}{% set missing = missing | plus: step %}{% set captured = captured | plus: step %}{% endfor %}{{ n }}/{{ missing }}/{{ captured }};{% endlet %}{{ title(1) }}{{ title(1) }}{{ captured }}'
assert_equal('4/3/8;4/3/8;5', parse(source).render!({}, strict_variables: true))
end
def test_each_table_row_initializer_reads_outer_binding
source = '{% let n = 10 %}{% tablerow i in (1..2) %}{% let n = n | plus: 1 %}{{ n }}{% endtablerow %}{{ n }}'
assert_render("<tr class=\"row1\">\n<td class=\"col1\">11</td><td class=\"col2\">11</td></tr>\n10", source)
end
def test_table_row_callbacks_retain_iteration_local_bindings
source = '{% let first = nil %}{% let second = nil %}{% tablerow i in (1..2) %}{% let n = i %}{% let title %}{{ n }}{% endlet %}{% if i == 1 %}{% set first = title %}{% else %}{% set second = title %}{% endif %}{% endtablerow %}{{ first() }}/{{ second() }}'
assert_render("<tr class=\"row1\">\n<td class=\"col1\"></td><td class=\"col2\"></td></tr>\n1/2", source)
end
def test_set_traverses_table_row_frames_to_invocation_root
source = '{% let title(n) %}{% tablerow i in (1..2) %}{% let step = i %}{% set n = n | plus: step %}{% set missing = missing | plus: step %}{% endtablerow %}{{ n }}/{{ missing }}{% endlet %}{{ title(1) }}'
assert_render("<tr class=\"row1\">\n<td class=\"col1\"></td><td class=\"col2\"></td></tr>\n4/3", source)
end
def test_empty_loop_branch_has_its_own_let_scope
source = '{% let n = 1 %}{% for i in missing %}{% else %}{% let n = 2 %}{{ n }}{% endfor %}{{ n }}'
assert_render('21', source)
end
def test_table_row_scope_can_shadow_outer_let
source = '{% let n = 10 %}{% tablerow i in (1..2) %}{% let n = i %}{{ n }}{% endtablerow %}{{ n }}'
assert_render("<tr class=\"row1\">\n<td class=\"col1\">1</td><td class=\"col2\">2</td></tr>\n10", source)
end
def test_body_declaration_can_execute_on_every_iteration
assert_render('123', '{% for i in (1..3) %}{% let title %}{{ i }}{% endlet %}{{ title() }}{% endfor %}')
end
def test_set_updates_nearest_owned_let
source = '{% let title(n) %}{% for i in (1..2) %}{% let n = i %}{% set n = n | plus: 10 %}{{ n }},{% endfor %}{{ n }}{% endlet %}{{ title(5) }}'
assert_render('11,12,5', source)
end
def test_nested_closures_have_live_read_only_captures
source = '{% let outer(n) %}{% let inner %}{% set n = n | plus: 10 %}{{ n }}{% endlet %}{% set n = n | plus: 1 %}{{ inner() }}{{ n }}{% endlet %}{{ outer(1) }}{{ outer(2) }}'
assert_render('122133', source)
end
def test_captures_search_all_parents_with_strict_variables
source = '{% let outer %}{% let inner %}{{ title }}{% endlet %}{{ inner() }}{% endlet %}{{ outer() }}'
assert_equal('hello', parse(source).render!({ 'title' => 'hello' }, strict_variables: true))
end
def test_repeated_renders_start_with_fresh_bindings
template = parse('{% set n = n | plus: 1 %}{% let title %}{% set n = n | plus: 1 %}{{ n }}{% endlet %}{{ title() }}{{ n }}')
2.times { assert_equal('21', template.render!) }
end
def test_repeated_renders_use_current_inputs
template = parse('{% let title = title | upcase %}{% let render_title %}{{ title }}{% endlet %}{{ render_title() }}')
assert_equal('FIRST', template.render!('title' => 'first'))
assert_equal('SECOND', template.render!('title' => 'second'))
end
def test_nil_and_false_bindings_are_owned
source = '{% let title %}{% let n = nil %}{% set n = n | default: 3 %}{% let flag = false %}{% set flag = true %}{{ n }}{{ flag }}{% endlet %}{{ title() }}'
assert_render('3true', source)
end
def test_nested_call_arguments_and_filters
source = '{% let title(text) %}[{{ text }}]{% endlet %}{{ title(title("hello")) | upcase }}'
assert_render('[[HELLO]]', source)
end
def test_calls_in_filter_arguments_conditions_and_dynamic_lookups
source = '{% let key %}title{% endlet %}{% if key() == "title" %}{{ product[key()] | append: key() }}{% endif %}'
assert_render('hellotitle', source, 'product' => { 'title' => 'hello' })
end
def test_calls_in_ranges
assert_render('123', '{% let finish %}3{% endlet %}{% for i in (1..finish()) %}{{ i }}{% endfor %}')
end
def test_callable_ranges_share_loop_continuation
source = '{% let finish %}3{% endlet %}{% for i in (1..finish()) limit: 1 %}{{ i }}{% endfor %}{% for i in ( 1 .. finish( ) ) limit: 1 offset: continue %}{{ i }}{% endfor %}'
assert_render('12', source)
end
def test_callable_dynamic_lookups_share_loop_continuation
source = '{% let key %}items{% endlet %}{% for i in groups[key()] limit: 1 %}{{ i }}{% endfor %}{% for i in groups[ key( ) ] limit: 1 offset: continue %}{{ i }}{% endfor %}'
assert_render('12', source, 'groups' => { 'items' => [1, 2, 3] })
end
def test_callable_collection_loop_names_preserve_syntax
source = '{% let identity(value) %}{{ value }}{% endlet %}{% let key %}items{% endlet %}{% for i in identity("a b") %}{{ forloop.name }}{% endfor %}/{% for i in groups[key()] limit: 1 %}{{ forloop.name }}{% endfor %}/{% for i in (1..identity(3)) limit: 1 %}{{ forloop.name }}{% endfor %}'
assert_render('i-identity("a b")/i-groups[key()]/i-(1..identity(3))', source, 'groups' => { 'items' => [1, 2, 3] })
end
def test_distinct_callable_arguments_have_separate_loop_continuations
source = '{% let finish(value) %}{{ value }}{% endlet %}{% for i in (1..finish(2)) limit: 1 %}{{ i }}{% endfor %}{% for i in (1..finish(3)) limit: 1 offset: continue %}{{ i }}{% endfor %}'
assert_render('11', source)
end
def test_callable_value_can_be_passed_through_let
assert_render('hello', '{% let title %}hello{% endlet %}{% let alias = title %}{{ alias() }}')
end
def test_multiline_liquid
source = <<~LIQUID
{% liquid
let n = 1
let title(step)
set n = n | plus: step
echo n
endlet
echo title(2)
echo n
%}
LIQUID
assert_render("31\n", source)
end
def test_same_scope_redeclarations_are_parse_errors
[
'{% let n = 1 %}{% let n = 2 %}',
'{% let n %}{% endlet %}{% let n = 2 %}',
'{% let title(n) %}{% let n = 1 %}{% endlet %}',
'{% let title %}{% let n = 1 %}{% if false %}{% let n = 2 %}{% endif %}{% endlet %}',
].each do |source|
assert_raises(Liquid::SyntaxError, source) { parse(source) }
end
end
def test_invalid_headers_and_complete_input_validation
[
'{% let %}',
'{% let 1 = 2 %}',
'{% let a.b = 2 %}',
'{% let a == 2 %}',
'{% let a = %}',
'{% let a = 1 junk %}',
'{% let a = 1 | %}',
'{% let a(x) = 2 %}',
'{% let a(x,x) %}{% endlet %}',
'{% let a(,x) %}{% endlet %}',
'{% let a(x,,) %}{% endlet %}',
'{% let a(x) junk %}{% endlet %}',
'{% let a %}',
'{% set %}',
'{% set a.b = 1 %}',
'{% set a == 1 %}',
'{% set a = %}',
'{% set a = 1 junk %}',
'{{ title(,1) }}',
'{{ title(1,,) }}',
'{{ title(1) junk }}',
'{{ title(1 }}',
].each do |source|
assert_raises(Liquid::SyntaxError, source) { parse(source) }
end
end
def test_assign_is_rejected_through_nested_control_flow
[
'{% assign x = 1 %}',
'{% if false %}{% assign x = 1 %}{% endif %}',
'{% for i in (1..2) %}{% unless false %}{% assign x = 1 %}{% endunless %}{% endfor %}',
'{% case n %}{% when 1 %}{% assign x = 1 %}{% endcase %}',
"{% liquid\nif false\nassign x = 1\nendif\n%}",
].each do |body|
error = assert_raises(Liquid::SyntaxError) { parse("{% let title %}#{body}{% endlet %}") }
assert_match('assign is not allowed', error.message)
end
end
def test_assign_still_works_outside_closures
assert_render('12', '{% assign n = 1 %}{% let title %}{{ n }}{% endlet %}{{ title() }}{% assign n = 2 %}{{ title() }}')
end
def test_capture_tag_does_not_mutate_parent_bindings
assert_render('localparent', '{% let n = "parent" %}{% let title %}{% capture n %}local{% endcapture %}{{ n }}{% endlet %}{{ title() }}{{ n }}')
end
def test_counters_in_closures_have_fresh_shadows
source = '{% let n = 5 %}{% let title %}{% increment n %}{{ n }}{% decrement n %}{{ n }}{% endlet %}{{ title() }}{{ title() }}{{ n }}'
assert_render('565556555', source)
end
def test_tags_are_unavailable_in_every_other_mode
[:lax, :warn, :strict, :rigid].each do |mode|
['{% let n = 1 %}', '{% let title %}{% endlet %}', '{% set n = 1 %}'].each do |source|
assert_raises(Liquid::SyntaxError, "#{mode}: #{source}") { Liquid::Template.parse(source, error_mode: mode) }
end
end
end
def test_calls_preserve_legacy_parsing_behavior
[:lax, :warn].each do |mode|
template = Liquid::Template.parse('{{ title(1) }}', error_mode: mode)
assert_equal('', template.render!('title' => 'value'))
end
[:strict, :rigid].each do |mode|
assert_raises(Liquid::SyntaxError) { Liquid::Template.parse('{{ title(1) }}', error_mode: mode) }
end
end
def test_effective_template_mode_overrides_environment
environment = Liquid::Environment.build(error_mode: :strict)
assert_equal('1', parse('{% let n = 1 %}{{ n }}', environment: environment).render!)
environment = Liquid::Environment.build(error_mode: :strict2)
assert_raises(Liquid::SyntaxError) { Liquid::Template.parse('{% let n = 1 %}', environment: environment, error_mode: :strict) }
end
def test_effective_partial_mode_enables_feature
environment = Liquid::Environment.build(error_mode: :strict2, file_system: StubFileSystem.new('snippet' => '{% let title %}ok{% endlet %}{{ title() }}'))
template = Liquid::Template.parse('{% render "snippet" %}', environment: environment, error_mode: :strict, include_options_blacklist: [:error_mode])
assert_equal('ok', template.render!)
end
def test_effective_partial_mode_disables_feature
[:lax, :warn, :strict, :rigid].each do |mode|
environment = Liquid::Environment.build(error_mode: mode, file_system: StubFileSystem.new('snippet' => '{% let n = 1 %}'))
template = parse('{% render "snippet" %}', environment: environment, include_options_blacklist: [:error_mode])
assert_raises(Liquid::SyntaxError) { template.render! }
end
end
def test_callable_security
callable = Object.new
def callable.to_liquid
self
end
def callable.call(*)
raise 'must not call arbitrary Ruby objects'
end
[callable, Liquid::Drop.new, 'text', 1, nil].each do |value|
assert_raises(Liquid::ArgumentError) { parse('{{ title(1) }}').render!('title' => value) }
end
end
def test_proc_lookup_is_unchanged_and_receives_no_callable_arguments
values = { 'title' => -> { 'resolved' } }
template = parse('{{ title(1) }}')
assert_raises(Liquid::ArgumentError) { template.render!(values) }
assert_equal('resolved', values['title'])
assert_equal('resolved', parse('{{ title }}').render!(values))
end
def test_proc_captures_keep_existing_lookup_and_memoization
values = { 'title' => -> { 'resolved' } }
assert_render('resolvedresolved', '{% let show %}{{ title }}{% endlet %}{{ show() }}{{ show() }}', values)
assert_equal('resolved', values['title'])
end
def test_nil_proc_captures_are_memoized_before_environment_fallback
values = { 'n' => -> { nil } }
context = Liquid::Context.build(environments: [values, { 'n' => 'fallback' }])
source = '{% let show %}{{ n }}{% endlet %}{{ show() }}/{{ show() }}'
assert_equal('fallback/fallback', parse(source).render!(context))
assert_nil(values['n'])
end
def test_nested_proc_captures_receive_the_current_invocation_context
values = { 'n' => ->(context) { context['argument'] } }
source = '{% let outer %}{% let inner(argument) %}{{ n }}{% endlet %}{{ inner("value") }}{% endlet %}{{ outer() }}'
assert_render('value', source, values)
assert_equal('value', values['n'])
end
def test_argument_count_is_validated
['title()', 'title(1,2)'].each do |call|
assert_raises(Liquid::ArgumentError) { parse("{% let title(n) %}{{ n }}{% endlet %}{{ #{call} }}").render! }
end
end
def test_recursion_uses_liquid_runtime_depth_limit
template = parse('{% let title %}{{ title() }}{% endlet %}{{ title() }}')
assert_raises(Liquid::StackLevelError) { template.render! }
end
def test_output_buffer_and_output_limit
template = parse('{% let title %}hello{% endlet %}{{ title() }}{{ title() }}')
output = +'prefix:'
assert_same(output, template.render!({}, output: output))
assert_equal('prefix:hellohello', output)
template.resource_limits.render_length_limit = 9
assert_raises(Liquid::MemoryError) { template.render! }
end
def test_assignment_limits_include_let_set_and_callable_initializers
['{% let n = "hello" %}', '{% set n = "hello" %}', '{% let title %}hello{% endlet %}{% let n = title() %}'].each do |source|
template = parse(source)
template.resource_limits.assign_score_limit = 4
assert_raises(Liquid::MemoryError) { template.render! }
end
end
def test_render_score_limit_includes_callable_body
template = parse('{% let title %}hello{% endlet %}{{ title() }}{{ title() }}')
template.resource_limits.render_score_limit = 4
assert_raises(Liquid::MemoryError) { template.render! }
end
def test_visitor_sees_initializers_arguments_and_bodies_without_captures
template = parse('{% let n = source | plus: step %}{% let title(x) %}{% set n = other %}{{ x }}{% endlet %}{{ title(product.name) }}')
visitor = Liquid::ParseTreeVisitor.for(template.root).add_callback_for(Liquid::VariableLookup) { |node| node.name } # rubocop:disable Style/SymbolProc
expected = ['source', 'step', 'other', 'x', 'title', 'product']
assert_equal(expected, visitor.visit.flatten.compact)
template.render!('source' => 1, 'step' => 2, 'other' => 3, 'product' => { 'name' => 'hello' })
assert_equal(expected, visitor.visit.flatten.compact)
end
def test_visitor_includes_initializer_filter_keyword_arguments
template = parse('{% let n = source | default: fallback, allow_false: flag %}')
visitor = Liquid::ParseTreeVisitor.for(template.root).add_callback_for(Liquid::VariableLookup) { |node| node.name } # rubocop:disable Style/SymbolProc
assert_equal(['source', 'fallback', 'flag'], visitor.visit.flatten.compact)
end
def test_custom_closure_body_parameters_are_writable_and_fresh
with_custom_tag('parameterized', ParameterizedBody) do
source = '{% let value = 10 %}{% parameterized %}{% set value = value | plus: 1 %}{{ value }}{% endparameterized %}{{ value }}'
assert_render('2310', source)
end
end
def test_custom_closure_body_exports_only_direct_child_body_declarations
source = '{% let n = 1 %}{% compose %}{% set n = n | plus: 1 %}{% let value = "hidden" %}{% if true %}{% let nested %}hidden{% endlet %}{% endif %}{% let item(text) %}{{ n }}:{{ text }}{% endlet %}{% endcompose %}{{ n }}'
with_custom_tag('compose', Compose) do
template = parse(source)
assert_equal('2:a|2:a|1', template.render!)
assert_equal('2:a|2:a|1', template.render!)
end
end
def test_custom_closure_bodies_can_nest
source = '{% compose %}{% let item(text) %}{% compose %}{% let item(inner) %}{{ text }}{{ inner }}{% endlet %}{% endcompose %}{% endlet %}{% endcompose %}'
with_custom_tag('compose', Compose) { assert_render('aa|aa||aa|aa||', source) }
end
def test_multiline_liquid_declarations_are_direct_child_exports
source = "{% compose %}{% liquid\nlet item(text)\necho text\nendlet\n%}{% endcompose %}"
with_custom_tag('compose', Compose) { assert_render('a|a|', source) }
end
def test_custom_closure_body_api_requires_strict2
with_custom_tag('compose', Compose) do
[:lax, :warn, :strict, :rigid].each do |mode|
assert_raises(Liquid::SyntaxError) { Liquid::Template.parse('{% compose %}{% endcompose %}', error_mode: mode) }
end
end
end
def test_custom_closure_body_rejects_nested_assign
with_custom_tag('compose', Compose) do
assert_raises(Liquid::SyntaxError) { parse('{% compose %}{% if false %}{% assign n = 1 %}{% endif %}{% endcompose %}') }
end
end
private
def parse(source, **options)
Liquid::Template.parse(source, **options, error_mode: :strict2)
end
def assert_render(expected, source, values = {})
assert_equal(expected, parse(source).render!(values))
end
class ParameterizedBody < Liquid::Block
def parse(tokens)
@closure_body = parse_closure_body(tokens, parameters: ['value'])
end
def render_to_output_buffer(context, output)
body = @closure_body.bind(context)
body.render_to_output_buffer(context, output, [1])
body.render_to_output_buffer(context, output, [2])
end
end
class Compose < Liquid::Block
def initialize(tag_name, markup, parse_context)
super
@component = Liquid::Template.parse("{{ item('a') }}|", environment: parse_context.environment, error_mode: :strict2)
end
def parse(tokens)
@closure_body = parse_closure_body(tokens)
end
def render_to_output_buffer(context, output)
body = @closure_body.bind(context)
2.times do
exports = body.invoke(context, output: output)
raise Liquid::ArgumentError, 'Unexpected exports' unless exports.keys == ['item']
inner = context.new_isolated_subcontext
inner.merge(exports)
@component.root.render_to_output_buffer(inner, output)
end
output
end
end
end
+120
View File
@@ -0,0 +1,120 @@
# frozen_string_literal: true
require 'test_helper'
class SelfDropContextTest < Minitest::Test
include Liquid
def test_self_drop_passed_as_render_param_preserves_original_scope
source = <<~LIQUID
{%- assign var = 42 -%}
{%- assign s = self -%}
{%- render "snippet1", other_self: s -%}
LIQUID
partials = {
'snippet1' => <<~LIQUID,
{%- assign var = 43 -%}
{{- other_self.var }}|{{ self.var -}}
LIQUID
}
assert_template_result('42|43', source, partials: partials)
end
def test_self_drop_in_render_without_passing_resolves_inner_scope
source = <<~LIQUID
{%- assign var = 42 -%}
{%- render "snippet1" -%}
LIQUID
partials = {
'snippet1' => <<~LIQUID,
{%- assign var = 99 -%}
{{- self.var -}}
LIQUID
}
assert_template_result('99', source, partials: partials)
end
def test_self_drop_passed_to_nested_renders_preserves_each_level
source = <<~LIQUID
{%- assign a = 1 -%}
{%- assign s1 = self -%}
{%- render "snippet1", outer: s1 -%}
LIQUID
partials = {
'snippet1' => <<~LIQUID,
{%- assign a = 2 -%}
{%- assign s2 = self -%}
{%- render "snippet2", outer: outer, middle: s2 -%}
LIQUID
'snippet2' => <<~LIQUID,
{%- assign a = 3 -%}
{{- outer.a }}|{{ middle.a }}|{{ self.a -}}
LIQUID
}
assert_template_result('1|2|3', source, partials: partials)
end
def test_self_drop_reflects_variables_assigned_after_creation
source = <<~LIQUID
{%- assign s = self -%}
{%- assign x = 42 %}{{ s.x -}}
LIQUID
assert_template_result('42', source)
end
def test_self_drop_context_setter_is_undefined
context = Context.new
drop = SelfDrop.new(context)
refute(drop.respond_to?(:context=))
assert_template_result('42', '{{ self.x }}', { 'x' => 42 })
end
def test_self_drop_repeated_lookups_compare_equal_for_same_context
context = Context.new
drop = context.find_variable("self")
cached_drop = context.find_variable("self")
assert_same(drop, cached_drop)
assert_equal(drop.object_id, cached_drop.object_id)
assert_equal(drop, cached_drop)
end
def test_assigned_self_drop_compares_equal_to_itself
assert_template_result('T', '{% assign s = self %}{% if s == s %}T{% else %}F{% endif %}')
end
def test_distinct_self_assignments_compare_equal_for_same_context
assert_template_result('T', '{% assign a = self %}{% assign b = self %}{% if a == b %}T{% else %}F{% endif %}')
end
def test_bare_self_compares_equal_to_bare_self
assert_template_result('T', '{% if self == self %}T{% else %}F{% endif %}')
end
def test_self_drop_with_strict_variables_does_not_raise_for_defined_var
t = Template.parse('{{ self.x }}')
result = t.render({ 'x' => 42 }, strict_variables: true)
assert_equal('42', result)
end
def test_self_drop_with_strict_variables_returns_nil_for_undefined_var
t = Template.parse('{{ self.x }}')
result = t.render({}, strict_variables: true)
assert_equal('', result)
end
def test_self_drop_can_be_passed_as_bare_drop_to_render
t = Template.parse('{{ self.x }}')
drop = SelfDrop.new(Context.new({ 'x' => 42 }))
result = t.render(drop)
assert_equal('42', result)
end
end
+2 -4
View File
@@ -105,10 +105,8 @@ class CycleTagTest < Minitest::Test
error1 = assert_raises(Liquid::SyntaxError) { Template.parse(template1) }
error2 = assert_raises(Liquid::SyntaxError) { Template.parse(template2) }
expected_error = /Liquid syntax error: \[:dot, "."\] is not a valid expression/
assert_match(expected_error, error1.message)
assert_match(expected_error, error2.message)
assert_match(/Liquid syntax error:/, error1.message)
assert_match(/Liquid syntax error: \[:dot, "."\] is not a valid expression/, error2.message)
end
end
+45
View File
@@ -439,4 +439,49 @@ class IncludeTagTest < Minitest::Test
assert_match(/Unexpected character =/, error.message)
end
end
def test_include_for_loop_true_with_for_keyword
with_error_modes(:lax, :strict, :strict2) do
template = Template.parse("{% include 'product' for products %}")
include_node = template.root.nodelist.first
assert(include_node.for_loop?, "Expected for_loop? to be true for 'for' keyword")
end
end
def test_include_for_loop_false_with_with_keyword
with_error_modes(:lax, :strict, :strict2) do
template = Template.parse("{% include 'product' with product %}")
include_node = template.root.nodelist.first
refute(include_node.for_loop?, "Expected for_loop? to be false for 'with' keyword")
end
end
def test_include_for_loop_false_without_keyword
with_error_modes(:lax, :strict, :strict2) do
template = Template.parse("{% include 'header' %}")
include_node = template.root.nodelist.first
refute(include_node.for_loop?, "Expected for_loop? to be false when no keyword")
end
end
def test_include_for_loop_with_alias
with_error_modes(:lax, :strict, :strict2) do
template = Template.parse("{% include 'product' for products as item %}")
include_node = template.root.nodelist.first
assert(include_node.for_loop?, "Expected for_loop? to be true for 'for' with alias")
end
end
def test_include_with_keyword_and_alias
with_error_modes(:lax, :strict, :strict2) do
template = Template.parse("{% include 'product' with products[0] as item %}")
include_node = template.root.nodelist.first
refute(include_node.for_loop?, "Expected for_loop? to be false for 'with' with alias")
end
end
end # IncludeTagTest
@@ -27,4 +27,50 @@ class IncrementTagTest < Minitest::Test
'{%decrement starboard %}',
)
end
def test_increment_strict2_rejects_invalid_variable_name
assert_raises(Liquid::SyntaxError) do
Template.parse('{% increment foo bar %}', error_mode: :strict2)
end
end
def test_increment_strict2_rejects_variable_starting_with_number
assert_raises(Liquid::SyntaxError) do
Template.parse('{% increment 11aa %}', error_mode: :strict2)
end
end
def test_increment_strict2_accepts_valid_variable_name
template = Template.parse('{% increment my-var %}', error_mode: :strict2)
assert_equal('0', template.render)
end
def test_decrement_strict2_rejects_invalid_variable_name
assert_raises(Liquid::SyntaxError) do
Template.parse('{% decrement foo bar %}', error_mode: :strict2)
end
end
def test_decrement_strict2_rejects_variable_starting_with_number
assert_raises(Liquid::SyntaxError) do
Template.parse('{% decrement 11aa %}', error_mode: :strict2)
end
end
def test_decrement_strict2_accepts_valid_variable_name
template = Template.parse('{% decrement my-var %}', error_mode: :strict2)
assert_equal('-1', template.render)
end
def test_increment_strict2_rejects_empty_variable_name
assert_raises(Liquid::SyntaxError) do
Template.parse('{% increment %}', error_mode: :strict2)
end
end
def test_decrement_strict2_rejects_empty_variable_name
assert_raises(Liquid::SyntaxError) do
Template.parse('{% decrement %}', error_mode: :strict2)
end
end
end
+4 -4
View File
@@ -174,16 +174,16 @@ class RenderTagTest < Minitest::Test
def test_increment_is_isolated_between_renders
assert_template_result(
'010',
'{% increment %}{% increment %}{% render "incr" %}',
partials: { 'incr' => '{% increment %}' },
'{% increment port %}{% increment port %}{% render "incr" %}',
partials: { 'incr' => '{% increment port %}' },
)
end
def test_decrement_is_isolated_between_renders
assert_template_result(
'-1-2-1',
'{% decrement %}{% decrement %}{% render "decr" %}',
partials: { 'decr' => '{% decrement %}' },
'{% decrement port %}{% decrement port %}{% render "decr" %}',
partials: { 'decr' => '{% decrement port %}' },
)
end
+5 -5
View File
@@ -55,7 +55,7 @@ class VariableTest < Minitest::Test
def test_expression_with_whitespace_in_square_brackets
assert_template_result('result', "{{ a[ 'b' ] }}", { 'a' => { 'b' => 'result' } })
assert_template_result('result', "{{ a[ [ 'b' ] ] }}", { 'b' => 'c', 'a' => { 'c' => 'result' } })
assert_template_result('result', "{{ a[ self[ 'b' ] ] }}", { 'b' => 'c', 'a' => { 'c' => 'result' } })
end
def test_ignore_unknown
@@ -135,17 +135,17 @@ class VariableTest < Minitest::Test
end
def test_dynamic_find_var
assert_template_result('bar', '{{ [key] }}', { 'key' => 'foo', 'foo' => 'bar' })
assert_template_result('bar', '{{ self[key] }}', { 'key' => 'foo', 'foo' => 'bar' })
end
def test_raw_value_variable
assert_template_result('bar', '{{ [key] }}', { 'key' => 'foo', 'foo' => 'bar' })
assert_template_result('bar', '{{ self[key] }}', { 'key' => 'foo', 'foo' => 'bar' })
end
def test_dynamic_find_var_with_drop
assert_template_result(
'bar',
'{{ [list[settings.zero]] }}',
'{{ self[list[settings.zero]] }}',
{
'list' => ['foo'],
'settings' => SettingsDrop.new("zero" => 0),
@@ -155,7 +155,7 @@ class VariableTest < Minitest::Test
assert_template_result(
'foo',
'{{ [list[settings.zero]["foo"]] }}',
'{{ self[list[settings.zero]["foo"]] }}',
{
'list' => [{ 'foo' => 'bar' }],
'settings' => SettingsDrop.new("zero" => 0),
+7
View File
@@ -26,6 +26,13 @@ class LexerUnitTest < Minitest::Test
)
end
def test_assignment_and_comparison_tokens
assert_equal(
[[:id, 'n'], [:assign, '='], [:number, '1'], [:comparison, '=='], [:number, '2'], [:end_of_string]],
tokenize('n=1==2'),
)
end
def test_comparison
assert_equal(
[[:comparison, '=='], [:comparison, '<>'], [:comparison, 'contains'], [:end_of_string]],
+72
View File
@@ -82,6 +82,78 @@ class ParserUnitTest < Minitest::Test
end
end
def test_assignment_delimiter
parser = new_parser('title = product.title')
assert_equal('title', parser.consume(:id))
assert_equal('=', parser.consume(:assign))
assert_equal('product.title', parser.expression)
parser.consume(:end_of_string)
end
def test_callable_expression_consumes_only_its_arguments
context = ParseContext.new(error_mode: :strict2)
parser = context.new_parser('title(product, nested(1),), next')
expression = context.safe_parse_expression(parser)
assert_instance_of(CallExpression, expression)
assert_equal('title', expression.target.name)
assert_equal('product', expression.arguments.first.name)
assert_instance_of(CallExpression, expression.arguments.last)
assert_equal([1], expression.arguments.last.arguments)
parser.consume(:comma)
assert_equal('next', parser.consume(:id))
parser.consume(:end_of_string)
end
def test_expression_markup_preserves_callable_syntax_without_consuming_attributes
[
['(1 .. finish( ))', '(1..finish())', RangeLookup],
['groups[ key( ) ].items', 'groups[key()].items', VariableLookup],
['identity( "a b", nested(1) )', 'identity("a b",nested(1))', CallExpression],
].each do |source, expected_markup, expected_class|
parser = ParseContext.new(error_mode: :strict2).new_parser("#{source} limit: 1")
value, markup = parser.expression_with_markup
assert_instance_of(expected_class, value)
assert_equal(expected_markup, markup)
assert_equal('limit', parser.consume(:id))
parser.consume(:colon)
assert_equal('1', parser.consume(:number))
parser.consume(:end_of_string)
end
end
def test_expression_markup_preserves_non_callable_loop_names
['items', 'groups[key].items', '(1..finish)', '"a b"'].each do |source|
value, markup = new_parser(source).expression_with_markup
assert_equal(source, value)
assert_equal(source, markup)
end
end
def test_parameters_consume_parentheses_and_optional_trailing_comma
parser = new_parser('(first, second,)')
assert_equal(['first', 'second'], parser.parameters)
parser.consume(:end_of_string)
assert_equal([], new_parser('()').parameters)
end
def test_callable_grammar_uses_effective_partial_mode
environment = Environment.build(error_mode: :strict)
context = ParseContext.new(environment: environment, error_mode: :strict2, include_options_blacklist: [:error_mode])
assert_instance_of(CallExpression, context.safe_parse_expression(context.new_parser('title()')))
context.partial = true
parser = context.new_parser('title()')
assert_equal('title', parser.expression)
assert_raises(SyntaxError) { parser.consume(:end_of_string) }
end
def test_callable_grammar_is_explicitly_disabled_in_other_modes
[:lax, :warn, :strict, :rigid].each do |mode|
parser = ParseContext.new(error_mode: mode).new_parser('title(1)')
assert_equal('title', parser.expression)
assert_raises(SyntaxError) { parser.consume(:end_of_string) }
end
end
private
def new_parser(str)
-27
View File
@@ -48,33 +48,6 @@ class TokenizerTest < Minitest::Test
assert_equal(["{%%}", "}"], tokenize('{%%}}'))
end
# Regression: lone '{' at or near end of string previously caused an infinite
# loop. The stray-{ else branch left `pos` unchanged when no further '{{' or
# '{%' existed, so the outer loop found the same '{' on every iteration.
def test_lone_brace_does_not_loop
assert_equal(["{"], tokenize('{'))
assert_equal(["a{"], tokenize('a{'))
assert_equal(["hello { world {"], tokenize('hello { world {'))
assert_equal(["{ world"], tokenize('{ world'))
assert_equal(["x{y"], tokenize('x{y'))
assert_equal(["{b{c"], tokenize('{b{c'))
end
def test_lone_brace_before_real_token
assert_equal(
["a { b ", "{% if x %}", "yes", "{% endif %}", " c"],
tokenize('a { b {% if x %}yes{% endif %} c'),
)
assert_equal(
["x { ", "{{ var }}", " y"],
tokenize('x { {{ var }} y'),
)
assert_equal(
["{ ", "{{ var }}"],
tokenize('{ {{ var }}'),
)
end
private
def new_tokenizer(source, parse_context: Liquid::ParseContext.new, start_line_number: nil)
-126
View File
@@ -1,126 +0,0 @@
# frozen_string_literal: true
require 'test_helper'
# Tests that the fast-path parser (try_fast_parse) produces the same result as the
# full Lexer → Parser pipeline for every input we expect it to handle.
#
# This protects against silent regressions where a change to try_fast_parse causes it
# to produce different output from the slow path (the existing test suite would still
# pass because the slow path catches it, but correctness would be silently lost).
class VariableFastParseTest < Minitest::Test
include Liquid
EQUIVALENCE_CASES = [
# Simple lookups
"product",
"product.title",
"product.variants.first.title",
# Quoted string literals
"'hello'",
'"hello"',
# Variables with no-arg filters
"product | upcase",
"product | upcase | downcase",
"product | strip | upcase | downcase",
# Variables with single-arg filters
"product | truncate: 50",
"product | plus: 1",
"product | plus: -3",
"product | round: 2",
"product | append: ' world'",
# Variables with multi-arg filters
"product | replace: 'a', 'b'",
"product | pluralize: 'item', 'items'",
"product | slice: 0, 5",
# Chained mixed filters
"product.title | truncate: 50",
"'hello' | append: ' world' | upcase",
"name | prepend: 'Dr. ' | append: ' PhD' | upcase",
# Numeric args
"count | plus: 1.5",
"price | minus: 0.99",
# No whitespace around pipe
"x|upcase",
"x|replace:'a','b'|upcase",
# Leading/trailing whitespace
" product ",
" product.title | upcase ",
].freeze
EQUIVALENCE_CASES.each_with_index do |markup, i|
define_method(:"test_fast_parse_equivalence_#{i.to_s.rjust(2, "0")}") do
lax_ctx = Liquid::ParseContext.new(error_mode: :lax)
strict_ctx = Liquid::ParseContext.new(error_mode: :strict)
lax_var = Liquid::Variable.new(markup, lax_ctx)
strict_var = Liquid::Variable.new(markup, strict_ctx)
assert_equal strict_var.name,
lax_var.name,
"Name mismatch for #{markup.inspect}: " \
"lax=#{lax_var.name.inspect} strict=#{strict_var.name.inspect}"
assert_equal strict_var.filters.length,
lax_var.filters.length,
"Filter count mismatch for #{markup.inspect}: " \
"lax=#{lax_var.filters.inspect} strict=#{strict_var.filters.inspect}"
strict_var.filters.each_with_index do |(s_name, *), i|
l_name = lax_var.filters[i][0]
assert_equal s_name,
l_name,
"Filter name mismatch at index #{i} for #{markup.inspect}"
end
end
end
# Verify the fast path is actually taken for simple variables (i.e. filters is the
# shared frozen EMPTY_ARRAY, not a newly allocated array).
def test_fast_path_taken_for_simple_variable
ctx = Liquid::ParseContext.new(error_mode: :lax)
var = Liquid::Variable.new("product.title", ctx)
assert_same(
Liquid::Const::EMPTY_ARRAY,
var.filters,
"Expected fast path (frozen EMPTY_ARRAY) for simple variable",
)
end
def test_fast_path_taken_for_no_arg_filter
ctx = Liquid::ParseContext.new(error_mode: :lax)
var = Liquid::Variable.new("product | upcase", ctx)
assert_equal(1, var.filters.length)
assert_equal("upcase", var.filters[0][0])
# The no-arg filter tuple should come from NO_ARG_FILTER_CACHE (frozen)
assert_predicate(var.filters[0], :frozen?)
end
def test_fast_path_taken_for_single_arg_filter
ctx = Liquid::ParseContext.new(error_mode: :lax)
var = Liquid::Variable.new("product | truncate: 50", ctx)
assert_equal(1, var.filters.length)
assert_equal("truncate", var.filters[0][0])
assert_equal([50], var.filters[0][1])
end
# Keyword args must fall through to the Lexer — verify the result is still correct.
def test_keyword_arg_falls_to_lexer_and_parses_correctly
ctx = Liquid::ParseContext.new(error_mode: :lax)
var = Liquid::Variable.new("img | img_tag: class: 'hero'", ctx)
assert_equal(1, var.filters.length)
assert_equal("img_tag", var.filters[0][0])
end
# Numeric filter arguments: integers and floats
def test_numeric_filter_args
ctx = Liquid::ParseContext.new(error_mode: :lax)
int_var = Liquid::Variable.new("price | plus: 3", ctx)
assert_equal([3], int_var.filters[0][1])
neg_var = Liquid::Variable.new("price | minus: -1", ctx)
assert_equal([-1], neg_var.filters[0][1])
float_var = Liquid::Variable.new("price | round: 2.5", ctx)
assert_equal([2.5], float_var.filters[0][1])
end
end