Compare commits

...
Author SHA1 Message Date
Chris Pak 731f64deef Improves performance on hot parse and render paths
Splits the render loop in block_body.rb on the loop-invariant check_write
condition. The common case (no resource limits) now pays zero branch cost
per node.

Rewrites truncatewords in standardfilters.rb to scan word positions into a
flat int array and builds the result string only when truncation is confirmed.
No string allocation in the common no-truncation case beyond the array itself.

Simplifies rest_blank? in cursor.rb: replaces manual save/skip/restore of
StringScanner position with [email protected]?(/\S/). exist? does not advance
position; returns nil when no non-whitespace remains; handles EOS correctly.

Removes the nl newline counter from skip_ws in cursor.rb -- all callers
discarded the return value. NL now handled in the same when-branch as the
other whitespace bytes.
2026-04-04 22:14:19 -07:00
Chris Pak 84779f8a28 Removes dead code, tightens idioms, adds clarifying comments
Dead code removed:
  cursor.rb:     parse_for_markup + attr_reader :for_var/collection/reversed
                 (zero callers in lib/; method also incomplete -- no limit/offset)
  for.rb:        Syntax regex, REVERSED_BYTES constant (both unreferenced by
                 lax_parse and strict_parse)
  block_body.rb: BLANK_STRING_REGEX (exact duplicate of WhitespaceOrNothing)

Idioms:
  expression.rb:    parse_number returns nil on failure, not false
                    (Ruby convention for 'no result'; callers use if (num = parse_number))
  block_body.rb:    freeze (idempotent by spec); whitespace_handler marked private;
                    render_node gets rationale comment; redundant comments collapsed
  variable_lookup.rb: COMMAND_METHODS -> %w[]; initialize loop uses each_with_index;
                    removes redundant &. on second clause of &&
  for.rb:           strict2_parse -> alias_method; nil-guard if/else -> ternary;
                    render_else -> ternary; include? checks unconsumed rest only
  condition.rb:     loop/break chain -> while condition.child_relation
  utils.rb:         tightens slice_collection_using_each loop

Comments:
  variable.rb:         backward pipe-walk algorithm; SPACE-only whitespace asymmetry
  cursor.rb:           COMPARISON_OPS identity-map exists for frozen string interning
  if.rb:               include? pre-check is both correctness guard and perf gate
  for.rb:              cursor->regex fallback for limit:/offset: attributes
  strainer_template.rb: __LINE__+1 limitation in module_eval loop
2026-04-04 22:13:43 -07:00
Chris Pak ba11b85875 Decomposes, extracts, and names the important concepts
Decomposes the 211-line try_fast_parse monolith in variable.rb into four
named private methods -- fast_scan_name, fast_resolve_name, fast_scan_filters,
fast_scan_filter_args -- and simplifies simple_variable_markup from a 55-line
byte scanner to a 10-line regex with fast pre-checks.

Generates invoke_single/invoke_two via module_eval in strainer_template.rb
and context.rb instead of duplicating near-identical method bodies. The two
arity variants differ only in parameter lists; generating them makes the
pattern explicit and eliminates copy-paste drift.

Extracts identical 4-line text-token handling block in block_body.rb into
private append_text_token(token, parse_context). The block appeared in
both the stray-{ fallback and the plain-text branch of parse_for_document.

Extracts accessible?(object, key) predicate from the 4-line inline ternary
in variable_lookup.rb that checked hash/array key presence; extracts
liquidize(object, context) from the duplicated to_liquid + context= wiring
that appeared in both the key-found and command-method branches.

Extracts find_in_envs(envs, key, raise_on_not_found:) from
try_variable_find_in_environments in context.rb, which looped over
@environments then @static_environments with identical loop bodies.

Adds 28 fast-path equivalence tests for Variable (variable_fast_parse_test.rb).
2026-04-04 22:13:43 -07:00
Chris Pak 03e5e29b0b Adds ByteTables, moves cursor load order, consolidates byte constants
Adds ByteTables (lib/liquid/byte_tables.rb): four frozen 256-entry boolean
lookup arrays replacing inline byte-range comparisons throughout:
  IDENT_START, IDENT_CONT, DIGIT, WHITESPACE

Moves require 'liquid/cursor' to immediately after byte_tables in liquid.rb.
Cursor has zero Liquid dependencies; loading it early lets every subsequent
file reference Cursor:: constants directly.

Removes all local byte-constant definitions that duplicated Cursor::
  tokenizer.rb   OPEN_CURLEY / CLOSE_CURLEY / PERCENTAGE
  block_body.rb  OPEN_CURLEY_BYTE / PERCENT_BYTE / DASH_BYTE / CLOSE_CURLEY_BYTE
  expression.rb  DOT / DASH / ZERO / NINE / INTEGER_REGEX / FLOAT_REGEX

Replaces inline byte-range comparisons with ByteTables lookups in:
  cursor.rb, variable_lookup.rb, expression.rb, standardfilters.rb

Removes incidental dead code alongside the constant consolidation:
  tokenizer.rb: require 'strscan', unused string_scanner: param,
                @ss = nil, .to_s.to_str, 'tokenize if @source' guard
  block_body.rb: require 'English'
2026-04-04 22:09:14 -07:00
Chris Pak d9c42fd2eb Fixes infinite loop in tokenizer on trailing stray '{'
The stray-{ else branch in tokenize_fast had a nested while loop that could
exit via its condition (when next_open >= len) without advancing pos.
The outer while pos < len loop would then find the same { again forever.

Reproduced by: Liquid::Template.parse('a{') -- hangs indefinitely.

Replaces the nested scan loop with two String#byteindex calls to find the
next '{%' and '{{' directly, then takes the minimum. Always O(n), eliminates
the nested loop entirely, and impossible to leave pos stranded.

Adds regression tests covering the three inputs that previously hung
plus adjacent stray-brace cases.
2026-04-04 21:25:59 -07:00
Tobi LutkeandChris Pak bc60deb671 update autoresearch experiment log 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 17b691ee32 Replace @blocks.each with while loop in If render — avoids block proc allocation per render\n\nResult: {"status":"keep","combined_µs":3496,"parse_µs":2356,"render_µs":1140,"allocations":24530} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak de95af5414 Inline to_liquid_value in If render — avoids one method dispatch per condition evaluation\n\nResult: {"status":"keep","combined_µs":3459,"parse_µs":2318,"render_µs":1141,"allocations":24647} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 6faa62697b Replace simple_lookup? byte scan with match? regex — 8x faster per call, cleaner code\n\nResult: {"status":"keep","combined_µs":3489,"parse_µs":2353,"render_µs":1136,"allocations":24647} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak f39fad8934 Condition#evaluate: skip loop block for simple conditions (no child_relation) — saves 235 allocs\n\nResult: {"status":"keep","combined_µs":3445,"parse_µs":2284,"render_µs":1161,"allocations":24647} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 1aa5854f44 Clean confirmation run: 3,314µs (-55% from main), stable\n\nResult: {"status":"keep","combined_µs":3314,"parse_µs":2203,"render_µs":1111,"allocations":24882} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak feb7036664 update autoresearch docs with current progress 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 111aeed8f1 parse_tag_token without StringScanner: pure byte ops avoid reset(token) overhead, -12% combined\n\nResult: {"status":"keep","combined_µs":3350,"parse_µs":2212,"render_µs":1138,"allocations":24882} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 9cbfae22b0 Clean up tokenizer: remove unused StringScanner setup and regex constants\n\nResult: {"status":"keep","combined_µs":3490,"parse_µs":2331,"render_µs":1159,"allocations":24882} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 89076dbf3a Confirmation run: byteindex tokenizer consistently 3,400-3,600µs\n\nResult: {"status":"keep","combined_µs":3464,"parse_µs":2335,"render_µs":1129,"allocations":24882} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak f2c0fbfa0b Replace StringScanner tokenizer with String#byteindex — 12% faster parse, no regex overhead for delimiter finding\n\nResult: {"status":"keep","combined_µs":3556,"parse_µs":2388,"render_µs":1168,"allocations":24882} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 78550c0e0d Baseline: 3,818µs combined, 24,881 allocs\n\nResult: {"status":"keep","combined_µs":3818,"parse_µs":2722,"render_µs":1096,"allocations":24881} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 68d697d1c5 Skip context.evaluate for String lookup keys in VariableLookup — avoids respond_to? dispatch\n\nResult: {"status":"keep","combined_µs":4103,"parse_µs":2881,"render_µs":1222,"allocations":24881} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak a206932c4a update autoresearch.md with current progress 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 8a39c3aa33 Cache no-arg filter tuples [name, EMPTY_ARRAY] — reuse frozen tuples across templates\n\nResult: {"status":"keep","combined_µs":4147,"parse_µs":2992,"render_µs":1155,"allocations":24881} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 89fcea371e Replace manual blank_string? with regex match — cleaner code\n\nResult: {"status":"keep","combined_µs":4196,"parse_µs":3042,"render_µs":1154,"allocations":25535} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak de8675f8fb Skip to_liquid_value for String/Integer keys in VariableLookup — avoids respond_to? dispatch\n\nResult: {"status":"keep","combined_µs":4131,"parse_µs":2893,"render_µs":1238,"allocations":25535} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak e3ba4aa958 Minor cleanup: optimize expect_id with while loop and early return\n\nResult: {"status":"keep","combined_µs":4184,"parse_µs":2921,"render_µs":1263,"allocations":25535} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak c3e55e92d9 Replace manual scan_dotted_id with regex\n\nResult: {"status":"keep","combined_µs":4121,"parse_µs":2812,"render_µs":1309,"allocations":25535} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 0924e59840 Replace manual scan_quoted_string with regex capture groups\n\nResult: {"status":"keep","combined_µs":4102,"parse_µs":2849,"render_µs":1253,"allocations":25535} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 8d1f030825 Replace manual rest_blank? with regex skip + eos? check\n\nResult: {"status":"keep","combined_µs":4047,"parse_µs":2795,"render_µs":1252,"allocations":25535} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 4896d6d497 Replace manual scan_comparison_op with regex — cleaner and avoids byteslice allocation for op strings\n\nResult: {"status":"keep","combined_µs":4007,"parse_µs":2808,"render_µs":1199,"allocations":25535} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak edd8fabb3a Replace manual scan_fragment/scan_quoted_string_raw/skip_fragment with regex — cleaner, same/better perf\n\nResult: {"status":"keep","combined_µs":4132,"parse_µs":2890,"render_µs":1242,"allocations":25535} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 28ede9c539 Replace manual byte-level scan_number with regex — cleaner code, same performance\n\nResult: {"status":"keep","combined_µs":4184,"parse_µs":2931,"render_µs":1253,"allocations":25535} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 14ac591403 Replace manual byte-level scan_id/skip_id with regex — C-level StringScanner.scan is faster than Ruby-level byte scanning\n\nResult: {"status":"keep","combined_µs":4185,"parse_µs":2943,"render_µs":1242,"allocations":25535} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 3f9b8916b2 Fast-path Hash lookups in VariableLookup#evaluate — skip respond_to? checks for Hash objects\n\nResult: {"status":"keep","combined_µs":4110,"parse_µs":2922,"render_µs":1188,"allocations":25535} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 41de814045 Skip to_liquid/context= for primitives in VariableLookup#evaluate\n\nResult: {"status":"keep","combined_µs":4334,"parse_µs":3062,"render_µs":1272,"allocations":25535} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 91d9a509f9 Fast return for primitive types in find_variable — skip to_liquid and respond_to?(:context=)\n\nResult: {"status":"keep","combined_µs":4225,"parse_µs":3009,"render_µs":1216,"allocations":25535} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 37a4c19405 Skip find_index when only one scope in find_variable — go straight to environments\n\nResult: {"status":"keep","combined_µs":4323,"parse_µs":3055,"render_µs":1268,"allocations":25535} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak dc35b765e8 Skip respond_to?(:context=) for primitive types in find_variable — avoids method lookup overhead\n\nResult: {"status":"keep","combined_µs":4207,"parse_µs":2943,"render_µs":1264,"allocations":25535} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 1dfdce82dc Use EMPTY_ARRAY for empty static_environments in Context — avoids 60 array allocs per render cycle\n\nResult: {"status":"keep","combined_µs":4262,"parse_µs":3079,"render_µs":1183,"allocations":25535} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak bf16d291c9 Lazy @changes hash in Registers — only allocate when a register is actually written\n\nResult: {"status":"keep","combined_µs":4287,"parse_µs":3059,"render_µs":1228,"allocations":25595} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 2da3ae330a Cache block_delimiter strings per tag name — avoids repeated string interpolation\n\nResult: {"status":"keep","combined_µs":4372,"parse_µs":3127,"render_µs":1245,"allocations":25605} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 1800cffd3b Lazy Context init: defer StringScanner and @interrupts array allocation until needed\n\nResult: {"status":"keep","combined_µs":4299,"parse_µs":3057,"render_µs":1242,"allocations":26015} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 02764d28f7 Cache small integer to_s (0-999): avoids 267 Integer#to_s allocations per render cycle\n\nResult: {"status":"keep","combined_µs":4158,"parse_µs":2920,"render_µs":1238,"allocations":26128} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak f08fe63042 Replace split+join in truncatewords with manual word scan — avoids array + string allocations\n\nResult: {"status":"keep","combined_µs":4280,"parse_µs":3009,"render_µs":1271,"allocations":26395} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 588966fd1c Extend fast-path filter parsing to handle comma-separated multi-arg filters (e.g. pluralize: 'item', 'items')\n\nResult: {"status":"keep","combined_µs":4266,"parse_µs":3032,"render_µs":1234,"allocations":26480} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 0e5edcc9da Avoid expr_markup byteslice when name is entire markup string (no whitespace, no filters)\n\nResult: {"status":"keep","combined_µs":4277,"parse_µs":3057,"render_µs":1220,"allocations":27026} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak 3f10ac702c Fast-path single-arg filter parsing: handle quoted strings, numbers, identifiers without Lexer/Parser\n\nResult: {"status":"keep","combined_µs":4427,"parse_µs":3181,"render_µs":1246,"allocations":27235} 2026-04-04 17:42:33 -07:00
Tobi LutkeandChris Pak f31930555e fix rubocop offenses: autocorrect style/layout violations 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak f0ac941ac6 update autoresearch.md with full progress log 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak 19bf49c116 For tag: migrate lax_parse to Cursor with zero-alloc skip_id/expect_id 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak 5204033588 Cursor: add skip_id, expect_id, skip_fragment for zero-alloc scanning 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak b0c9f576af REVERTED: Cursor for For tag adds 148 allocs from scan_id/scan_fragment string creation\n\nResult: {"status":"discard","combined_µs":5049,"parse_us":3765,"render_us":1284,"allocations":29793,"parse_µs":3765,"render_µs":1284} 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak 343ae1da56 remove dead BlockBody.parse_tag_token and If SIMPLE_CONDITION - now in Cursor 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak f42593dc50 introduce Cursor class: centralize byte-level scanning for tag/variable/condition parsing 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak 80edd212a0 add parse_simple to skip simple_lookup? check when caller validates 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak 65d7568403 fast-path VariableLookup init: skip scan_variable for simple identifier chains 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak c2ba6b0676 avoid allocating seen={} hash in Utils.to_s/inspect when not needed 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak 92ca381fb1 update autoresearch.sh: 3-run best-of, skip liquid-spec for speed 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak 1032d57532 optimize Context init: avoid unnecessary array wrapping for environments 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak e1a0e7e716 use frozen EMPTY_ARRAY/EMPTY_HASH for Context @filters/@disabled_tags 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak 1f309b19a7 replace INTEGER_REGEX/FLOAT_REGEX with byte-level parse_number 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak c6617accc5 replace SIMPLE_CONDITION regex with manual byte parser in if/elsif lax_parse 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak cf062e1da2 fast-path slice_collection: skip copy for full Array without offset/limit 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak 06a718cd69 add invoke_two fast path for single-arg filter invocation, avoids splat chain 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak 50888426a1 fast-path find_variable: check top scope first before find_index 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak 1a29584f96 add invoke_single fast path for no-arg filter invocation, avoids splat alloc 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak 9d16eb3d05 fast-path simple if truthiness: use byte scanner before SIMPLE_CONDITION regex 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak f879183a23 update autoresearch.md progress log 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak 34971c0314 replace WhitespaceOrNothing regex with byte-level blank_string? check 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak f24ca042a2 avoid array allocation in parse_tag_token: return tag_name, store markup/newlines as class ivars 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak 4ea4350775 clean up filter parsing: Lexer fallback for args, no-arg fast scan stays 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak aedd2dded6 autoresearch.md: add strategic direction toward single-pass scanner architecture 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak e1f363c72c add security constraint to autoresearch.md, fix strict mode gate 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak 808dad6fab split filter parsing: scan no-arg filters directly, only invoke Lexer when args present 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak c82b6e58ef autoresearch: add autoresearch.md/sh, increase benchmark warmup to 20 iterations 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak b994f22983 extend fast-path to handle quoted string literal variables (262 more fast-pathed) 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak 62877e666c skip filter arg splat for no-arg filters, trim render loop comments 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak 48a2fae5f6 hoist write score check out of render loop: skip increment_write_score when no limits active 2026-04-04 17:42:32 -07:00
Tobi LutkeandChris Pak 27fcb3a4c9 use frozen EMPTY_ARRAY for disabled_tags in Variable 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak 6e9782c5cc return [tag_name, markup, newlines] from parse_tag_token: avoid 2 whitespace string allocs 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak da57336be9 use getbyte dispatch instead of start_with? in parse_for_document 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak 28f29433e4 avoid empty array allocation in evaluate_filter_expressions for no-arg filters 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak 4ff5523713 replace For tag Syntax regex with manual byte-level parser 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak f388c873de expose expression_cache/string_scanner via attr_reader, skip regex in filter args without colon 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak 543c1e1fe7 unified fast-path Variable parsing: handle both plain lookups and filter chains without full Lexer pass for name 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak 885c8df159 fast-path render for filter-less variables: skip render method overhead 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak 8f67d81023 skip TagAttributes scan in for tag when no colon present 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak 01f33e96e8 fast-path simple if conditions: skip ExpressionsAndOperators scan for single conditions 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak 9e6f93a494 replace SIMPLE_VARIABLE regex with byte-level scanner to avoid MatchData 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak dc6e9799e9 fast-path simple variable parsing: skip Lexer/Parser for plain dot-separated lookups 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak 98e29aa4a7 use frozen EMPTY_ARRAY for Variable filters when no filters present 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak a404cf10d0 fast-path variable_lookups: skip mutable string alloc when no dot/bracket follows 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak e73b41f3b6 fast-path String in render_obj_to_output, avoid Utils.to_s dispatch for common case 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak 9d9d094e20 short-circuit parse_number with first-byte check before regex 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak 4c96b5f682 avoid unnecessary strip allocation in Expression.parse, use byteslice for string literals 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak 0b19c0ce44 use equal? for frozen array comparison in Lexer, skip whitespace with \s+ 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak a43d970473 use getbyte instead of string indexing in whitespace_handler and create_variable 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak 3751f95da2 add auto/bench.sh: unit tests + liquid-spec + perf benchmark 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak 6ffd0e2339 replace VariableParser regex scan with manual byte parser in VariableLookup 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak dc7dda2253 replace FullToken regex with manual byte parsing in parse_for_document 2026-04-04 17:33:53 -07:00
Tobi LutkeandChris Pak 2479b26f5c add quick benchmark script for autoresearch 2026-04-04 17:33:53 -07:00
Kevin MenardandGitHub a9c85622dd Merge pull request #2066 from eregon/skip-slow-test
Skip slow test raising many exceptions on non-CRuby
2026-03-26 02:14:10 -04:00
Benoit Daloze 9f4d7e78b8 Skip slow test raising many exceptions on non-CRuby 2026-03-25 17:20:30 +01:00
Kevin MenardandGitHub fd68d076dd Merge pull request #2038 from eregon/truffleruby-ci
Add TruffleRuby in CI
2026-03-25 11:53:01 -04:00
Benoit Daloze 96aa47d13f Add TruffleRuby in CI 2026-03-25 16:46:47 +01:00
Benoit Daloze ad70c5c459 Improve no Symbol leak tests to be more reliable 2026-03-25 16:46:47 +01:00
Benoit Daloze d824de701c Adapt slice filter to work on non-64 bit platforms 2026-03-25 16:46:47 +01:00
Alok SwamyandGitHub dd37353cca Merge pull request #2062 from Shopify/update-setup-ruby
Update ruby/setup-ruby to v1.295.0 for ubuntu-24.04 support
2026-03-19 15:29:47 -04:00
Alok SwamyandClaude Opus 4.6 e80f775f89 Update ruby/setup-ruby from v1.273.0 to v1.295.0
The pinned version (v1.273.0) does not have prebuilt `ruby-head` binaries
for `ubuntu-24.04`, which `ubuntu-latest` now resolves to. This causes CI
to fail with "Unavailable version head for ruby on ubuntu-24.04".

Updating to v1.295.0 picks up ubuntu-24.04 support for all Ruby versions
including head builds.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-19 15:22:56 -04:00
Ian Ker-SeymerandGitHub 59d8d0d22d Add cumulative resource score tracking across partial renders (#2058)
* feat: add cumulative resource score tracking across partial renders

Add cumulative_render_score and cumulative_assign_score counters to
ResourceLimits that accumulate across reset() calls, with optional
cumulative_render_score_limit and cumulative_assign_score_limit to
cap total work across all partial renders.

Also add a reached? check in BlockBody's render loop so that once a
cumulative limit triggers, the parent template stops processing
further nodes.

Bump version to 5.12.0.

* refactor: move cumulative limit enforcement into reset()

Instead of checking reached? in BlockBody's render loop, enforce
cumulative limits in reset() itself. Since reset() is called before
the begin/rescue MemoryError block in Template#render, the raise
propagates to the parent naturally — no changes to BlockBody needed.
2026-03-18 11:54:46 -04:00
Gray GilmoreandGitHub 5fa36267aa Merge pull request #2054 from Shopify/gg-fix-rubocop-offenses
Fix rubocop offenses in test file
2026-03-06 13:28:54 -08:00
Gray Gilmore a72b604680 Fix rubocop offenses in test file 2026-03-06 13:27:05 -08:00
Gray GilmoreandGitHub 3e76244cd2 Merge pull request #2050 from bakura10/squish-filter
Add squish filter
2026-03-06 13:22:18 -08:00
Michaël Gallego d589c51697 Add squish filter 2026-02-19 10:03:52 +09:00
CP ClermontandGitHub d897899f66 Merge pull request #2036 from Shopify/cp-fix-rubocop
Update the specs to new signature and fix CI
2026-01-14 09:14:01 -05:00
Charles-P. ClermontandClaude Opus 4.5 aa817c4cfd Update liquid-spec adapters for new ctx-based API
liquid-spec main changed the adapter API:
- compile block now receives (ctx, source, options) and should store
  the template in ctx[:template]
- render block now receives (ctx, assigns, options) and retrieves
  the template from ctx[:template]

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-01-13 13:06:55 -05:00
Charles-P. Clermont 7d90b524ea Remove on pull_request trigger. It's redundant. 2026-01-12 13:27:53 -05:00
Charles-P. Clermont bbcf8d6ad8 Better matrix CI check names 2026-01-12 13:27:52 -05:00
Charles-P. Clermont 51ff08db7b Fix CI 2026-01-12 13:06:13 -05:00
Tobi LutkeandClaude Opus 4.5 eaa9f215bf Add lax and YJIT liquid-spec adapters
- ruby_liquid_lax.rb: Tests lax parsing mode with :lax_parsing feature
- ruby_liquid_yjit.rb: Tests YJIT + strict mode + ActiveSupport

Matrix results: 4483 matched, 18 different (lax edge cases), 61 skipped

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-01-05 15:11:29 -10:00
Tobi LutkeandClaude Opus 4.5 ccd05e869c Make blank/empty comparisons invariant to ActiveSupport
- Implement liquid_blank? and liquid_empty? methods in Condition
  to emulate ActiveSupport's behavior when it's not loaded
- This ensures templates like `{% if x == blank %}` work identically
  whether ActiveSupport is loaded or not
- Update liquid-spec adapters for new API (ctx parameter)
- Add rake spec task for running liquid-spec matrix

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-01-05 14:54:43 -10:00
Tobias LütkeandGitHub a4a29f3e08 Merge pull request #2034 from Shopify/fix-liquid-spec-without-activesupport
Require liquid-spec to be run on commit automatically and related fixes
2026-01-01 22:32:38 -05:00
Tobi Lutke 0058e4322b Add fail-fast: false to prevent job cancellation 2026-01-01 22:30:14 -05:00
Tobi Lutke 50e1789537 Use liquid-spec feature branch until PR is merged 2026-01-01 22:29:32 -05:00
Tobi Lutke 79a2e042ff Use liquid-spec main branch 2026-01-01 22:24:00 -05:00
Tobi Lutke ddee08fb95 Add activesupport feature to with_active_support adapter
Both adapters now pass with 0 failures:
- ruby_liquid.rb: 4194 passed (skips activesupport and shopify_error_handling specs)
- ruby_liquid_with_active_support.rb: 4203 passed (skips shopify_error_handling specs)
2026-01-01 22:21:47 -05:00
Tobi Lutke 2988f1a500 Update liquid-spec to branch with per-spec required_features support 2026-01-01 22:15:38 -05:00
Tobi Lutke ef13b2dfd5 Fix empty? semantics and string first/last for empty strings
- nil is NOT empty (but IS blank) - matches Shopify production
- String first/last returns '' for empty strings, not nil - matches ActiveSupport
- Add test for nil not being empty
2026-01-01 22:06:22 -05:00
Tobi Lutke b0fb0ad83f Run liquid-spec for all adapters in spec/*.rb 2026-01-01 22:02:42 -05:00
Tobi Lutke ae26cb29ac Disable auto-require for activesupport gem 2026-01-01 22:00:49 -05:00
Tobi Lutke 608a877053 Add spec adapter with ActiveSupport for comparison testing 2026-01-01 22:00:38 -05:00
Tobi Lutke d321adae77 Fix spec adapter for liquid-spec API (template, assigns, options) 2026-01-01 21:59:16 -05:00
Tobi Lutke 53641e19ce Pin liquid-spec to minimum required commit 3d1b492 2026-01-01 21:57:34 -05:00
Tobi Lutke ccd10a986a Pin liquid-spec to main branch 2026-01-01 21:56:39 -05:00
Tobi Lutke f4890de9d5 hm 2026-01-01 21:54:08 -05:00
Tobi Lutke 7e3ccbc188 test 2026-01-01 21:49:33 -05:00
Tobi Lutke e0b46049af Add Ruby 3.4 yjit, 4.0 zjit, and head zjit to CI matrix 2026-01-01 21:44:28 -05:00
Tobi Lutke 19528a9b3f Update CI matrix: remove Ruby 3.0-3.2, add Ruby 4.0 2026-01-01 21:42:47 -05:00
Tobi Lutke 34c274d314 Fix rubocop: rename ruby-liquid.rb to ruby_liquid.rb and add trailing comma 2026-01-01 21:42:11 -05:00
Tobi Lutke 533d470723 Fix spec job to include :spec bundle group 2026-01-01 21:35:13 -05:00
Tobi Lutke 05f9c2a030 Add liquid-spec for conformance testing
- Add liquid-spec gem from GitHub to :spec group
- Create spec/ruby-liquid.rb adapter for the reference implementation
- Add spec job to CI workflow to run liquid-spec tests
2026-01-01 21:34:10 -05:00
Tobi Lutke af58800c16 Update rubocop-shopify to 2.18.0 and fix new offenses 2026-01-01 20:22:19 -05:00
Tobi Lutke 361d1d52b1 Fix rubocop offenses from 1.82 upgrade 2026-01-01 20:20:50 -05:00
Tobi Lutke 391c0df57a Update rubocop to 1.82.0 for Ruby 4.0 support 2026-01-01 20:19:21 -05:00
Tobi Lutke 0ed29760c0 Add benchmark gem for Ruby 4.0 compatibility 2026-01-01 20:18:35 -05:00
Tobi Lutke 0e3548d39e Remove redundant else-clause 2026-01-01 20:16:24 -05:00
Tobi Lutke 33bac87a5c Address liquid-spec issues without ActiveSupport loaded
Implement ActiveSupport-compatible behaviors internally so Liquid works
correctly without ActiveSupport being loaded:

1. String first/last via property access (name.first, name.last)
   - VariableLookup now handles string[0] and string[-1] for first/last

2. String first/last via filters (name | first, name | last)
   - StandardFilters#first and #last now handle strings

3. blank?/empty? comparisons for types without these methods
   - Condition now implements liquid_blank? and liquid_empty? internally
   - blank? matches ActiveSupport: nil, false, empty/whitespace strings,
     empty arrays/hashes are all blank
   - empty? checks length == 0 only (whitespace is NOT empty)

This fixes spec failures for templates like:
- {{ name.first }} / {{ name | first }} on strings
- {% if x == blank %} for whitespace strings, empty hashes/arrays
- {% case ' ' %}{% when blank %} matching whitespace
2026-01-01 20:14:21 -05:00
CP ClermontandGitHub a60a6c0d93 Merge pull request #2027 from Shopify/cp-make-new-tests-serializable-to-liquid-spec
Make new tests serializable to liquid-spec
2025-12-18 15:41:04 -05:00
Charles-P. Clermont bad29caaae Fixup GH action 2025-12-18 15:39:13 -05:00
Charles-P. Clermont cbeff64708 Make new tests serializable to liquid-spec
(No anonymous classes)
2025-12-18 13:56:02 -05:00
Ian Ker-SeymerandGitHub 22e979a6fa Use floating-point format for BigDecimal stringification (#2022)
## Summary

- BigDecimal values now stringify using `to_s("F")` format instead of the default `to_s`

## Why

Ruby's default `BigDecimal#to_s` produces engineering/scientific notation for certain values:

```ruby
BigDecimal("0.00001").to_s      # => "0.1E-4"
BigDecimal("12345678.9").to_s   # => "0.123456789E8"
```

This is rarely the desired output in templates. Using `to_s("F")` produces the expected floating-point format:

```ruby
BigDecimal("0.00001").to_s("F")      # => "0.00001"
BigDecimal("12345678.9").to_s("F")   # => "12345678.9"
```
2025-12-05 15:54:52 -05:00
JuliaandGitHub 735d551168 Merge pull request #2016 from Shopify/jb-method-literals
Preserve literal semantics in strict2 case/when
2025-12-04 09:11:58 -07:00
Julia Boutin fa27bfe6e0 Preserve literal semantics in strict2 case/when
Previously, strict2 case/when used `safe_parse_expression`
to parse when expressions causing `blank`/`empty` to be
treated as string literals (Expression::LITERALS maps 'empty' => ''),
rather than method literals

This caused unexpected behavior:

```
{%- case empty_obj -%}
{%- when empty -%}
  previously: doesn't render (empty_obj == '' is false)
  now: renders (empty_obj.empty? is true)
{%- endcase -%}
```

This commit instead calls `Condition.parse_expression`
with `safe: true`, which will correctly handle `blank`
and `empty`
2025-11-28 15:51:48 -07:00
Guilherme CarreiroandGitHub 32b50ecafe Bump Liquid to 5.11.0 (#2012)
This commit reverts the Inline Snippets tag (#2001) and bumps
Liquid to 5.11. For now, the inclusion of Inline Snippets
in the latest Liquid release is being treated as a bug.

While #2001 does implement the scope contained in RFC#1916,
we need to take a step back to make sure we’re setting our
sights high enough with this feature, and that we’re truly
supporting theme developers in the ways they need.

If you have any feedback, please leave a comment on RFC#1916.

- Liquid Developer Tools
2025-11-19 18:03:23 +01:00
Gray GilmoreandGitHub cbd8a0a2ee Merge pull request #2007 from Shopify/gg-change-key-behavior
Don't raise if no variable found when `using context.key?` with `strict_variables`
2025-11-04 13:01:20 -08:00
Gray Gilmore 9973f3399e Don't raise if no variable found when using context.key?
Previously if you set `strict_variables` to `true` on the context using
`key?('key_name')` would raise a `Liquid::UndefinedVariable` error.
Raising this error makes sense if you're trying to access the variable
directly with something like  `context['key_name']` but by using `key?`
you're safely checking if it exists first.

You should be able to enable `strict_variables` and use `key?` in
combination with each other to ensure code safety.
2025-10-30 15:33:07 -07:00
Guilherme CarreiroandGuilherme Carreiro 248f3a412f Bump to 5.10.0 2025-10-30 12:28:52 +01:00
Guilherme CarreiroandGuilherme Carreiro a16ec56a40 Update error handling for keeping backward-compatibility on error messages in the render tag 2025-10-30 12:00:44 +01:00
Julia BoutinandGuilherme Carreiro 12fd93fbe2 Missing inline snippets should display same error as filebased 2025-10-30 12:00:44 +01:00
Julia BoutinandGuilherme Carreiro 5ceb0e9cec Raise error on invalid snippet name 2025-10-30 12:00:44 +01:00
Julia BoutinandGuilherme Carreiro 98fbd985d8 Remove unneeded read method 2025-10-30 12:00:44 +01:00
Julia BoutinandGuilherme Carreiro ae05ba071c Add liquid_public_docs yard tag to snippet tag 2025-10-30 12:00:44 +01:00
Julia BoutinandGuilherme Carreiro 4205131148 Extract snippet resource scoring logic into assign_score_of 2025-10-30 12:00:44 +01:00
Julia BoutinandGuilherme Carreiro db350c54ff Allow render tag to recognize drops that respond to to_partial 2025-10-30 12:00:44 +01:00
Julia BoutinandGuilherme Carreiro 0cc6cdd553 Remove ... syntax references 2025-10-30 12:00:44 +01:00
Julia BoutinandGuilherme Carreiro 40e45e32ac Raise syntax error on incorrect render identifier type 2025-10-30 12:00:44 +01:00
Julia BoutinandGuilherme Carreiro d4d2237b90 Support prop spreading 2025-10-30 12:00:44 +01:00
Julia BoutinandGuilherme Carreiro 0ceeefba02 Implement resource limits and remove leftover string references 2025-10-30 12:00:44 +01:00
Julia BoutinandGuilherme Carreiro 65fb80a347 Render arguments should maintain correct precedence 2025-10-30 12:00:44 +01:00
Julia BoutinandGuilherme Carreiro 489a03118c Remove inline snippet specific example files 2025-10-30 12:00:44 +01:00
Julia BoutinandGuilherme Carreiro 99116638fd Support with, for, and as inline snippet syntax
This commit updates the render method to share parts
of the snippet and block rendering logic to enable
inline snippets to support `with`, `for`, and `as`
syntax
2025-10-30 12:00:44 +01:00
Julia BoutinandGuilherme Carreiro 9bcfd32e65 Support ... inline snippet syntax 2025-10-30 12:00:44 +01:00
Julia BoutinandGuilherme Carreiro c7ad1c90ca Change inline snippet identifier from string to variable
Currently, snippet files identified by strings. This
PR makes changes to render to allow for new inline
snippets to use variables as identifiers instead
2025-10-30 12:00:44 +01:00
Julia BoutinandGuilherme Carreiro 12bbbc4537 Create SnippetDrop and set in scope 2025-10-30 12:00:44 +01:00
1eca707c4a Update inline snippets syntax
Previously, inline snippets syntax looked a bit
different, they:

- used strings as tag identifiers
- defined tag arguments {% snippet "input" |type| %}

This PR updates snippets to better reflect
the currently proposed syntax

Co-authored-by: Orlando Qiu <[email protected]>
2025-10-30 12:00:44 +01:00
Josh FaiganandGuilherme Carreiro ed9c4e31c4 Introduce new inline snippets tag
Inline snippets will reduce code duplication and
improve the developer experience, eliminating the
need for one-off snippet files
2025-10-30 12:00:44 +01:00
Guilherme CarreiroandGuilherme Carreiro c357f91e0c Bump to 5.9.0 2025-10-27 17:25:36 +01:00
Guilherme CarreiroandGuilherme Carreiro 1f58216f48 Add unit test mixing positional and kwargs arguments 2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGray Gilmore d38795168b Update test/integration/tags/table_row_test.rb
Co-authored-by: Gray Gilmore <[email protected]>
2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGray Gilmore 51f312b220 Update README.md
Co-authored-by: Gray Gilmore <[email protected]>
2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro 44f0429c08 Extract /\w+:0x\h{8}/ regex to UNNAMED_CYCLE_PATTERN constant 2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro e57b7efe4e Simplify render/include tags following PR review feedback 2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro 15430c0770 Add rigid mode to rake benchmark task
The benchmark results show that rigid mode performs a bit better than both strict
and lax modes across most metrics, including tokenization, parsing, rendering,
and their combined operations. Rigid mode consistently delivers the highest
number of iterations per second, with performance differences staying within
1–2% compared to the other modes

```
================================================================================
/opt/rubies/3.4.1/bin/ruby ./performance/benchmark.rb lax

Running benchmark for 20 seconds (with 10 seconds warmup).

ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
           tokenize:   332.000 i/100ms
              parse:    14.000 i/100ms
             render:    61.000 i/100ms
     parse & render:    11.000 i/100ms
Calculating -------------------------------------
           tokenize:      3.325k (± 1.0%) i/s  (300.73 μs/i) -    66.732k in  20.070562s
              parse:    148.166 (± 0.7%) i/s    (6.75 ms/i)  -     2.968k in  20.032971s
             render:    654.428 (± 4.0%) i/s    (1.53 ms/i)  -    13.115k in  20.090452s
     parse & render:    116.108 (± 1.7%) i/s    (8.61 ms/i)  -     2.332k in  20.089221s

================================================================================
/opt/rubies/3.4.1/bin/ruby ./performance/benchmark.rb strict

Running benchmark for 20 seconds (with 10 seconds warmup).

ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
           tokenize:   332.000 i/100ms
              parse:    14.000 i/100ms
             render:    61.000 i/100ms
     parse & render:    11.000 i/100ms
Calculating -------------------------------------
           tokenize:      3.332k (± 0.2%) i/s  (300.14 μs/i) -    66.732k in  20.029095s
              parse:    145.674 (± 0.0%) i/s    (6.86 ms/i)  -     2.926k in  20.086104s
             render:    656.711 (± 4.6%) i/s    (1.52 ms/i)  -    13.115k in  20.050810s
     parse & render:    114.705 (± 0.0%) i/s    (8.72 ms/i)  -     2.299k in  20.043028s

================================================================================
/opt/rubies/3.4.1/bin/ruby ./performance/benchmark.rb rigid

Running benchmark for 20 seconds (with 10 seconds warmup).

ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
           tokenize:   333.000 i/100ms
              parse:    14.000 i/100ms
             render:    62.000 i/100ms
     parse & render:    11.000 i/100ms
Calculating -------------------------------------
           tokenize:      3.334k (± 0.3%) i/s  (299.93 μs/i) -    66.933k in  20.075484s
              parse:    148.349 (± 2.0%) i/s    (6.74 ms/i)  -     2.968k in  20.019775s
             render:    663.752 (± 2.6%) i/s    (1.51 ms/i)  -    13.268k in  20.010303s
     parse & render:    116.464 (± 2.6%) i/s    (8.59 ms/i)  -     2.332k in  20.037869s
liquid$
```
2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro 6aa4041c0f Rename with_error_mode(...) to with_error_modes(...) 2025-10-27 16:33:31 +01:00
Guilherme CarreiroandAlok Swamy d63dd104de Update test/integration/tags/render_tag_test.rb
Co-authored-by: Alok Swamy <[email protected]>
2025-10-27 16:33:31 +01:00
Guilherme CarreiroandAlok Swamy 1733586242 Update test/unit/tags/case_tag_unit_test.rb
Co-authored-by: Alok Swamy <[email protected]>
2025-10-27 16:33:31 +01:00
Guilherme CarreiroandAlok Swamy 5a210fc8f6 Update Rakefile
Co-authored-by: Alok Swamy <[email protected]>
2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro 8d7bb9b6f8 Update History.md (5.8.8 -> 5.9.0) 2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro 4f35764d44 Update History.md 2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro 018442b7d1 Covered changes with more tests, remove redundant cases, and the new with_error_mode(*modes)
Most of changes update this:
```
[:lax, :strict].each do |mode|
  with_error_mode(mode) do
    assert_template_result(...
```

to be this:
```
with_error_mode(:lax, :strict) do
  assert_template_result(...
```
2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro a23c71e40b Fix variable to keep it backward-compatible in strict mode
* lax_parse    - no changes
  * strict_parse - uses the `lax_parse_filter_expressions` (as it was doing before)
  * rigid_parse  - uses the `rigid_parse_filter_expressions`
2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro 738540a601 Update infrastructure that handles parsing switching:
* Remove development helpers from parse context
* Simplify strict_parse_with_error_mode_fallback and update
  documentation
* Add unit tests for `Liquid::Expression` and `Liquid::ParseContext`
* Update test helpers to work better with the `:rigid` mode
2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro 4cd367d971 * Update bin/render script to present an error when no template is passed
* Remove `bin/example.liquid` as it's not executable
2025-10-27 16:33:31 +01:00
Charles-P. ClermontandGuilherme Carreiro 34ab3bdc8e Fix assert_template_result tests not picking up Liquid::Environment.default.error_mode
The `rake test` command gave us the impression that we were running all the tests
on all the error modes, that was false.
2025-10-27 16:33:31 +01:00
Charles-P. ClermontandGuilherme Carreiro 1be1e36a8d Fixup cycle rigid parsing to be backwards compatible 2025-10-27 16:33:31 +01:00
Charles-P. ClermontandGuilherme Carreiro 902ff978a6 Fixup include parsing of with expression 2025-10-27 16:33:31 +01:00
Charles-P. ClermontandGuilherme Carreiro 2ba81b3f1a Fix alias parsing 2025-10-27 16:33:31 +01:00
Charles-P. ClermontandGuilherme Carreiro 5660ce6945 render end of string is not optional 2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro 5ec3008b37 Add rigid parser to tablerow tag 2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro 5248025439 Remove redundant tests where rigid and strict modes have the same
behavior
2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro 0bc69b86b7 No longer test ParseContext directly on RigidModeUnitTest as
now the `safe: true` calls are considered safe

Test the entire template instead
2025-10-27 16:33:31 +01:00
Charles-P. ClermontandGuilherme Carreiro b5fbad08c6 rigid set_attribute in for parsing 2025-10-27 16:33:31 +01:00
Charles-P. ClermontandGuilherme Carreiro b8958f626d Stricter 1:1 refactor of strict_parse for Variable 2025-10-27 16:33:31 +01:00
Charles-P. ClermontandGuilherme Carreiro 94bbf6ca32 Make it possible to safe_parse subsets of expressions
e.g. sometimes you want to only accept strings | lookups.
{% render snippetName %} for example. snippetName is a string right now.
We don't want safe_parse_expression because this would allow snippetName
to be a number, a boolean, etc. But we still want to strict parse this.

So what we'll do is use parse_expression(string, safe: true), this is
an optional opt-in to say "I know what I'm doing". Usually that's
because you're using the output of Parser#something as the input of
parse_expression.

It is true that Parser#expression is subset of Expression.parse, it is
not true of the opposite (e.g. Expression.parse doesn't care about .5
and happily parses that as a global lookup of the variable named "5",
Parser#expression throws for that.)

diff --git a/lib/liquid/condition.rb b/lib/liquid/condition.rb
index e5c321dc..9ab350f0 100644
--- a/lib/liquid/condition.rb
+++ b/lib/liquid/condition.rb
@@ -48,8 +48,8 @@ module Liquid
       @@operators
     end

-    def self.parse_expression(parse_context, markup)
-      @@method_literals[markup] || parse_context.parse_expression(markup)
+    def self.parse_expression(parse_context, markup, safe: false)
+      @@method_literals[markup] || parse_context.parse_expression(markup, safe: safe)
     end

     attr_reader :attachment, :child_condition
diff --git a/lib/liquid/parse_context.rb b/lib/liquid/parse_context.rb
index 1c59fe4a..82cf5768 100644
--- a/lib/liquid/parse_context.rb
+++ b/lib/liquid/parse_context.rb
@@ -51,13 +51,13 @@ module Liquid
     end

     def safe_parse_expression(parser)
-      Expression.safe_parse(parser)
+      Expression.safe_parse(parser, @string_scanner, @expression_cache)
     end

-    def parse_expression(markup)
+    def parse_expression(markup, safe: false)
       # todo(guilherme): remove this once rigid mode is fully using safe_parse_expression
-      # raise Liquid::InternalError, "parse_expression is not supported in rigid mode" if @error_mode == :rigid
-      puts("🚨 parse_expression used in rigid mode") if @error_mode == :rigid
+      # raise Liquid::InternalError, "parse_expression is not supported in rigid mode" if !safe && @error_mode == :rigid
+      puts("🚨 parse_expression used in rigid mode") if !safe && @error_mode == :rigid

       Expression.parse(markup, @string_scanner, @expression_cache)
     end
diff --git a/lib/liquid/tag.rb b/lib/liquid/tag.rb
index 656d2e47..374ee511 100644
--- a/lib/liquid/tag.rb
+++ b/lib/liquid/tag.rb
@@ -72,8 +72,8 @@ module Liquid
       parse_context.safe_parse_expression(parser)
     end

-    def parse_expression(markup)
-      parse_context.parse_expression(markup)
+    def parse_expression(markup, safe: false)
+      parse_context.parse_expression(markup, safe: safe)
     end
   end
 end
diff --git a/lib/liquid/tags/for.rb b/lib/liquid/tags/for.rb
index c2be5db1..3182983b 100644
--- a/lib/liquid/tags/for.rb
+++ b/lib/liquid/tags/for.rb
@@ -93,7 +93,7 @@ module Liquid
       raise SyntaxError, options[:locale].t("errors.syntax.for_invalid_in") unless p.id?('in')

       collection_name  = p.expression
-      @collection_name = parse_expression(collection_name)
+      @collection_name = parse_expression(collection_name, safe: true)

       @name     = "#{@variable_name}-#{collection_name}"
       @reversed = p.id?('reversed')
diff --git a/lib/liquid/tags/if.rb b/lib/liquid/tags/if.rb
index 342374f1..e25d6250 100644
--- a/lib/liquid/tags/if.rb
+++ b/lib/liquid/tags/if.rb
@@ -81,8 +81,8 @@ module Liquid
       block.attach(new_body)
     end

-    def parse_expression(markup)
-      Condition.parse_expression(parse_context, markup)
+    def parse_expression(markup, safe: false)
+      Condition.parse_expression(parse_context, markup, safe: safe)
     end

     def lax_parse(markup)
@@ -124,9 +124,9 @@ module Liquid
     end

     def parse_comparison(p)
-      a = parse_expression(p.expression)
+      a = parse_expression(p.expression, safe: true)
       if (op = p.consume?(:comparison))
-        b = parse_expression(p.expression)
+        b = parse_expression(p.expression, safe: true)
         Condition.new(a, op, b)
       else
         Condition.new(a)
diff --git a/lib/liquid/tags/include.rb b/lib/liquid/tags/include.rb
index 6cdbfd6f..b72a235b 100644
--- a/lib/liquid/tags/include.rb
+++ b/lib/liquid/tags/include.rb
@@ -87,10 +87,11 @@ module Liquid
     def rigid_parse(markup)
       p = @parse_context.new_parser(markup)

-      template_name = p.expression
+      @template_name_expr = safe_parse_expression(p)
       with_or_for = p.id?("for") || p.id?("with") || nil
+      @variable_name_expr = nil
       if with_or_for
-        variable_name = p.expression
+        @variable_name_expr = parse_expression(p.consume(:id), safe: true)
       end

       alias_name = nil
@@ -98,8 +99,6 @@ module Liquid
         alias_name = p.consume(:id)
       end

-      @template_name_expr = parse_expression(template_name)
-      @variable_name_expr = variable_name ? parse_expression(variable_name) : nil
       @alias_name = alias_name

       # optional comma
@@ -109,7 +108,7 @@ module Liquid
       while p.look(:id)
         key = p.consume
         p.consume(:colon)
-        @attributes[key] = parse_expression(p.expression)
+        @attributes[key] = safe_parse_expression(p)
         p.consume?(:comma) # optional comma
       end
     end
diff --git a/lib/liquid/tags/render.rb b/lib/liquid/tags/render.rb
index 89c11063..4f716b24 100644
--- a/lib/liquid/tags/render.rb
+++ b/lib/liquid/tags/render.rb
@@ -88,10 +88,11 @@ module Liquid
     def rigid_parse(markup)
       p = @parse_context.new_parser(markup)

-      template_name = rigid_template_name(p)
+      @template_name_expr = parse_expression(rigid_template_name(p), safe: true)
+      @variable_name_expr = nil
       with_or_for = p.id?("for") || p.id?("with") || nil
       if with_or_for
-        variable_name = p.expression
+        @variable_name_expr = safe_parse_expression(p)
       end

       alias_name = nil
@@ -99,8 +100,6 @@ module Liquid
         alias_name = p.consume(:id)
       end

-      @template_name_expr = parse_expression(template_name)
-      @variable_name_expr = variable_name ? parse_expression(variable_name) : nil
       @alias_name = alias_name
       @is_for_loop = (with_or_for == FOR)

@@ -111,7 +110,7 @@ module Liquid
       while p.look(:id)
         key = p.consume
         p.consume(:colon)
-        @attributes[key] = parse_expression(p.expression)
+        @attributes[key] = safe_parse_expression(p)
         p.consume?(:comma) # optional comma
       end
     end
diff --git a/lib/liquid/variable.rb b/lib/liquid/variable.rb
index 20957065..a3623bc5 100644
--- a/lib/liquid/variable.rb
+++ b/lib/liquid/variable.rb
@@ -65,11 +65,11 @@ module Liquid

       return if p.look(:end_of_string)

-      @name = parse_context.parse_expression(p.expression)
+      @name = parse_context.safe_parse_expression(p)
       while p.consume?(:pipe)
         filtername = p.consume(:id)
         filterargs = p.consume?(:colon) ? parse_filterargs(p) : Const::EMPTY_ARRAY
-        @filters << parse_filter_expressions(filtername, filterargs)
+        @filters << parse_filter_expressions(filtername, filterargs, safe: true)
       end
       p.consume(:end_of_string)
     end
@@ -122,15 +122,15 @@ module Liquid

     private

-    def parse_filter_expressions(filter_name, unparsed_args)
+    def parse_filter_expressions(filter_name, unparsed_args, safe: false)
       filter_args  = []
       keyword_args = nil
       unparsed_args.each do |a|
-        if (matches = a.match(JustTagAttributes))
+        if (matches = a.match(JustTagAttributes)) # we'll need to fix this
           keyword_args           ||= {}
-          keyword_args[matches[1]] = parse_context.parse_expression(matches[2])
+          keyword_args[matches[1]] = parse_context.parse_expression(matches[2], safe: false)
         else
-          filter_args << parse_context.parse_expression(a)
+          filter_args << parse_context.parse_expression(a, safe: safe)
         end
       end
       result = [filter_name, filter_args]
2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro 4f7dafbcac Use safe_parse_expression instead of parse_expression 2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro d77662cd0e Remove unnecessary skips 2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro 65a1c167b3 Add rigid_parse to case/when 2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro e2a15334f0 Fail with trailing elements in the cycle tag 2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro 327790cdce Fix an int the cycle tag, add extra unit tests, and updated parser switcher:
- Fixed NoMethod error with .peek (using look instead)
  - Add friendlier error message when {% cycle %}
2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro 75c95d0791 Fix cycle tag
- `respond_to?` was returning `false` in the parser switcher
     because `rigid_parse` was private
     It was working before because `parse_context` was doing
     the double-parsing thing, but when we removed that, this
     test fairly started breaking
2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro e413104e78 Remove ExpressionParser in favor of ParseContext#safe_parse 2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro d56e3c50f9 Use ExpressionParser in the ParseContext when parsing in :rigid mode 2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro 1bff382ebc Add ExpressionParser and ExpressionConsumer 2025-10-27 16:33:31 +01:00
Charles-P. ClermontandGuilherme Carreiro 6a9e46dd19 Add rigid_parse method to include 2025-10-27 16:33:31 +01:00
Charles-P. ClermontandGuilherme Carreiro e58ac0e75b Add rigid_parse to render 2025-10-27 16:33:31 +01:00
Charles-P. ClermontandGuilherme Carreiro c78bf20010 Add a rigid_parse method to cycle 2025-10-27 16:33:31 +01:00
Charles-P. ClermontandGuilherme Carreiro 6d585a24f1 Add rigid_parse_with_error_context and clarifications 2025-10-27 16:33:31 +01:00
Guilherme CarreiroandGuilherme Carreiro edf06c2882 Introduce :rigid parsing mode 2025-10-27 16:33:31 +01:00
Michael GoandGitHub 1c1e711906 Merge pull request #1995 from Shopify/avoid-regex-allocation
avoid new regex allocation in util functions
2025-10-15 10:43:52 -03:00
Michael Go b5b36665e6 avoid new regex allocation in util functions 2025-10-15 10:08:13 -03:00
Alok SwamyandGitHub 9942592ea8 Merge pull request #1984 from Shopify/update-loop-named-params
Update liquid docs for loop's named params
2025-09-08 06:24:06 -04:00
Alok Swamy 786381c762 Update liquid docs for loop's named params 2025-09-05 16:43:48 -04:00
Gray GilmoreandGitHub 8ad91b5e36 Merge pull request #1952 from Shopify/jus/template.rb-typo-docs
template.rb: Correct typo in docs
2025-09-03 09:03:54 -07:00
iainandGitHub 9bb7fbf123 Merge pull request #1968 from Shopify/shopify-dev-docs-formatting
Inline some information that previously lived at category level
2025-07-03 10:59:58 -04:00
Iain Campbell 8555fd8a20 inline warning previuosly at category level 2025-06-27 16:57:00 -04:00
James MengandGitHub 9bd408f5d0 Merge pull request #1965 from Shopify/jm/bump_liquid
Bump Liquid to 5.8.7
2025-06-09 12:42:27 -07:00
James Meng 79b831d96c Bump Liquid to 5.8.7 2025-06-09 12:38:04 -07:00
James MengandGitHub aebd75e5e8 Merge pull request #1954 from Shopify/jm/doc_body
Expose tag body in the Doc tag
2025-06-09 12:25:19 -07:00
James Meng 7f2f8a226b Add tests for new public methods 2025-06-09 12:24:59 -07:00
James Meng 7b2b25fda1 Fix Doc tag blank? method to check body content
Previously the blank? method always returned true. Now it properly checks
if the body is empty, making the tag behavior consistent with other tags.

Also updated test to use whitespace control for cleaner assertions.
2025-06-09 11:36:18 -07:00
James Meng 8548b96a97 Remove body attr_reader and initiliaze @body instance variable in parse method 2025-06-06 13:08:06 -07:00
upgrade-umpire[bot]andGitHub fc96e66e14 Merge pull request #1953 from Shopify/actions-commit
Applying Merge
2025-06-05 20:15:14 +00:00
James Meng 79a771d724 Add test for doc tag capturing token before enddoc 2025-06-04 19:30:47 -07:00
James Meng 65b1dedac5 Expose tag body in the Doc tag 2025-06-04 12:04:41 -07:00
Ian Ker-SeymerandGitHub f375d7b3aa Add unit test for custom Liquid tag registration (#1960)
Adds EnvironmentTest to verify custom tag registration and rendering.
2025-05-22 11:38:10 -04:00
James MengandGitHub c6f05eaf83 Merge pull request #1955 from Shopify/jm/trim_doc_tag_desc
Remove {{ foo }} and {{ bar }} from `doc` tag description
2025-05-06 15:29:29 -07:00
James Meng eabbd5cb6f Remove {{ foo }} and {{ bar }} from doc tag description 2025-05-06 15:20:35 -07:00
Brian Chen 6b3f6c6fb4 update github actions to commits 2025-04-29 14:04:22 -04:00
Justin D. HarrisandGitHub aefd48e341 template.rb: Correct typo in docs 2025-04-22 09:50:47 -04:00
Marco Concetto RudilossoandGitHub ea864f1177 Merge pull request #1951 from Shopify/actual-revert
Fully revert calling to_s on filter array
2025-04-14 17:49:47 +02:00
Marco Concetto Rudilosso c34dd812c5 Fully revert calling to_s on filter array 2025-04-14 17:36:45 +02:00
Marco Concetto RudilossoandGitHub cc04892e54 Merge pull request #1943 from Shopify/revert-to-s
Revert `Utils.to_s` on all array filters
2025-04-14 17:22:10 +02:00
Marco Concetto Rudilosso f676e699a4 rollback changes to tests 2025-04-14 11:24:01 +02:00
Marco Concetto Rudilosso fc0f7f44c1 use Utils.to_s on property error 2025-04-14 11:17:59 +02:00
Marco Concetto Rudilosso b459eb656f remove test 2025-04-14 11:17:46 +02:00
Marco Concetto Rudilosso c6dcf3e714 add test 2025-04-14 11:17:46 +02:00
Marco Concetto Rudilosso 4dae678c63 Revert "Stringify properties before filtering (#1929)"
This reverts commit f5d6a36574.
2025-04-14 11:17:46 +02:00
Marco Concetto Rudilosso 4c07ff920b Revert "Always stringify properties in all array filters (#1936)"
This reverts commit aa1640035f.
2025-04-14 11:17:46 +02:00
Ian Ker-SeymerandGitHub dbe709c3bf Use to_liquid_value in uniq filter (#1948)
* Use to_liquid_value in uniq filter

* Bump version to 5.8.4
2025-04-09 15:00:01 -04:00
2b75bfaff4 Fix regression when using empty/nil properties with array filters (#1944)
* Make all array filters that use `filter_array` util process empty string and nil correctly

* up version

* fix ordering of checks

* also do it for map

* Do not raise property error

* Gracefully empty property in map filter

---------

Co-authored-by: Marco Concetto Rudilosso <[email protected]>
2025-04-04 11:27:29 -04:00
James MengandGitHub 87bc6e7cfa Merge pull request #1940 from Shopify/jm/update_snippet_url_links
Update snippet URLs in documentation links to point to newly created snippet page
2025-04-01 10:58:54 -07:00
James Meng 7f122aeed2 Fix snippet URLs in documentation links 2025-03-31 17:08:31 -07:00
aa1640035f Always stringify properties in all array filters (#1936)
* Always stringify sum property.

* Add test

* always stringify properties in all array filters

* fix syntax error

* up version

---------

Co-authored-by: Dominic Petrick <[email protected]>
2025-03-19 13:50:01 -04:00
Ian Ker-SeymerandGitHub f5d6a36574 Stringify properties before filtering (#1929) 2025-03-17 17:43:57 -04:00
Ian Ker-SeymerandGitHub c5711c095f Improve docs of date filter (#1920) 2025-03-13 17:36:29 -04:00
James MengandGitHub 284f5fb647 Merge pull request #1928 from Shopify/jm/add_ld_link_doc_tag
Add a link to the LiquidDoc tooling reference in `doc` tag documentation
2025-03-13 09:48:40 -07:00
James MengandGitHub 21432928d0 Use full relative path for hyperlink 2025-03-13 09:47:57 -07:00
James Meng 1783c0c084 Add a link to the LiquidDoc reference in doc tag documentation comment 2025-03-12 15:31:28 -07:00
Guilherme CarreiroandGuilherme Carreiro e38f730c00 Update LiquidDoc documentation 2025-03-07 18:59:59 +01:00
2d0442798b chore: Add RUBYOPT configuration to the tests (#1859)
Co-authored-by: Ian Ker-Seymer <[email protected]>
2025-02-26 14:28:17 -05:00
Guilherme CarreiroandGuilherme Carreiro 6453a0ea48 Implement nodelist in the Doc tag so it may be visited 2025-02-26 13:14:39 +01:00
Guilherme CarreiroandGuilherme Carreiro a398b4cc74 Fix History.md 2025-02-25 08:50:46 +01:00
Guilherme CarreiroandGuilherme Carreiro cca9fe99cf Bump version to 5.8.0 2025-02-25 08:50:46 +01:00
Guilherme CarreiroandGuilherme Carreiro 17d327988d Rename {% doc %} constant strictly validates the abscense of args 2025-02-20 12:37:09 +01:00
Guilherme CarreiroandGuilherme Carreiro f643af4bac Update the implementation to make {% doc %} as strict as {% raw %} 2025-02-20 12:37:09 +01:00
Guilherme CarreiroandGuilherme Carreiro ae8a0a86ac Remove misleading unit test (thank you, @EvilGenius13) 2025-02-20 12:37:09 +01:00
Guilherme CarreiroandGuilherme Carreiro b439d0da53 Update {% doc %} to no longer support nested tags (as {% comment %} does) 2025-02-20 12:37:09 +01:00
Guilherme CarreiroandGuilherme Carreiro 16592cfb8f Add support to LiquidDoc with the new {% doc %} tag 2025-02-20 12:37:09 +01:00
Chris AtLeeandGitHub da4afd4156 Merge pull request #1905 from Shopify/catlee/invalid_utf8
Raise SyntaxError on invalid UTF8 strings in lexer/tokenizer
2025-02-13 09:24:11 -05:00
Chris AtLeeandGitHub 1bb3091208 Merge pull request #1909 from Shopify/catlee/5.7.3
Bump version to 5.7.3
2025-02-13 09:22:47 -05:00
Max StoiberandGitHub 040801b32c Fix array has filters referring to some (#1910) 2025-02-12 17:37:49 +01:00
Chris AtLee 550135c0b9 Raise SyntaxError on invalid UTF8 strings in lexer/tokenizer 2025-02-11 14:23:15 -05:00
Chris AtLee aec966eed7 Bump version to 5.7.3 2025-02-11 14:21:14 -05:00
Michael GoandGitHub bfe29e11be Merge pull request #1907 from Shopify/nested-properties
Fix array filters to not support nested properties
2025-01-31 12:35:03 -04:00
Guilherme Carreiro f9454d8cf3 Fix array filters to not support nested properties 2025-01-31 13:53:17 +01:00
Guilherme CarreiroandGuilherme Carreiro 8dd9279265 Fix release date on History.md 2025-01-24 15:34:48 +01:00
Guilherme CarreiroandGuilherme Carreiro bf1419b8ac Apply the same fix for find_index and has 2025-01-24 08:39:01 +01:00
Guilherme CarreiroandGuilherme Carreiro 5718c4cee2 Fix the find filter to return nil when filtering empty arrays 2025-01-24 08:39:01 +01:00
Guilherme CarreiroandGitHub b0dbc62696 Fix bundle exec rake example (#1900)
* Fix `bundle exec rake example`

* Move 'webrick' from 'benchmark/test' to 'development'
2025-01-23 08:21:38 +01:00
03aafa974c Bump to 5.7.0 (#1894)
* Bump to 5.7.0

* Update 'History.md'

* Update 'Releasing' steps

* Update CONTRIBUTING.md

Co-authored-by: Gray Gilmore <[email protected]>

* Update History.md

Co-authored-by: Ian Ker-Seymer <[email protected]>

---------

Co-authored-by: Gray Gilmore <[email protected]>
Co-authored-by: Ian Ker-Seymer <[email protected]>
2025-01-17 12:10:18 +01:00
Ian Ker-SeymerandGitHub 6372289ba3 Ensure we use InputIterator#each when in join filter (#1898) 2025-01-16 11:36:36 -05:00
Ian Ker-SeymerandGitHub 0ec52a40b5 Use Liquid::Utils.to_s for join filter (#1897) 2025-01-16 11:21:58 -05:00
Ian Ker-SeymerandGitHub 74af735f0e Allow for custom < Hash classes to override #to_s (#1896) 2025-01-16 11:16:13 -05:00
4b65a28722 Implement logic for stringify Hashes to keep compat with 3.4 (#1892)
* Exploring

* Bump to v5.6.5

---------

Co-authored-by: Dominic Petrick <[email protected]>
2025-01-15 16:36:53 -05:00
ecf25ea83d Add the "Releasing" section to CONTRIBUTING.md (#1891)
* Add 'Releasing' workflow to 'CONTRIBUTING.md'

* Update CONTRIBUTING.md

Co-authored-by: Michael Go <[email protected]>

---------

Co-authored-by: Michael Go <[email protected]>
2025-01-14 19:07:11 +01:00
6909570f8e Add find, find_index, has, and reject filters to arrays (#1869)
* Add reject filter #1573

* Add deep search for filter taking in properties #1749

* Update branch with main

* Add `find`, `find_index`, `has`, and `reject` filters to arrays

* Refactor: avoid usage of public_send

---------

Co-authored-by: Anders Søgaard <[email protected]>
Co-authored-by: Anders Søgaard <[email protected]>
2025-01-14 10:33:31 +01:00
Michael GoandGitHub cd9971579f Merge pull request #1889 from Shopify/default-variable-lookup-string-scanner
Add default string_scanner to `Liquid::VariableLookup.parse`
2025-01-13 23:28:11 -04:00
Ian Ker-Seymer 8e37c5e18b Bump to v5.6.4 2025-01-13 21:57:52 -05:00
Ian Ker-Seymer b3f9639e7d Add default string_scanner to Liquid::VariableLookup.parse 2025-01-13 21:56:39 -05:00
Michael GoandGitHub fe3da0e17d Merge pull request #1887 from Shopify/remove-lru-redux
remove lru-redux
2025-01-13 18:37:17 -04:00
Michael Go e200c4544b remove expression caching while rendering 2025-01-13 18:36:30 -04:00
Michael Go 7124540563 remove lru-redux 2025-01-13 18:27:56 -04:00
Michael GoandGitHub 0558bd12c4 Merge pull request #1886 from Shopify/fix-parsing-float-with-leading-point
float has to start with a digit
2025-01-13 17:17:15 -04:00
Michael Go 0639a094a8 bump version to 5.6.2 2025-01-13 17:16:33 -04:00
Michael Go 58777bcd93 float has to start with a digit 2025-01-13 17:13:27 -04:00
Michael GoandGitHub 323951b36f Merge pull request #1844 from Shopify/fast-expression-parse
Faster Expression parser / Tokenizer with StringScanner
2025-01-10 15:34:06 -04:00
Benjamin SteinandGitHub a5b91e83ae Update README.md with default environment (#1879)
Smaller projects, especially when upgrading from earlier versions, may not need different environment scopes, so just adding to the docs.
2025-01-10 14:28:46 -05:00
Michael Go 10114b333e minor version bump 2025-01-10 15:28:02 -04:00
Michael Go a07ae90523 use Ruby Hash as default expression cache 2025-01-10 15:07:57 -04:00
Michael Go 2e236b0a0e fix expression cache to be compatible with Ruby Hash and LruCache 2025-01-10 14:58:23 -04:00
Michael Go bd05dfbd1c use lru_redux getset for simplicity 2025-01-10 14:41:35 -04:00
Michael Go 2a5ecf068f remove lru_redux from Gemfile 2025-01-10 14:38:50 -04:00
Michael Go 0b8e30b819 update unit test to be compatible with new source_location format 2025-01-10 14:37:54 -04:00
Michael Go 97cada61f9 remove Expression2 strscan workaround 2025-01-10 14:23:57 -04:00
Michael Go 7c9b77e8cf remove StringScanner version check in Lexer and Tokenizer 2025-01-07 14:51:29 -04:00
Michael Go d48708ae46 skip expression caching with Expression1 2025-01-07 14:32:46 -04:00
Michael Go 42d822bda9 users can provide optional expression cache 2025-01-07 14:32:46 -04:00
Michael Go 2c7d686690 adding caching to context variable lookup 2025-01-07 14:32:46 -04:00
Michael Go 313d01706a code styling 2025-01-07 14:32:46 -04:00
Michael Go e9f86724f6 code styling 2025-01-07 14:32:46 -04:00
Michael Go 0a17c15289 freeze '-' VariableLookup in Expression 2025-01-07 14:32:46 -04:00
Michael Go 04bd9dbe90 refactor Lexer to only take StringScanner 2025-01-07 14:32:46 -04:00
Michael Go 253ec81b56 don't add a breaking change to Parser constructor 2025-01-07 14:32:46 -04:00
Michael Go e05719fb82 use Kernel Integer parsing for simple Integer expression 2025-01-07 14:32:46 -04:00
Michael Go ac374a208f freeze Expression2 parse results 2025-01-07 14:32:46 -04:00
Michael Go 91be3dd75e keep the expression parse caching per document parsing 2025-01-07 14:32:46 -04:00
Michael Go 40191022e8 initialize StringScanner with input for default 2025-01-07 14:32:46 -04:00
Michael Go 71506ad54e avoid setting up StringScanner for parsing simple numbers 2025-01-07 14:32:46 -04:00
Michael Go cef64e277e early match number expressions with Regex 2025-01-07 14:32:46 -04:00
Michael Go 7c592c1c00 store StringScanner in ParseContext and reuse it through parsing 2025-01-07 14:32:45 -04:00
Michael Go 002e4caea7 refactor Lexer to be static class function 2025-01-07 14:32:07 -04:00
Michael Go 3c16c27ee1 add StringScannerPool for thread safety 2025-01-07 14:32:07 -04:00
Michael Go eff5c5de8e code clean up 2025-01-07 14:32:07 -04:00
Michael Go 19adfbd863 don't force strscan 3.1 2025-01-07 14:32:07 -04:00
Michael Go a672f3836c don't cache literals for Expression parser 2025-01-07 14:32:07 -04:00
Michael Go 92fa334192 don't cache string expressions to be more perfomant 2025-01-07 14:32:07 -04:00
Michael Go 7c8a269b4d fix cycle tag not resetting 2025-01-07 14:32:07 -04:00
Michael Go 0b1dc295ff fix quirky negative sign expression markup parsing 2025-01-07 14:32:07 -04:00
Michael Go 1de6025362 decrease the size of Expression2 LRU cache to store 5 themes 2025-01-07 14:32:07 -04:00
Michael Go 8ecb703d4d use StringScanner to improve Expression Parsing and Tokenizer 2025-01-07 14:32:06 -04:00
Ian Ker-SeymerandGitHub b4667adadf Bump to v5.6.0 (#1876) 2024-12-19 15:01:16 -05:00
Bahar PourazarandGitHub 94e02d765f Merge pull request #1874 from Shopify/bp/bump-version
Bump version patch
2024-12-17 17:39:35 -05:00
Bahar Pourazar 60701f865d Bump version patch 2024-12-17 17:37:00 +00:00
Bahar Pourazar 0c49dd592f bring back to_s.to_str 2024-12-17 17:37:00 +00:00
Bahar PourazarandGitHub c3ac0e0127 Merge pull request #1873 from Shopify/bp/tokenizer-fix
Fix bug in tokenizer with nil source value
2024-12-17 11:43:16 -05:00
Bahar Pourazar a0a4307e7d Fix bug in tokenizer with nil value 2024-12-17 15:34:13 +00:00
Ian Ker-SeymerandGitHub fdd8c714b2 Stop testing against liquid-c (#1868)
* Stop testing against `liquid-c`

* Bump to `v5.6.0.rc2`
2024-12-11 12:23:50 -05:00
Ian Ker-SeymerandGitHub 63583ffe5b Write one value at a time for array variables (#1863)
* Write one value at a time for array variables

* Handle recursive array
2024-12-11 10:16:58 -05:00
Benjamin SehlandGitHub 9a06cedbba Merge pull request #1634 from tjoyal/patch-1
Update homepage url
2024-12-11 09:20:15 -05:00
Ian Ker-Seymer 42b6763546 Bump to v5.6.0.rc1 2024-11-04 15:26:56 -05:00
Michael GoandGitHub e5d18c83bb Merge pull request #1848 from Shopify/env-warn-cleanup
clean up all warnings by using new Environment
2024-11-04 16:17:17 -04:00
Michael Go c77ff68573 clean up all warnings by using new Environment 2024-11-04 16:15:05 -04:00
Ian Ker-SeymerandGitHub b0cba0bfd2 Remove Liquid.cache_classes option (#1847) 2024-11-04 14:41:56 -05:00
Michael GoandGitHub 8d8661349a Merge pull request #1843 from Shopify/empty-array
avoid allocating new empty array
2024-11-04 15:36:46 -04:00
Michael Go 1f3ea7322b avoid allocating new empty array 2024-11-04 15:35:45 -04:00
Michael GoandGitHub 06f44226c0 Merge pull request #1846 from Shopify/env-propgating
propagate Environment on new Context creation
2024-11-04 15:33:50 -04:00
Michael GoandGitHub 4bd22a26dc Merge pull request #1845 from Shopify/remove-tag-registry
remove TagRegistry
2024-11-04 15:33:30 -04:00
Michael Go 3ed54bfdf9 propagate Environment on new Context creation 2024-11-04 15:32:07 -04:00
Michael Go 29986d3704 remove TagRegistry 2024-11-04 15:22:56 -04:00
Michael GoandGitHub 8e40f8050a Merge pull request #1838 from Shopify/quirky-lexer-parsing
fix parsing quirky incomplete expressions
2024-10-30 13:52:47 -03:00
Michael Go ffce6de8bb avoid using StringScanner eos 2024-10-30 13:45:10 -03:00
Michael Go f00670cb01 refactor lexer unit test 2024-10-30 13:44:36 -03:00
Michael Go f6a3e25e2e fix parsing quirky incomplete expressions 2024-10-30 13:44:35 -03:00
Michael GoandGitHub f6ffc37cf2 Merge pull request #1840 from Shopify/fix-lexer-contains-as-id
fix lexer parsing ID 'contains' as comparison
2024-10-30 13:43:48 -03:00
Michael Go 1375a9e4dc fix lexer parsing ID 'contains' as comparison 2024-10-30 13:39:55 -03:00
Michael GoandGitHub c626dfa1a1 Merge pull request #1839 from Shopify/lexer-parse-error-with-utf8
raise syntax error from lexer parser with UTF-8 character
2024-10-30 13:39:05 -03:00
Michael Go 8a9f33a060 raise syntax error from lexer parser with utf8 character 2024-10-29 22:04:37 -03:00
Michael GoandGitHub 1943441361 Merge pull request #1835 from Shopify/fix-multibyte-variable-parsing
fix parsing Variable blockbody with multibyte character
2024-10-28 19:31:21 -03:00
Michael GoandGitHub 36251e640c Merge pull request #1837 from Shopify/lexer-comparison-fix
fix lexer parsing comparison without whitespaces
2024-10-28 19:31:11 -03:00
Michael Go d94293a464 fix lexer parsing comparison without whitespaces 2024-10-28 19:30:12 -03:00
Michael Go 6c13805a60 fix parsing Variable blockbody with multibyte character 2024-10-28 17:33:55 -03:00
Michael GoandGitHub b4196489c2 Merge pull request #1833 from Shopify/fast-variable-parse
Faster Variable BlockBody Matching
2024-10-28 15:28:04 -03:00
Gray GilmoreandGitHub 6d58c41440 Merge pull request #1831 from Shopify/gg-add-named-params-docs
Update liquid docs for named parameters
2024-10-28 09:20:55 -07:00
Michael Go fb6ac72520 use byteslice to create Variable BlockBody 2024-10-25 15:41:35 -03:00
Michael Go cb16219552 faster BlockBody variable matching 2024-10-25 15:22:56 -03:00
Gray Gilmore 8d7ed706f4 Update liquid docs for named parameters
The YARD liquid gem now supports specifying named parameters. For the
core liquid tags and filters this is the only object I could find that
needed to be updated.
2024-10-24 09:56:50 -07:00
Ian Ker-SeymerandGitHub b3553787c8 Speed up the lexer for Ruby 3.4+ (#1832)
* Speed up lexing

* Bump msrv to 3.0 (from 2.7)

* Normalize test for ruby-head compat

* Fix bug when parsing negative numbers
2024-10-23 14:15:33 -04:00
Ian Ker-SeymerandGitHub b233b3d081 Bump to v5.6.0.alpha (#1819) 2024-08-15 16:12:16 -04:00
Alex CocoandGitHub ac91d31268 Merge pull request #1818 from Shopify/tablerow-interrupt
Handle interrupts in table row
2024-08-15 14:04:35 -04:00
Alex Coco 9067e5167a Handle interrupts in table row 2024-08-15 11:56:29 -04:00
Ian Ker-SeymerandGitHub fb6634f454 Add concept of Liquid::Environment (#1815)
* Add concept of `Liquid::World`

* Rename `World` to `Environment`
2024-08-07 15:00:44 -04:00
Ian Ker-SeymerandIan Ker-Seymer a0411e0927 Bump to v5.5.1 2024-07-22 23:25:56 -04:00
Ian Ker-SeymerandGitHub ed421202e2 Merge pull request #1811 from Shopify/marco-cycle-update
Add named? method to Cycle class for checking if the cycle is named
2024-07-16 15:58:21 -04:00
Ian Ker-Seymer d6ca569e8a Require base64 2024-07-16 15:56:47 -04:00
Marco Concetto RudilossoandIan Ker-Seymer d36937d17f Add named? method to Cycle class for checking if the cycle is named 2024-07-16 15:52:25 -04:00
Peter ZhuandGitHub 77bc56a1c2 Merge pull request #1792 from Shopify/centralize_ruby_version
Centralize Ruby Version to `.ruby-version`
2024-04-11 11:12:08 -04:00
Jenny Shen 88d013c8da Use latest compatible Bundler of each Ruby version run 2024-04-05 10:33:27 -04:00
Jenny Shen 36c7fc8e07 Remove ruby version definition in CI/CD
it will be read from .ruby-version
2024-04-05 10:33:06 -04:00
Jenny Shen 2b4810006b Remove TargetRubyVersion in RuboCop config
reads from required_ruby_version in RuboCop 1.61+
2024-04-05 10:23:39 -04:00
Jenny Shen fc4f19471e Update rubocop to 1.61.0 2024-04-05 10:23:37 -04:00
Jenny Shen 8596bb2e38 Commit a .ruby-version 2024-04-05 10:23:05 -04:00
Ian Ker-SeymerandGitHub 6bf18775e7 Merge pull request #1791 from Shopify/v5.5.0
Bump to v5.5.0
2024-03-21 15:50:59 -04:00
Ian Ker-Seymer 56a0b7c42b Bump to v5.5.0 2024-03-21 15:48:54 -04:00
Ian Ker-SeymerandGitHub dba733084e Merge pull request #1760 from mtasaka/nil-nil-comparison-fix
change: make nil_safe_casecmp judge compatible for nil-nil comparison
2024-03-21 15:38:13 -04:00
Michael GoandGitHub 4a4fe3c72a Merge pull request #1781 from Shopify/contextualize-before-to-liquid
update variable's context before invoking its to_liquid
2024-02-12 16:58:54 -04:00
Michael Go 4f35b0bc66 update liquid-c 2024-02-12 16:49:52 -04:00
Michael Go a5e5fab82a update variable's context before invoking its to_liquid 2024-02-12 16:48:20 -04:00
Ian Ker-SeymerandGitHub 02ecaab9d1 Merge pull request #1783 from Shopify/contains-encoding
Fallback to binary comparison when `contains` RHS is  UTF8 encoded
2024-01-31 11:14:01 -05:00
Ian Ker-Seymer 1b2b62964e Allow for binary comparison of incompatible strings 2024-01-30 19:03:05 -05:00
Michael GoandGitHub 9b38a15282 Merge pull request #1776 from Shopify/refactor/invalid-encoding-error
add new TemplateEncodingError
2024-01-12 12:38:34 -04:00
Michael Go 7b25b770af add new TemplateEncodingError 2024-01-12 12:17:20 -04:00
Michael GoandGitHub cf76c0bbec Merge pull request #1775 from Shopify/allow-nil-template-source
allow non-string template source
2024-01-12 10:48:30 -04:00
Michael Go 6a0fe3f7e3 convert template source to string to ensure encoding validity 2024-01-10 16:50:31 -04:00
Michael Go 730ad3684a allow nil template source 2024-01-10 16:32:08 -04:00
Michael GoandGitHub 3ac7e470e6 Merge pull request #1774 from Shopify/check-utf8-validity
check template UTF8 validity before parsing
2024-01-10 15:27:49 -04:00
Michael Go 369a6c55e3 check template UTF8 validity before parsing 2024-01-10 18:58:15 +00:00
Michael GoandGitHub f5ed5404b5 Merge pull request #1773 from Shopify/liquid-tag-whitespace-control-with-comment
don't reset Liquid tag's whitespace control from comment tag
2024-01-05 10:54:32 -04:00
Michael Go a3c837687e don't reset Liquid tag's whitespace control from comment tag 2024-01-05 10:21:33 -04:00
Michael GoandGitHub 96a036372d Merge pull request #1770 from Shopify/revert-invalid-comment-body
don't allow invalid syntax inside comment tag
2024-01-02 11:07:05 -04:00
Mamoru TASAKA 4924822c88 change: make nil_safe_casecmp judge compatible for nil-nil comparison
Ruby returns 0 (not nil) for nil <=> nil, i.e. nil and nil are judged
as equal for comparison, and so returns nil_safe_compare .
ref: https://github.com/Shopify/liquid/pull/1476

To make the behavior of nil_safe_casecmp consistent with
nil_safe_compare , change nil_safe_casecmp so that comparison between
nil <=> nil return 0 (equal).

Also change testsuite to reflect this change.

Fixes #1759 .
2023-12-15 15:09:57 +09:00
Michael Go fc9c338682 refactor: rename comment tag unit test 2023-12-14 17:08:26 -04:00
Michael Go 3c5ad7db61 don't allow invalid syntax inside comment tag 2023-12-14 17:01:03 -04:00
Michael GoandGitHub c658bf970a Merge pull request #1769 from Shopify/comment-tag-with-extra-string
fix parsing comment tag with extra string
2023-12-13 14:16:44 -04:00
Michael Go c618ac1c9f fix parsing comment tag with extra string 2023-12-12 16:23:37 -04:00
Ian Ker-SeymerandGitHub 0f0d5d889f Merge pull request #1750 from IevaGraz/fix/custom_tag_rendering
Fix for custom tag rendering
2023-12-10 11:28:50 -05:00
Michael GoandGitHub 11a1f8e673 Merge pull request #1764 from Shopify/fix-comment-tag-whitespace-control
implement whitespace control to comment tag
2023-12-06 14:30:16 -05:00
Michael Go 24d461a9e3 implement whitespace control to comment tag 2023-12-01 16:04:03 -04:00
Michael GoandGitHub cbb422e5d3 Merge pull request #1763 from Shopify/fix-comment-tag-delim-parsing
fix parsing comment tag delimiter
2023-12-01 15:35:32 -04:00
Michael Go bf0f79f36c fix parsing comment tag delimiter 2023-11-30 13:24:54 -04:00
Michael GoandGitHub cf2787791e Merge pull request #1755 from Shopify/non-parsing-comment
Don't parse nodes inside a comment tag
2023-11-29 14:10:20 -04:00
Michael Go 6a5ebb0e85 fix parsing nested comment tag with extra strings 2023-11-11 12:19:44 -04:00
Michael Go 6dafc19b6d fix parsing comment tag delimiter with extra strings 2023-11-10 15:42:54 -04:00
Michael Go 41f65173b0 fix parsing comment block body inside a liquid tag 2023-11-09 16:27:23 -04:00
Michael Go e180535784 allow incomplete tags inside a comment tag 2023-11-08 16:50:41 -04:00
Michael Go a681e73aec refactor comment tag unit test to use test helper 2023-11-08 16:45:11 -04:00
Michael Go 2abf52d546 add a quirky comment tag unit test 2023-11-08 11:23:00 -04:00
eada2b65a2 clean up comment tag body parsing
Co-authored-by: Peter Zhu <[email protected]>
2023-11-08 11:13:13 -04:00
Michael GoandAlex Coco dbf0aa8cd5 don't parse nodes inside a comment tag
Co-authored-by: Alex Coco <[email protected]>
2023-11-07 18:38:13 -04:00
Ieva Grazuleviciute 407a8e5b0f Fix for custom tag rendering 2023-10-20 11:42:54 +03:00
Ateş GöralandGitHub e3dcc75ab5 Merge pull request #1748 from Shopify/atesgoral/update-ci-status-badge
Update CI status badge
2023-10-16 11:23:04 -04:00
Ates Goral ceb7a4237f Update CI status badge 2023-10-16 11:17:24 -04:00
Guillaume MaletteandGitHub 7b60b7fef5 Merge pull request #1746 from Shopify/gm/base64-encoding
[base64] Respect string encoding of input in base64_decode filters
2023-09-29 11:13:18 -04:00
Guillaume Malette 33e1a8ffbc fix style 2023-09-29 09:50:42 -04:00
Guillaume Malette cc47fa8f03 [base64] Respect string encoding of input in base64_decode filters 2023-09-29 09:39:59 -04:00
Adam KlingbaumandGitHub b1b9b9f691 Merge pull request #1739 from Shopify/klingbaum/fix-sum-filter-float-output
Fix `BigDecimal` output in `sum` filter
2023-09-27 13:34:27 -04:00
Adam Klingbaum 75e7725f57 Streamline new tests for sum filter 2023-08-08 16:15:48 +00:00
James PriorandAdam Klingbaum de6d15a73e Test sum floats from properties. 2023-08-08 16:10:04 +00:00
James PriorandAdam Klingbaum 2c5d2be193 Don't render sum filter results in scientific notation. 2023-08-08 16:10:03 +00:00
Thierry JoyalandGitHub 347a2418c4 Update homepage url
`http://www.liquidmarkup.org` is `http` 
`http://www.liquidmarkup.org` redirects to `https://shopify.github.io/liquid/`
`https://www.liquidmarkup.org` can’t provide a secure connection (ERR_SSL_PROTOCOL_ERROR)
2022-10-03 16:52:22 -04:00
117 changed files with 7054 additions and 951 deletions
+6
View File
@@ -0,0 +1,6 @@
version: 2
updates:
- package-ecosystem: github-actions
directory: "/"
schedule:
interval: weekly
+48 -10
View File
@@ -1,5 +1,5 @@
name: Liquid
on: [push, pull_request]
on: [push]
env:
BUNDLE_JOBS: 4
@@ -9,27 +9,65 @@ jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
entry:
- { ruby: 2.7, allowed-failure: false } # minimum supported
- { ruby: 3.2, allowed-failure: false } # latest
- { ruby: ruby-head, allowed-failure: true }
name: Test Ruby ${{ matrix.entry.ruby }}
- { ruby: 3.3, allowed-failure: false } # minimum supported
- { ruby: 3.4, allowed-failure: false, rubyopt: "--yjit" }
- { ruby: 4.0, allowed-failure: false } # latest stable
- {
ruby: 4.0,
allowed-failure: false,
rubyopt: "--enable-frozen-string-literal",
}
- { ruby: 4.0, allowed-failure: false, rubyopt: "--yjit" }
- { ruby: 4.0, allowed-failure: false, rubyopt: "--zjit" }
- { ruby: truffleruby, allowed-failure: false }
# Head can have failures due to being in development
- { ruby: head, allowed-failure: true }
- {
ruby: head,
allowed-failure: true,
rubyopt: "--enable-frozen-string-literal",
}
- { ruby: head, allowed-failure: true, rubyopt: "--yjit" }
- { ruby: head, allowed-failure: true, rubyopt: "--zjit" }
name: Test Ruby ${{ matrix.entry.ruby }} ${{ matrix.entry.rubyopt }} --${{ matrix.entry.allowed-failure && 'allowed-failure' || 'strict' }}
steps:
- uses: actions/checkout@v3
- uses: ruby/setup-ruby@v1
- uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # v3.6.0
- uses: ruby/setup-ruby@319994f95fa847cf3fb3cd3dbe89f6dcde9f178f # v1.295.0
with:
ruby-version: ${{ matrix.entry.ruby }}
bundler-cache: true
bundler: latest
- run: bundle exec rake
continue-on-error: ${{ matrix.entry.allowed-failure }}
env:
RUBYOPT: ${{ matrix.entry.rubyopt }}
spec:
runs-on: ubuntu-latest
env:
BUNDLE_WITH: spec
steps:
- uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # v3.6.0
- uses: ruby/setup-ruby@319994f95fa847cf3fb3cd3dbe89f6dcde9f178f # v1.295.0
with:
bundler-cache: true
bundler: latest
- name: Run liquid-spec for all adapters
run: |
for adapter in spec/*.rb; do
echo "=== Running $adapter ==="
bundle exec liquid-spec run "$adapter" --no-max-failures
done
memory_profile:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: ruby/setup-ruby@v1
- uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # v3.6.0
- uses: ruby/setup-ruby@319994f95fa847cf3fb3cd3dbe89f6dcde9f178f # v1.295.0
with:
ruby-version: 2.7
bundler-cache: true
- run: bundle exec rake memory_profile:run
+1 -2
View File
@@ -4,7 +4,6 @@
pkg
*.rbc
.rvmrc
.ruby-version
Gemfile.lock
.bundle
.byebug_history
Gemfile.lock
-1
View File
@@ -10,7 +10,6 @@ Performance:
Enabled: true
AllCops:
TargetRubyVersion: 2.7
NewCops: disable
SuggestExtensions: false
Exclude:
+10 -1
View File
@@ -174,7 +174,16 @@ Style/WordArray:
# Offense count: 117
# This cop supports safe auto-correction (--auto-correct).
# Configuration parameters: AllowHeredoc, AllowURI, URISchemes, IgnoreCopDirectives, AllowedPatterns, IgnoredPatterns.
# Configuration parameters: AllowHeredoc, AllowURI, URISchemes, AllowCopDirectives, AllowedPatterns.
# URISchemes: http, https
Layout/LineLength:
Max: 260
Naming/PredicatePrefix:
Enabled: false
# Offense count: 1
# This is intentional - early return from begin/rescue in assignment context
Lint/NoReturnInBeginEndBlocks:
Exclude:
- 'lib/liquid/standardfilters.rb'
+1
View File
@@ -0,0 +1 @@
3.4.1
+8
View File
@@ -26,3 +26,11 @@
* If it makes sense, add tests for your code and/or run a performance benchmark
* Make sure all tests pass (`bundle exec rake`)
* Create a pull request
## Releasing
* Bump the version in `lib/liquid/version.rb`
* Update the `History.md` file
* Open a PR like [this one](https://github.com/Shopify/liquid/pull/1894) and merge it to `main`
* Create a new release using the [GitHub UI](https://github.com/Shopify/liquid/releases/new)
+17 -8
View File
@@ -7,22 +7,31 @@ end
gemspec
gem "base64"
group :benchmark, :test do
gem 'benchmark-ips'
gem 'memory_profiler'
gem 'terminal-table'
gem "lru_redux"
install_if -> { RUBY_PLATFORM !~ /mingw|mswin|java/ && RUBY_ENGINE != 'truffleruby' } do
gem 'stackprof'
end
end
group :test do
gem 'rubocop', '~> 1.44.0'
gem 'rubocop-shopify', '~> 2.12.0', require: false
gem 'rubocop-performance', require: false
platform :mri, :truffleruby do
gem 'liquid-c', github: 'Shopify/liquid-c', ref: 'master'
end
group :development do
gem "webrick"
end
group :test do
gem 'benchmark'
gem 'rubocop', '~> 1.82.0'
gem 'rubocop-shopify', '~> 2.18.0', require: false
gem 'rubocop-performance', require: false
end
group :spec do
gem 'liquid-spec', github: 'Shopify/liquid-spec', branch: 'main'
gem 'activesupport', require: false
end
+86
View File
@@ -1,5 +1,91 @@
# Liquid Change Log
## 5.11.0
* Revert the Inline Snippets tag (#2001), treat its inclusion in the latest Liquid release as a bug, and allow for feedback on RFC#1916 to better support Liquid developers [Guilherme Carreiro]
* Rename the `:rigid` error mode to `:strict2` and display a warning when users attempt to use the `:rigid` mode [Guilherme Carreiro]
## 5.10.0
* Introduce support for Inline Snippets [Julia Boutin]
## 5.9.0
* Introduce `:rigid` error mode for stricter, safer parsing of all tags [CP Clermont, Guilherme Carreiro]
## 5.8.7
* Expose body content in the `Doc` tag [James Meng]
## 5.8.1
* Fix `{% doc %}` tag to be visitable [Guilherme Carreiro]
## 5.8.0
* Introduce the new `{% doc %}` tag [Guilherme Carreiro]
## 5.7.3
* Raise Liquid::SyntaxError when parsing invalidly encoded strings [Chris AtLee]
## 5.7.2 2025-01-31
* Fix array filters to not support nested properties [Guilherme Carreiro]
## 5.7.1 2025-01-24
* Fix the `find` and `find_index`filters to return `nil` when filtering empty arrays [Guilherme Carreiro]
* Fix the `has` filter to return `false` when filtering empty arrays [Guilherme Carreiro]
## 5.7.0 2025-01-16
### Features
* Add `find`, `find_index`, `has`, and `reject` filters to arrays [Guilherme Carreiro]
* Compatibility with Ruby 3.4 [Ian Ker-Seymer]
## 5.6.4 2025-01-14
### Fixes
* Add a default `string_scanner` to avoid errors with `Liquid::VariableLookup.parse("foo.bar")` [Ian Ker-Seymer]
## 5.6.3 2025-01-13
* Remove `lru_redux` dependency [Michael Go]
## 5.6.2 2025-01-13
### Fixes
* Preserve the old behavior of requiring floats to start with a digit [Michael Go]
## 5.6.1 2025-01-13
### Performance improvements
* Faster Expression parser / Tokenizer with StringScanner [Michael Go]
## 5.6.0 2024-12-19
### Architectural changes
* Added new `Environment` class to manage configuration and state that was previously stored in `Template` [Ian Ker-Seymer]
* Moved tag registration from `Template` to `Environment` [Ian Ker-Seymer]
* Removed `StrainerFactory` in favor of `Environment`-based strainer creation [Ian Ker-Seymer]
* Consolidated standard tags into a new `Tags` module with `STANDARD_TAGS` constant [Ian Ker-Seymer]
### Performance improvements
* Optimized `Lexer` with a new `Lexer2` implementation using jump tables for faster tokenization, requires Ruby 3.4 [Ian Ker-Seymer]
* Improved variable rendering with specialized handling for different types [Michael Go]
* Reduced array allocations by using frozen empty constants [Michael Go]
### API changes
* Deprecated several `Template` class methods in favor of `Environment` methods [Ian Ker-Seymer]
* Added deprecation warnings system [Ian Ker-Seymer]
* Changed how filters and tags are registered to use Environment [Ian Ker-Seymer]
### Fixes
* Fixed table row handling of break interrupts [Alex Coco]
* Improved variable output handling for arrays [Ian Ker-Seymer]
* Fix Tokenizer to handle null source value (#1873) [Bahar Pourazar]
## 5.5.0 2024-03-21
Please reference the GitHub release for more information.
## 5.4.0 2022-07-29
### Breaking Changes
+47 -5
View File
@@ -1,4 +1,4 @@
[![Build Status](https://api.travis-ci.org/Shopify/liquid.svg?branch=master)](http://travis-ci.org/Shopify/liquid)
[![Build status](https://github.com/Shopify/liquid/actions/workflows/liquid.yml/badge.svg)](https://github.com/Shopify/liquid/actions/workflows/liquid.yml)
[![Inline docs](http://inch-ci.org/github/Shopify/liquid.svg?branch=master)](http://inch-ci.org/github/Shopify/liquid)
# Liquid template engine
@@ -52,19 +52,61 @@ For standard use you can just pass it the content of a file and call render with
@template.render('name' => 'tobi') # => "hi tobi"
```
### Concept of Environments
In Liquid, a "Environment" is a scoped environment that encapsulates custom tags, filters, and other configurations. This allows you to define and isolate different sets of functionality for different contexts, avoiding global overrides that can lead to conflicts and unexpected behavior.
By using environments, you can:
1. **Encapsulate Logic**: Keep the logic for different parts of your application separate.
2. **Avoid Conflicts**: Prevent custom tags and filters from clashing with each other.
3. **Improve Maintainability**: Make it easier to manage and understand the scope of customizations.
4. **Enhance Security**: Limit the availability of certain tags and filters to specific contexts.
We encourage the use of Environments over globally overriding things because it promotes better software design principles such as modularity, encapsulation, and separation of concerns.
Here's an example of how you can define and use Environments in Liquid:
```ruby
user_environment = Liquid::Environment.build do |environment|
environment.register_tag("renderobj", RenderObjTag)
end
Liquid::Template.parse(<<~LIQUID, environment: user_environment)
{% renderobj src: "path/to/model.obj" %}
LIQUID
```
In this example, `RenderObjTag` is a custom tag that is only available within the `user_environment`.
Similarly, you can define another environment for a different context, such as email templates:
```ruby
email_environment = Liquid::Environment.build do |environment|
environment.register_tag("unsubscribe_footer", UnsubscribeFooter)
end
Liquid::Template.parse(<<~LIQUID, environment: email_environment)
{% unsubscribe_footer %}
LIQUID
```
By using Environments, you ensure that custom tags and filters are only available in the contexts where they are needed, making your Liquid templates more robust and easier to manage. For smaller projects, a global environment is available via `Liquid::Environment.default`.
### Error Modes
Setting the error mode of Liquid lets you specify how strictly you want your templates to be interpreted.
Normally the parser is very lax and will accept almost anything without error. Unfortunately this can make
it very hard to debug and can lead to unexpected behaviour.
Liquid also comes with a stricter parser that can be used when editing templates to give better error messages
Liquid also comes with different parsers that can be used when editing templates to give better error messages
when templates are invalid. You can enable this new parser like this:
```ruby
Liquid::Template.error_mode = :strict # Raises a SyntaxError when invalid syntax is used
Liquid::Template.error_mode = :warn # Adds strict errors to template.errors but continues as normal
Liquid::Template.error_mode = :lax # The default mode, accepts almost anything.
Liquid::Environment.default.error_mode = :strict2 # Raises a SyntaxError when invalid syntax is used in all tags
Liquid::Environment.default.error_mode = :strict # Raises a SyntaxError when invalid syntax is used in some tags
Liquid::Environment.default.error_mode = :warn # Adds strict errors to template.errors but continues as normal
Liquid::Environment.default.error_mode = :lax # The default mode, accepts almost anything.
```
If you want to set the error mode only on specific templates you can pass `:error_mode` as an option to `parse`:
+48 -4
View File
@@ -33,7 +33,7 @@ task :rubocop do
end
end
desc('runs test suite with both strict and lax parsers')
desc('runs test suite with lax, strict, and strict2 parsers')
task :test do
ENV['LIQUID_PARSER_MODE'] = 'lax'
Rake::Task['base_test'].invoke
@@ -42,9 +42,11 @@ task :test do
Rake::Task['base_test'].reenable
Rake::Task['base_test'].invoke
if RUBY_ENGINE == 'ruby' || RUBY_ENGINE == 'truffleruby'
ENV['LIQUID_C'] = '1'
ENV['LIQUID_PARSER_MODE'] = 'strict2'
Rake::Task['base_test'].reenable
Rake::Task['base_test'].invoke
if RUBY_ENGINE == 'ruby' || RUBY_ENGINE == 'truffleruby'
ENV['LIQUID_PARSER_MODE'] = 'lax'
Rake::Task['integration_test'].reenable
Rake::Task['integration_test'].invoke
@@ -52,6 +54,10 @@ task :test do
ENV['LIQUID_PARSER_MODE'] = 'strict'
Rake::Task['integration_test'].reenable
Rake::Task['integration_test'].invoke
ENV['LIQUID_PARSER_MODE'] = 'strict2'
Rake::Task['integration_test'].reenable
Rake::Task['integration_test'].invoke
end
end
@@ -73,7 +79,7 @@ end
namespace :benchmark do
desc "Run the liquid benchmark with lax parsing"
task :run do
task :lax do
ruby "./performance/benchmark.rb lax"
end
@@ -81,6 +87,38 @@ namespace :benchmark do
task :strict do
ruby "./performance/benchmark.rb strict"
end
desc "Run the liquid benchmark with strict2 parsing"
task :strict2 do
ruby "./performance/benchmark.rb strict2"
end
desc "Run the liquid benchmark with lax, strict, and strict2 parsing"
task run: [:lax, :strict, :strict2]
desc "Run unit benchmarks"
namespace :unit do
task :all do
Dir["./performance/unit/*_benchmark.rb"].each do |file|
puts "🧪 Running #{file}"
ruby file
end
end
task :lexer do
Dir["./performance/unit/lexer_benchmark.rb"].each do |file|
puts "🧪 Running #{file}"
ruby file
end
end
task :expression do
Dir["./performance/unit/expression_benchmark.rb"].each do |file|
puts "🧪 Running #{file}"
ruby file
end
end
end
end
namespace :profile do
@@ -110,3 +148,9 @@ end
task :console do
exec 'irb -I lib -r liquid'
end
desc('run liquid-spec suite across all adapters')
task :spec do
adapters = Dir['./spec/*.rb'].join(',')
sh "bundle exec liquid-spec matrix --adapters=#{adapters} --reference=ruby_liquid"
end
+30
View File
@@ -0,0 +1,30 @@
# Autoresearch Ideas
## Dead Ends (tried and failed)
- **Tag name interning** (skip+byte dispatch): saves 878 allocs but verification loop overhead kills speed
- **String dedup (-@)** for filter names: no alloc savings, creates temp strings anyway
- **Split-based tokenizer**: 2.5x faster C-level split but can't handle {{ followed by %} nesting
- **Streaming tokenizer**: needs own StringScanner (+alloc), per-shift overhead worse than eager array
- **Merge simple_lookup? into initialize**: logic overhead offsets saved index call
- **Cursor for filter scanning**: cursor.reset overhead worse than inline byte loops
- **Direct strainer call**: YJIT already inlines context.invoke_single well
- **TruthyCondition subclass**: YJIT polymorphism at evaluate call site hurts more than 115 saved allocs
- **Index loop for filters**: YJIT optimizes each+destructure MUCH better than manual filter[0]/filter[1]
## Key Insights
- YJIT monomorphism > allocation reduction at this scale
- C-level StringScanner.scan/skip > Ruby-level byte loops (already applied)
- String#split is 2.5x faster than manual tokenization, but Liquid's grammar is too complex for regex
- 74% of total CPU time is GC — alloc reduction is the highest-leverage optimization
- But YJIT-deoptimization from polymorphism costs more than the GC savings
## Remaining Ideas
- **Tokenizer: use String#index + byteslice instead of StringScanner**: avoid the StringScanner overhead entirely for the simple case of finding {%/{{ delimiters
- **Pre-freeze all Condition operator lambdas**: reduce alloc in Condition initialization
- **Avoid `@blocks = []` in If with single-element optimization**: use `@block` ivar for single condition, only create array for elsif
- **Reduce ForloopDrop allocation**: reuse ForloopDrop objects across iterations or use a lighter-weight object
- **VariableLookup: single-segment optimization**: for "product.title" (1 lookup), use an ivar instead of 1-element Array
+109
View File
@@ -0,0 +1,109 @@
# Autoresearch: Liquid Parse+Render Performance
## Objective
Optimize the Shopify Liquid template engine's parse and render performance.
The workload is the ThemeRunner benchmark which parses and renders real Shopify
theme templates (dropify, ripen, tribble, vogue) with realistic data from
`performance/shopify/database.rb`. We measure parse time, render time, and
object allocations. The optimization target is combined parse+render time (µs).
## How to Run
Run `./auto/autoresearch.sh` — it runs unit tests, liquid-spec conformance,
then the performance benchmark, outputting metrics in parseable format.
## Metrics
- **Primary (optimization target)**: `combined_µs` (µs, lower is better) — sum of parse + render time
- **Secondary (tradeoff monitoring)**:
- `parse_µs` — time to parse all theme templates (Liquid::Template#parse)
- `render_µs` — time to render all pre-compiled templates
- `allocations` — total object allocations for one parse+render cycle
Parse dominates (~70-75% of combined). Allocations correlate with GC pressure.
## Files in Scope
- `lib/liquid/*.rb` — core Liquid library (parser, lexer, context, expression, etc.)
- `lib/liquid/tags/*.rb` — tag implementations (for, if, assign, etc.)
- `performance/bench_quick.rb` — benchmark script
## Off Limits
- `test/` — tests must continue to pass unchanged
- `performance/tests/` — benchmark templates, do not modify
- `performance/shopify/` — benchmark data/filters, do not modify
## Constraints
- All unit tests must pass (`bundle exec rake base_test`)
- liquid-spec failures must not increase beyond 2 (pre-existing UTF-8 edge cases)
- No new gem dependencies
- Semantic correctness must be preserved — templates must render identical output
- **Security**: Liquid runs untrusted user code. See Strategic Direction for details.
## Strategic Direction
The long-term goal is to converge toward a **single-pass, forward-only parsing
architecture** using one shared StringScanner instance. The current system has
multiple redundant passes: Tokenizer → BlockBody → Lexer → Parser → Expression
→ VariableLookup, each re-scanning portions of the source. A unified scanner
approach would:
1. **One StringScanner** flows through the entire parse — no intermediate token
arrays, no re-lexing filter chains, no string reconstruction in Parser#expression.
2. **Emit a lightweight IL or normalized AST** during the single forward pass,
decoupling strictness checking from the hot parse path. The LiquidIL project
(`~/src/tries/2026-01-05-liquid-il`) demonstrated this: a recursive-descent
parser emitting IL directly achieved significant speedups.
3. **Minimal backtracking** — the scanner advances forward, byte-checking as it
goes. liquid-c (`~/src/tries/2026-01-16-Shopify-liquid-c`) showed that a
C-level cursor-based tokenizer eliminates most allocation overhead.
Current fast-path optimizations (byte-level tag/variable/for/if parsing) are
steps toward this goal. Each one replaces a regex+MatchData pattern with
forward-only byte scanning. The remaining Lexer→Parser path for filter args
is the next target for elimination.
**Security note**: Liquid executes untrusted user templates. All parsing must
use explicit byte-range checks. Never use eval, send on user input, dynamic
method dispatch, const_get, or any pattern that lets template authors escape
the sandbox.
## Baseline
- **Commit**: 4ea835a (original, before any optimizations)
- **combined_µs**: 7,374
- **parse_µs**: 5,928
- **render_µs**: 1,446
- **allocations**: 62,620
## Progress Log
- 3329b09: Replace FullToken regex with manual byte parsing → combined 7,262 (-1.5%)
- 97e6893: Replace VariableParser regex with manual byte scanner → combined 6,945 (-5.8%), allocs 58,009
- 2b78e4b: getbyte instead of string indexing in whitespace_handler/create_variable → allocs 51,477
- d291e63: Lexer equal? for frozen arrays, \s+ whitespace skip → combined ~6,331
- d79b9fa: Avoid strip alloc in Expression.parse, byteslice for strings → allocs 49,151
- fa41224: Short-circuit parse_number with first-byte check → allocs 48,240
- c1113ad: Fast-path String in render_obj_to_output → combined ~6,071
- 25f9224: Fast-path simple variable parsing (skip Lexer/Parser) → combined ~5,860, allocs 45,202
- 3939d74: Replace SIMPLE_VARIABLE regex with byte scanner → combined ~5,717, allocs 42,763
- fe7a2f5: Fast-path simple if conditions → combined ~5,444, allocs 41,490
- cfa0dfe: Replace For tag Syntax regex with manual byte parser → combined ~4,974, allocs 39,847
- 8a92a4e: Unified fast-path Variable: parse name directly, only lex filter chain → combined ~5,060, allocs 40,520
- 58d2514: parse_tag_token returns [tag_name, markup, newlines] → combined ~4,815, allocs 37,355
- db43492: Hoist write score check out of render loop → render ~1,345
- 17daac9: Extend fast-path to quoted string literal variables → all 1,197 variables fast-pathed
- 9fd7cec: Split filter parsing: no-arg filters scanned directly, Lexer only for args → combined ~4,595, allocs 35,159
- e5933fc: Avoid array alloc in parse_tag_token via class ivars → allocs 34,281
- 2e207e6: Replace WhitespaceOrNothing regex with byte-level blank_string? → combined ~4,800
- 526af22: invoke_single fast path for no-arg filter invocation → allocs 32,621
- 76ae8f1: find_variable top-scope fast path → combined ~4,740
- 4cda1a5: slice_collection: skip copy for full Array → allocs 32,004
- 79840b1: Replace SIMPLE_CONDITION regex with manual byte parser → combined ~4,663, allocs 31,465
- 69430e9: Replace INTEGER_REGEX/FLOAT_REGEX with byte-level parse_number → allocs 31,129
- 405e3dc: Frozen EMPTY_ARRAY/EMPTY_HASH for Context @filters/@disabled_tags → allocs 31,009
- b90d7f0: Avoid unnecessary array wrapping for Context environments → allocs 30,709
- 3799d4c: Lazy seen={} hash in Utils.to_s/inspect → allocs 30,169
- 0b07487: Fast-path VariableLookup: skip scan_variable for simple identifiers → allocs 29,711
- 9de1527: Introduce Cursor class for centralized byte-level scanning
- dd4a100: Remove dead parse_tag_token/SIMPLE_CONDITION (now in Cursor)
- cdc3438: For tag: migrate lax_parse to Cursor with zero-alloc scanning → allocs 29,620
## Current Best
- **combined_µs**: ~3,400 (-54% from original 7,374 baseline)
- **parse_µs**: ~2,300
- **render_µs**: ~1,100
- **allocations**: 24,882 (-60% from original 62,620 baseline)
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
# Autoresearch benchmark runner for Liquid performance optimization
# Runs: unit tests → performance benchmark (3 runs, takes best)
# Outputs METRIC lines for the agent to parse
# Exit code 0 = all good, non-zero = broken
set -euo pipefail
cd "$(dirname "$0")/.."
# ── Step 1: Unit tests (fast gate) ──────────────────────────────────
echo "=== Unit Tests ==="
TEST_OUT=$(bundle exec rake base_test 2>&1)
TEST_RESULT=$(echo "$TEST_OUT" | tail -1)
if echo "$TEST_OUT" | grep -q 'failures\|errors' && ! echo "$TEST_RESULT" | grep -q '0 failures, 0 errors'; then
echo "$TEST_OUT" | grep -E 'Failure|Error|failures|errors' | head -20
echo "FATAL: unit tests failed"
exit 1
fi
echo "$TEST_RESULT"
# ── Step 2: Performance benchmark (3 runs, take best) ──────────────
echo ""
echo "=== Performance Benchmark (3 runs) ==="
BEST_COMBINED=999999
BEST_PARSE=0
BEST_RENDER=0
BEST_ALLOC=0
for i in 1 2 3; do
OUT=$(bundle exec ruby performance/bench_quick.rb 2>&1)
P=$(echo "$OUT" | grep '^parse_us=' | cut -d= -f2)
R=$(echo "$OUT" | grep '^render_us=' | cut -d= -f2)
C=$(echo "$OUT" | grep '^combined_us=' | cut -d= -f2)
A=$(echo "$OUT" | grep '^allocations=' | cut -d= -f2)
echo " run $i: combined=${C}µs (parse=${P} render=${R}) allocs=${A}"
if [ "$C" -lt "$BEST_COMBINED" ]; then
BEST_COMBINED=$C
BEST_PARSE=$P
BEST_RENDER=$R
BEST_ALLOC=$A
fi
done
echo ""
echo "METRIC combined_us=$BEST_COMBINED"
echo "METRIC parse_us=$BEST_PARSE"
echo "METRIC render_us=$BEST_RENDER"
echo "METRIC allocations=$BEST_ALLOC"
Executable
+40
View File
@@ -0,0 +1,40 @@
#!/usr/bin/env bash
# Auto-research benchmark script for Liquid
# Runs: unit tests → liquid-spec → performance benchmark
# Outputs machine-readable metrics on success
# Exit code 0 = all good, non-zero = broken
set -euo pipefail
cd "$(dirname "$0")/.."
# ── Step 1: Unit tests (fast gate) ──────────────────────────────────
echo "=== Unit Tests ==="
if ! bundle exec rake base_test 2>&1; then
echo "FATAL: unit tests failed"
exit 1
fi
# ── Step 2: liquid-spec (correctness gate) ──────────────────────────
echo ""
echo "=== Liquid Spec ==="
SPEC_OUTPUT=$(bundle exec liquid-spec run spec/ruby_liquid.rb 2>&1 || true)
echo "$SPEC_OUTPUT" | tail -3
# Extract failure count from "Total: N passed, N failed, N errors" line
# Allow known pre-existing failures (≤2)
TOTAL_LINE=$(echo "$SPEC_OUTPUT" | grep "^Total:" || echo "Total: 0 passed, 0 failed, 0 errors")
FAILURES=$(echo "$TOTAL_LINE" | sed -n 's/.*\([0-9][0-9]*\) failed.*/\1/p')
ERRORS=$(echo "$TOTAL_LINE" | sed -n 's/.*\([0-9][0-9]*\) error.*/\1/p')
FAILURES=${FAILURES:-0}
ERRORS=${ERRORS:-0}
TOTAL_BAD=$((FAILURES + ERRORS))
if [ "$TOTAL_BAD" -gt 2 ]; then
echo "FATAL: liquid-spec has $FAILURES failures and $ERRORS errors (threshold: 2)"
exit 1
fi
# ── Step 3: Performance benchmark ──────────────────────────────────
echo ""
echo "=== Performance Benchmark ==="
bundle exec ruby performance/bench_quick.rb 2>&1
+30
View File
@@ -0,0 +1,30 @@
{"type":"config","name":"Liquid parse+render performance (tenderlove-inspired)","metricName":"combined_µs","metricUnit":"µs","bestDirection":"lower"}
{"run":1,"commit":"c09e722","metric":3818,"metrics":{"parse_µs":2722,"render_µs":1096,"allocations":24881},"status":"keep","description":"Baseline: 3,818µs combined, 24,881 allocs","timestamp":1773348490227}
{"run":2,"commit":"c09e722","metric":4063,"metrics":{"parse_µs":2901,"render_µs":1162,"allocations":24003},"status":"discard","description":"Tag name interning via skip+byte dispatch: saves 878 allocs but verification loop slower than scan","timestamp":1773348738557,"segment":0}
{"run":3,"commit":"c09e722","metric":3881,"metrics":{"parse_µs":2720,"render_µs":1161,"allocations":24881},"status":"discard","description":"String dedup (-@) for filter names: no alloc savings, no speed benefit","timestamp":1773348781481,"segment":0}
{"run":4,"commit":"c09e722","metric":3970,"metrics":{"parse_µs":2829,"render_µs":1141,"allocations":24881},"status":"discard","description":"Streaming tokenizer: needs own StringScanner (+1 alloc), per-shift overhead worse than saved array","timestamp":1773348883093,"segment":0}
{"run":5,"commit":"c09e722","metric":0,"metrics":{"parse_µs":0,"render_µs":0,"allocations":0},"status":"crash","description":"REVERTED: split-based tokenizer — regex can't handle unclosed tags inside raw blocks","timestamp":1773349089230,"segment":0}
{"run":6,"commit":"c09e722","metric":0,"metrics":{"parse_µs":0,"render_µs":0,"allocations":0},"status":"crash","description":"REVERTED: split regex tokenizer v2 — can't handle {{ followed by %} (variable-becomes-tag nesting)","timestamp":1773349248313,"segment":0}
{"run":7,"commit":"c09e722","metric":3861,"metrics":{"parse_µs":2744,"render_µs":1117,"allocations":24881},"status":"discard","description":"Merge simple_lookup? dot position into initialize — logic overhead offsets saved index call","timestamp":1773349376707,"segment":0}
{"run":8,"commit":"c09e722","metric":4048,"metrics":{"parse_µs":2929,"render_µs":1119,"allocations":24881},"status":"discard","description":"Use Cursor regex for filter name scanning — cursor.reset + method dispatch overhead worse than inline bytes","timestamp":1773349447172,"segment":0}
{"run":9,"commit":"c09e722","metric":3872,"metrics":{"parse_µs":2744,"render_µs":1128,"allocations":24881},"status":"discard","description":"Direct strainer call in Variable#render — YJIT already inlines context.invoke_single well","timestamp":1773349497593,"segment":0}
{"run":10,"commit":"c09e722","metric":3839,"metrics":{"parse_µs":2732,"render_µs":1107,"allocations":24879},"status":"discard","description":"Array#[] fast path for slice_collection with limit/offset — only 2 alloc savings, not meaningful","timestamp":1773349555348,"segment":0}
{"run":11,"commit":"c09e722","metric":3889,"metrics":{"parse_µs":2770,"render_µs":1119,"allocations":24766},"status":"discard","description":"TruthyCondition for simple if checks: -115 allocs but YJIT polymorphism at evaluate call site hurts speed","timestamp":1773349649377,"segment":0}
{"run":12,"commit":"c09e722","metric":4150,"metrics":{"parse_µs":2769,"render_µs":1381,"allocations":24881},"status":"discard","description":"Index loop for filters: YJIT optimizes each+destructure better than manual indexing","timestamp":1773349699285,"segment":0}
{"run":13,"commit":"b7ae55f","metric":3556,"metrics":{"parse_µs":2388,"render_µs":1168,"allocations":24882},"status":"keep","description":"Replace StringScanner tokenizer with String#byteindex — 12% faster parse, no regex overhead for delimiter finding","timestamp":1773349875890,"segment":0}
{"run":14,"commit":"e25f2f1","metric":3464,"metrics":{"parse_µs":2335,"render_µs":1129,"allocations":24882},"status":"keep","description":"Confirmation run: byteindex tokenizer consistently 3,400-3,600µs","timestamp":1773349889465,"segment":0}
{"run":15,"commit":"b37fa98","metric":3490,"metrics":{"parse_µs":2331,"render_µs":1159,"allocations":24882},"status":"keep","description":"Clean up tokenizer: remove unused StringScanner setup and regex constants","timestamp":1773349928672,"segment":0}
{"run":16,"commit":"b37fa98","metric":3638,"metrics":{"parse_µs":2460,"render_µs":1178,"allocations":24882},"status":"discard","description":"Single-char byteindex for %} search: Ruby loop overhead worse for nearby targets","timestamp":1773349985509,"segment":0}
{"run":17,"commit":"b37fa98","metric":3553,"metrics":{"parse_µs":2431,"render_µs":1122,"allocations":25256},"status":"discard","description":"Regex simple_variable_markup: MatchData creates 374 extra allocs, offsetting speed gain","timestamp":1773350066627,"segment":0}
{"run":18,"commit":"b37fa98","metric":3629,"metrics":{"parse_µs":2455,"render_µs":1174,"allocations":25002},"status":"discard","description":"String.new(capacity: 4096) for output buffer: allocates more objects, not fewer","timestamp":1773350101852,"segment":0}
{"run":19,"commit":"f6baeae","metric":3350,"metrics":{"parse_µs":2212,"render_µs":1138,"allocations":24882},"status":"keep","description":"parse_tag_token without StringScanner: pure byte ops avoid reset(token) overhead, -12% combined","timestamp":1773350230252,"segment":0}
{"run":20,"commit":"f6baead","metric":0,"metrics":{"parse_µs":0,"render_µs":0,"allocations":0},"status":"crash","description":"REVERTED: regex ultra-fast path for Variable — name pattern too broad, matches invalid trailing dots","timestamp":1773350472859,"segment":0}
{"run":21,"commit":"ae9a2e2","metric":3314,"metrics":{"parse_µs":2203,"render_µs":1111,"allocations":24882},"status":"keep","description":"Clean confirmation run: 3,314µs (-55% from main), stable","timestamp":1773350544354,"segment":0}
{"run":22,"commit":"ae9a2e2","metric":3497,"metrics":{"parse_µs":2336,"render_µs":1161,"allocations":24882},"status":"discard","description":"Regex fast path for no-filter variables: include? + match? overhead exceeds byte scan savings","timestamp":1773350641375,"segment":0}
{"run":23,"commit":"ca327b0","metric":3445,"metrics":{"parse_µs":2284,"render_µs":1161,"allocations":24647},"status":"keep","description":"Condition#evaluate: skip loop block for simple conditions (no child_relation) — saves 235 allocs","timestamp":1773350691752,"segment":0}
{"run":24,"commit":"99454a9","metric":3489,"metrics":{"parse_µs":2353,"render_µs":1136,"allocations":24647},"status":"keep","description":"Replace simple_lookup? byte scan with match? regex — 8x faster per call, cleaner code","timestamp":1773350837721,"segment":0}
{"run":25,"commit":"99454a9","metric":3797,"metrics":{"parse_µs":2636,"render_µs":1161,"allocations":29627},"status":"discard","description":"Regex name extraction in try_fast_parse: MatchData creates 5K extra allocs, much worse","timestamp":1773351048938,"segment":0}
{"run":26,"commit":"db348e0","metric":3459,"metrics":{"parse_µs":2318,"render_µs":1141,"allocations":24647},"status":"keep","description":"Inline to_liquid_value in If render — avoids one method dispatch per condition evaluation","timestamp":1773351080001,"segment":0}
{"run":27,"commit":"b195d09","metric":3496,"metrics":{"parse_µs":2356,"render_µs":1140,"allocations":24530},"status":"keep","description":"Replace @blocks.each with while loop in If render — avoids block proc allocation per render","timestamp":1773351101134,"segment":0}
{"run":28,"commit":"b195d09","metric":3648,"metrics":{"parse_µs":2457,"render_µs":1191,"allocations":24530},"status":"discard","description":"While loop in For render: YJIT optimizes each well for hot loops with many iterations","timestamp":1773351142275,"segment":0}
{"run":29,"commit":"b195d09","metric":3966,"metrics":{"parse_µs":2641,"render_µs":1325,"allocations":24060},"status":"discard","description":"While loop for environment search: -470 allocs but YJIT deopt makes render 16% slower","timestamp":1773351193863,"segment":0}
Executable
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env ruby
# frozen_string_literal: true
require 'bundler/setup'
require 'liquid'
class VirtualFileSystem
def initialize
snippet_1 = <<~LIQUID
<h1>
{{- greating | default: 'Hello' }}, {{ name | default: 'world' -}}!
</h1>
LIQUID
snippet_2 = <<~LIQUID
{%- for i in (1..5) -%}
> {{ i }}
{%- endfor -%}
LIQUID
@templates = {
'snippet-1' => snippet_1,
'snippet-2' => snippet_2,
}
end
def read_template_file(key)
@templates[key] || raise(Liquid::FileSystemError, "No such template '#{key}'")
end
end
def source
File.read(ARGV[0])
rescue StandardError
'Usage: bin/render example/server/templates/index.liquid'
end
def assigns
{
'date' => Time.now,
}
end
puts Liquid::Template
.parse(source, error_mode: :strict2)
.tap { |t| t.registers[:file_system] = VirtualFileSystem.new }
.render(assigns)
+16 -15
View File
@@ -21,6 +21,8 @@
# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
require "strscan"
module Liquid
FilterSeparator = /\|/
ArgumentSeparator = ','
@@ -44,13 +46,23 @@ module Liquid
VariableParser = /\[(?>[^\[\]]+|\g<0>)*\]|#{VariableSegment}+\??/o
RAISE_EXCEPTION_LAMBDA = ->(_e) { raise }
singleton_class.send(:attr_accessor, :cache_classes)
self.cache_classes = true
HAS_STRING_SCANNER_SCAN_BYTE = StringScanner.instance_methods.include?(:scan_byte)
end
require "liquid/version"
require "liquid/deprecations"
require "liquid/const"
require 'liquid/byte_tables'
require 'liquid/cursor'
require 'liquid/standardfilters'
require 'liquid/file_system'
require 'liquid/parser_switching'
require 'liquid/tag'
require 'liquid/block'
require 'liquid/parse_tree_visitor'
require 'liquid/interrupts'
require 'liquid/tags'
require "liquid/environment"
require 'liquid/lexer'
require 'liquid/parser'
require 'liquid/i18n'
@@ -61,23 +73,16 @@ require 'liquid/extensions'
require 'liquid/errors'
require 'liquid/interrupts'
require 'liquid/strainer_template'
require 'liquid/strainer_factory'
require 'liquid/expression'
require 'liquid/context'
require 'liquid/parser_switching'
require 'liquid/tag'
require 'liquid/tag/disabler'
require 'liquid/tag/disableable'
require 'liquid/block'
require 'liquid/block_body'
require 'liquid/document'
require 'liquid/variable'
require 'liquid/variable_lookup'
require 'liquid/range_lookup'
require 'liquid/file_system'
require 'liquid/resource_limits'
require 'liquid/expression'
require 'liquid/template'
require 'liquid/standardfilters'
require 'liquid/condition'
require 'liquid/utils'
require 'liquid/tokenizer'
@@ -86,7 +91,3 @@ require 'liquid/partial_cache'
require 'liquid/usage'
require 'liquid/registers'
require 'liquid/template_factory'
# Load all the tags of the standard library
#
Dir["#{__dir__}/liquid/tags/*.rb"].each { |f| require f }
+4 -1
View File
@@ -60,8 +60,11 @@ module Liquid
@tag_name
end
# Cache block delimiters per tag name to avoid repeated string allocation
BLOCK_DELIMITER_CACHE = Hash.new { |h, k| h[k] = "end#{k}".freeze }
def block_delimiter
@block_delimiter ||= "end#{block_name}"
@block_delimiter ||= BLOCK_DELIMITER_CACHE[block_name]
end
private
+109 -65
View File
@@ -1,11 +1,10 @@
# frozen_string_literal: true
require 'English'
module Liquid
class BlockBody
LiquidTagToken = /\A\s*(#{TagName})\s*(.*?)\z/o
FullToken = /\A#{TagStart}#{WhitespaceControl}?(\s*)(#{TagName})(\s*)(.*?)#{WhitespaceControl}?#{TagEnd}\z/om
FullTokenPossiblyInvalid = /\A(.*)#{TagStart}#{WhitespaceControl}?\s*(\w+)\s*(.*)?#{WhitespaceControl}?#{TagEnd}\z/om
ContentOfVariable = /\A#{VariableStart}#{WhitespaceControl}?(.*?)#{WhitespaceControl}?#{VariableEnd}\z/om
WhitespaceOrNothing = /\A\s*\z/
TAGSTART = "{%"
@@ -37,7 +36,7 @@ module Liquid
private def parse_for_liquid_tag(tokenizer, parse_context)
while (token = tokenizer.shift)
unless token.empty? || token.match?(WhitespaceOrNothing)
unless token.empty? || BlockBody.blank_string?(token)
unless token =~ LiquidTagToken
# line isn't empty but didn't match tag syntax, yield and let the
# caller raise a syntax error
@@ -51,9 +50,8 @@ module Liquid
next parse_liquid_tag(markup, parse_context)
end
unless (tag = registered_tags[tag_name])
# end parsing if we reach an unknown tag and let the caller decide
# determine how to proceed
unless (tag = parse_context.environment.tag_for_name(tag_name))
# end parsing if we reach an unknown tag; let the caller determine how to proceed
return yield tag_name, markup
end
new_tag = tag.parse(tag_name, markup, tokenizer, parse_context)
@@ -123,48 +121,38 @@ module Liquid
end
end
def self.blank_string?(str)
str.match?(WhitespaceOrNothing)
end
private def parse_for_document(tokenizer, parse_context, &block)
while (token = tokenizer.shift)
next if token.empty?
case
when token.start_with?(TAGSTART)
whitespace_handler(token, parse_context)
unless token =~ FullToken
return handle_invalid_tag_token(token, parse_context, &block)
end
tag_name = Regexp.last_match(2)
markup = Regexp.last_match(4)
if parse_context.line_number
# newlines inside the tag should increase the line number,
# particularly important for multiline {% liquid %} tags
parse_context.line_number += Regexp.last_match(1).count("\n") + Regexp.last_match(3).count("\n")
first_byte = token.getbyte(0)
if first_byte == Cursor::LCURLY
second_byte = token.getbyte(1)
if second_byte == Cursor::PCT
# handle_tag_token returns:
# nil — tag parsed normally, continue (update line number)
# :next — 'liquid' inline tag; skip line number update
# :unknown — end tag or unknown tag; yield to caller and return
# :invalid — malformed tag token; delegate to handle_invalid_tag_token
result = handle_tag_token(token, parse_context, tokenizer)
next unless result # nil: normal
next if result == :next # :next: 'liquid'
return yield(@_unknown_tag_name, parse_context.cursor.tag_markup) if result == :unknown
return handle_invalid_tag_token(token, parse_context, &block) # :invalid
elsif second_byte == Cursor::LCURLY
whitespace_handler(token, parse_context)
@nodelist << create_variable(token, parse_context)
@blank = false
else
# Fallback: text token starting with '{'
append_text_token(token, parse_context)
end
if tag_name == 'liquid'
parse_liquid_tag(markup, parse_context)
next
end
unless (tag = registered_tags[tag_name])
# end parsing if we reach an unknown tag and let the caller decide
# determine how to proceed
return yield tag_name, markup
end
new_tag = tag.parse(tag_name, markup, tokenizer, parse_context)
@blank &&= new_tag.blank?
@nodelist << new_tag
when token.start_with?(VARSTART)
whitespace_handler(token, parse_context)
@nodelist << create_variable(token, parse_context)
@blank = false
else
if parse_context.trim_whitespace
token.lstrip!
end
parse_context.trim_whitespace = false
@nodelist << token
@blank &&= token.match?(WhitespaceOrNothing)
append_text_token(token, parse_context)
end
parse_context.line_number = tokenizer.line_number
end
@@ -172,8 +160,54 @@ module Liquid
yield nil, nil
end
def whitespace_handler(token, parse_context)
if token[2] == WhitespaceControl
# Handles a {%...%} tag token. Does not receive the outer block — callers handle
# yield/block passing themselves, keeping the Proc off the hot path.
# Returns:
# nil — tag parsed, caller continues the loop
# :next — 'liquid' inline tag; caller skips line number update
# :unknown — unknown/end tag; @_unknown_tag_name holds the tag name;
# markup is in parse_context.cursor.tag_markup
# :invalid — malformed token; caller delegates to handle_invalid_tag_token
private def handle_tag_token(token, parse_context, tokenizer)
whitespace_handler(token, parse_context)
cursor = parse_context.cursor
tag_name = cursor.parse_tag_token(token)
return :invalid unless tag_name
markup = cursor.tag_markup
if parse_context.line_number
newlines = cursor.tag_newlines
parse_context.line_number += newlines if newlines > 0
end
if tag_name == 'liquid'
parse_liquid_tag(markup, parse_context)
return :next
end
tag = parse_context.environment.tag_for_name(tag_name)
unless tag
# end parsing if we reach an unknown tag; let the caller determine how to proceed
@_unknown_tag_name = tag_name
return :unknown
end
new_tag = tag.parse(tag_name, markup, tokenizer, parse_context)
@blank &&= new_tag.blank?
@nodelist << new_tag
nil
end
def append_text_token(token, parse_context)
token.lstrip! if parse_context.trim_whitespace
parse_context.trim_whitespace = false
@nodelist << token
@blank &&= BlockBody.blank_string?(token)
end
private :append_text_token
private def whitespace_handler(token, parse_context)
if token.getbyte(2) == Cursor::DASH
previous_token = @nodelist.last
if previous_token.is_a?(String)
first_byte = previous_token.getbyte(0)
@@ -183,7 +217,7 @@ module Liquid
end
end
end
parse_context.trim_whitespace = (token[-3] == WhitespaceControl)
parse_context.trim_whitespace = (token.getbyte(token.bytesize - 3) == Cursor::DASH)
end
def blank?
@@ -215,24 +249,35 @@ module Liquid
end
def render_to_output_buffer(context, output)
freeze unless frozen?
freeze
context.resource_limits.increment_render_score(@nodelist.length)
resource_limits = context.resource_limits
resource_limits.increment_render_score(@nodelist.length)
# Hot render loop — split on check_write so the common case (no resource
# limits) pays zero branch cost per node.
idx = 0
while (node = @nodelist[idx])
if node.instance_of?(String)
output << node
else
render_node(context, output, node)
# If we get an Interrupt that means the block must stop processing. An
# Interrupt is any command that stops block execution such as {% break %}
# or {% continue %}. These tags may also occur through Block or Include tags.
break if context.interrupt? # might have happened in a for-block
if resource_limits.render_length_limit || resource_limits.last_capture_length
while (node = @nodelist[idx])
if node.instance_of?(String)
output << node
else
render_node(context, output, node)
break if context.interrupt?
end
idx += 1
resource_limits.increment_write_score(output)
end
else
while (node = @nodelist[idx])
if node.instance_of?(String)
output << node
else
render_node(context, output, node)
break if context.interrupt?
end
idx += 1
end
idx += 1
context.resource_limits.increment_write_score(output)
end
output
@@ -240,15 +285,18 @@ module Liquid
private
# Indirection allows subclasses to intercept per-node rendering.
def render_node(context, output, node)
BlockBody.render_node(context, output, node)
end
def create_variable(token, parse_context)
if token =~ ContentOfVariable
markup = Regexp.last_match(1)
len = token.bytesize
if len >= 4 && token.getbyte(len - 1) == Cursor::RCURLY && token.getbyte(len - 2) == Cursor::RCURLY
markup = parse_context.cursor.parse_variable_token(token)
return Variable.new(markup, parse_context)
end
BlockBody.raise_missing_variable_terminator(token, parse_context)
end
@@ -261,9 +309,5 @@ module Liquid
def raise_missing_variable_terminator(token, parse_context)
BlockBody.raise_missing_variable_terminator(token, parse_context)
end
def registered_tags
Template.tags
end
end
end
+40
View File
@@ -0,0 +1,40 @@
# frozen_string_literal: true
module Liquid
# Pre-computed 256-entry boolean lookup tables for byte classification.
# Built once at load time; used as TABLE[byte] — a single array index
# instead of 3-5 comparison operators per check.
#
# Performance: neutral to slightly faster vs. chained comparisons.
# Readability: replaces expressions like
# (b >= 97 && b <= 122) || (b >= 65 && b <= 90) || b == 95
# with the intent-revealing
# ByteTables::IDENT_START[b]
module ByteTables
# [a-zA-Z_] — valid first byte of an identifier
IDENT_START = Array.new(256, false).tap do |t|
(97..122).each { |b| t[b] = true } # a-z
(65..90).each { |b| t[b] = true } # A-Z
t[95] = true # _
end.freeze
# [a-zA-Z0-9_-] — valid continuation byte of an identifier
IDENT_CONT = Array.new(256, false).tap do |t|
(97..122).each { |b| t[b] = true } # a-z
(65..90).each { |b| t[b] = true } # A-Z
(48..57).each { |b| t[b] = true } # 0-9
t[95] = true # _
t[45] = true # -
end.freeze
# [0-9] — ASCII digit
DIGIT = Array.new(256, false).tap do |t|
(48..57).each { |b| t[b] = true }
end.freeze
# [ \t\n\v\f\r] — ASCII whitespace (mirrors Ruby's \s)
WHITESPACE = Array.new(256, false).tap do |t|
[32, 9, 10, 11, 12, 13].each { |b| t[b] = true } # space, tab, \n, \v, \f, \r
end.freeze
end
end
+67 -20
View File
@@ -24,6 +24,9 @@ module Liquid
else
false
end
rescue Encoding::CompatibilityError
# "✅".b.include?("✅") raises Encoding::CompatibilityError despite being materially equal
left.b.include?(right.b)
end,
}
@@ -45,8 +48,8 @@ module Liquid
@@operators
end
def self.parse_expression(parse_context, markup)
@@method_literals[markup] || parse_context.parse_expression(markup)
def self.parse_expression(parse_context, markup, safe: false)
@@method_literals[markup] || parse_context.parse_expression(markup, safe: safe)
end
attr_reader :attachment, :child_condition
@@ -62,20 +65,21 @@ module Liquid
end
def evaluate(context = deprecated_default_context)
condition = self
result = nil
loop do
result = interpret_condition(condition.left, condition.right, condition.operator, context)
result = interpret_condition(@left, @right, @operator, context)
# Fast path: no child conditions (most common)
return result unless @child_relation
condition = self
while condition.child_relation
case condition.child_relation
when :or
break if Liquid::Utils.to_liquid_value(result)
when :and
break unless Liquid::Utils.to_liquid_value(result)
else
break
end
condition = condition.child_condition
result = interpret_condition(condition.left, condition.right, condition.operator, context)
end
result
end
@@ -110,24 +114,67 @@ module Liquid
def equal_variables(left, right)
if left.is_a?(MethodLiteral)
if right.respond_to?(left.method_name)
return right.send(left.method_name)
else
return nil
end
return call_method_literal(left, right)
end
if right.is_a?(MethodLiteral)
if left.respond_to?(right.method_name)
return left.send(right.method_name)
else
return nil
end
return call_method_literal(right, left)
end
left == right
end
def call_method_literal(literal, value)
method_name = literal.method_name
# If the object responds to the method (e.g., ActiveSupport is loaded), use it
if value.respond_to?(method_name)
value.send(method_name)
else
# Emulate ActiveSupport's blank?/empty? to make Liquid invariant
# to whether ActiveSupport is loaded or not
case method_name
when :blank?
liquid_blank?(value)
when :empty?
liquid_empty?(value)
else
false
end
end
end
# Implement blank? semantics matching ActiveSupport
# blank? returns true for nil, false, empty strings, whitespace-only strings,
# empty arrays, and empty hashes
def liquid_blank?(value)
case value
when NilClass, FalseClass
true
when TrueClass, Numeric
false
when String
# Blank if empty or whitespace only (matches ActiveSupport)
value.empty? || value.match?(/\A\s*\z/)
when Array, Hash
value.empty?
else
# Fall back to empty? if available, otherwise false
value.respond_to?(:empty?) ? value.empty? : false
end
end
# Implement empty? semantics
# Note: nil is NOT empty. empty? checks if a collection has zero elements.
def liquid_empty?(value)
case value
when String, Array, Hash
value.empty?
else
value.respond_to?(:empty?) ? value.empty? : false
end
end
def interpret_condition(left, right, op, context)
# If the operator is empty this means that the decision statement is just
# a single variable. We can just poll this variable from the context and
@@ -151,8 +198,8 @@ module Liquid
end
def deprecated_default_context
warn("DEPRECATION WARNING: Condition#evaluate without a context argument is deprecated" \
" and will be removed from Liquid 6.0.0.")
warn("DEPRECATION WARNING: Condition#evaluate without a context argument is deprecated " \
"and will be removed from Liquid 6.0.0.")
Context.new
end
+8
View File
@@ -0,0 +1,8 @@
# frozen_string_literal: true
module Liquid
module Const
EMPTY_HASH = {}.freeze
EMPTY_ARRAY = [].freeze
end
end
+83 -38
View File
@@ -15,35 +15,44 @@ module Liquid
# context['bob'] #=> nil class Context
class Context
attr_reader :scopes, :errors, :registers, :environments, :resource_limits, :static_registers, :static_environments
attr_accessor :exception_renderer, :template_name, :partial, :global_filter, :strict_variables, :strict_filters
attr_accessor :exception_renderer, :template_name, :partial, :global_filter, :strict_variables, :strict_filters, :environment
# rubocop:disable Metrics/ParameterLists
def self.build(environments: {}, outer_scope: {}, registers: {}, rethrow_errors: false, resource_limits: nil, static_environments: {}, &block)
new(environments, outer_scope, registers, rethrow_errors, resource_limits, static_environments, &block)
def self.build(environment: Environment.default, environments: {}, outer_scope: {}, registers: {}, rethrow_errors: false, resource_limits: nil, static_environments: {}, &block)
new(environments, outer_scope, registers, rethrow_errors, resource_limits, static_environments, environment, &block)
end
def initialize(environments = {}, outer_scope = {}, registers = {}, rethrow_errors = false, resource_limits = nil, static_environments = {})
@environments = [environments]
@environments.flatten!
def initialize(environments = {}, outer_scope = {}, registers = {}, rethrow_errors = false, resource_limits = nil, static_environments = {}, environment = Environment.default)
@environment = environment
@environments = environments.is_a?(Array) ? environments : [environments]
@static_environments = [static_environments].flatten(1).freeze
@scopes = [(outer_scope || {})]
@static_environments = if static_environments.is_a?(Array)
static_environments.frozen? ? static_environments : static_environments.freeze
elsif static_environments.empty?
Const::EMPTY_ARRAY
else
[static_environments].freeze
end
@scopes = [outer_scope || {}]
@registers = registers.is_a?(Registers) ? registers : Registers.new(registers)
@errors = []
@partial = false
@strict_variables = false
@resource_limits = resource_limits || ResourceLimits.new(Template.default_resource_limits)
@resource_limits = resource_limits || ResourceLimits.new(environment.default_resource_limits)
@base_scope_depth = 0
@interrupts = []
@filters = []
@interrupts = Const::EMPTY_ARRAY
@filters = Const::EMPTY_ARRAY
@global_filter = nil
@disabled_tags = {}
@disabled_tags = Const::EMPTY_HASH
# Lazy-init StringScanner — only needed if Context#[] is called during render
@string_scanner = nil
@registers.static[:cached_partials] ||= {}
@registers.static[:file_system] ||= Liquid::Template.file_system
@registers.static[:file_system] ||= environment.file_system
@registers.static[:template_factory] ||= Liquid::TemplateFactory.new
self.exception_renderer = Template.default_exception_renderer
self.exception_renderer = environment.exception_renderer
if rethrow_errors
self.exception_renderer = Liquid::RAISE_EXCEPTION_LAMBDA
end
@@ -60,7 +69,7 @@ module Liquid
end
def strainer
@strainer ||= StrainerFactory.create(self, @filters)
@strainer ||= @environment.create_strainer(self, @filters)
end
# Adds filters to this context.
@@ -68,7 +77,7 @@ module Liquid
# Note that this does not register the filters with the main Template object. see <tt>Template.register_filter</tt>
# for that
def add_filters(filters)
filters = [filters].flatten.compact
filters = Array(filters).flatten.compact
@filters += filters
@strainer = nil
end
@@ -79,11 +88,12 @@ module Liquid
# are there any not handled interrupts?
def interrupt?
!@interrupts.empty?
!@interrupts.equal?(Const::EMPTY_ARRAY) && @interrupts.any?
end
# push an interrupt to the stack. this interrupt is considered not handled.
def push_interrupt(e)
@interrupts = [] if @interrupts.frozen?
@interrupts.push(e)
end
@@ -104,6 +114,20 @@ module Liquid
strainer.invoke(method, *args).to_liquid
end
# Arity-specialized filter delegation — generated to match StrainerTemplate's specializations.
# The pattern (avoid *args splat) is the same for each arity; generating makes it explicit.
{
invoke_single: ['input'],
invoke_two: ['input', 'arg1'],
}.each do |method_name, params|
all_params = (["method"] + params).join(", ")
module_eval(<<~RUBY, __FILE__, __LINE__ + 1)
def #{method_name}(#{all_params})
strainer.#{method_name}(#{all_params}).to_liquid
end
RUBY
end
# Push new local scope on the stack. use <tt>Context#stack</tt> instead
def push(new_scope = {})
@scopes.unshift(new_scope)
@@ -142,6 +166,7 @@ module Liquid
check_overflow
self.class.build(
environment: @environment,
resource_limits: resource_limits,
static_environments: static_environments,
registers: Registers.new(registers),
@@ -174,11 +199,11 @@ module Liquid
# Example:
# products == empty #=> products.empty?
def [](expression)
evaluate(Expression.parse(expression))
evaluate(Expression.parse(expression, @string_scanner ||= StringScanner.new("")))
end
def key?(key)
self[key] != nil
!find_variable(key, raise_on_not_found: false).nil?
end
def evaluate(object)
@@ -187,20 +212,40 @@ module Liquid
# Fetches an object starting at the local scope and then moving up the hierachy
def find_variable(key, raise_on_not_found: true)
# This was changed from find() to find_index() because this is a very hot
# path and find_index() is optimized in MRI to reduce object allocation
index = @scopes.find_index { |s| s.key?(key) }
variable = if index
lookup_and_evaluate(@scopes[index], key, raise_on_not_found: raise_on_not_found)
# Fast path: check top scope first (most common in for loops)
scope = @scopes[0]
if scope.key?(key)
variable = lookup_and_evaluate(scope, key, raise_on_not_found: raise_on_not_found)
elsif @scopes.length == 1
# Only one scope and key not found — go straight to environments
variable = try_variable_find_in_environments(key, raise_on_not_found: raise_on_not_found)
else
try_variable_find_in_environments(key, raise_on_not_found: raise_on_not_found)
# Multiple scopes — search through all of them
scope = @scopes.find { |s| s.key?(key) }
variable = if scope
lookup_and_evaluate(scope, key, raise_on_not_found: raise_on_not_found)
else
try_variable_find_in_environments(key, raise_on_not_found: raise_on_not_found)
end
end
# update variable's context before invoking #to_liquid
# Fast path: primitive types don't need context= or to_liquid conversion
case variable
when String, Integer, Float, NilClass, TrueClass, FalseClass, Array, Hash, Time
return variable
end
variable = variable.to_liquid
variable.context = self if variable.respond_to?(:context=)
variable
liquid_variable = variable.to_liquid
if variable != liquid_variable
liquid_variable.context = self if liquid_variable.respond_to?(:context=)
end
liquid_variable
end
def lookup_and_evaluate(obj, key, raise_on_not_found: true)
@@ -218,6 +263,7 @@ module Liquid
end
def with_disabled_tags(tag_names)
@disabled_tags = {} if @disabled_tags.frozen?
tag_names.each do |name|
@disabled_tags[name] = @disabled_tags.fetch(name, 0) + 1
end
@@ -241,17 +287,16 @@ module Liquid
attr_reader :base_scope_depth
def try_variable_find_in_environments(key, raise_on_not_found:)
@environments.each do |environment|
found = find_in_envs(@environments, key, raise_on_not_found: raise_on_not_found)
return found unless found.nil? && !(@strict_variables && raise_on_not_found)
find_in_envs(@static_environments, key, raise_on_not_found: raise_on_not_found)
end
def find_in_envs(envs, key, raise_on_not_found:)
envs.each do |environment|
found_variable = lookup_and_evaluate(environment, key, raise_on_not_found: raise_on_not_found)
if !found_variable.nil? || @strict_variables && raise_on_not_found
return found_variable
end
end
@static_environments.each do |environment|
found_variable = lookup_and_evaluate(environment, key, raise_on_not_found: raise_on_not_found)
if !found_variable.nil? || @strict_variables && raise_on_not_found
return found_variable
end
return found_variable if !found_variable.nil? || (@strict_variables && raise_on_not_found)
end
nil
end
+317
View File
@@ -0,0 +1,317 @@
# frozen_string_literal: true
require "strscan"
module Liquid
# Single-pass forward-only scanner for Liquid parsing.
# Wraps StringScanner with higher-level methods for common Liquid constructs.
# One Cursor per template parse — threaded through all parsing code.
class Cursor
# Byte constants
SPACE = 32
TAB = 9
NL = 10
CR = 13
FF = 12
DASH = 45 # '-'
DOT = 46 # '.'
COLON = 58 # ':'
PIPE = 124 # '|'
QUOTE_S = 39 # "'"
QUOTE_D = 34 # '"'
LBRACK = 91 # '['
RBRACK = 93 # ']'
LPAREN = 40 # '('
RPAREN = 41 # ')'
QMARK = 63 # '?'
HASH = 35 # '#'
USCORE = 95 # '_'
COMMA = 44
ZERO = 48
NINE = 57
PCT = 37 # '%'
LCURLY = 123 # '{'
RCURLY = 125 # '}'
attr_reader :ss
def initialize(source)
@source = source
@ss = StringScanner.new(source)
end
# ── Position ────────────────────────────────────────────────────
def pos = @ss.pos
def pos=(n)
@ss.pos = n
end
def eos? = @ss.eos?
def peek_byte = @ss.peek_byte
def scan_byte = @ss.scan_byte
# Reset scanner to a new string (for reuse on sub-markup)
def reset(source)
@source = source
@ss.string = source
end
# Extract a slice from the source (deferred allocation)
def slice(start, len)
@source.byteslice(start, len)
end
# ── Whitespace ──────────────────────────────────────────────────
# Skip spaces/tabs/newlines/cr
def skip_ws
while (b = @ss.peek_byte)
case b
when SPACE, TAB, CR, FF, NL then @ss.scan_byte
else break
end
end
end
# Check if remaining bytes are all whitespace (or EOS).
# exist?(/\S/) returns nil when no non-whitespace remains, without advancing position.
def rest_blank?
!@ss.exist?(/\S/)
end
# Regex for identifier: [a-zA-Z_][\w-]*\??
ID_REGEX = /[a-zA-Z_][\w-]*\??/
# ── Identifiers ─────────────────────────────────────────────────
# Skip an identifier without allocating a string. Returns length skipped, or 0.
def skip_id
@ss.skip(ID_REGEX) || 0
end
# Check if next id matches expected string, consume if so. No allocation.
def expect_id(expected)
start = @ss.pos
len = @ss.skip(ID_REGEX)
if len == expected.bytesize
# Compare bytes directly without allocating a string
i = 0
while i < len
unless @source.getbyte(start + i) == expected.getbyte(i)
@ss.pos = start
return false
end
i += 1
end
return true
end
@ss.pos = start if len
false
end
# Scan a single identifier: [a-zA-Z_][\w-]*\??
# Returns the string or nil if not at an identifier
def scan_id
@ss.scan(ID_REGEX)
end
# Scan a tag name: '#' or \w+
def scan_tag_name
if @ss.peek_byte == HASH
@ss.scan_byte
"#"
else
scan_id
end
end
# Regex for numbers: -?\d+(\.\d+)?
FLOAT_REGEX = /-?\d+\.\d+/
INT_REGEX = /-?\d+/
# ── Numbers ─────────────────────────────────────────────────────
# Try to scan an integer or float. Returns the number or nil.
def scan_number
if (s = @ss.scan(FLOAT_REGEX))
s.to_f
elsif (s = @ss.scan(INT_REGEX))
s.to_i
end
end
# Regex for quoted string content (without quotes)
SINGLE_QUOTED_CONTENT = /'([^']*)'/
DOUBLE_QUOTED_CONTENT = /"([^"]*)"/
# ── Strings ─────────────────────────────────────────────────────
# Scan a quoted string ('...' or "..."). Returns the content without quotes, or nil.
def scan_quoted_string
if @ss.scan(SINGLE_QUOTED_CONTENT) || @ss.scan(DOUBLE_QUOTED_CONTENT)
@ss[1]
end
end
# Regex for quoted strings (single or double quoted, including quotes)
QUOTED_STRING_RAW = /"[^"]*"|'[^']*'/
# Scan a quoted string including quotes. Returns the full "..." or '...' string, or nil.
def scan_quoted_string_raw
@ss.scan(QUOTED_STRING_RAW)
end
# Regex for dotted identifier: name(.name)*
DOTTED_ID_REGEX = /[a-zA-Z_][\w-]*\??(?:\.[a-zA-Z_][\w-]*\??)*/
# ── Expressions ─────────────────────────────────────────────────
# Scan a simple variable lookup: name(.name)* — no brackets, no filters
# Returns the string or nil
def scan_dotted_id
@ss.scan(DOTTED_ID_REGEX)
end
# Skip a fragment without allocating. Returns length skipped, or 0.
def skip_fragment
@ss.skip(QUOTED_STRING_RAW) || @ss.skip(UNQUOTED_FRAGMENT) || 0
end
# Regex for unquoted fragment: non-whitespace/comma/pipe sequence
UNQUOTED_FRAGMENT = /[^\s,|]+/
# Scan a "QuotedFragment" — a quoted string or non-whitespace/comma/pipe run
def scan_fragment
@ss.scan(QUOTED_STRING_RAW) || @ss.scan(UNQUOTED_FRAGMENT)
end
# ── Comparison operators ────────────────────────────────────────
# Identity map used for frozen string interning: StringScanner#scan returns a
# new unfrozen String on every call. Indexing into this hash returns the frozen
# literal stored here, avoiding a separate allocation and enabling faster
# equality checks downstream (frozen strings can be compared by identity).
COMPARISON_OPS = {
'==' => '==',
'!=' => '!=',
'<>' => '<>',
'<=' => '<=',
'>=' => '>=',
'<' => '<',
'>' => '>',
'contains' => 'contains',
}.freeze
# Scan a comparison operator. Returns frozen string or nil.
# Regex for comparison operators
COMPARISON_OP_REGEX = /==|!=|<>|<=|>=|<|>|contains(?!\w)/
def scan_comparison_op
if (op = @ss.scan(COMPARISON_OP_REGEX))
COMPARISON_OPS[op]
end
end
# ── Tag parsing helpers ─────────────────────────────────────────
# Results from last parse_tag_token call (avoids array allocation)
attr_reader :tag_markup, :tag_newlines
# Parse the interior of a tag token: "{%[-] tag_name markup [-]%}"
# Pure byte operations — avoids StringScanner reset overhead.
# Returns tag_name string or nil. Sets tag_markup and tag_newlines.
def parse_tag_token(token)
len = token.bytesize
pos = 2 # skip "{%"
pos += 1 if token.getbyte(pos) == DASH # skip '-'
nl = 0
# Skip whitespace, count newlines
while pos < len
b = token.getbyte(pos)
case b
when SPACE, TAB, CR, FF then pos += 1
when NL then pos += 1
nl += 1
else break
end
end
# Scan tag name: '#' or [a-zA-Z_][\w-]*
name_start = pos
b = token.getbyte(pos)
if b == HASH
pos += 1
elsif b && ByteTables::IDENT_START[b]
pos += 1
while pos < len
b = token.getbyte(pos)
break unless ByteTables::IDENT_CONT[b]
pos += 1
end
pos += 1 if pos < len && token.getbyte(pos) == QMARK
else
return
end
tag_name = token.byteslice(name_start, pos - name_start)
# Skip whitespace after tag name, count newlines
while pos < len
b = token.getbyte(pos)
case b
when SPACE, TAB, CR, FF then pos += 1
when NL then pos += 1
nl += 1
else break
end
end
# markup is everything up to optional '-' before '%}'
markup_end = len - 2
markup_end -= 1 if markup_end > pos && token.getbyte(markup_end - 1) == DASH
@tag_markup = pos >= markup_end ? "" : token.byteslice(pos, markup_end - pos)
@tag_newlines = nl
tag_name
end
# Parse variable token interior: extract markup from "{{[-] ... [-]}}"
def parse_variable_token(token)
len = token.bytesize
return if len < 4
i = 2
i = 3 if token.getbyte(i) == DASH
parse_end = len - 3
parse_end -= 1 if token.getbyte(parse_end) == DASH
markup_len = parse_end - i + 1
markup_len <= 0 ? "" : token.byteslice(i, markup_len)
end
# ── Simple condition parser ─────────────────────────────────────
# Results from last parse_simple_condition call
attr_reader :cond_left, :cond_op, :cond_right
# Parse "expr [op expr]" from current position to end.
# Returns true on success, nil on failure. Sets cond_left, cond_op, cond_right.
def parse_simple_condition
skip_ws
@cond_left = scan_fragment
return unless @cond_left
skip_ws
if eos?
@cond_op = nil
@cond_right = nil
return true
end
@cond_op = scan_comparison_op
return unless @cond_op
skip_ws
@cond_right = scan_fragment
return unless @cond_right
skip_ws
return unless eos? # trailing junk
true
end
end
end
+22
View File
@@ -0,0 +1,22 @@
# frozen_string_literal: true
require "set"
module Liquid
class Deprecations
class << self
attr_accessor :warned
Deprecations.warned = Set.new
def warn(name, alternative)
return if warned.include?(name)
warned << name
caller_location = caller_locations(2, 1).first
Warning.warn("[DEPRECATION] #{name} is deprecated. Use #{alternative} instead. Called from #{caller_location}\n")
end
end
end
end
+1 -1
View File
@@ -31,7 +31,7 @@ module Liquid
# Catch all for the method
def liquid_method_missing(method)
return nil unless @context&.strict_variables
return unless @context&.strict_variables
raise Liquid::UndefinedDropMethod, "undefined method #{method}"
end
+159
View File
@@ -0,0 +1,159 @@
# frozen_string_literal: true
module Liquid
# The Environment is the container for all configuration options of Liquid, such as
# the registered tags, filters, and the default error mode.
class Environment
# The default error mode for all templates. This can be overridden on a
# per-template basis.
attr_accessor :error_mode
# The tags that are available to use in the template.
attr_accessor :tags
# The strainer template which is used to store filters that are available to
# use in templates.
attr_accessor :strainer_template
# The exception renderer that is used to render exceptions that are raised
# when rendering a template
attr_accessor :exception_renderer
# The default file system that is used to load templates from.
attr_accessor :file_system
# The default resource limits that are used to limit the resources that a
# template can consume.
attr_accessor :default_resource_limits
class << self
# Creates a new environment instance.
#
# @param tags [Hash] The tags that are available to use in
# the template.
# @param file_system The default file system that is used
# to load templates from.
# @param error_mode [Symbol] The default error mode for all templates
# (either :strict2, :strict, :warn, or :lax).
# @param exception_renderer [Proc] The exception renderer that is used to
# render exceptions.
# @yieldparam environment [Environment] The environment instance that is being built.
# @return [Environment] The new environment instance.
def build(tags: nil, file_system: nil, error_mode: nil, exception_renderer: nil)
ret = new
ret.tags = tags if tags
ret.file_system = file_system if file_system
ret.error_mode = error_mode if error_mode
ret.exception_renderer = exception_renderer if exception_renderer
yield ret if block_given?
ret.freeze
end
# Returns the default environment instance.
#
# @return [Environment] The default environment instance.
def default
@default ||= new
end
# Sets the default environment instance for the duration of the block
#
# @param environment [Environment] The environment instance to use as the default for the
# duration of the block.
# @yield
# @return [Object] The return value of the block.
def dangerously_override(environment)
original_default = @default
@default = environment
yield
ensure
@default = original_default
end
end
# Initializes a new environment instance.
# @api private
def initialize
@tags = Tags::STANDARD_TAGS.dup
@error_mode = :lax
@strainer_template = Class.new(StrainerTemplate).tap do |klass|
klass.add_filter(StandardFilters)
end
@exception_renderer = ->(exception) { exception }
@file_system = BlankFileSystem.new
@default_resource_limits = Const::EMPTY_HASH
@strainer_template_class_cache = {}
end
# Registers a new tag with the environment.
#
# @param name [String] The name of the tag.
# @param klass [Liquid::Tag] The class that implements the tag.
# @return [void]
def register_tag(name, klass)
@tags[name] = klass
end
# Registers a new filter with the environment.
#
# @param filter [Module] The module that contains the filter methods.
# @return [void]
def register_filter(filter)
@strainer_template_class_cache.clear
@strainer_template.add_filter(filter)
end
# Registers multiple filters with this environment.
#
# @param filters [Array<Module>] The modules that contain the filter methods.
# @return [self]
def register_filters(filters)
@strainer_template_class_cache.clear
filters.each { |f| @strainer_template.add_filter(f) }
self
end
# Creates a new strainer instance with the given filters, caching the result
# for faster lookup.
#
# @param context [Liquid::Context] The context that the strainer will be
# used in.
# @param filters [Array<Module>] The filters that the strainer will have
# access to.
# @return [Liquid::Strainer] The new strainer instance.
def create_strainer(context, filters = Const::EMPTY_ARRAY)
return @strainer_template.new(context) if filters.empty?
strainer_template = @strainer_template_class_cache[filters] ||= begin
klass = Class.new(@strainer_template)
filters.each { |f| klass.add_filter(f) }
klass
end
strainer_template.new(context)
end
# Returns the names of all the filter methods that are available to use in
# the strainer template.
#
# @return [Array<String>] The names of all the filter methods.
def filter_method_names
@strainer_template.filter_method_names
end
# Returns the tag class for the given tag name.
#
# @param name [String] The name of the tag.
# @return [Liquid::Tag] The tag class.
def tag_for_name(name)
@tags[name]
end
def freeze
@tags.freeze
# TODO: freeze the tags, currently this is not possible because of liquid-c
# @strainer_template.freeze
super
end
end
end
+16 -15
View File
@@ -40,19 +40,20 @@ module Liquid
end
end
ArgumentError = Class.new(Error)
ContextError = Class.new(Error)
FileSystemError = Class.new(Error)
StandardError = Class.new(Error)
SyntaxError = Class.new(Error)
StackLevelError = Class.new(Error)
MemoryError = Class.new(Error)
ZeroDivisionError = Class.new(Error)
FloatDomainError = Class.new(Error)
UndefinedVariable = Class.new(Error)
UndefinedDropMethod = Class.new(Error)
UndefinedFilter = Class.new(Error)
MethodOverrideError = Class.new(Error)
DisabledError = Class.new(Error)
InternalError = Class.new(Error)
ArgumentError = Class.new(Error)
ContextError = Class.new(Error)
FileSystemError = Class.new(Error)
StandardError = Class.new(Error)
SyntaxError = Class.new(Error)
StackLevelError = Class.new(Error)
MemoryError = Class.new(Error)
ZeroDivisionError = Class.new(Error)
FloatDomainError = Class.new(Error)
UndefinedVariable = Class.new(Error)
UndefinedDropMethod = Class.new(Error)
UndefinedFilter = Class.new(Error)
MethodOverrideError = Class.new(Error)
DisabledError = Class.new(Error)
InternalError = Class.new(Error)
TemplateEncodingError = Class.new(Error)
end
+131 -24
View File
@@ -10,38 +10,145 @@ module Liquid
'true' => true,
'false' => false,
'blank' => '',
'empty' => ''
'empty' => '',
# in lax mode, minus sign can be a VariableLookup
# For simplicity and performace, we treat it like a literal
'-' => VariableLookup.parse("-", nil).freeze,
}.freeze
INTEGERS_REGEX = /\A(-?\d+)\z/
FLOATS_REGEX = /\A(-?\d[\d\.]+)\z/
# Use an atomic group (?>...) to avoid pathological backtracing from
# malicious input as described in https://github.com/Shopify/liquid/issues/1357
RANGES_REGEX = /\A\(\s*(?>(\S+)\s*\.\.)\s*(\S+)\s*\)\z/
RANGES_REGEX = /\A\(\s*(?>(\S+)\s*\.\.)\s*(\S+)\s*\)\z/
def self.parse(markup)
return nil unless markup
markup = markup.strip
if (markup.start_with?('"') && markup.end_with?('"')) ||
(markup.start_with?("'") && markup.end_with?("'"))
return markup[1..-2]
class << self
def safe_parse(parser, ss = StringScanner.new(""), cache = nil)
parse(parser.expression, ss, cache)
end
case markup
when INTEGERS_REGEX
Regexp.last_match(1).to_i
when RANGES_REGEX
RangeLookup.parse(Regexp.last_match(1), Regexp.last_match(2))
when FLOATS_REGEX
Regexp.last_match(1).to_f
else
if LITERALS.key?(markup)
LITERALS[markup]
else
VariableLookup.parse(markup)
def parse(markup, ss = StringScanner.new(""), cache = nil)
return unless markup
# Only strip if there's leading/trailing whitespace (avoids allocation)
first_byte = markup.getbyte(0)
if first_byte && ByteTables::WHITESPACE[first_byte]
markup = markup.strip
elsif first_byte
markup = markup.strip if ByteTables::WHITESPACE[markup.getbyte(markup.bytesize - 1)]
end
if (markup.start_with?('"') && markup.end_with?('"')) ||
(markup.start_with?("'") && markup.end_with?("'"))
return markup.byteslice(1, markup.bytesize - 2)
elsif LITERALS.key?(markup)
return LITERALS[markup]
end
# Cache only exists during parsing
if cache
return cache[markup] if cache.key?(markup)
cache[markup] = inner_parse(markup, ss, cache).freeze
else
inner_parse(markup, ss, nil).freeze
end
end
def inner_parse(markup, ss, cache)
if markup.start_with?("(") && markup.end_with?(")") && markup =~ RANGES_REGEX
return RangeLookup.parse(
Regexp.last_match(1),
Regexp.last_match(2),
ss,
cache,
)
end
if (num = parse_number(markup, ss))
num
else
VariableLookup.parse(markup, ss, cache)
end
end
def parse_number(markup, _ss = nil)
len = markup.bytesize
return if len == 0
# Quick reject: first byte must be digit or dash
pos = 0
first = markup.getbyte(pos)
if first == Cursor::DASH
pos += 1
return if pos >= len
b = markup.getbyte(pos)
return unless ByteTables::DIGIT[b]
pos += 1
elsif ByteTables::DIGIT[first]
pos += 1
else
return
end
# Scan digits
while pos < len
b = markup.getbyte(pos)
break unless ByteTables::DIGIT[b]
pos += 1
end
# If we consumed everything, it's a simple integer
if pos == len
return Integer(markup, 10)
end
# Check for dot (float)
if markup.getbyte(pos) == Cursor::DOT
dot_pos = pos
pos += 1
# Must have at least one digit after dot
digit_after_dot = pos
while pos < len
b = markup.getbyte(pos)
break unless ByteTables::DIGIT[b]
pos += 1
end
if pos > digit_after_dot && pos == len
# Simple float like "123.456"
return markup.to_f
elsif pos > digit_after_dot
# Float followed by more content: "1.2.3.4" — scan to find where the
# numeric portion ends (stop at next dot or non-digit).
return scan_float_with_trailing(markup, pos, len)
else
# dot at end: "123."
return markup.byteslice(0, dot_pos).to_f
end
end
# Not a number (has non-digit, non-dot characters)
nil
end
private
# Scans forward from `pos` through digits, returning the float up to the
# next dot or the end of string. Returns nil when a non-digit, non-dot
# byte is found (not a valid number). Used by parse_number for inputs
# like "1.2.3.4" where the float literal ends at the second dot.
def scan_float_with_trailing(markup, pos, len)
while pos < len
b = markup.getbyte(pos)
return markup.byteslice(0, pos).to_f if b == Cursor::DOT
return unless ByteTables::DIGIT[b]
pos += 1
end
markup.byteslice(0, pos).to_f
end
end
end
+1 -1
View File
@@ -28,7 +28,7 @@ module Liquid
def interpolate(name, vars)
name.gsub(/%\{(\w+)\}/) do
# raise TranslationError, "Undefined key #{$1} for interpolation in translation #{name}" unless vars[$1.to_sym]
(vars[Regexp.last_match(1).to_sym]).to_s
vars[Regexp.last_match(1).to_sym].to_s
end
end
+161 -42
View File
@@ -1,61 +1,180 @@
# frozen_string_literal: true
require "strscan"
module Liquid
class Lexer
SPECIALS = {
'|' => :pipe,
'.' => :dot,
':' => :colon,
',' => :comma,
'[' => :open_square,
']' => :close_square,
'(' => :open_round,
')' => :close_round,
'?' => :question,
'-' => :dash,
}.freeze
IDENTIFIER = /[a-zA-Z_][\w-]*\??/
SINGLE_STRING_LITERAL = /'[^\']*'/
CLOSE_ROUND = [:close_round, ")"].freeze
CLOSE_SQUARE = [:close_square, "]"].freeze
COLON = [:colon, ":"].freeze
COMMA = [:comma, ","].freeze
COMPARISION_NOT_EQUAL = [:comparison, "!="].freeze
COMPARISON_CONTAINS = [:comparison, "contains"].freeze
COMPARISON_EQUAL = [:comparison, "=="].freeze
COMPARISON_GREATER_THAN = [:comparison, ">"].freeze
COMPARISON_GREATER_THAN_OR_EQUAL = [:comparison, ">="].freeze
COMPARISON_LESS_THAN = [:comparison, "<"].freeze
COMPARISON_LESS_THAN_OR_EQUAL = [:comparison, "<="].freeze
COMPARISON_NOT_EQUAL_ALT = [:comparison, "<>"].freeze
DASH = [:dash, "-"].freeze
DOT = [:dot, "."].freeze
DOTDOT = [:dotdot, ".."].freeze
DOT_ORD = ".".ord
DOUBLE_STRING_LITERAL = /"[^\"]*"/
STRING_LITERAL = Regexp.union(SINGLE_STRING_LITERAL, DOUBLE_STRING_LITERAL)
EOS = [:end_of_string].freeze
IDENTIFIER = /[a-zA-Z_][\w-]*\??/
NUMBER_LITERAL = /-?\d+(\.\d+)?/
DOTDOT = /\.\./
COMPARISON_OPERATOR = /==|!=|<>|<=?|>=?|contains(?=\s)/
OPEN_ROUND = [:open_round, "("].freeze
OPEN_SQUARE = [:open_square, "["].freeze
PIPE = [:pipe, "|"].freeze
QUESTION = [:question, "?"].freeze
RUBY_WHITESPACE = [" ", "\t", "\r", "\n", "\f"].freeze
SINGLE_STRING_LITERAL = /'[^\']*'/
WHITESPACE_OR_NOTHING = /\s*/
WHITESPACE = /\s+/
def initialize(input)
@ss = StringScanner.new(input)
SINGLE_COMPARISON_TOKENS = [].tap do |table|
table["<".ord] = COMPARISON_LESS_THAN
table[">".ord] = COMPARISON_GREATER_THAN
table.freeze
end
def tokenize
@output = []
TWO_CHARS_COMPARISON_JUMP_TABLE = [].tap do |table|
table["=".ord] = [].tap do |sub_table|
sub_table["=".ord] = COMPARISON_EQUAL
sub_table.freeze
end
table["!".ord] = [].tap do |sub_table|
sub_table["=".ord] = COMPARISION_NOT_EQUAL
sub_table.freeze
end
table.freeze
end
until @ss.eos?
@ss.skip(WHITESPACE_OR_NOTHING)
break if @ss.eos?
tok = if (t = @ss.scan(COMPARISON_OPERATOR))
[:comparison, t]
elsif (t = @ss.scan(STRING_LITERAL))
[:string, t]
elsif (t = @ss.scan(NUMBER_LITERAL))
[:number, t]
elsif (t = @ss.scan(IDENTIFIER))
[:id, t]
elsif (t = @ss.scan(DOTDOT))
[:dotdot, t]
else
c = @ss.getch
if (s = SPECIALS[c])
[s, c]
COMPARISON_JUMP_TABLE = [].tap do |table|
table["<".ord] = [].tap do |sub_table|
sub_table["=".ord] = COMPARISON_LESS_THAN_OR_EQUAL
sub_table[">".ord] = COMPARISON_NOT_EQUAL_ALT
sub_table.freeze
end
table[">".ord] = [].tap do |sub_table|
sub_table["=".ord] = COMPARISON_GREATER_THAN_OR_EQUAL
sub_table.freeze
end
table.freeze
end
NEXT_MATCHER_JUMP_TABLE = [].tap do |table|
"a".upto("z") do |c|
table[c.ord] = [:id, IDENTIFIER].freeze
table[c.upcase.ord] = [:id, IDENTIFIER].freeze
end
table["_".ord] = [:id, IDENTIFIER].freeze
"0".upto("9") do |c|
table[c.ord] = [:number, NUMBER_LITERAL].freeze
end
table["-".ord] = [:number, NUMBER_LITERAL].freeze
table["'".ord] = [:string, SINGLE_STRING_LITERAL].freeze
table["\"".ord] = [:string, DOUBLE_STRING_LITERAL].freeze
table.freeze
end
SPECIAL_TABLE = [].tap do |table|
table["|".ord] = PIPE
table[".".ord] = DOT
table[":".ord] = COLON
table[",".ord] = COMMA
table["[".ord] = OPEN_SQUARE
table["]".ord] = CLOSE_SQUARE
table["(".ord] = OPEN_ROUND
table[")".ord] = CLOSE_ROUND
table["?".ord] = QUESTION
table["-".ord] = DASH
end
NUMBER_TABLE = [].tap do |table|
"0".upto("9") do |c|
table[c.ord] = true
end
table.freeze
end
# rubocop:disable Metrics/BlockNesting
class << self
def tokenize(ss)
output = []
until ss.eos?
ss.skip(WHITESPACE)
break if ss.eos?
start_pos = ss.pos
peeked = ss.peek_byte
if (special = SPECIAL_TABLE[peeked])
ss.scan_byte
# Special case for ".."
if special.equal?(DOT) && ss.peek_byte == DOT_ORD
ss.scan_byte
output << DOTDOT
elsif special.equal?(DASH)
# Special case for negative numbers
if (peeked_byte = ss.peek_byte) && NUMBER_TABLE[peeked_byte]
ss.pos -= 1
output << [:number, ss.scan(NUMBER_LITERAL)]
else
output << special
end
else
output << special
end
elsif (sub_table = TWO_CHARS_COMPARISON_JUMP_TABLE[peeked])
ss.scan_byte
if (peeked_byte = ss.peek_byte) && (found = sub_table[peeked_byte])
output << found
ss.scan_byte
else
raise_syntax_error(start_pos, ss)
end
elsif (sub_table = COMPARISON_JUMP_TABLE[peeked])
ss.scan_byte
if (peeked_byte = ss.peek_byte) && (found = sub_table[peeked_byte])
output << found
ss.scan_byte
else
output << SINGLE_COMPARISON_TOKENS[peeked]
end
else
raise SyntaxError, "Unexpected character #{c}"
type, pattern = NEXT_MATCHER_JUMP_TABLE[peeked]
if type && (t = ss.scan(pattern))
# Special case for "contains"
output << if type == :id && t == "contains" && output.last&.first != :dot
COMPARISON_CONTAINS
else
[type, t]
end
else
raise_syntax_error(start_pos, ss)
end
end
end
@output << tok
# rubocop:enable Metrics/BlockNesting
output << EOS
rescue ::ArgumentError => e
if e.message == "invalid byte sequence in #{ss.string.encoding}"
raise SyntaxError, "Invalid byte sequence in #{ss.string.encoding}"
else
raise
end
end
@output << [:end_of_string]
def raise_syntax_error(start_pos, ss)
ss.pos = start_pos
# the character could be a UTF-8 character, use getch to get all the bytes
raise SyntaxError, "Unexpected character #{ss.getch}"
end
end
end
end
+4
View File
@@ -2,12 +2,14 @@
errors:
syntax:
tag_unexpected_args: "Syntax Error in '%{tag}' - Valid syntax: %{tag}"
block_tag_unexpected_args: "Syntax Error in '%{tag}' - Valid syntax: {% %{tag} %}{% end%{tag} %}"
assign: "Syntax Error in 'assign' - Valid syntax: assign [var] = [source]"
capture: "Syntax Error in 'capture' - Valid syntax: capture [var]"
case: "Syntax Error in 'case' - Valid syntax: case [condition]"
case_invalid_when: "Syntax Error in tag 'case' - Valid when condition: {% when [condition] [or condition2...] %}"
case_invalid_else: "Syntax Error in tag 'case' - Valid else condition: {% else %} (no parameters) "
cycle: "Syntax Error in 'cycle' - Valid syntax: cycle [name :] var [, var2, var3 ...]"
doc_invalid_nested: "Syntax Error in 'doc' - Nested doc tags are not allowed"
for: "Syntax Error in 'for loop' - Valid syntax: for [item] in [collection]"
for_invalid_in: "For loops require an 'in' clause"
for_invalid_attribute: "Invalid attribute in for loop. Valid attributes are limit and offset"
@@ -15,8 +17,10 @@
include: "Error in tag 'include' - Valid syntax: include '[template]' (with|for) [object|collection]"
inline_comment_invalid: "Syntax error in tag '#' - Each line of comments must be prefixed by the '#' character"
invalid_delimiter: "'%{tag}' is not a valid delimiter for %{block_name} tags. use %{block_delimiter}"
invalid_template_encoding: "Invalid template encoding"
render: "Syntax error in tag 'render' - Template name must be a quoted string"
table_row: "Syntax Error in 'table_row loop' - Valid syntax: table_row [item] in [collection] cols=3"
table_row_invalid_attribute: "Invalid attribute '%{attribute}' in tablerow loop. Valid attributes are cols, limit, offset, and range"
tag_never_closed: "'%{block_name}' tag was never closed"
tag_termination: "Tag '%{token}' was not properly terminated with regexp: %{tag_end}"
unexpected_else: "%{block_name} tag does not expect 'else' tag"
+47 -7
View File
@@ -3,14 +3,29 @@
module Liquid
class ParseContext
attr_accessor :locale, :line_number, :trim_whitespace, :depth
attr_reader :partial, :warnings, :error_mode
attr_reader :partial, :warnings, :error_mode, :environment, :expression_cache, :string_scanner, :cursor
def initialize(options = {})
def initialize(options = Const::EMPTY_HASH)
@environment = options.fetch(:environment, Environment.default)
@template_options = options ? options.dup : {}
@locale = @template_options[:locale] ||= I18n.new
@warnings = []
# constructing new StringScanner in Lexer, Tokenizer, etc is expensive
# This StringScanner will be shared by all of them
@string_scanner = StringScanner.new("")
@expression_cache = if options[:expression_cache].nil?
{}
elsif options[:expression_cache].respond_to?(:[]) && options[:expression_cache].respond_to?(:[]=)
options[:expression_cache]
elsif options[:expression_cache]
{}
end
@cursor = Cursor.new("")
self.depth = 0
self.partial = false
end
@@ -23,19 +38,44 @@ module Liquid
Liquid::BlockBody.new
end
def new_tokenizer(markup, start_line_number: nil, for_liquid_tag: false)
Tokenizer.new(markup, line_number: start_line_number, for_liquid_tag: for_liquid_tag)
def new_parser(input)
@string_scanner.string = input
Parser.new(@string_scanner)
end
def parse_expression(markup)
Expression.parse(markup)
def new_tokenizer(source, start_line_number: nil, for_liquid_tag: false)
Tokenizer.new(
source: source,
string_scanner: @string_scanner,
line_number: start_line_number,
for_liquid_tag: for_liquid_tag,
)
end
def safe_parse_expression(parser)
Expression.safe_parse(parser, @string_scanner, @expression_cache)
end
def parse_expression(markup, safe: false)
if !safe && @error_mode == :strict2
# parse_expression is a widely used API. To maintain backward
# compatibility while raising awareness about strict2 parser standards,
# the safe flag supports API users make a deliberate decision.
#
# In strict2 mode, markup MUST come from a string returned by the parser
# (e.g., parser.expression). We're not calling the parser here to
# prevent redundant parser overhead.
raise Liquid::InternalError, "unsafe parse_expression cannot be used in strict2 mode"
end
Expression.parse(markup, @string_scanner, @expression_cache)
end
def partial=(value)
@partial = value
@options = value ? partial_options : @template_options
@error_mode = @options[:error_mode] || Template.error_mode
@error_mode = @options[:error_mode] || @environment.error_mode
end
def partial_options
+1 -1
View File
@@ -36,7 +36,7 @@ module Liquid
protected
def children
@node.respond_to?(:nodelist) ? Array(@node.nodelist) : []
@node.respond_to?(:nodelist) ? Array(@node.nodelist) : Const::EMPTY_ARRAY
end
end
end
+6 -3
View File
@@ -3,8 +3,8 @@
module Liquid
class Parser
def initialize(input)
l = Lexer.new(input)
@tokens = l.tokenize
ss = input.is_a?(StringScanner) ? input : StringScanner.new(input)
@tokens = Lexer.tokenize(ss)
@p = 0 # pointer to current location
end
@@ -53,7 +53,7 @@ module Liquid
str = consume
str << variable_lookups
when :open_square
str = consume
str = consume.dup
str << expression
str << consume(:close_square)
str << variable_lookups
@@ -83,6 +83,9 @@ module Liquid
end
def variable_lookups
# Fast path: no lookups at all (most common case for simple identifiers)
return "" unless look(:dot) || look(:open_square)
str = +""
loop do
if look(:open_square)
+36 -3
View File
@@ -2,10 +2,25 @@
module Liquid
module ParserSwitching
# Do not use this.
#
# It's basically doing the same thing the {#parse_with_selected_parser},
# except this will try the strict parser regardless of the error mode,
# and fall back to the lax parser if the error mode is lax or warn,
# except when in strict2 mode where it uses the strict2 parser.
#
# @deprecated Use {#parse_with_selected_parser} instead.
def strict_parse_with_error_mode_fallback(markup)
return strict2_parse_with_error_context(markup) if strict2_mode?
strict_parse_with_error_context(markup)
rescue SyntaxError => e
case parse_context.error_mode
when :rigid
rigid_warn
raise
when :strict2
raise
when :strict
raise
when :warn
@@ -16,11 +31,13 @@ module Liquid
def parse_with_selected_parser(markup)
case parse_context.error_mode
when :strict then strict_parse_with_error_context(markup)
when :lax then lax_parse(markup)
when :rigid then rigid_warn && strict2_parse_with_error_context(markup)
when :strict2 then strict2_parse_with_error_context(markup)
when :strict then strict_parse_with_error_context(markup)
when :lax then lax_parse(markup)
when :warn
begin
strict_parse_with_error_context(markup)
strict2_parse_with_error_context(markup)
rescue SyntaxError => e
parse_context.warnings << e
lax_parse(markup)
@@ -28,8 +45,24 @@ module Liquid
end
end
def strict2_mode?
parse_context.error_mode == :strict2 || parse_context.error_mode == :rigid
end
private
def rigid_warn
Deprecations.warn(':rigid', ':strict2')
end
def strict2_parse_with_error_context(markup)
strict2_parse(markup)
rescue SyntaxError => e
e.line_number = line_number
e.markup_context = markup_context(markup)
raise e
end
def strict_parse_with_error_context(markup)
strict_parse(markup)
rescue SyntaxError => e
+3 -3
View File
@@ -2,9 +2,9 @@
module Liquid
class RangeLookup
def self.parse(start_markup, end_markup)
start_obj = Expression.parse(start_markup)
end_obj = Expression.parse(end_markup)
def self.parse(start_markup, end_markup, string_scanner, cache = nil)
start_obj = Expression.parse(start_markup, string_scanner, cache)
end_obj = Expression.parse(end_markup, string_scanner, cache)
if start_obj.respond_to?(:evaluate) || end_obj.respond_to?(:evaluate)
new(start_obj, end_obj)
else
+6 -6
View File
@@ -6,15 +6,15 @@ module Liquid
def initialize(registers = {})
@static = registers.is_a?(Registers) ? registers.static : registers
@changes = {}
@changes = nil
end
def []=(key, value)
@changes[key] = value
(@changes ||= {})[key] = value
end
def [](key)
if @changes.key?(key)
if @changes&.key?(key)
@changes[key]
else
@static[key]
@@ -22,13 +22,13 @@ module Liquid
end
def delete(key)
@changes.delete(key)
@changes&.delete(key)
end
UNDEFINED = Object.new
def fetch(key, default = UNDEFINED, &block)
if @changes.key?(key)
if @changes&.key?(key)
@changes.fetch(key)
elsif default != UNDEFINED
if block_given?
@@ -42,7 +42,7 @@ module Liquid
end
def key?(key)
@changes.key?(key) || @static.key?(key)
@changes&.key?(key) || @static.key?(key)
end
end
+23 -5
View File
@@ -2,24 +2,40 @@
module Liquid
class ResourceLimits
attr_accessor :render_length_limit, :render_score_limit, :assign_score_limit
attr_reader :render_score, :assign_score
attr_accessor :render_length_limit,
:render_score_limit,
:assign_score_limit,
:cumulative_render_score_limit,
:cumulative_assign_score_limit
attr_reader :render_score,
:assign_score,
:last_capture_length,
:cumulative_render_score,
:cumulative_assign_score
def initialize(limits)
@render_length_limit = limits[:render_length_limit]
@render_score_limit = limits[:render_score_limit]
@assign_score_limit = limits[:assign_score_limit]
@render_length_limit = limits[:render_length_limit]
@render_score_limit = limits[:render_score_limit]
@assign_score_limit = limits[:assign_score_limit]
@cumulative_render_score_limit = limits[:cumulative_render_score_limit]
@cumulative_assign_score_limit = limits[:cumulative_assign_score_limit]
@cumulative_render_score = 0
@cumulative_assign_score = 0
reset
end
def increment_render_score(amount)
@render_score += amount
@cumulative_render_score += amount
raise_limits_reached if @render_score_limit && @render_score > @render_score_limit
raise_limits_reached if @cumulative_render_score_limit && @cumulative_render_score > @cumulative_render_score_limit
end
def increment_assign_score(amount)
@assign_score += amount
@cumulative_assign_score += amount
raise_limits_reached if @assign_score_limit && @assign_score > @assign_score_limit
raise_limits_reached if @cumulative_assign_score_limit && @cumulative_assign_score > @cumulative_assign_score_limit
end
# update either render_length or assign_score based on whether or not the writes are captured
@@ -47,6 +63,8 @@ module Liquid
@reached_limit = false
@last_capture_length = nil
@render_score = @assign_score = 0
raise_limits_reached if @cumulative_render_score_limit && @cumulative_render_score > @cumulative_render_score_limit
raise_limits_reached if @cumulative_assign_score_limit && @cumulative_assign_score > @cumulative_assign_score_limit
end
def with_capture
+269 -81
View File
@@ -3,16 +3,24 @@
require 'cgi'
require 'base64'
require 'bigdecimal'
module Liquid
module StandardFilters
MAX_I32 = (1 << 31) - 1
private_constant :MAX_I32
MIN_I64 = -(1 << 63)
MAX_I64 = (1 << 63) - 1
I64_RANGE = MIN_I64..MAX_I64
private_constant :MIN_I64, :MAX_I64, :I64_RANGE
supports_64bit_indices = begin
[][1 << 33, 1 << 33]
true
rescue RangeError
false
end
INDEX_RANGE = if supports_64bit_indices
(-(1 << 63))..((1 << 63) - 1)
else
(-(1 << 31))..((1 << 31) - 1)
end
private_constant :INDEX_RANGE
HTML_ESCAPE = {
'&' => '&amp;',
@@ -29,6 +37,19 @@ module Liquid
)
STRIP_HTML_TAGS = /<.*?>/m
class << self
def try_coerce_encoding(input, encoding:)
original_encoding = input.encoding
if input.encoding != encoding
input.force_encoding(encoding)
unless input.valid_encoding?
input.force_encoding(original_encoding)
end
end
input
end
end
# @liquid_public_docs
# @liquid_type filter
# @liquid_category array
@@ -51,7 +72,7 @@ module Liquid
# @liquid_syntax string | downcase
# @liquid_return [string]
def downcase(input)
input.to_s.downcase
Utils.to_s(input).downcase
end
# @liquid_public_docs
@@ -62,7 +83,7 @@ module Liquid
# @liquid_syntax string | upcase
# @liquid_return [string]
def upcase(input)
input.to_s.upcase
Utils.to_s(input).upcase
end
# @liquid_public_docs
@@ -73,7 +94,7 @@ module Liquid
# @liquid_syntax string | capitalize
# @liquid_return [string]
def capitalize(input)
input.to_s.capitalize
Utils.to_s(input).capitalize
end
# @liquid_public_docs
@@ -84,7 +105,7 @@ module Liquid
# @liquid_syntax string | escape
# @liquid_return [string]
def escape(input)
CGI.escapeHTML(input.to_s) unless input.nil?
CGI.escapeHTML(Utils.to_s(input)) unless input.nil?
end
alias_method :h, :escape
@@ -96,7 +117,7 @@ module Liquid
# @liquid_syntax string | escape_once
# @liquid_return [string]
def escape_once(input)
input.to_s.gsub(HTML_ESCAPE_ONCE_REGEXP, HTML_ESCAPE)
Utils.to_s(input).gsub(HTML_ESCAPE_ONCE_REGEXP, HTML_ESCAPE)
end
# @liquid_public_docs
@@ -111,7 +132,7 @@ module Liquid
# @liquid_syntax string | url_encode
# @liquid_return [string]
def url_encode(input)
CGI.escape(input.to_s) unless input.nil?
CGI.escape(Utils.to_s(input)) unless input.nil?
end
# @liquid_public_docs
@@ -125,7 +146,7 @@ module Liquid
def url_decode(input)
return if input.nil?
result = CGI.unescape(input.to_s)
result = CGI.unescape(Utils.to_s(input))
raise Liquid::ArgumentError, "invalid byte sequence in #{result.encoding}" unless result.valid_encoding?
result
@@ -139,7 +160,7 @@ module Liquid
# @liquid_syntax string | base64_encode
# @liquid_return [string]
def base64_encode(input)
Base64.strict_encode64(input.to_s)
Base64.strict_encode64(Utils.to_s(input))
end
# @liquid_public_docs
@@ -150,7 +171,8 @@ module Liquid
# @liquid_syntax string | base64_decode
# @liquid_return [string]
def base64_decode(input)
Base64.strict_decode64(input.to_s)
input = Utils.to_s(input)
StandardFilters.try_coerce_encoding(Base64.strict_decode64(input), encoding: input.encoding)
rescue ::ArgumentError
raise Liquid::ArgumentError, "invalid base64 provided to base64_decode"
end
@@ -163,7 +185,7 @@ module Liquid
# @liquid_syntax string | base64_url_safe_encode
# @liquid_return [string]
def base64_url_safe_encode(input)
Base64.urlsafe_encode64(input.to_s)
Base64.urlsafe_encode64(Utils.to_s(input))
end
# @liquid_public_docs
@@ -174,7 +196,8 @@ module Liquid
# @liquid_syntax string | base64_url_safe_decode
# @liquid_return [string]
def base64_url_safe_decode(input)
Base64.urlsafe_decode64(input.to_s)
input = Utils.to_s(input)
StandardFilters.try_coerce_encoding(Base64.urlsafe_decode64(input), encoding: input.encoding)
rescue ::ArgumentError
raise Liquid::ArgumentError, "invalid base64 provided to base64_url_safe_decode"
end
@@ -197,14 +220,14 @@ module Liquid
if input.is_a?(Array)
input.slice(offset, length) || []
else
input.to_s.slice(offset, length) || ''
Utils.to_s(input).slice(offset, length) || ''
end
rescue RangeError
if I64_RANGE.cover?(length) && I64_RANGE.cover?(offset)
if INDEX_RANGE.cover?(length) && INDEX_RANGE.cover?(offset)
raise # unexpected error
end
offset = offset.clamp(I64_RANGE)
length = length.clamp(I64_RANGE)
offset = offset.clamp(INDEX_RANGE)
length = length.clamp(INDEX_RANGE)
retry
end
end
@@ -221,10 +244,10 @@ module Liquid
# @liquid_return [string]
def truncate(input, length = 50, truncate_string = "...")
return if input.nil?
input_str = input.to_s
input_str = Utils.to_s(input)
length = Utils.to_integer(length)
truncate_string_str = truncate_string.to_s
truncate_string_str = Utils.to_s(truncate_string)
l = length - truncate_string_str.length
l = 0 if l < 0
@@ -248,21 +271,75 @@ module Liquid
# @liquid_return [string]
def truncatewords(input, words = 15, truncate_string = "...")
return if input.nil?
input = input.to_s
input = Utils.to_s(input)
words = Utils.to_integer(words)
words = 1 if words <= 0
wordlist = begin
input.split(" ", words + 1)
rescue RangeError
# integer too big for String#split, but we can semantically assume no truncation is needed
return input if words + 1 > MAX_I32
raise # unexpected error
end
return input if wordlist.length <= words
return input if words + 1 > MAX_I32
wordlist.pop
wordlist.join(" ").concat(truncate_string.to_s)
# Scan words tracking byte positions; build the normalized (single-space)
# result string only when truncation is actually needed.
len = input.bytesize
pos = 0
word_count = 0
# Flat array of [start, end, start, end, ...] for up to `words` words.
# Avoids allocating a result string in the common no-truncation case.
positions = []
# Skip leading whitespace
while pos < len
break unless ByteTables::WHITESPACE[input.getbyte(pos)]
pos += 1
end
while pos < len
word_start = pos
word_count += 1
# Scan to end of word
while pos < len
break if ByteTables::WHITESPACE[input.getbyte(pos)]
pos += 1
end
if word_count <= words
positions.push(word_start, pos) # [start, end, start, end, ...]
else
# Truncation confirmed — build normalized result from stored positions
result = +input.byteslice(positions[0], positions[1] - positions[0])
i = 2
while i < positions.length
result << " " << input.byteslice(positions[i], positions[i + 1] - positions[i])
i += 2
end
return result << Utils.to_s(truncate_string)
end
# Skip whitespace between words
while pos < len
break unless ByteTables::WHITESPACE[input.getbyte(pos)]
pos += 1
end
end
# Fewer words than requested — no truncation needed, return original unchanged.
return input if word_count < words
# Exactly `words` words. Ruby's split(" ", words+1) would produce a words+1-th
# empty element when input has trailing whitespace, triggering the truncation path.
# Match that behaviour: if the input ends with whitespace, normalize and append
# truncate_string even though no word was cut.
if len > 0 && ByteTables::WHITESPACE[input.getbyte(len - 1)]
result = +input.byteslice(positions[0], positions[1] - positions[0])
i = 2
while i < positions.length
result << " " << input.byteslice(positions[i], positions[i + 1] - positions[i])
i += 2
end
return result << Utils.to_s(truncate_string)
end
input
end
# @liquid_public_docs
@@ -273,7 +350,22 @@ module Liquid
# @liquid_syntax string | split: string
# @liquid_return [array[string]]
def split(input, pattern)
input.to_s.split(pattern.to_s)
pattern = Utils.to_s(pattern)
input = Utils.to_s(input)
input.split(pattern)
end
# @liquid_public_docs
# @liquid_type filter
# @liquid_category string
# @liquid_summary
# Removes leading and trailing whitespace and collapses consecutive whitespace to a single space.
# @liquid_syntax string | squish
# @liquid_return [string]
def squish(input)
return if input.nil?
Utils.to_s(input).strip.gsub(/\s+/, ' ')
end
# @liquid_public_docs
@@ -284,7 +376,8 @@ module Liquid
# @liquid_syntax string | strip
# @liquid_return [string]
def strip(input)
input.to_s.strip
input = Utils.to_s(input)
input.strip
end
# @liquid_public_docs
@@ -295,7 +388,8 @@ module Liquid
# @liquid_syntax string | lstrip
# @liquid_return [string]
def lstrip(input)
input.to_s.lstrip
input = Utils.to_s(input)
input.lstrip
end
# @liquid_public_docs
@@ -306,7 +400,8 @@ module Liquid
# @liquid_syntax string | rstrip
# @liquid_return [string]
def rstrip(input)
input.to_s.rstrip
input = Utils.to_s(input)
input.rstrip
end
# @liquid_public_docs
@@ -317,8 +412,9 @@ module Liquid
# @liquid_syntax string | strip_html
# @liquid_return [string]
def strip_html(input)
input = Utils.to_s(input)
empty = ''
result = input.to_s.gsub(STRIP_HTML_BLOCKS, empty)
result = input.gsub(STRIP_HTML_BLOCKS, empty)
result.gsub!(STRIP_HTML_TAGS, empty)
result
end
@@ -331,7 +427,8 @@ module Liquid
# @liquid_syntax string | strip_newlines
# @liquid_return [string]
def strip_newlines(input)
input.to_s.gsub(/\r?\n/, '')
input = Utils.to_s(input)
input.gsub(/\r?\n/, '')
end
# @liquid_public_docs
@@ -342,6 +439,7 @@ module Liquid
# @liquid_syntax array | join
# @liquid_return [string]
def join(input, glue = ' ')
glue = Utils.to_s(glue)
InputIterator.new(input, context).join(glue)
end
@@ -409,29 +507,59 @@ module Liquid
# @liquid_syntax array | where: string, string
# @liquid_return [array[untyped]]
def where(input, property, target_value = nil)
ary = InputIterator.new(input, context)
filter_array(input, property, target_value) { |ary, &block| ary.select(&block) }
end
if ary.empty?
[]
elsif target_value.nil?
ary.select do |item|
item[property]
rescue TypeError
raise_property_error(property)
rescue NoMethodError
return nil unless item.respond_to?(:[])
raise
end
else
ary.select do |item|
item[property] == target_value
rescue TypeError
raise_property_error(property)
rescue NoMethodError
return nil unless item.respond_to?(:[])
raise
end
end
# @liquid_public_docs
# @liquid_type filter
# @liquid_category array
# @liquid_summary
# Filters an array to exclude items with a specific property value.
# @liquid_description
# This requires you to provide both the property name and the associated value.
# @liquid_syntax array | reject: string, string
# @liquid_return [array[untyped]]
def reject(input, property, target_value = nil)
filter_array(input, property, target_value) { |ary, &block| ary.reject(&block) }
end
# @liquid_public_docs
# @liquid_type filter
# @liquid_category array
# @liquid_summary
# Tests if any item in an array has a specific property value.
# @liquid_description
# This requires you to provide both the property name and the associated value.
# @liquid_syntax array | has: string, string
# @liquid_return [boolean]
def has(input, property, target_value = nil)
filter_array(input, property, target_value, false) { |ary, &block| ary.any?(&block) }
end
# @liquid_public_docs
# @liquid_type filter
# @liquid_category array
# @liquid_summary
# Returns the first item in an array with a specific property value.
# @liquid_description
# This requires you to provide both the property name and the associated value.
# @liquid_syntax array | find: string, string
# @liquid_return [untyped]
def find(input, property, target_value = nil)
filter_array(input, property, target_value, nil) { |ary, &block| ary.find(&block) }
end
# @liquid_public_docs
# @liquid_type filter
# @liquid_category array
# @liquid_summary
# Returns the index of the first item in an array with a specific property value.
# @liquid_description
# This requires you to provide both the property name and the associated value.
# @liquid_syntax array | find_index: string, string
# @liquid_return [number]
def find_index(input, property, target_value = nil)
filter_array(input, property, target_value, nil) { |ary, &block| ary.find_index(&block) }
end
# @liquid_public_docs
@@ -528,7 +656,10 @@ module Liquid
# @liquid_syntax string | replace: string, string
# @liquid_return [string]
def replace(input, string, replacement = '')
input.to_s.gsub(string.to_s, replacement.to_s)
string = Utils.to_s(string)
replacement = Utils.to_s(replacement)
input = Utils.to_s(input)
input.gsub(string, replacement)
end
# @liquid_public_docs
@@ -539,7 +670,10 @@ module Liquid
# @liquid_syntax string | replace_first: string, string
# @liquid_return [string]
def replace_first(input, string, replacement = '')
input.to_s.sub(string.to_s, replacement.to_s)
string = Utils.to_s(string)
replacement = Utils.to_s(replacement)
input = Utils.to_s(input)
input.sub(string, replacement)
end
# @liquid_public_docs
@@ -550,9 +684,9 @@ module Liquid
# @liquid_syntax string | replace_last: string, string
# @liquid_return [string]
def replace_last(input, string, replacement)
input = input.to_s
string = string.to_s
replacement = replacement.to_s
input = Utils.to_s(input)
string = Utils.to_s(string)
replacement = Utils.to_s(replacement)
start_index = input.rindex(string)
@@ -604,7 +738,9 @@ module Liquid
# @liquid_syntax string | append: string
# @liquid_return [string]
def append(input, string)
input.to_s + string.to_s
input = Utils.to_s(input)
string = Utils.to_s(string)
input + string
end
# @liquid_public_docs
@@ -633,7 +769,9 @@ module Liquid
# @liquid_syntax string | prepend: string
# @liquid_return [string]
def prepend(input, string)
string.to_s + input.to_s
input = Utils.to_s(input)
string = Utils.to_s(string)
string + input
end
# @liquid_public_docs
@@ -644,10 +782,20 @@ module Liquid
# @liquid_syntax string | newline_to_br
# @liquid_return [string]
def newline_to_br(input)
input.to_s.gsub(/\r?\n/, "<br />\n")
input = Utils.to_s(input)
input.gsub(/\r?\n/, "<br />\n")
end
# Reformat a date using Ruby's core Time#strftime( string ) -> string
# @liquid_public_docs
# @liquid_type filter
# @liquid_category date
# @liquid_summary
# Formats a date according to a specified format string.
# @liquid_description
# This filter formats a date using various format specifiers. If the format string is empty,
# the original input is returned. If the input cannot be converted to a date, the original input is returned.
#
# The following format specifiers can be used:
#
# %a - The abbreviated weekday name (``Sun'')
# %A - The full weekday name (``Sunday'')
@@ -676,14 +824,15 @@ module Liquid
# %Y - Year with century
# %Z - Time zone name
# %% - Literal ``%'' character
#
# See also: http://www.ruby-doc.org/core/Time.html#method-i-strftime
# @liquid_syntax date | date: string
# @liquid_return [string]
def date(input, format)
return input if format.to_s.empty?
str_format = Utils.to_s(format)
return input if str_format.empty?
return input unless (date = Utils.to_date(input))
date.strftime(format.to_s)
date.strftime(str_format)
end
# @liquid_public_docs
@@ -694,6 +843,8 @@ module Liquid
# @liquid_syntax array | first
# @liquid_return [untyped]
def first(array)
# ActiveSupport returns "" for empty strings, not nil
return array[0] || "" if array.is_a?(String)
array.first if array.respond_to?(:first)
end
@@ -705,6 +856,8 @@ module Liquid
# @liquid_syntax array | last
# @liquid_return [untyped]
def last(array)
# ActiveSupport returns "" for empty strings, not nil
return array[-1] || "" if array.is_a?(String)
array.last if array.respond_to?(:last)
end
@@ -862,7 +1015,7 @@ module Liquid
# - [`nil`](/docs/api/liquid/basics#nil)
# @liquid_syntax variable | default: variable
# @liquid_return [untyped]
# @liquid_optional_param allow_false [boolean] Whether to use false values instead of the default.
# @liquid_optional_param allow_false: [boolean] Whether to use false values instead of the default.
def default(input, default_value = '', options = {})
options = {} unless options.is_a?(Hash)
false_check = options['allow_false'] ? input.nil? : !Liquid::Utils.to_liquid_value(input)
@@ -892,17 +1045,38 @@ module Liquid
raise_property_error(property)
end
InputIterator.new(values_for_sum, context).sum do |item|
result = InputIterator.new(values_for_sum, context).sum do |item|
Utils.to_number(item)
end
result.is_a?(BigDecimal) ? result.to_f : result
end
private
attr_reader :context
def filter_array(input, property, target_value, default_value = [], &block)
ary = InputIterator.new(input, context)
return default_value if ary.empty?
block.call(ary) do |item|
if target_value.nil?
item[property]
else
item[property] == target_value
end
rescue TypeError
raise_property_error(property)
rescue NoMethodError
return nil unless item.respond_to?(:[])
raise
end
end
def raise_property_error(property)
raise Liquid::ArgumentError, "cannot select the property '#{property}'"
raise Liquid::ArgumentError, "cannot select the property '#{Utils.to_s(property)}'"
end
def apply_operation(input, operand, operation)
@@ -927,6 +1101,8 @@ module Liquid
def nil_safe_casecmp(a, b)
if !a.nil? && !b.nil?
a.to_s.casecmp(b.to_s)
elsif a.nil? && b.nil?
0
else
a.nil? ? 1 : -1
end
@@ -949,7 +1125,18 @@ module Liquid
end
def join(glue)
to_a.join(glue.to_s)
first = true
output = +""
each do |item|
if first
first = false
else
output << glue
end
output << Liquid::Utils.to_s(item)
end
output
end
def concat(args)
@@ -961,7 +1148,10 @@ module Liquid
end
def uniq(&block)
to_a.uniq(&block)
to_a.uniq do |item|
item = Utils.to_liquid_value(item)
block ? yield(item) : item
end
end
def compact
@@ -982,6 +1172,4 @@ module Liquid
end
end
end
Template.register_filter(StandardFilters)
end
-41
View File
@@ -1,41 +0,0 @@
# frozen_string_literal: true
module Liquid
# StrainerFactory is the factory for the filters system.
module StrainerFactory
extend self
def add_global_filter(filter)
strainer_class_cache.clear
GlobalCache.add_filter(filter)
end
def create(context, filters = [])
strainer_from_cache(filters).new(context)
end
def global_filter_names
GlobalCache.filter_method_names
end
GlobalCache = Class.new(StrainerTemplate)
private
def strainer_from_cache(filters)
if filters.empty?
GlobalCache
else
strainer_class_cache[filters] ||= begin
klass = Class.new(GlobalCache)
filters.each { |f| klass.add_filter(f) }
klass
end
end
end
def strainer_class_cache
@strainer_class_cache ||= {}
end
end
end
+26
View File
@@ -58,5 +58,31 @@ module Liquid
rescue ::ArgumentError => e
raise Liquid::ArgumentError, e.message, e.backtrace
end
# Arity-specialized filter invocation.
# Avoids *args splat allocation for the common 0-arg and 1-arg cases.
# `invoke` (general case) still uses *args for 2+ extra arguments.
{
invoke_single: ['input'],
invoke_two: ['input', 'arg1'],
}.each do |method_name, params|
all_params = (["method"] + params).join(", ")
send_params = params.join(", ")
# __LINE__ + 1 is a parse-time constant; both generated methods will report
# the same file:line in backtraces. The method name in the trace distinguishes them.
module_eval(<<~RUBY, __FILE__, __LINE__ + 1)
def #{method_name}(#{all_params})
if self.class.invokable?(method)
send(method, #{send_params})
elsif @context.strict_filters
raise Liquid::UndefinedFilter, "undefined filter \#{method}"
else
input
end
rescue ::ArgumentError => e
raise Liquid::ArgumentError, e.message, e.backtrace
end
RUBY
end
end
end
+11 -3
View File
@@ -1,5 +1,8 @@
# frozen_string_literal: true
require 'liquid/tag/disabler'
require 'liquid/tag/disableable'
module Liquid
class Tag
attr_reader :nodelist, :tag_name, :line_number, :parse_context
@@ -54,7 +57,8 @@ module Liquid
# of the `render_to_output_buffer` method will become the default and the `render`
# method will be removed.
def render_to_output_buffer(context, output)
output << render(context)
render_result = render(context)
output << render_result if render_result
output
end
@@ -64,8 +68,12 @@ module Liquid
private
def parse_expression(markup)
parse_context.parse_expression(markup)
def safe_parse_expression(parser)
parse_context.safe_parse_expression(parser)
end
def parse_expression(markup, safe: false)
parse_context.parse_expression(markup, safe: safe)
end
end
end
+49
View File
@@ -0,0 +1,49 @@
# frozen_string_literal: true
require_relative "tags/table_row"
require_relative "tags/echo"
require_relative "tags/if"
require_relative "tags/break"
require_relative "tags/inline_comment"
require_relative "tags/for"
require_relative "tags/assign"
require_relative "tags/ifchanged"
require_relative "tags/case"
require_relative "tags/include"
require_relative "tags/continue"
require_relative "tags/capture"
require_relative "tags/decrement"
require_relative "tags/unless"
require_relative "tags/increment"
require_relative "tags/comment"
require_relative "tags/raw"
require_relative "tags/render"
require_relative "tags/cycle"
require_relative "tags/doc"
module Liquid
module Tags
STANDARD_TAGS = {
'cycle' => Cycle,
'render' => Render,
'raw' => Raw,
'comment' => Comment,
'increment' => Increment,
'unless' => Unless,
'decrement' => Decrement,
'capture' => Capture,
'continue' => Continue,
'include' => Include,
'case' => Case,
'ifchanged' => Ifchanged,
'assign' => Assign,
'for' => For,
'#' => InlineComment,
'break' => Break,
'if' => If,
'echo' => Echo,
'tablerow' => TableRow,
'doc' => Doc,
}.freeze
end
end
+4 -2
View File
@@ -9,6 +9,10 @@ module Liquid
# Creates a new variable.
# @liquid_description
# You can create variables of any [basic type](/docs/api/liquid/basics#types), [object](/docs/api/liquid/objects), or object property.
#
# > Caution:
# > Predefined Liquid objects can be overridden by variables with the same name.
# > To make sure that you can access all Liquid objects, make sure that your variable name doesn't match a predefined object's name.
# @liquid_syntax
# {% assign variable_name = value %}
# @liquid_syntax_keyword variable_name The name of the variable being created.
@@ -72,6 +76,4 @@ module Liquid
end
end
end
Template.register_tag('assign', Assign)
end
-2
View File
@@ -26,6 +26,4 @@ module Liquid
output
end
end
Template.register_tag('break', Break)
end
+4 -2
View File
@@ -9,6 +9,10 @@ module Liquid
# Creates a new variable with a string value.
# @liquid_description
# You can create complex strings with Liquid logic and variables.
#
# > Caution:
# > Predefined Liquid objects can be overridden by variables with the same name.
# > To make sure that you can access all Liquid objects, make sure that your variable name doesn't match a predefined object's name.
# @liquid_syntax
# {% capture variable %}
# value
@@ -39,6 +43,4 @@ module Liquid
true
end
end
Template.register_tag('capture', Capture)
end
+42 -8
View File
@@ -31,12 +31,7 @@ module Liquid
def initialize(tag_name, markup, options)
super
@blocks = []
if markup =~ Syntax
@left = parse_expression(Regexp.last_match(1))
else
raise SyntaxError, options[:locale].t("errors.syntax.case")
end
parse_with_selected_parser(markup)
end
def parse(tokens)
@@ -91,9 +86,50 @@ module Liquid
private
def strict2_parse(markup)
parser = @parse_context.new_parser(markup)
@left = safe_parse_expression(parser)
parser.consume(:end_of_string)
end
def strict_parse(markup)
lax_parse(markup)
end
def lax_parse(markup)
if markup =~ Syntax
@left = parse_expression(Regexp.last_match(1))
else
raise SyntaxError, options[:locale].t("errors.syntax.case")
end
end
def record_when_condition(markup)
body = new_body
if strict2_mode?
parse_strict2_when(markup, body)
else
parse_lax_when(markup, body)
end
end
def parse_strict2_when(markup, body)
parser = @parse_context.new_parser(markup)
loop do
expr = Condition.parse_expression(parse_context, parser.expression, safe: true)
block = Condition.new(@left, '==', expr)
block.attach(body)
@blocks << block
break unless parser.id?('or') || parser.consume?(:comma)
end
parser.consume(:end_of_string)
end
def parse_lax_when(markup, body)
while markup
unless markup =~ WhenSyntax
raise SyntaxError, options[:locale].t("errors.syntax.case_invalid_when")
@@ -123,6 +159,4 @@ module Liquid
end
end
end
Template.register_tag('case', Case)
end
+59 -2
View File
@@ -25,7 +25,64 @@ module Liquid
def blank?
true
end
end
Template.register_tag('comment', Comment)
private
def parse_body(body, tokenizer)
if parse_context.depth >= MAX_DEPTH
raise StackLevelError, "Nesting too deep"
end
parse_context.depth += 1
comment_tag_depth = 1
begin
# Consume tokens without creating child nodes.
# The children tag doesn't require to be a valid Liquid except the comment and raw tag.
# The child comment and raw tag must be closed.
while (token = tokenizer.send(:shift))
tag_name = if tokenizer.for_liquid_tag
next if token.empty? || token.match?(BlockBody::WhitespaceOrNothing)
tag_name_match = BlockBody::LiquidTagToken.match(token)
next if tag_name_match.nil?
tag_name_match[1]
else
token =~ BlockBody::FullToken
Regexp.last_match(2)
end
case tag_name
when "raw"
parse_raw_tag_body(tokenizer)
when "comment"
comment_tag_depth += 1
when "endcomment"
comment_tag_depth -= 1
end
if comment_tag_depth.zero?
parse_context.trim_whitespace = (token[-3] == WhitespaceControl) unless tokenizer.for_liquid_tag
return false
end
end
raise_tag_never_closed(block_name)
ensure
parse_context.depth -= 1
end
false
end
def parse_raw_tag_body(tokenizer)
while (token = tokenizer.send(:shift))
return if token =~ BlockBody::FullTokenPossiblyInvalid && "endraw" == Regexp.last_match(2)
end
raise_tag_never_closed("raw")
end
end
end
-2
View File
@@ -17,6 +17,4 @@ module Liquid
output
end
end
Template.register_tag('continue', Continue)
end
+76 -13
View File
@@ -17,21 +17,17 @@ module Liquid
class Cycle < Tag
SimpleSyntax = /\A#{QuotedFragment}+/o
NamedSyntax = /\A(#{QuotedFragment})\s*\:\s*(.*)/om
UNNAMED_CYCLE_PATTERN = /\w+:0x\h{8}/
attr_reader :variables
def initialize(tag_name, markup, options)
super
case markup
when NamedSyntax
@variables = variables_from_string(Regexp.last_match(2))
@name = parse_expression(Regexp.last_match(1))
when SimpleSyntax
@variables = variables_from_string(markup)
@name = @variables.to_s
else
raise SyntaxError, options[:locale].t("errors.syntax.cycle")
end
parse_with_selected_parser(markup)
end
def named?
@is_named
end
def render_to_output_buffer(context, output)
@@ -59,19 +55,86 @@ module Liquid
private
# cycle [name:] expression(, expression)*
def strict2_parse(markup)
p = @parse_context.new_parser(markup)
@variables = []
raise SyntaxError, options[:locale].t("errors.syntax.cycle") if p.look(:end_of_string)
first_expression = safe_parse_expression(p)
if p.look(:colon)
# cycle name: expr1, expr2, ...
@name = first_expression
@is_named = true
p.consume(:colon)
# After the colon, parse the first variable (required for named cycles)
@variables << maybe_dup_lookup(safe_parse_expression(p))
else
# cycle expr1, expr2, ...
@variables << maybe_dup_lookup(first_expression)
end
# Parse remaining comma-separated expressions
while p.consume?(:comma)
break if p.look(:end_of_string)
@variables << maybe_dup_lookup(safe_parse_expression(p))
end
p.consume(:end_of_string)
unless @is_named
@name = @variables.to_s
@is_named = !@name.match?(UNNAMED_CYCLE_PATTERN)
end
end
def strict_parse(markup)
lax_parse(markup)
end
def lax_parse(markup)
case markup
when NamedSyntax
@variables = variables_from_string(Regexp.last_match(2))
@name = parse_expression(Regexp.last_match(1))
@is_named = true
when SimpleSyntax
@variables = variables_from_string(markup)
@name = @variables.to_s
@is_named = !@name.match?(UNNAMED_CYCLE_PATTERN)
else
raise SyntaxError, options[:locale].t("errors.syntax.cycle")
end
end
def variables_from_string(markup)
markup.split(',').collect do |var|
var =~ /\s*(#{QuotedFragment})\s*/o
Regexp.last_match(1) ? parse_expression(Regexp.last_match(1)) : nil
next unless Regexp.last_match(1)
var = parse_expression(Regexp.last_match(1))
maybe_dup_lookup(var)
end.compact
end
# For backwards compatibility, whenever a lookup is used in an unnamed cycle,
# we make it so that the @variables.to_s produces different strings for cycles
# called with the same arguments (since @variables.to_s is used as the cycle counter key)
# This makes it so {% cycle a, b %} and {% cycle a, b %} have independent counters even if a and b share value.
# This is not true for literal values, {% cycle "a", "b" %} and {% cycle "a", "b" %} share the same counter.
# I was really scratching my head about this one, but migrating away from this would be more headache
# than it's worth. So we're keeping this quirk for now.
def maybe_dup_lookup(var)
var.is_a?(VariableLookup) ? var.dup : var
end
class ParseTreeVisitor < Liquid::ParseTreeVisitor
def children
Array(@node.variables)
end
end
end
Template.register_tag('cycle', Cycle)
end
+5 -3
View File
@@ -7,10 +7,14 @@ module Liquid
# @liquid_name decrement
# @liquid_summary
# Creates a new variable, with a default value of -1, that's decreased by 1 with each subsequent call.
#
# > Caution:
# > Predefined Liquid objects can be overridden by variables with the same name.
# > To make sure that you can access all Liquid objects, make sure that your variable name doesn't match a predefined object's name.
# @liquid_description
# Variables that are declared with `decrement` are unique to the [layout](/themes/architecture/layouts), [template](/themes/architecture/templates),
# or [section](/themes/architecture/sections) file that they're created in. However, the variable is shared across
# [snippets](/themes/architecture#snippets) included in the file.
# [snippets](/themes/architecture/snippets) included in the file.
#
# Similarly, variables that are created with `decrement` are independent from those created with [`assign`](/docs/api/liquid/tags/assign)
# and [`capture`](/docs/api/liquid/tags/capture). However, `decrement` and [`increment`](/docs/api/liquid/tags/increment) share
@@ -35,6 +39,4 @@ module Liquid
output
end
end
Template.register_tag('decrement', Decrement)
end
+81
View File
@@ -0,0 +1,81 @@
# frozen_string_literal: true
module Liquid
# @liquid_public_docs
# @liquid_type tag
# @liquid_category syntax
# @liquid_name doc
# @liquid_summary
# Documents template elements with annotations.
# @liquid_description
# The `doc` tag allows developers to include documentation within Liquid
# templates. Any content inside `doc` tags is not rendered or outputted.
# Liquid code inside will be parsed but not executed. This facilitates
# tooling support for features like code completion, linting, and inline
# documentation.
#
# For detailed documentation syntax and examples, see the
# [`LiquidDoc` reference](/docs/storefronts/themes/tools/liquid-doc).
#
# @liquid_syntax
# {% doc %}
# Renders a message.
#
# @param {string} foo - A string value.
# @param {string} [bar] - An optional string value.
#
# @example
# {% render 'message', foo: 'Hello', bar: 'World' %}
# {% enddoc %}
class Doc < Block
NO_UNEXPECTED_ARGS = /\A\s*\z/
def initialize(tag_name, markup, parse_context)
super
ensure_valid_markup(tag_name, markup, parse_context)
end
def parse(tokens)
@body = +""
while (token = tokens.shift)
tag_name = token =~ BlockBody::FullTokenPossiblyInvalid && Regexp.last_match(2)
raise_nested_doc_error if tag_name == @tag_name
if tag_name == block_delimiter
parse_context.trim_whitespace = (token[-3] == WhitespaceControl)
@body << Regexp.last_match(1) if Regexp.last_match(1) != ""
return
end
@body << token unless token.empty?
end
raise_tag_never_closed(block_name)
end
def render_to_output_buffer(_context, output)
output
end
def blank?
@body.empty?
end
def nodelist
[@body]
end
private
def ensure_valid_markup(tag_name, markup, parse_context)
unless NO_UNEXPECTED_ARGS.match?(markup)
raise SyntaxError, parse_context.locale.t("errors.syntax.block_tag_unexpected_args", tag: tag_name)
end
end
def raise_nested_doc_error
raise SyntaxError, parse_context.locale.t("errors.syntax.doc_invalid_nested")
end
end
end
-2
View File
@@ -36,6 +36,4 @@ module Liquid
end
end
end
Template.register_tag('echo', Echo)
end
+56 -36
View File
@@ -20,13 +20,11 @@ module Liquid
# @liquid_syntax_keyword variable The current item in the array.
# @liquid_syntax_keyword array The array to iterate over.
# @liquid_syntax_keyword expression The expression to render for each iteration.
# @liquid_optional_param limit [number] The number of iterations to perform.
# @liquid_optional_param offset [number] The 1-based index to start iterating at.
# @liquid_optional_param limit: [number] The number of iterations to perform.
# @liquid_optional_param offset: [number] The 1-based index to start iterating at.
# @liquid_optional_param range [untyped] A custom numeric range to iterate over.
# @liquid_optional_param reversed [untyped] Iterate in reverse order.
class For < Block
Syntax = /\A(#{VariableSegment}+)\s+in\s+(#{QuotedFragment}+)\s*(reversed)?/o
attr_reader :collection_name, :variable_name, :limit, :from
def initialize(tag_name, markup, options)
@@ -72,28 +70,62 @@ module Liquid
protected
# Fast byte-level parser for "var in collection [reversed] [limit:N] [offset:N]"
def lax_parse(markup)
if markup =~ Syntax
@variable_name = Regexp.last_match(1)
collection_name = Regexp.last_match(2)
@reversed = !!Regexp.last_match(3)
@name = "#{@variable_name}-#{collection_name}"
@collection_name = parse_expression(collection_name)
markup.scan(TagAttributes) do |key, value|
set_attribute(key, value)
c = @parse_context.cursor
c.reset(markup)
c.skip_ws
# Parse variable name
var_start = c.pos
var_len = c.skip_id
raise SyntaxError, options[:locale].t("errors.syntax.for") if var_len == 0
@variable_name = c.slice(var_start, var_len)
# Expect "in"
c.skip_ws
raise SyntaxError, options[:locale].t("errors.syntax.for") unless c.expect_id("in")
c.skip_ws
# Parse collection name
col_start = c.pos
if c.peek_byte == Cursor::LPAREN
# Parenthesized range: (1..10)
depth = 1
c.scan_byte
while !c.eos? && depth > 0
b = c.scan_byte
depth += 1 if b == Cursor::LPAREN
depth -= 1 if b == Cursor::RPAREN
end
else
raise SyntaxError, options[:locale].t("errors.syntax.for")
c.skip_fragment
end
collection_name = c.slice(col_start, c.pos - col_start)
@name = "#{@variable_name}-#{collection_name}"
@collection_name = parse_expression(collection_name)
c.skip_ws
@reversed = c.expect_id("reversed")
c.skip_ws
# Parse limit:/offset: if present.
# Cursor doesn't handle key:value attributes — delegate to regex for limit:/offset:.
if !c.eos? && (rest = c.slice(c.pos, markup.bytesize - c.pos)).include?(':')
rest.scan(TagAttributes) do |key, value|
set_attribute(key, value)
end
end
end
def strict_parse(markup)
p = Parser.new(markup)
p = @parse_context.new_parser(markup)
@variable_name = p.consume(:id)
raise SyntaxError, options[:locale].t("errors.syntax.for_invalid_in") unless p.id?('in')
collection_name = p.expression
@collection_name = parse_expression(collection_name)
@collection_name = parse_expression(collection_name, safe: true)
@name = "#{@variable_name}-#{collection_name}"
@reversed = p.id?('reversed')
@@ -104,13 +136,15 @@ module Liquid
raise SyntaxError, options[:locale].t("errors.syntax.for_invalid_attribute")
end
p.consume(:colon)
set_attribute(attribute, p.expression)
set_attribute(attribute, p.expression, safe: true)
end
p.consume(:end_of_string)
end
private
alias_method :strict2_parse, :strict_parse
def collection_segment(context)
offsets = context.registers[:for] ||= {}
@@ -118,22 +152,14 @@ module Liquid
offsets[@name].to_i
else
from_value = context.evaluate(@from)
if from_value.nil?
0
else
Utils.to_integer(from_value)
end
from_value.nil? ? 0 : Utils.to_integer(from_value)
end
collection = context.evaluate(@collection_name)
collection = collection.to_a if collection.is_a?(Range)
limit_value = context.evaluate(@limit)
to = if limit_value.nil?
nil
else
Utils.to_integer(limit_value) + from
end
to = limit_value && (Utils.to_integer(limit_value) + from)
segment = Utils.slice_collection(collection, from, to)
segment.reverse! if @reversed
@@ -174,25 +200,21 @@ module Liquid
output
end
def set_attribute(key, expr)
def set_attribute(key, expr, safe: false)
case key
when 'offset'
@from = if expr == 'continue'
:continue
else
parse_expression(expr)
parse_expression(expr, safe: safe)
end
when 'limit'
@limit = parse_expression(expr)
@limit = parse_expression(expr, safe: safe)
end
end
def render_else(context, output)
if @else_block
@else_block.render_to_output_buffer(context, output)
else
output
end
@else_block ? @else_block.render_to_output_buffer(context, output) : output
end
class ParseTreeVisitor < Liquid::ParseTreeVisitor
@@ -201,6 +223,4 @@ module Liquid
end
end
end
Template.register_tag('for', For)
end
+38 -12
View File
@@ -51,14 +51,17 @@ module Liquid
end
def render_to_output_buffer(context, output)
@blocks.each do |block|
result = Liquid::Utils.to_liquid_value(
block.evaluate(context),
)
idx = 0
blocks = @blocks
while idx < blocks.length
block = blocks[idx]
result = block.evaluate(context)
result = result.to_liquid_value if result.respond_to?(:to_liquid_value)
if result
return block.attachment.render_to_output_buffer(context, output)
end
idx += 1
end
output
@@ -66,6 +69,10 @@ module Liquid
private
def strict2_parse(markup)
strict_parse(markup)
end
def push_block(tag, markup)
block = if tag == 'else'
ElseCondition.new
@@ -77,11 +84,32 @@ module Liquid
block.attach(new_body)
end
def parse_expression(markup)
Condition.parse_expression(parse_context, markup)
def parse_expression(markup, safe: false)
Condition.parse_expression(parse_context, markup, safe: safe)
end
def lax_parse(markup)
# Fastest path: simple identifier truthiness like "product.available" or "forloop.first"
if (simple = Variable.simple_variable_markup(markup))
return Condition.new(parse_expression(simple))
end
# Fast path: simple condition without and/or — use Cursor.
# The include? pre-checks are both a correctness guard (parse_simple_condition
# only handles a single comparison) and a perf gate (avoids cursor allocation
# for the compound-condition case that will always fall through to lax_parse).
if !markup.include?(' and ') && !markup.include?(' or ')
cursor = @parse_context.cursor
cursor.reset(markup)
if cursor.parse_simple_condition
return Condition.new(
parse_expression(cursor.cond_left),
cursor.cond_op,
cursor.cond_right ? parse_expression(cursor.cond_right) : nil,
)
end
end
expressions = markup.scan(ExpressionsAndOperators)
raise SyntaxError, options[:locale].t("errors.syntax.if") unless expressions.pop =~ Syntax
@@ -102,7 +130,7 @@ module Liquid
end
def strict_parse(markup)
p = Parser.new(markup)
p = @parse_context.new_parser(markup)
condition = parse_binary_comparisons(p)
p.consume(:end_of_string)
condition
@@ -111,7 +139,7 @@ module Liquid
def parse_binary_comparisons(p)
condition = parse_comparison(p)
first_condition = condition
while (op = (p.id?('and') || p.id?('or')))
while (op = p.id?('and') || p.id?('or'))
child_condition = parse_comparison(p)
condition.send(op, child_condition)
condition = child_condition
@@ -120,9 +148,9 @@ module Liquid
end
def parse_comparison(p)
a = parse_expression(p.expression)
a = parse_expression(p.expression, safe: true)
if (op = p.consume?(:comparison))
b = parse_expression(p.expression)
b = parse_expression(p.expression, safe: true)
Condition.new(a, op, b)
else
Condition.new(a)
@@ -135,6 +163,4 @@ module Liquid
end
end
end
Template.register_tag('if', If)
end
-2
View File
@@ -14,6 +14,4 @@ module Liquid
output
end
end
Template.register_tag('ifchanged', Ifchanged)
end
+45 -21
View File
@@ -6,7 +6,7 @@ module Liquid
# @liquid_category theme
# @liquid_name include
# @liquid_summary
# Renders a [snippet](/themes/architecture#snippets).
# Renders a [snippet](/themes/architecture/snippets).
# @liquid_description
# Inside the snippet, you can access and alter variables that are [created](/docs/api/liquid/tags/variable-tags) outside of the
# snippet.
@@ -27,24 +27,7 @@ module Liquid
def initialize(tag_name, markup, options)
super
if markup =~ SYNTAX
template_name = Regexp.last_match(1)
variable_name = Regexp.last_match(3)
@alias_name = Regexp.last_match(5)
@variable_name_expr = variable_name ? parse_expression(variable_name) : nil
@template_name_expr = parse_expression(template_name)
@attributes = {}
markup.scan(TagAttributes) do |key, value|
@attributes[key] = parse_expression(value)
end
else
raise SyntaxError, options[:locale].t("errors.syntax.include")
end
parse_with_selected_parser(markup)
end
def parse(_tokens)
@@ -101,6 +84,49 @@ module Liquid
alias_method :parse_context, :options
private :parse_context
def strict2_parse(markup)
p = @parse_context.new_parser(markup)
@template_name_expr = safe_parse_expression(p)
@variable_name_expr = safe_parse_expression(p) if p.id?("for") || p.id?("with")
@alias_name = p.consume(:id) if p.id?("as")
p.consume?(:comma)
@attributes = {}
while p.look(:id)
key = p.consume
p.consume(:colon)
@attributes[key] = safe_parse_expression(p)
p.consume?(:comma)
end
p.consume(:end_of_string)
end
def strict_parse(markup)
lax_parse(markup)
end
def lax_parse(markup)
if markup =~ SYNTAX
template_name = Regexp.last_match(1)
variable_name = Regexp.last_match(3)
@alias_name = Regexp.last_match(5)
@variable_name_expr = variable_name ? parse_expression(variable_name) : nil
@template_name_expr = parse_expression(template_name)
@attributes = {}
markup.scan(TagAttributes) do |key, value|
@attributes[key] = parse_expression(value)
end
else
raise SyntaxError, options[:locale].t("errors.syntax.include")
end
end
class ParseTreeVisitor < Liquid::ParseTreeVisitor
def children
[
@@ -110,6 +136,4 @@ module Liquid
end
end
end
Template.register_tag('include', Include)
end
+5 -3
View File
@@ -7,10 +7,14 @@ module Liquid
# @liquid_name increment
# @liquid_summary
# Creates a new variable, with a default value of 0, that's increased by 1 with each subsequent call.
#
# > Caution:
# > Predefined Liquid objects can be overridden by variables with the same name.
# > To make sure that you can access all Liquid objects, make sure that your variable name doesn't match a predefined object's name.
# @liquid_description
# Variables that are declared with `increment` are unique to the [layout](/themes/architecture/layouts), [template](/themes/architecture/templates),
# or [section](/themes/architecture/sections) file that they're created in. However, the variable is shared across
# [snippets](/themes/architecture#snippets) included in the file.
# [snippets](/themes/architecture/snippets) included in the file.
#
# Similarly, variables that are created with `increment` are independent from those created with [`assign`](/docs/api/liquid/tags/assign)
# and [`capture`](/docs/api/liquid/tags/capture). However, `increment` and [`decrement`](/docs/api/liquid/tags/decrement) share
@@ -35,6 +39,4 @@ module Liquid
output
end
end
Template.register_tag('increment', Increment)
end
-2
View File
@@ -25,6 +25,4 @@ module Liquid
true
end
end
Template.register_tag('#', InlineComment)
end
+1 -4
View File
@@ -14,7 +14,6 @@ module Liquid
# @liquid_syntax_keyword expression The expression to be output without being rendered.
class Raw < Block
Syntax = /\A\s*\z/
FullTokenPossiblyInvalid = /\A(.*)#{TagStart}#{WhitespaceControl}?\s*(\w+)\s*(.*)?#{WhitespaceControl}?#{TagEnd}\z/om
def initialize(tag_name, markup, parse_context)
super
@@ -25,7 +24,7 @@ module Liquid
def parse(tokens)
@body = +''
while (token = tokens.shift)
if token =~ FullTokenPossiblyInvalid && block_delimiter == Regexp.last_match(2)
if token =~ BlockBody::FullTokenPossiblyInvalid && block_delimiter == Regexp.last_match(2)
parse_context.trim_whitespace = (token[-3] == WhitespaceControl)
@body << Regexp.last_match(1) if Regexp.last_match(1) != ""
return
@@ -57,6 +56,4 @@ module Liquid
end
end
end
Template.register_tag('raw', Raw)
end
+51 -19
View File
@@ -6,7 +6,7 @@ module Liquid
# @liquid_category theme
# @liquid_name render
# @liquid_summary
# Renders a [snippet](/themes/architecture#snippets) or [app block](/themes/architecture/sections/section-schema#render-app-blocks).
# Renders a [snippet](/themes/architecture/snippets) or [app block](/themes/architecture/sections/section-schema#render-app-blocks).
# @liquid_description
# Inside snippets and app blocks, you can't directly access variables that are [created](/docs/api/liquid/tags/variable-tags) outside
# of the snippet or app block. However, you can [specify variables as parameters](/docs/api/liquid/tags/render#render-passing-variables-to-a-snippet)
@@ -35,22 +35,7 @@ module Liquid
def initialize(tag_name, markup, options)
super
raise SyntaxError, options[:locale].t("errors.syntax.render") unless markup =~ SYNTAX
template_name = Regexp.last_match(1)
with_or_for = Regexp.last_match(3)
variable_name = Regexp.last_match(4)
@alias_name = Regexp.last_match(6)
@variable_name_expr = variable_name ? parse_expression(variable_name) : nil
@template_name_expr = parse_expression(template_name)
@is_for_loop = (with_or_for == FOR)
@attributes = {}
markup.scan(TagAttributes) do |key, value|
@attributes[key] = parse_expression(value)
end
parse_with_selected_parser(markup)
end
def for_loop?
@@ -99,6 +84,55 @@ module Liquid
output
end
# render (string) (with|for expression)? (as id)? (key: value)*
def strict2_parse(markup)
p = @parse_context.new_parser(markup)
@template_name_expr = parse_expression(strict2_template_name(p), safe: true)
with_or_for = p.id?("for") || p.id?("with")
@variable_name_expr = safe_parse_expression(p) if with_or_for
@alias_name = p.consume(:id) if p.id?("as")
@is_for_loop = (with_or_for == FOR)
p.consume?(:comma)
@attributes = {}
while p.look(:id)
key = p.consume
p.consume(:colon)
@attributes[key] = safe_parse_expression(p)
p.consume?(:comma)
end
p.consume(:end_of_string)
end
def strict2_template_name(p)
p.consume(:string)
end
def strict_parse(markup)
lax_parse(markup)
end
def lax_parse(markup)
raise SyntaxError, options[:locale].t("errors.syntax.render") unless markup =~ SYNTAX
template_name = Regexp.last_match(1)
with_or_for = Regexp.last_match(3)
variable_name = Regexp.last_match(4)
@alias_name = Regexp.last_match(6)
@variable_name_expr = variable_name ? parse_expression(variable_name) : nil
@template_name_expr = parse_expression(template_name)
@is_for_loop = (with_or_for == FOR)
@attributes = {}
markup.scan(TagAttributes) do |key, value|
@attributes[key] = parse_expression(value)
end
end
class ParseTreeVisitor < Liquid::ParseTreeVisitor
def children
[
@@ -108,6 +142,4 @@ module Liquid
end
end
end
Template.register_tag('render', Render)
end
+46 -5
View File
@@ -19,17 +19,54 @@ module Liquid
# @liquid_syntax_keyword variable The current item in the array.
# @liquid_syntax_keyword array The array to iterate over.
# @liquid_syntax_keyword expression The expression to render.
# @liquid_optional_param cols [number] The number of columns that the table should have.
# @liquid_optional_param limit [number] The number of iterations to perform.
# @liquid_optional_param offset [number] The 1-based index to start iterating at.
# @liquid_optional_param cols: [number] The number of columns that the table should have.
# @liquid_optional_param limit: [number] The number of iterations to perform.
# @liquid_optional_param offset: [number] The 1-based index to start iterating at.
# @liquid_optional_param range [untyped] A custom numeric range to iterate over.
class TableRow < Block
Syntax = /(\w+)\s+in\s+(#{QuotedFragment}+)/o
ALLOWED_ATTRIBUTES = ['cols', 'limit', 'offset', 'range'].freeze
attr_reader :variable_name, :collection_name, :attributes
def initialize(tag_name, markup, options)
super
parse_with_selected_parser(markup)
end
def strict2_parse(markup)
p = @parse_context.new_parser(markup)
@variable_name = p.consume(:id)
unless p.id?("in")
raise SyntaxError, options[:locale].t("errors.syntax.for_invalid_in")
end
@collection_name = safe_parse_expression(p)
p.consume?(:comma)
@attributes = {}
while p.look(:id)
key = p.consume
unless ALLOWED_ATTRIBUTES.include?(key)
raise SyntaxError, options[:locale].t("errors.syntax.table_row_invalid_attribute", attribute: key)
end
p.consume(:colon)
@attributes[key] = safe_parse_expression(p)
p.consume?(:comma)
end
p.consume(:end_of_string)
end
def strict_parse(markup)
lax_parse(markup)
end
def lax_parse(markup)
if markup =~ Syntax
@variable_name = Regexp.last_match(1)
@collection_name = parse_expression(Regexp.last_match(2))
@@ -65,6 +102,12 @@ module Liquid
super
output << '</td>'
# Handle any interrupts if they exist.
if context.interrupt?
interrupt = context.pop_interrupt
break if interrupt.is_a?(BreakInterrupt)
end
if tablerowloop.col_last && !tablerowloop.last
output << "</tr>\n<tr class=\"row#{tablerowloop.row + 1}\">"
end
@@ -91,6 +134,4 @@ module Liquid
raise Liquid::ArgumentError, "invalid integer"
end
end
Template.register_tag('tablerow', TableRow)
end
-2
View File
@@ -44,6 +44,4 @@ module Liquid
output
end
end
Template.register_tag('unless', Unless)
end
+64 -62
View File
@@ -2,7 +2,7 @@
module Liquid
# Templates are central to liquid.
# Interpretating templates is a two step process. First you compile the
# Interpreting templates is a two step process. First you compile the
# source code you got. During compile time some extensive error checking is performed.
# your code should expect to get some SyntaxErrors.
#
@@ -18,95 +18,91 @@ module Liquid
attr_accessor :root, :name
attr_reader :resource_limits, :warnings
class TagRegistry
include Enumerable
def initialize
@tags = {}
@cache = {}
end
def [](tag_name)
return nil unless @tags.key?(tag_name)
return @cache[tag_name] if Liquid.cache_classes
lookup_class(@tags[tag_name]).tap { |o| @cache[tag_name] = o }
end
def []=(tag_name, klass)
@tags[tag_name] = klass.name
@cache[tag_name] = klass
end
def delete(tag_name)
@tags.delete(tag_name)
@cache.delete(tag_name)
end
def each(&block)
@tags.each(&block)
end
private
def lookup_class(name)
Object.const_get(name)
end
end
attr_reader :profiler
class << self
# Sets how strict the parser should be.
# :lax acts like liquid 2.5 and silently ignores malformed tags in most cases.
# :warn is the default and will give deprecation warnings when invalid syntax is used.
# :strict will enforce correct syntax.
attr_accessor :error_mode
Template.error_mode = :lax
attr_accessor :default_exception_renderer
Template.default_exception_renderer = lambda do |exception|
exception
# :strict enforces correct syntax for most tags
# :strict2 enforces correct syntax for all tags
def error_mode=(mode)
Deprecations.warn("Template.error_mode=", "Environment#error_mode=")
Environment.default.error_mode = mode
end
attr_accessor :file_system
Template.file_system = BlankFileSystem.new
def error_mode
Environment.default.error_mode
end
attr_accessor :tags
Template.tags = TagRegistry.new
private :tags=
def default_exception_renderer=(renderer)
Deprecations.warn("Template.default_exception_renderer=", "Environment#exception_renderer=")
Environment.default.exception_renderer = renderer
end
def default_exception_renderer
Environment.default.exception_renderer
end
def file_system=(file_system)
Deprecations.warn("Template.file_system=", "Environment#file_system=")
Environment.default.file_system = file_system
end
def file_system
Environment.default.file_system
end
def tags
Environment.default.tags
end
def register_tag(name, klass)
tags[name.to_s] = klass
Deprecations.warn("Template.register_tag", "Environment#register_tag")
Environment.default.register_tag(name, klass)
end
# Pass a module with filter methods which should be available
# to all liquid views. Good for registering the standard library
def register_filter(mod)
StrainerFactory.add_global_filter(mod)
Deprecations.warn("Template.register_filter", "Environment#register_filter")
Environment.default.register_filter(mod)
end
attr_accessor :default_resource_limits
Template.default_resource_limits = {}
private :default_resource_limits=
private def default_resource_limits=(limits)
Deprecations.warn("Template.default_resource_limits=", "Environment#default_resource_limits=")
Environment.default.default_resource_limits = limits
end
def default_resource_limits
Environment.default.default_resource_limits
end
# creates a new <tt>Template</tt> object from liquid source code
# To enable profiling, pass in <tt>profile: true</tt> as an option.
# See Liquid::Profiler for more information
def parse(source, options = {})
new.parse(source, options)
environment = options[:environment] || Environment.default
new(environment: environment).parse(source, options)
end
end
def initialize
def initialize(environment: Environment.default)
@environment = environment
@rethrow_errors = false
@resource_limits = ResourceLimits.new(Template.default_resource_limits)
@resource_limits = ResourceLimits.new(environment.default_resource_limits)
end
# Parse source code.
# Returns self for easy chaining
def parse(source, options = {})
parse_context = configure_options(options)
source = source.to_s.to_str
unless source.valid_encoding?
raise TemplateEncodingError, parse_context.locale.t("errors.syntax.invalid_template_encoding")
end
tokenizer = parse_context.new_tokenizer(source, start_line_number: @line_numbers && 1)
@root = Document.parse(tokenizer, parse_context)
self
@@ -156,11 +152,11 @@ module Liquid
c
when Liquid::Drop
drop = args.shift
drop.context = Context.new([drop, assigns], instance_assigns, registers, @rethrow_errors, @resource_limits)
drop.context = Context.new([drop, assigns], instance_assigns, registers, @rethrow_errors, @resource_limits, {}, @environment)
when Hash
Context.new([args.shift, assigns], instance_assigns, registers, @rethrow_errors, @resource_limits)
Context.new([args.shift, assigns], instance_assigns, registers, @rethrow_errors, @resource_limits, {}, @environment)
when nil
Context.new(assigns, instance_assigns, registers, @rethrow_errors, @resource_limits)
Context.new(assigns, instance_assigns, registers, @rethrow_errors, @resource_limits, {}, @environment)
else
raise ArgumentError, "Expected Hash or Liquid::Context as parameter"
end
@@ -220,8 +216,14 @@ module Liquid
@options = options
@profiling = profiling
@line_numbers = options[:line_numbers] || @profiling
parse_context = options.is_a?(ParseContext) ? options : ParseContext.new(options)
@warnings = parse_context.warnings
parse_context = if options.is_a?(ParseContext)
options
else
opts = options.key?(:environment) ? options : options.merge(environment: @environment)
ParseContext.new(opts)
end
@warnings = parse_context.warnings
parse_context
end
+123 -16
View File
@@ -4,17 +4,26 @@ module Liquid
class Tokenizer
attr_reader :line_number, :for_liquid_tag
def initialize(source, line_numbers = false, line_number: nil, for_liquid_tag: false)
@source = source.to_s.to_str
@line_number = line_number || (line_numbers ? 1 : nil)
def initialize(
source:,
string_scanner: nil,
line_numbers: false,
line_number: nil,
for_liquid_tag: false
)
@line_number = line_number || (line_numbers ? 1 : nil)
@for_liquid_tag = for_liquid_tag
@offset = 0
@tokens = tokenize
@source = source.to_s
@offset = 0
@tokens = []
tokenize
end
def shift
token = @tokens[@offset]
return nil unless token
return unless token
@offset += 1
@@ -28,18 +37,116 @@ module Liquid
private
def tokenize
return [] if @source.empty?
return @source.split("\n") if @for_liquid_tag
tokens = @source.split(TemplateParser)
# removes the rogue empty element at the beginning of the array
if tokens[0]&.empty?
@offset += 1
if @for_liquid_tag
@tokens = @source.split("\n")
else
tokenize_fast
end
tokens
@source = nil
end
# Fast tokenizer using String#byteindex instead of StringScanner regex.
# String#byteindex is ~40% faster for finding { delimiters.
def tokenize_fast
src = @source
unless src.valid_encoding?
raise SyntaxError, "Invalid byte sequence in #{src.encoding}"
end
len = src.bytesize
pos = 0
while pos < len
# Find next { which could start a tag or variable
idx = src.byteindex('{', pos)
unless idx
# No more tags/variables — rest is text
@tokens << src.byteslice(pos, len - pos) if pos < len
break
end
next_byte = idx + 1 < len ? src.getbyte(idx + 1) : nil
if next_byte == Cursor::PCT # {%
# Emit text before tag
@tokens << src.byteslice(pos, idx - pos) if idx > pos
# Find %} to close the tag
close = src.byteindex('%}', idx + 2)
if close
@tokens << src.byteslice(idx, close + 2 - idx)
pos = close + 2
else
# Emit malformed token to propagate a missing-terminator error in the parser
@tokens << "{%"
pos = idx + 2
end
elsif next_byte == Cursor::LCURLY # {{
# Emit text before variable, then scan for the closing }}.
@tokens << src.byteslice(pos, idx - pos) if idx > pos
pos = scan_variable_token(src, idx, len)
else
# Lone '{' — not the start of a tag or variable.
# Find the next '{{' or '{%' to know where this text token ends.
# Using two byteindex calls avoids a nested loop and is always O(n).
tag_start = src.byteindex('{%', idx + 1)
var_start = src.byteindex('{{', idx + 1)
next_token = [tag_start, var_start].compact.min
if next_token
@tokens << src.byteslice(pos, next_token - pos)
pos = next_token
else
@tokens << src.byteslice(pos, len - pos)
pos = len
end
end
end
end
# Scans a {{ ... }} variable token starting at `idx` in `src`.
# Emits the token to @tokens and returns the new position after the token.
# Handles }}, single }, and embedded {% ... %} (nested tag inside variable).
private def scan_variable_token(src, idx, len)
# Byte-by-byte scan: find } or {, then inspect the next byte.
scan_pos = idx + 2
while scan_pos < len
b = src.getbyte(scan_pos)
if b == Cursor::RCURLY # }
if scan_pos + 1 >= len
# } at end of string — emit token up to here
@tokens << src.byteslice(idx, scan_pos + 1 - idx)
return scan_pos + 1
end
b2 = src.getbyte(scan_pos + 1)
if b2 == Cursor::RCURLY
# Found }} — close variable
@tokens << src.byteslice(idx, scan_pos + 2 - idx)
return scan_pos + 2
else
# } followed by non-} — emit token up to here (matches original: @ss.pos -= 1)
@tokens << src.byteslice(idx, scan_pos + 1 - idx)
return scan_pos + 1
end
elsif b == Cursor::LCURLY && scan_pos + 1 < len && src.getbyte(scan_pos + 1) == Cursor::PCT
# Found {% inside {{ — scan to %} and emit as one token
close = src.byteindex('%}', scan_pos + 2)
if close
@tokens << src.byteslice(idx, close + 2 - idx)
return close + 2
else
@tokens << src.byteslice(idx, len - idx)
return len
end
else
scan_pos += 1
end
end
# Reached end without finding }} — malformed
@tokens << "{{"
idx + 2
end
end
end
+117 -13
View File
@@ -2,9 +2,15 @@
module Liquid
module Utils
DECIMAL_REGEX = /\A-?\d+\.\d+\z/
UNIX_TIMESTAMP_REGEX = /\A\d+\z/
def self.slice_collection(collection, from, to)
if (from != 0 || !to.nil?) && collection.respond_to?(:load_slice)
collection.load_slice(from, to)
elsif from == 0 && to.nil? && collection.is_a?(Array)
# Fast path: no offset/limit on an Array — return as-is (avoid copy)
collection
else
slice_collection_using_each(collection, from, to)
end
@@ -12,23 +18,17 @@ module Liquid
def self.slice_collection_using_each(collection, from, to)
segments = []
index = 0
# Maintains Ruby 1.8.7 String#each behaviour on 1.9
# String is Enumerable but #each is not defined; handle it as a single-element collection
if collection.is_a?(String)
return collection.empty? ? [] : [collection]
end
return [] unless collection.respond_to?(:each)
index = 0
collection.each do |item|
if to && to <= index
break
end
if from <= index
segments << item
end
break if to && to <= index
segments << item if from <= index
index += 1
end
@@ -52,7 +52,7 @@ module Liquid
when Numeric
obj
when String
/\A-?\d+\.\d+\z/.match?(obj.strip) ? BigDecimal(obj) : obj.to_i
DECIMAL_REGEX.match?(obj.strip) ? BigDecimal(obj) : obj.to_i
else
if obj.respond_to?(:to_number)
obj.to_number
@@ -66,14 +66,14 @@ module Liquid
return obj if obj.respond_to?(:strftime)
if obj.is_a?(String)
return nil if obj.empty?
return if obj.empty?
obj = obj.downcase
end
case obj
when 'now', 'today'
Time.now
when /\A\d+\z/, Integer
when UNIX_TIMESTAMP_REGEX, Integer
Time.at(obj.to_i)
when String
Time.parse(obj)
@@ -89,5 +89,109 @@ module Liquid
# Otherwise return the object itself
obj
end
# Cached string representations for common small integers (0-999)
# Avoids repeated Integer#to_s allocations during rendering
SMALL_INT_STRINGS = Array.new(1000) { |i| i.to_s.freeze }.freeze
def self.to_s(obj, seen = nil)
case obj
when Integer
obj >= 0 && obj < 1000 ? SMALL_INT_STRINGS[obj] : obj.to_s
when BigDecimal
obj.to_s("F")
when Hash
# If the custom hash implementation overrides `#to_s`, use their
# custom implementation. Otherwise we use Liquid's default
# implementation.
if obj.class.instance_method(:to_s) == HASH_TO_S_METHOD
hash_inspect(obj, seen || {})
else
obj.to_s
end
when Array
array_inspect(obj, seen || {})
else
obj.to_s
end
end
def self.inspect(obj, seen = nil)
case obj
when Hash
# If the custom hash implementation overrides `#inspect`, use their
# custom implementation. Otherwise we use Liquid's default
# implementation.
if obj.class.instance_method(:inspect) == HASH_INSPECT_METHOD
hash_inspect(obj, seen || {})
else
obj.inspect
end
when Array
array_inspect(obj, seen || {})
else
obj.inspect
end
end
def self.array_inspect(arr, seen = {})
if seen[arr.object_id]
return "[...]"
end
seen[arr.object_id] = true
str = +"["
cursor = 0
len = arr.length
while cursor < len
if cursor > 0
str << ", "
end
item_str = inspect(arr[cursor], seen)
str << item_str
cursor += 1
end
str << "]"
str
ensure
seen.delete(arr.object_id)
end
def self.hash_inspect(hash, seen = {})
if seen[hash.object_id]
return "{...}"
end
seen[hash.object_id] = true
str = +"{"
first = true
hash.each do |key, value|
if first
first = false
else
str << ", "
end
key_str = inspect(key, seen)
str << key_str
str << "=>"
value_str = inspect(value, seen)
str << value_str
end
str << "}"
str
ensure
seen.delete(hash.object_id)
end
HASH_TO_S_METHOD = Hash.instance_method(:to_s)
private_constant :HASH_TO_S_METHOD
HASH_INSPECT_METHOD = Hash.instance_method(:inspect)
private_constant :HASH_INSPECT_METHOD
end
end
+393 -22
View File
@@ -12,6 +12,26 @@ module Liquid
# {{ user | link }}
#
class Variable
# Checks if markup is a simple "name.lookup.chain" with no filters/brackets/quotes.
# Returns the trimmed markup string, or nil if not simple.
def self.simple_variable_markup(markup)
return if markup.empty?
return unless markup.match?(SIMPLE_VARIABLE_RE)
# Avoid allocation when there's no surrounding whitespace (the common case)
first = markup.getbyte(0)
last = markup.getbyte(markup.bytesize - 1)
needs_strip = first == Cursor::SPACE || first == Cursor::TAB || first == Cursor::NL || first == Cursor::CR ||
last == Cursor::SPACE || last == Cursor::TAB || last == Cursor::NL || last == Cursor::CR
needs_strip ? markup.strip : markup
end
# Cache for [filtername, EMPTY_ARRAY] tuples — avoids repeated array creation
NO_ARG_FILTER_CACHE = Hash.new { |h, k| h[k] = [k, Const::EMPTY_ARRAY].freeze }
# Regex for a simple variable lookup with optional surrounding whitespace.
# Shares the identifier grammar with VariableLookup::SIMPLE_LOOKUP_RE.
SIMPLE_VARIABLE_RE = /\A\s*[\w-]+\??(?:\.[\w-]+\??)*\s*\z/
FilterMarkupRegex = /#{FilterSeparator}\s*(.*)/om
FilterParser = /(?:\s+|#{QuotedFragment}|#{ArgumentSeparator})+/o
FilterArgsRegex = /(?:#{FilterArgumentSeparator}|#{ArgumentSeparator})\s*((?:\w+\s*\:\s*)?#{QuotedFragment})/o
@@ -30,7 +50,278 @@ module Liquid
@parse_context = parse_context
@line_number = parse_context.line_number
strict_parse_with_error_mode_fallback(markup)
# Fast path: try to parse without going through Lexer → Parser
# Skip for strict2/rigid modes which require different parsing
# Fast path only for lax/warn modes — strict modes need full error checking
error_mode = parse_context.error_mode
if error_mode == :strict2 || error_mode == :rigid || error_mode == :strict || !try_fast_parse(markup, parse_context)
strict_parse_with_error_mode_fallback(markup)
end
end
private def try_fast_parse(markup, parse_context)
pos = fast_scan_name(markup)
return false unless pos
# fast_resolve_name calls VariableLookup.parse_simple / Expression::LITERALS — the
# only sites that can raise SyntaxError on malformed input. The byte scanners return
# false instead of raising.
begin
fast_resolve_name(markup, parse_context)
rescue SyntaxError
return false
end
# End of markup — no filters
if pos >= markup.bytesize
@filters = Const::EMPTY_ARRAY
return true
end
# Must be followed by a pipe filter separator
return false unless markup.getbyte(pos) == Cursor::PIPE
fast_scan_filters(markup, pos, parse_context)
end
# Scan the variable name (quoted string or identifier chain) at the start of markup.
# Returns the position after the name + trailing whitespace, or false on failure.
# Sets @_fast_name_start and @_fast_name_end for fast_resolve_name.
private def fast_scan_name(markup)
len = markup.bytesize
return false if len == 0
# Skip leading whitespace
pos = 0
while pos < len
b = markup.getbyte(pos)
break unless b == Cursor::SPACE || b == Cursor::TAB || b == Cursor::NL || b == Cursor::CR
pos += 1
end
return false if pos >= len
b = markup.getbyte(pos)
if b == Cursor::QUOTE_S || b == Cursor::QUOTE_D
# Quoted string literal: scan to matching close quote
quote = b
@_fast_name_start = pos
pos += 1
pos += 1 while pos < len && markup.getbyte(pos) != quote
pos += 1 if pos < len # skip closing quote
@_fast_name_end = pos
elsif ByteTables::IDENT_START[b]
# Identifier chain: [a-zA-Z_][a-zA-Z0-9_-]*(.[a-zA-Z_][a-zA-Z0-9_-]*)*
@_fast_name_start = pos
pos += 1
while pos < len
b = markup.getbyte(pos)
if ByteTables::IDENT_CONT[b]
pos += 1
elsif b == Cursor::DOT
pos += 1
return false if pos >= len
b = markup.getbyte(pos)
return false unless ByteTables::IDENT_START[b]
pos += 1
else
break
end
end
@_fast_name_end = pos
else
return false
end
# Skip whitespace after name
while pos < len
b = markup.getbyte(pos)
break unless b == Cursor::SPACE || b == Cursor::TAB || b == Cursor::NL || b == Cursor::CR
pos += 1
end
pos
end
# Resolve the scanned name bytes to a Liquid expression object.
# Reads @_fast_name_start / @_fast_name_end set by fast_scan_name.
# Sets @name. May raise SyntaxError (rescued in try_fast_parse).
private def fast_resolve_name(markup, parse_context)
name_start = @_fast_name_start
name_end = @_fast_name_end
len = markup.bytesize
# Avoid byteslice when the name spans the whole markup (no surrounding whitespace/filters)
expr_markup = name_start == 0 && name_end == len ? markup : markup.byteslice(name_start, name_end - name_start)
cache = parse_context.expression_cache
ss = parse_context.string_scanner
first_byte = expr_markup.getbyte(0)
@name = if first_byte == Cursor::QUOTE_S || first_byte == Cursor::QUOTE_D
# String literal — strip enclosing quotes
expr_markup.byteslice(1, expr_markup.bytesize - 2)
elsif Expression::LITERALS.key?(expr_markup)
Expression::LITERALS[expr_markup]
elsif cache
cache[expr_markup] || (cache[expr_markup] = VariableLookup.parse_simple(expr_markup, ss, cache).freeze)
else
VariableLookup.parse_simple(expr_markup, ss || StringScanner.new(""), nil).freeze
end
end
# Scan the filter chain starting at `pos` (the first '|').
# Returns true on success (sets @filters), false to fall back to the Lexer.
# Rescues SyntaxError from Expression.parse inside fast_scan_filter_args.
private def fast_scan_filters(markup, pos, parse_context)
len = markup.bytesize
@filters = []
filter_pos = pos
while filter_pos < len && markup.getbyte(filter_pos) == Cursor::PIPE
filter_pos += 1
# Skip spaces after pipe (tabs/newlines handled in the between-filters skip below)
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) == Cursor::SPACE
# Scan filter name: must start with [a-zA-Z_]
fname_start = filter_pos
b = filter_pos < len ? markup.getbyte(filter_pos) : nil
break unless b && ByteTables::IDENT_START[b]
filter_pos += 1
while filter_pos < len
b = markup.getbyte(filter_pos)
break unless ByteTables::IDENT_CONT[b]
filter_pos += 1
end
filtername = markup.byteslice(fname_start, filter_pos - fname_start)
# Skip whitespace after filter name
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) == Cursor::SPACE
if filter_pos < len && markup.getbyte(filter_pos) == Cursor::COLON
# Has arguments — fast-scan positional args; fall to Lexer on keyword args
filter_pos += 1 # skip ':'
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) == Cursor::SPACE
result = fast_scan_filter_args(markup, filter_pos, parse_context)
return fall_to_lexer_filters(markup, pos, fname_start, len, parse_context) if result == :fall_to_lexer
filter_args, filter_pos = result
@filters << [filtername, filter_args]
else
# No-arg filter — reuse the cached [name, EMPTY_ARRAY] tuple
@filters << NO_ARG_FILTER_CACHE[filtername]
end
# Skip whitespace (including tabs and newlines) between filters
filter_pos += 1 while filter_pos < len && (
markup.getbyte(filter_pos) == Cursor::SPACE ||
markup.getbyte(filter_pos) == Cursor::TAB ||
markup.getbyte(filter_pos) == Cursor::NL ||
markup.getbyte(filter_pos) == Cursor::CR
)
end
# Trailing bytes that aren't a pipe mean something the fast path doesn't handle
return false if filter_pos < len
@filters = Const::EMPTY_ARRAY if @filters.empty?
true
rescue SyntaxError
# Expression.parse (called inside fast_scan_filter_args for identifier args) can
# raise SyntaxError on malformed input. Fall back to full Lexer parse.
@name = nil
@filters = nil
false
end
# Called when fast_scan_filter_args encounters keyword args or an unrecognised
# token. Hands the remaining filter chain (from the pipe before fname_start)
# to the full Lexer-based parser, merges results into @filters, and returns true.
private def fall_to_lexer_filters(markup, pos, fname_start, len, parse_context)
# Walk back from fname_start to find the pipe that opened this filter.
# Equivalent to: markup.rindex('|', fname_start), bounded by pos.
rest_start = fname_start
rest_start -= 1 while rest_start > pos && markup.getbyte(rest_start) != Cursor::PIPE
rest_markup = markup.byteslice(rest_start, len - rest_start)
p = parse_context.new_parser(rest_markup)
while p.consume?(:pipe)
fn = p.consume(:id)
fa = p.consume?(:colon) ? parse_filterargs(p) : Const::EMPTY_ARRAY
@filters << lax_parse_filter_expressions(fn, fa)
end
p.consume(:end_of_string)
@filters = Const::EMPTY_ARRAY if @filters.empty?
true
end
# Scan positional filter arguments starting at `filter_pos`.
# Returns [filter_args_array, new_filter_pos] on success, or :fall_to_lexer when
# keyword args or unrecognised tokens are encountered.
private def fast_scan_filter_args(markup, filter_pos, parse_context)
len = markup.bytesize
filter_args = []
loop do
arg_start = filter_pos
b = filter_pos < len ? markup.getbyte(filter_pos) : nil
if b == Cursor::QUOTE_S || b == Cursor::QUOTE_D
# Quoted string argument
quote = b
filter_pos += 1
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) != quote
filter_pos += 1 if filter_pos < len # skip closing quote
filter_args << markup.byteslice(arg_start + 1, filter_pos - arg_start - 2)
elsif b && (ByteTables::DIGIT[b] ||
(b == Cursor::DASH && filter_pos + 1 < len && ByteTables::DIGIT[markup.getbyte(filter_pos + 1)]))
# Numeric argument (integer or float, optionally negative)
filter_pos += 1 if b == Cursor::DASH
filter_pos += 1 while filter_pos < len && ByteTables::DIGIT[markup.getbyte(filter_pos)]
if filter_pos < len && markup.getbyte(filter_pos) == Cursor::DOT # float
filter_pos += 1
filter_pos += 1 while filter_pos < len && ByteTables::DIGIT[markup.getbyte(filter_pos)]
end
num_str = markup.byteslice(arg_start, filter_pos - arg_start)
filter_args << (num_str.include?('.') ? num_str.to_f : num_str.to_i)
elsif b && ByteTables::IDENT_START[b]
# Identifier argument — may be a variable lookup or keyword arg
id_start = filter_pos
filter_pos += 1
while filter_pos < len
b2 = markup.getbyte(filter_pos)
break unless ByteTables::IDENT_CONT[b2] || b2 == Cursor::DOT
filter_pos += 1
end
filter_pos += 1 if filter_pos < len && markup.getbyte(filter_pos) == Cursor::QMARK
# Peek past whitespace: if followed by ':', this is a keyword arg → fall to Lexer
kw_check = filter_pos
kw_check += 1 while kw_check < len && markup.getbyte(kw_check) == Cursor::SPACE
return :fall_to_lexer if kw_check < len && markup.getbyte(kw_check) == Cursor::COLON
id_markup = markup.byteslice(id_start, filter_pos - id_start)
filter_args << Expression.parse(id_markup, parse_context.string_scanner, parse_context.expression_cache)
else
return :fall_to_lexer
end
# Skip whitespace after argument
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) == Cursor::SPACE
# Comma: more arguments follow; anything else: done with this filter's args
if filter_pos < len && markup.getbyte(filter_pos) == Cursor::COMMA
filter_pos += 1
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) == Cursor::SPACE
else
break
end
end
[filter_args, filter_pos]
end
def raw
@@ -42,7 +333,7 @@ module Liquid
end
def lax_parse(markup)
@filters = []
@filters = Const::EMPTY_ARRAY
return unless markup =~ MarkupWithQuotedFragment
name_markup = Regexp.last_match(1)
@@ -54,22 +345,38 @@ module Liquid
next unless f =~ /\w+/
filtername = Regexp.last_match(0)
filterargs = f.scan(FilterArgsRegex).flatten
@filters << parse_filter_expressions(filtername, filterargs)
@filters = [] if @filters.frozen?
@filters << lax_parse_filter_expressions(filtername, filterargs)
end
end
end
def strict_parse(markup)
@filters = []
p = Parser.new(markup)
@filters = Const::EMPTY_ARRAY
p = @parse_context.new_parser(markup)
return if p.look(:end_of_string)
@name = parse_context.parse_expression(p.expression)
@name = parse_context.safe_parse_expression(p)
while p.consume?(:pipe)
@filters = [] if @filters.frozen?
filtername = p.consume(:id)
filterargs = p.consume?(:colon) ? parse_filterargs(p) : []
@filters << parse_filter_expressions(filtername, filterargs)
filterargs = p.consume?(:colon) ? parse_filterargs(p) : Const::EMPTY_ARRAY
@filters << lax_parse_filter_expressions(filtername, filterargs)
end
p.consume(:end_of_string)
end
def strict2_parse(markup)
@filters = Const::EMPTY_ARRAY
p = @parse_context.new_parser(markup)
return if p.look(:end_of_string)
@name = parse_context.safe_parse_expression(p)
while p.consume?(:pipe)
@filters = [] if @filters.frozen?
@filters << strict2_parse_filter_expressions(p)
end
p.consume(:end_of_string)
end
@@ -86,41 +393,61 @@ module Liquid
obj = context.evaluate(@name)
@filters.each do |filter_name, filter_args, filter_kwargs|
filter_args = evaluate_filter_expressions(context, filter_args, filter_kwargs)
obj = context.invoke(filter_name, obj, *filter_args)
if filter_args.empty? && !filter_kwargs
obj = context.invoke_single(filter_name, obj)
elsif !filter_kwargs && filter_args.length == 1
# Single positional arg — most common after no-arg
obj = context.invoke_two(filter_name, obj, context.evaluate(filter_args[0]))
else
filter_args = evaluate_filter_expressions(context, filter_args, filter_kwargs)
obj = context.invoke(filter_name, obj, *filter_args)
end
end
context.apply_global_filter(obj)
end
def render_to_output_buffer(context, output)
obj = render(context)
if obj.is_a?(Array)
output << obj.join
elsif obj.nil?
# Fast path: no filters and no global filter
obj = if @filters.empty? && context.global_filter.nil?
context.evaluate(@name)
else
output << obj.to_s
render(context)
end
render_obj_to_output(obj, output)
output
end
def render_obj_to_output(obj, output)
if obj.instance_of?(String)
output << obj
elsif obj.nil?
# Do nothing
elsif obj.instance_of?(Array)
obj.each do |o|
render_obj_to_output(o, output)
end
else
output << Liquid::Utils.to_s(obj)
end
end
def disabled?(_context)
false
end
def disabled_tags
[]
Const::EMPTY_ARRAY
end
private
def parse_filter_expressions(filter_name, unparsed_args)
def lax_parse_filter_expressions(filter_name, unparsed_args)
filter_args = []
keyword_args = nil
unparsed_args.each do |a|
if (matches = a.match(JustTagAttributes))
# Fast check: keyword args must contain ':'
if a.include?(':') && (matches = a.match(JustTagAttributes))
keyword_args ||= {}
keyword_args[matches[1]] = parse_context.parse_expression(matches[2])
else
@@ -132,16 +459,60 @@ module Liquid
result
end
# Surprisingly, positional and keyword arguments can be mixed.
#
# filter = filtername [":" filterargs?]
# filterargs = argument ("," argument)*
# argument = (positional_argument | keyword_argument)
# positional_argument = expression
# keyword_argument = id ":" expression
def strict2_parse_filter_expressions(p)
filtername = p.consume(:id)
filter_args = []
keyword_args = {}
if p.consume?(:colon)
# Parse first argument (no leading comma)
argument(p, filter_args, keyword_args) unless end_of_arguments?(p)
# Parse remaining arguments (with leading commas) and optional trailing comma
argument(p, filter_args, keyword_args) while p.consume?(:comma) && !end_of_arguments?(p)
end
result = [filtername, filter_args]
result << keyword_args unless keyword_args.empty?
result
end
def argument(p, positional_arguments, keyword_arguments)
if p.look(:id) && p.look(:colon, 1)
key = p.consume(:id)
p.consume(:colon)
value = parse_context.safe_parse_expression(p)
keyword_arguments[key] = value
else
positional_arguments << parse_context.safe_parse_expression(p)
end
end
def end_of_arguments?(p)
p.look(:pipe) || p.look(:end_of_string)
end
def evaluate_filter_expressions(context, filter_args, filter_kwargs)
parsed_args = filter_args.map { |expr| context.evaluate(expr) }
if filter_kwargs
parsed_args = filter_args.map { |expr| context.evaluate(expr) }
parsed_kwargs = {}
filter_kwargs.each do |key, expr|
parsed_kwargs[key] = context.evaluate(expr)
end
parsed_args << parsed_kwargs
parsed_args
elsif filter_args.empty?
Const::EMPTY_ARRAY
else
filter_args.map { |expr| context.evaluate(expr) }
end
parsed_args
end
class ParseTreeVisitor < Liquid::ParseTreeVisitor
+157 -22
View File
@@ -6,26 +6,130 @@ module Liquid
attr_reader :name, :lookups
def self.parse(markup)
new(markup)
def self.parse(markup, string_scanner = StringScanner.new(""), cache = nil)
new(markup, string_scanner, cache)
end
def initialize(markup)
lookups = markup.scan(VariableParser)
# Fast parse that skips simple_lookup? check — caller guarantees simple identifier chain
def self.parse_simple(markup, string_scanner = nil, cache = nil)
new(markup, string_scanner, cache, true)
end
# Fast manual scanner replacing markup.scan(VariableParser)
# VariableParser = /\[(?>[^\[\]]+|\g<0>)*\]|[\w-]+\??/
# Splits "product.variants[0].title" into ["product", "variants", "[0]", "title"]
def self.scan_variable(markup)
result = []
pos = 0
len = markup.bytesize
while pos < len
byte = markup.getbyte(pos)
if byte == 91 # '['
# Scan balanced brackets
depth = 1
start = pos
pos += 1
while pos < len && depth > 0
b = markup.getbyte(pos)
depth += 1 if b == 91
depth -= 1 if b == 93
pos += 1
end
if depth == 0
result << markup.byteslice(start, pos - start)
else
# Unbalanced bracket - skip '[' and continue
pos = start + 1
end
elsif byte == 46 # '.'
pos += 1
elsif ByteTables::IDENT_CONT[byte] # [\w-]
start = pos
pos += 1
while pos < len
b = markup.getbyte(pos)
break unless ByteTables::IDENT_CONT[b]
pos += 1
end
# Check trailing '?'
if pos < len && markup.getbyte(pos) == 63
pos += 1
end
result << markup.byteslice(start, pos - start)
else
pos += 1
end
end
result
end
# Check if markup is a simple identifier chain: [\w-]+\??(.[\w-]+\??)*
# Uses C-level match? — 8x faster than Ruby byte scanning
SIMPLE_LOOKUP_RE = /\A[\w-]+\??(?:\.[\w-]+\??)*\z/
def self.simple_lookup?(markup)
markup.bytesize > 0 && markup.match?(SIMPLE_LOOKUP_RE)
end
def initialize(markup, string_scanner = StringScanner.new(""), cache = nil, simple = false)
# Fast path: simple identifier chain without brackets
if simple || self.class.simple_lookup?(markup)
dot_pos = markup.index('.')
if dot_pos.nil?
@name = markup
@lookups = Const::EMPTY_ARRAY
@command_flags = 0
return
end
@name = markup.byteslice(0, dot_pos)
# Build lookups array from remaining dot-separated segments
lookups = []
@command_flags = 0
pos = dot_pos + 1
len = markup.bytesize
while pos < len
seg_start = pos
while pos < len
b = markup.getbyte(pos)
break if b == 46 # '.'
pos += 1
end
seg = markup.byteslice(seg_start, pos - seg_start)
if COMMAND_METHODS.include?(seg)
@command_flags |= 1 << lookups.length
end
lookups << seg
pos += 1 # skip dot
end
@lookups = lookups
return
end
lookups = self.class.scan_variable(markup)
name = lookups.shift
if name&.start_with?('[') && name&.end_with?(']')
name = Expression.parse(name[1..-2])
if name&.start_with?('[') && name.end_with?(']')
name = Expression.parse(
name[1..-2],
string_scanner,
cache,
)
end
@name = name
@lookups = lookups
@command_flags = 0
@lookups.each_index do |i|
lookup = lookups[i]
if lookup&.start_with?('[') && lookup&.end_with?(']')
lookups[i] = Expression.parse(lookup[1..-2])
@lookups.each_with_index do |lookup, i|
if lookup&.start_with?('[') && lookup.end_with?(']')
lookups[i] = Expression.parse(
lookup[1..-2],
string_scanner,
cache,
)
elsif COMMAND_METHODS.include?(lookup)
@command_flags |= 1 << i
end
@@ -41,26 +145,39 @@ module Liquid
object = context.find_variable(name)
@lookups.each_index do |i|
key = context.evaluate(@lookups[i])
lookup = @lookups[i]
key = lookup.instance_of?(String) ? lookup : context.evaluate(lookup)
# Cast "key" to its liquid value to enable it to act as a primitive value
key = Liquid::Utils.to_liquid_value(key)
# Fast path: strings and integers (most common key types) don't need conversion
unless key.instance_of?(String) || key.instance_of?(Integer)
key = Liquid::Utils.to_liquid_value(key)
end
# If object is a hash- or array-like object we look for the
# presence of the key and if its available we return it
if object.respond_to?(:[]) &&
((object.respond_to?(:key?) && object.key?(key)) ||
(object.respond_to?(:fetch) && key.is_a?(Integer)))
if accessible?(object, key)
# if its a proc we will replace the entry with the proc
res = context.lookup_and_evaluate(object, key)
object = res.to_liquid
object = context.lookup_and_evaluate(object, key)
# Skip to_liquid for common primitive types (they return self)
unless object.instance_of?(String) || object.instance_of?(Integer) || object.instance_of?(Float) ||
object.instance_of?(Array) || object.instance_of?(Hash) || object.nil?
object = liquidize(object, context)
end
# Some special cases. If the part wasn't in square brackets and
# no key with the same name was found we interpret following calls
# as commands and call them on the current object
elsif lookup_command?(i) && object.respond_to?(key)
object = object.send(key).to_liquid
object = object.send(key)
unless object.instance_of?(String) || object.instance_of?(Integer) || object.instance_of?(Array) || object.nil?
object = liquidize(object, context)
end
# Handle string first/last like ActiveSupport does (returns first/last character)
# ActiveSupport returns "" for empty strings, not nil
elsif lookup_command?(i) && object.is_a?(String) && (key == "first" || key == "last")
object = key == "first" ? (object[0] || "") : (object[-1] || "")
# No key was present with the desired value and it wasn't one of the directly supported
# keywords either. The only thing we got left is to return nil or
@@ -69,9 +186,6 @@ module Liquid
return nil unless context.strict_variables
raise Liquid::UndefinedVariable, "undefined variable #{key}"
end
# If we are dealing with a drop here we have to
object.context = context if object.respond_to?(:context=)
end
object
@@ -81,6 +195,27 @@ module Liquid
self.class == other.class && state == other.state
end
private
# Returns true if +object+ has +key+ accessible via [] lookup.
def accessible?(object, key)
if object.instance_of?(Hash)
object.key?(key)
else
object.respond_to?(:[]) &&
((object.respond_to?(:key?) && object.key?(key)) ||
(object.respond_to?(:fetch) && key.is_a?(Integer)))
end
end
# Calls to_liquid on +object+ and wires up the context reference if needed.
# Skipped for primitive types that return self from to_liquid.
def liquidize(object, context)
object = object.to_liquid
object.context = context if object.respond_to?(:context=)
object
end
protected
def state
+1 -1
View File
@@ -2,5 +2,5 @@
# frozen_string_literal: true
module Liquid
VERSION = "5.4.0"
VERSION = "5.12.0"
end
+5 -2
View File
@@ -13,11 +13,11 @@ Gem::Specification.new do |s|
s.summary = "A secure, non-evaling end user template engine with aesthetic markup."
s.authors = ["Tobias Lütke"]
s.email = ["[email protected]"]
s.homepage = "http://www.liquidmarkup.org"
s.homepage = "https://shopify.github.io/liquid/"
s.license = "MIT"
# s.description = "A secure, non-evaling end user template engine with aesthetic markup."
s.required_ruby_version = ">= 2.7.0"
s.required_ruby_version = ">= 3.0.0"
s.required_rubygems_version = ">= 1.3.7"
s.metadata['allowed_push_host'] = 'https://rubygems.org'
@@ -28,6 +28,9 @@ Gem::Specification.new do |s|
s.require_path = "lib"
s.add_dependency("strscan", ">= 3.1.1")
s.add_dependency("bigdecimal")
s.add_development_dependency('rake', '~> 13.0')
s.add_development_dependency('minitest')
end
+62
View File
@@ -0,0 +1,62 @@
# frozen_string_literal: true
# Quick benchmark for autoresearch: measures parse µs, render µs, and object allocations
# Outputs machine-readable metrics to stdout
require_relative 'theme_runner'
RubyVM::YJIT.enable if defined?(RubyVM::YJIT)
runner = ThemeRunner.new
# Warmup — enough iterations for YJIT to fully optimize hot paths
20.times { runner.compile }
20.times { runner.render }
GC.start
GC.compact if GC.respond_to?(:compact)
# Measure parse
parse_times = []
10.times do
GC.disable
t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
runner.compile
t1 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
GC.enable
GC.start
parse_times << (t1 - t0) * 1_000_000 # µs
end
# Measure render
render_times = []
10.times do
GC.disable
t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
runner.render
t1 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
GC.enable
GC.start
render_times << (t1 - t0) * 1_000_000 # µs
end
# Measure object allocations for one parse+render cycle
require 'objspace'
GC.start
GC.disable
before = ObjectSpace.count_objects.values_at(:TOTAL).first - ObjectSpace.count_objects.values_at(:FREE).first
runner.compile
runner.render
after = ObjectSpace.count_objects.values_at(:TOTAL).first - ObjectSpace.count_objects.values_at(:FREE).first
GC.enable
allocations = after - before
parse_us = parse_times.min.round(0)
render_us = render_times.min.round(0)
combined_us = parse_us + render_us
puts "RESULTS"
puts "parse_us=#{parse_us}"
puts "render_us=#{render_us}"
puts "combined_us=#{combined_us}"
puts "allocations=#{allocations}"
+11 -6
View File
@@ -3,18 +3,23 @@
require 'benchmark/ips'
require_relative 'theme_runner'
Liquid::Template.error_mode = ARGV.first.to_sym if ARGV.first
RubyVM::YJIT.enable if defined?(RubyVM::YJIT)
Liquid::Environment.default.error_mode = ARGV.first.to_sym if ARGV.first
profiler = ThemeRunner.new
Benchmark.ips do |x|
x.time = 10
x.warmup = 5
x.time = 20
x.warmup = 10
puts
puts "Running benchmark for #{x.time} seconds (with #{x.warmup} seconds warmup)."
puts
x.report("parse:") { profiler.compile }
x.report("render:") { profiler.render }
x.report("parse & render:") { profiler.run }
phase = ENV["PHASE"] || "all"
x.report("tokenize:") { profiler.tokenize } if phase == "all" || phase == "tokenize"
x.report("parse:") { profiler.compile } if phase == "all" || phase == "parse"
x.report("render:") { profiler.render } if phase == "all" || phase == "render"
x.report("parse & render:") { profiler.run } if phase == "all" || phase == "run"
end
+8 -7
View File
@@ -11,11 +11,12 @@ require_relative 'shop_filter'
require_relative 'tag_filter'
require_relative 'weight_filter'
Liquid::Template.register_tag('paginate', Paginate)
Liquid::Template.register_tag('form', CommentForm)
default_environment = Liquid::Environment.default
default_environment.register_tag('paginate', Paginate)
default_environment.register_tag('form', CommentForm)
Liquid::Template.register_filter(JsonFilter)
Liquid::Template.register_filter(MoneyFilter)
Liquid::Template.register_filter(WeightFilter)
Liquid::Template.register_filter(ShopFilter)
Liquid::Template.register_filter(TagFilter)
default_environment.register_filter(JsonFilter)
default_environment.register_filter(MoneyFilter)
default_environment.register_filter(WeightFilter)
default_environment.register_filter(ShopFilter)
default_environment.register_filter(TagFilter)
+13
View File
@@ -48,6 +48,19 @@ class ThemeRunner
end
end
# `tokenize` will just test the tokenizen portion of liquid without any templates
def tokenize
ss = StringScanner.new("")
@tests.each do |test_hash|
tokenizer = Liquid::Tokenizer.new(
source: test_hash[:liquid],
string_scanner: ss,
line_numbers: true,
)
while tokenizer.shift; end
end
end
# `run` is called to benchmark rendering and compiling at the same time
def run
each_test do |liquid, layout, assigns, page_template, template_name|
+94
View File
@@ -0,0 +1,94 @@
# frozen_string_literal: true
require "benchmark/ips"
# benchmark liquid lexing
require 'liquid'
RubyVM::YJIT.enable
STRING_MARKUPS = [
"\"foo\"",
"\"fooooooooooo\"",
"\"foooooooooooooooooooooooooooooo\"",
"'foo'",
"'fooooooooooo'",
"'foooooooooooooooooooooooooooooo'",
]
VARIABLE_MARKUPS = [
"article",
"article.title",
"article.title.size",
"very_long_variable_name_2024_11_05",
"very_long_variable_name_2024_11_05.size",
]
NUMBER_MARKUPS = [
"0",
"35",
"1241891024912849",
"3.5",
"3.51214128409128",
"12381902839.123819283910283",
"123.456.789",
"-123",
"-12.33",
"-405.231",
"-0",
"0",
"0.0",
"0.0000000000000000000000",
"0.00000000001",
]
RANGE_MARKUPS = [
"(1..30)",
"(1...30)",
"(1..30..5)",
"(1.0...30.0)",
"(1.........30)",
"(1..foo)",
"(foo..30)",
"(foo..bar)",
"(foo...bar...100)",
"(foo...bar...100.0)",
]
LITERAL_MARKUPS = [
nil,
'nil',
'null',
'',
'true',
'false',
'blank',
'empty',
]
MARKUPS = {
"string" => STRING_MARKUPS,
"literal" => LITERAL_MARKUPS,
"variable" => VARIABLE_MARKUPS,
"number" => NUMBER_MARKUPS,
"range" => RANGE_MARKUPS,
}
Benchmark.ips do |x|
x.config(time: 5, warmup: 5)
MARKUPS.each do |type, markups|
x.report("Liquid::Expression#parse: #{type}") do
markups.each do |markup|
Liquid::Expression.parse(markup)
end
end
end
x.report("Liquid::Expression#parse: all") do
MARKUPS.values.flatten.each do |markup|
Liquid::Expression.parse(markup)
end
end
end
+43
View File
@@ -0,0 +1,43 @@
# frozen_string_literal: true
require "benchmark/ips"
# benchmark liquid lexing
require 'liquid'
RubyVM::YJIT.enable
EXPRESSIONS = [
"foo[1..2].baz",
"12.0",
"foo.bar.based",
"21 - 62",
"foo.bar.baz",
"foo > 12",
"foo < 12",
"foo <= 12",
"foo >= 12",
"foo <> 12",
"foo == 12",
"foo != 12",
"foo contains 12",
"foo contains 'bar'",
"foo != 'bar'",
"'foo' contains 'bar'",
'234089',
"foo | default: -1",
]
Benchmark.ips do |x|
x.config(time: 10, warmup: 5)
x.report("Liquid::Lexer#tokenize") do
EXPRESSIONS.each do |expr|
l = Liquid::Lexer.new(expr)
l.tokenize
end
end
x.compare!
end
+36
View File
@@ -0,0 +1,36 @@
# frozen_string_literal: true
# Liquid Spec Adapter for Shopify/liquid (Ruby reference implementation)
#
# Run with: bundle exec liquid-spec run spec/ruby_liquid.rb
$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
require 'liquid'
LiquidSpec.configure do |config|
# Run core Liquid specs
config.features = [:core]
end
# Compile a template string into a Liquid::Template
LiquidSpec.compile do |ctx, source, options|
ctx[:template] = Liquid::Template.parse(source, **options)
end
# Render a compiled template with the given context
# @param ctx [Hash] adapter context containing :template
# @param assigns [Hash] environment variables
# @param options [Hash] :registers, :strict_errors, :exception_renderer
LiquidSpec.render do |ctx, assigns, options|
registers = Liquid::Registers.new(options[:registers] || {})
context = Liquid::Context.build(
static_environments: assigns,
registers: registers,
rethrow_errors: options[:strict_errors],
)
context.exception_renderer = options[:exception_renderer] if options[:exception_renderer]
ctx[:template].render(context)
end
+34
View File
@@ -0,0 +1,34 @@
# frozen_string_literal: true
# Liquid Spec Adapter for Shopify/liquid with lax parsing mode
#
# Run with: bundle exec liquid-spec run spec/ruby_liquid_lax.rb
$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
require 'liquid'
LiquidSpec.configure do |config|
config.features = [:core, :lax_parsing]
end
# Compile a template string into a Liquid::Template
LiquidSpec.compile do |ctx, source, options|
# Force lax mode
options = options.merge(error_mode: :lax)
ctx[:template] = Liquid::Template.parse(source, **options)
end
# Render a compiled template with the given context
LiquidSpec.render do |ctx, assigns, options|
registers = Liquid::Registers.new(options[:registers] || {})
context = Liquid::Context.build(
static_environments: assigns,
registers: registers,
rethrow_errors: options[:strict_errors],
)
context.exception_renderer = options[:exception_renderer] if options[:exception_renderer]
ctx[:template].render(context)
end
+37
View File
@@ -0,0 +1,37 @@
# frozen_string_literal: true
# Liquid Spec Adapter for Shopify/liquid with ActiveSupport loaded
#
# Run with: bundle exec liquid-spec run spec/ruby_liquid_with_active_support.rb
$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
require 'active_support/all'
require 'liquid'
LiquidSpec.configure do |config|
# Run core Liquid specs plus ActiveSupport SafeBuffer tests
config.features = [:core, :activesupport]
end
# Compile a template string into a Liquid::Template
LiquidSpec.compile do |ctx, source, options|
ctx[:template] = Liquid::Template.parse(source, **options)
end
# Render a compiled template with the given context
# @param ctx [Hash] adapter context containing :template
# @param assigns [Hash] environment variables
# @param options [Hash] :registers, :strict_errors, :exception_renderer
LiquidSpec.render do |ctx, assigns, options|
registers = Liquid::Registers.new(options[:registers] || {})
context = Liquid::Context.build(
static_environments: assigns,
registers: registers,
rethrow_errors: options[:strict_errors],
)
context.exception_renderer = options[:exception_renderer] if options[:exception_renderer]
ctx[:template].render(context)
end
+41
View File
@@ -0,0 +1,41 @@
# frozen_string_literal: true
# Liquid Spec Adapter for Shopify/liquid with YJIT + strict mode + ActiveSupport
#
# Run with: bundle exec liquid-spec run spec/ruby_liquid_yjit.rb
$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
# Enable YJIT if available
if defined?(RubyVM::YJIT) && RubyVM::YJIT.respond_to?(:enable)
RubyVM::YJIT.enable
end
require 'active_support/all'
require 'liquid'
LiquidSpec.configure do |config|
config.features = [:core, :activesupport]
end
# Compile a template string into a Liquid::Template
LiquidSpec.compile do |ctx, source, options|
# Force strict mode
options = { error_mode: :strict }.merge(options)
ctx[:template] = Liquid::Template.parse(source, **options)
end
# Render a compiled template with the given context
LiquidSpec.render do |ctx, assigns, options|
registers = Liquid::Registers.new(options[:registers] || {})
context = Liquid::Context.build(
static_environments: assigns,
registers: registers,
rethrow_errors: options[:strict_errors],
)
context.exception_renderer = options[:exception_renderer] if options[:exception_renderer]
ctx[:template].render(context)
end
+68 -1
View File
@@ -36,6 +36,24 @@ class Category
end
end
class ProductsDrop < Liquid::Drop
def initialize(products)
@products = products
end
def size
@products.size
end
def to_liquid
if @context["forloop"]
@products.first(@context["forloop"].length)
else
@products
end
end
end
class CategoryDrop < Liquid::Drop
attr_accessor :category, :context
@@ -614,13 +632,28 @@ class ContextTest < Minitest::Test
end
def test_has_key_will_not_add_an_error_for_missing_keys
with_error_mode(:strict) do
with_error_modes(:strict) do
context = Context.new
context.key?('unknown')
assert_empty(context.errors)
end
end
def test_key_lookup_will_raise_for_missing_keys_when_strict_variables_is_enabled
context = Context.new
context.strict_variables = true
assert_raises(Liquid::UndefinedVariable) do
context['unknown']
end
end
def test_has_key_will_not_raise_for_missing_keys_when_strict_variables_is_enabled
context = Context.new
context.strict_variables = true
refute(context.key?('unknown'))
assert_empty(context.errors)
end
def test_context_always_uses_static_registers
registers = {
my_register: :my_value,
@@ -635,6 +668,40 @@ class ContextTest < Minitest::Test
assert_equal(:my_value, c.registers[:my_register])
end
def test_variable_to_liquid_returns_contextual_drop
context = {
"products" => ProductsDrop.new(["A", "B", "C", "D", "E"]),
}
template = Liquid::Template.parse(<<~LIQUID)
{%- for i in (1..3) -%}
for_loop_products_count: {{ products | size }}
{% endfor %}
unscoped_products_count: {{ products | size }}
LIQUID
result = template.render(context)
assert_includes(result, "for_loop_products_count: 3")
assert_includes(result, "unscoped_products_count: 5")
end
def test_new_isolated_context_inherits_parent_environment
global_environment = Liquid::Environment.build(tags: {})
context = Context.build(environment: global_environment)
subcontext = context.new_isolated_subcontext
assert_equal(global_environment, subcontext.environment)
end
def test_newly_built_context_inherits_parent_environment
global_environment = Liquid::Environment.build(tags: {})
context = Context.build(environment: global_environment)
assert_equal(global_environment, context.environment)
assert(context.environment.tags.each.to_a.empty?)
end
private
def assert_no_object_allocations
+23 -15
View File
@@ -67,7 +67,7 @@ class ErrorHandlingTest < Minitest::Test
end
def test_unrecognized_operator
with_error_mode(:strict) do
with_error_modes(:strict) do
assert_raises(SyntaxError) do
Liquid::Template.parse(' {% if 1 =! 2 %}ok{% endif %} ')
end
@@ -203,20 +203,34 @@ class ErrorHandlingTest < Minitest::Test
end
def test_setting_default_exception_renderer
old_exception_renderer = Liquid::Template.default_exception_renderer
exceptions = []
Liquid::Template.default_exception_renderer = ->(e) {
default_exception_renderer = ->(e) {
exceptions << e
''
}
template = Liquid::Template.parse('This is a runtime error: {{ errors.argument_error }}')
env = Liquid::Environment.build(exception_renderer: default_exception_renderer)
template = Liquid::Template.parse('This is a runtime error: {{ errors.argument_error }}', environment: env)
output = template.render('errors' => ErrorDrop.new)
assert_equal('This is a runtime error: ', output)
assert_equal([Liquid::ArgumentError], template.errors.map(&:class))
ensure
Liquid::Template.default_exception_renderer = old_exception_renderer if old_exception_renderer
end
def test_setting_exception_renderer_on_environment
exceptions = []
exception_renderer = ->(e) do
exceptions << e
''
end
environment = Liquid::Environment.build(exception_renderer: exception_renderer)
template = Liquid::Template.parse('This is a runtime error: {{ errors.argument_error }}', environment: environment)
output = template.render('errors' => ErrorDrop.new)
assert_equal('This is a runtime error: ', output)
assert_equal([Liquid::ArgumentError], template.errors.map(&:class))
end
def test_exception_renderer_exposing_non_liquid_error
@@ -242,16 +256,10 @@ class ErrorHandlingTest < Minitest::Test
end
def test_included_template_name_with_line_numbers
old_file_system = Liquid::Template.file_system
environment = Liquid::Environment.build(file_system: TestFileSystem.new)
template = Liquid::Template.parse("Argument error:\n{% include 'product' %}", line_numbers: true, environment: environment)
page = template.render('errors' => ErrorDrop.new)
begin
Liquid::Template.file_system = TestFileSystem.new
template = Liquid::Template.parse("Argument error:\n{% include 'product' %}", line_numbers: true)
page = template.render('errors' => ErrorDrop.new)
ensure
Liquid::Template.file_system = old_file_system
end
assert_equal("Argument error:\nLiquid error (product line 1): argument error", page)
assert_equal("product", template.errors.first.template_name)
end
+141
View File
@@ -1,6 +1,7 @@
# frozen_string_literal: true
require 'test_helper'
require 'lru_redux'
class ExpressionTest < Minitest::Test
def test_keyword_literals
@@ -13,6 +14,7 @@ class ExpressionTest < Minitest::Test
assert_template_result("double quoted", '{{"double quoted"}}')
assert_template_result("spaced", "{{ 'spaced' }}")
assert_template_result("spaced2", "{{ 'spaced2' }}")
assert_template_result("emoji🔥", "{{ 'emoji🔥' }}")
end
def test_int
@@ -22,8 +24,22 @@ class ExpressionTest < Minitest::Test
end
def test_float
assert_template_result("-17.42", "{{ -17.42 }}")
assert_template_result("2.5", "{{ 2.5 }}")
with_error_modes(:lax) do
assert_expression_result(0.0, "0.....5")
assert_expression_result(0.0, "-0..1")
end
assert_expression_result(1.5, "1.5")
# this is a unfortunate quirky behavior of Liquid
result = Expression.parse(".5")
assert_kind_of(Liquid::VariableLookup, result)
result = Expression.parse("-.5")
assert_kind_of(Liquid::VariableLookup, result)
end
def test_range
@@ -40,6 +56,131 @@ class ExpressionTest < Minitest::Test
)
end
def test_quirky_negative_sign_expression_markup
result = Expression.parse("-", nil)
assert(result.is_a?(VariableLookup))
assert_equal("-", result.name)
# for this template, the expression markup is "-"
assert_template_result(
"",
"{{ - 'theme.css' - }}",
error_mode: :lax,
)
end
def test_expression_cache
skip("Liquid-C does not support Expression caching") if defined?(Liquid::C) && Liquid::C.enabled
cache = {}
template = <<~LIQUID
{% assign x = 1 %}
{{ x }}
{% assign x = 2 %}
{{ x }}
{% assign y = 1 %}
{{ y }}
LIQUID
Liquid::Template.parse(template, expression_cache: cache).render
assert_equal(
["1", "2", "x", "y"],
cache.to_a.map { _1[0] }.sort,
)
end
def test_expression_cache_with_true_boolean
skip("Liquid-C does not support Expression caching") if defined?(Liquid::C) && Liquid::C.enabled
template = <<~LIQUID
{% assign x = 1 %}
{{ x }}
{% assign x = 2 %}
{{ x }}
{% assign y = 1 %}
{{ y }}
LIQUID
parse_context = ParseContext.new(expression_cache: true)
Liquid::Template.parse(template, parse_context).render
cache = parse_context.instance_variable_get(:@expression_cache)
assert_equal(
["1", "2", "x", "y"],
cache.to_a.map { _1[0] }.sort,
)
end
def test_expression_cache_with_lru_redux
skip("Liquid-C does not support Expression caching") if defined?(Liquid::C) && Liquid::C.enabled
cache = LruRedux::Cache.new(10)
template = <<~LIQUID
{% assign x = 1 %}
{{ x }}
{% assign x = 2 %}
{{ x }}
{% assign y = 1 %}
{{ y }}
LIQUID
Liquid::Template.parse(template, expression_cache: cache).render
assert_equal(
["1", "2", "x", "y"],
cache.to_a.map { _1[0] }.sort,
)
end
def test_disable_expression_cache
skip("Liquid-C does not support Expression caching") if defined?(Liquid::C) && Liquid::C.enabled
template = <<~LIQUID
{% assign x = 1 %}
{{ x }}
{% assign x = 2 %}
{{ x }}
{% assign y = 1 %}
{{ y }}
LIQUID
parse_context = Liquid::ParseContext.new(expression_cache: false)
Liquid::Template.parse(template, parse_context).render
assert(parse_context.instance_variable_get(:@expression_cache).nil?)
end
def test_safe_parse_with_variable_lookup
parse_context = Liquid::ParseContext.new
parser = parse_context.new_parser('product.title')
result = Liquid::Expression.safe_parse(parser)
assert_instance_of(Liquid::VariableLookup, result)
assert_equal('product', result.name)
assert_equal(['title'], result.lookups)
end
def test_safe_parse_with_number
parse_context = Liquid::ParseContext.new
parser = parse_context.new_parser('42')
result = Liquid::Expression.safe_parse(parser)
assert_equal(42, result)
end
def test_safe_parse_raises_syntax_error_for_invalid_expression
parse_context = Liquid::ParseContext.new
parser = parse_context.new_parser('')
error = assert_raises(Liquid::SyntaxError) do
Liquid::Expression.safe_parse(parser)
end
assert_match(/is not a valid expression/, error.message)
end
private
def assert_expression_result(expect, markup, **assigns)
+100
View File
@@ -0,0 +1,100 @@
# frozen_string_literal: true
require 'test_helper'
class HashRenderingTest < Minitest::Test
def test_render_empty_hash
assert_template_result("{}", "{{ my_hash }}", { "my_hash" => {} })
end
def test_render_hash_with_string_keys_and_values
assert_template_result("{\"key1\"=>\"value1\", \"key2\"=>\"value2\"}", "{{ my_hash }}", { "my_hash" => { "key1" => "value1", "key2" => "value2" } })
end
def test_render_hash_with_symbol_keys_and_integer_values
assert_template_result("{:key1=>1, :key2=>2}", "{{ my_hash }}", { "my_hash" => { key1: 1, key2: 2 } })
end
def test_render_nested_hash
assert_template_result("{\"outer\"=>{\"inner\"=>\"value\"}}", "{{ my_hash }}", { "my_hash" => { "outer" => { "inner" => "value" } } })
end
def test_render_hash_with_array_values
assert_template_result("{\"numbers\"=>[1, 2, 3]}", "{{ my_hash }}", { "my_hash" => { "numbers" => [1, 2, 3] } })
end
def test_render_recursive_hash
recursive_hash = { "self" => {} }
recursive_hash["self"]["self"] = recursive_hash
assert_template_result("{\"self\"=>{\"self\"=>{...}}}", "{{ my_hash }}", { "my_hash" => recursive_hash })
end
def test_hash_with_downcase_filter
assert_template_result("{\"key\"=>\"value\", \"anotherkey\"=>\"anothervalue\"}", "{{ my_hash | downcase }}", { "my_hash" => { "Key" => "Value", "AnotherKey" => "AnotherValue" } })
end
def test_hash_with_upcase_filter
assert_template_result("{\"KEY\"=>\"VALUE\", \"ANOTHERKEY\"=>\"ANOTHERVALUE\"}", "{{ my_hash | upcase }}", { "my_hash" => { "Key" => "Value", "AnotherKey" => "AnotherValue" } })
end
def test_hash_with_strip_filter
assert_template_result("{\"Key\"=>\"Value\", \"AnotherKey\"=>\"AnotherValue\"}", "{{ my_hash | strip }}", { "my_hash" => { "Key" => "Value", "AnotherKey" => "AnotherValue" } })
end
def test_hash_with_escape_filter
assert_template_result("{&quot;Key&quot;=&gt;&quot;Value&quot;, &quot;AnotherKey&quot;=&gt;&quot;AnotherValue&quot;}", "{{ my_hash | escape }}", { "my_hash" => { "Key" => "Value", "AnotherKey" => "AnotherValue" } })
end
def test_hash_with_url_encode_filter
assert_template_result("%7B%22Key%22%3D%3E%22Value%22%2C+%22AnotherKey%22%3D%3E%22AnotherValue%22%7D", "{{ my_hash | url_encode }}", { "my_hash" => { "Key" => "Value", "AnotherKey" => "AnotherValue" } })
end
def test_hash_with_strip_html_filter
assert_template_result("{\"Key\"=>\"Value\", \"AnotherKey\"=>\"AnotherValue\"}", "{{ my_hash | strip_html }}", { "my_hash" => { "Key" => "Value", "AnotherKey" => "AnotherValue" } })
end
def test_hash_with_truncate__20_filter
assert_template_result("{\"Key\"=>\"Value\", ...", "{{ my_hash | truncate: 20 }}", { "my_hash" => { "Key" => "Value", "AnotherKey" => "AnotherValue" } })
end
def test_hash_with_replace___key____replaced_key__filter
assert_template_result("{\"Key\"=>\"Value\", \"AnotherKey\"=>\"AnotherValue\"}", "{{ my_hash | replace: 'key', 'replaced_key' }}", { "my_hash" => { "Key" => "Value", "AnotherKey" => "AnotherValue" } })
end
def test_hash_with_append____appended_text__filter
assert_template_result("{\"Key\"=>\"Value\", \"AnotherKey\"=>\"AnotherValue\"} appended text", "{{ my_hash | append: ' appended text' }}", { "my_hash" => { "Key" => "Value", "AnotherKey" => "AnotherValue" } })
end
def test_hash_with_prepend___prepended_text___filter
assert_template_result("prepended text {\"Key\"=>\"Value\", \"AnotherKey\"=>\"AnotherValue\"}", "{{ my_hash | prepend: 'prepended text ' }}", { "my_hash" => { "Key" => "Value", "AnotherKey" => "AnotherValue" } })
end
def test_render_hash_with_array_values_empty
assert_template_result("{\"numbers\"=>[]}", "{{ my_hash }}", { "my_hash" => { "numbers" => [] } })
end
def test_render_hash_with_array_values_hash
assert_template_result("{\"numbers\"=>[{:foo=>42}]}", "{{ my_hash }}", { "my_hash" => { "numbers" => [{ foo: 42 }] } })
end
def test_join_filter_with_hash
array = [{ "key1" => "value1" }, { "key2" => "value2" }]
glue = { "lol" => "wut" }
assert_template_result("{\"key1\"=>\"value1\"}{\"lol\"=>\"wut\"}{\"key2\"=>\"value2\"}", "{{ my_array | join: glue }}", { "my_array" => array, "glue" => glue })
end
def test_render_hash_with_hash_key
assert_template_result("{{\"foo\"=>\"bar\"}=>42}", "{{ my_hash }}", { "my_hash" => { Hash["foo" => "bar"] => 42 } })
end
def test_rendering_hash_with_custom_to_s_method_uses_custom_to_s
assert_template_result("kewl", "{{ my_hash }}", { "my_hash" => HashWithCustomToS.new })
end
def test_rendering_hash_without_custom_to_s_uses_default_inspect
my_hash = HashWithoutCustomToS.new
my_hash[:foo] = :bar
assert_template_result("{:foo=>:bar}", "{{ my_hash }}", { "my_hash" => my_hash })
end
end
+29 -9
View File
@@ -31,18 +31,18 @@ class ParsingQuirksTest < Minitest::Test
def test_error_on_empty_filter
assert(Template.parse("{{test}}"))
with_error_mode(:lax) do
with_error_modes(:lax) do
assert(Template.parse("{{|test}}"))
end
with_error_mode(:strict) do
with_error_modes(:strict) do
assert_raises(SyntaxError) { Template.parse("{{|test}}") }
assert_raises(SyntaxError) { Template.parse("{{test |a|b|}}") }
end
end
def test_meaningless_parens_error
with_error_mode(:strict) do
with_error_modes(:strict) do
assert_raises(SyntaxError) do
markup = "a == 'foo' or (b == 'bar' and c == 'baz') or false"
Template.parse("{% if #{markup} %} YES {% endif %}")
@@ -51,7 +51,7 @@ class ParsingQuirksTest < Minitest::Test
end
def test_unexpected_characters_syntax_error
with_error_mode(:strict) do
with_error_modes(:strict) do
assert_raises(SyntaxError) do
markup = "true && false"
Template.parse("{% if #{markup} %} YES {% endif %}")
@@ -70,7 +70,7 @@ class ParsingQuirksTest < Minitest::Test
end
def test_meaningless_parens_lax
with_error_mode(:lax) do
with_error_modes(:lax) do
assigns = { 'b' => 'bar', 'c' => 'baz' }
markup = "a == 'foo' or (b == 'bar' and c == 'baz') or false"
assert_template_result(' YES ', "{% if #{markup} %} YES {% endif %}", assigns)
@@ -78,7 +78,7 @@ class ParsingQuirksTest < Minitest::Test
end
def test_unexpected_characters_silently_eat_logic_lax
with_error_mode(:lax) do
with_error_modes(:lax) do
markup = "true && false"
assert_template_result(' YES ', "{% if #{markup} %} YES {% endif %}")
markup = "false || true"
@@ -93,7 +93,7 @@ class ParsingQuirksTest < Minitest::Test
end
def test_unanchored_filter_arguments
with_error_mode(:lax) do
with_error_modes(:lax) do
assert_template_result('hi', "{{ 'hi there' | split$$$:' ' | first }}")
assert_template_result('x', "{{ 'X' | downcase) }}")
@@ -106,14 +106,14 @@ class ParsingQuirksTest < Minitest::Test
end
def test_invalid_variables_work
with_error_mode(:lax) do
with_error_modes(:lax) do
assert_template_result('bar', "{% assign 123foo = 'bar' %}{{ 123foo }}")
assert_template_result('123', "{% assign 123 = 'bar' %}{{ 123 }}")
end
end
def test_extra_dots_in_ranges
with_error_mode(:lax) do
with_error_modes(:lax) do
assert_template_result('12345', "{% for i in (1...5) %}{{ i }}{% endfor %}")
end
end
@@ -131,4 +131,24 @@ class ParsingQuirksTest < Minitest::Test
def test_contains_in_id
assert_template_result(' YES ', '{% if containsallshipments == true %} YES {% endif %}', { 'containsallshipments' => true })
end
def test_incomplete_expression
with_error_modes(:lax) do
assert_template_result("false", "{{ false - }}")
assert_template_result("false", "{{ false > }}")
assert_template_result("false", "{{ false < }}")
assert_template_result("false", "{{ false = }}")
assert_template_result("false", "{{ false ! }}")
assert_template_result("false", "{{ false 1 }}")
assert_template_result("false", "{{ false a }}")
assert_template_result("false", "{% liquid assign foo = false -\n%}{{ foo }}")
assert_template_result("false", "{% liquid assign foo = false >\n%}{{ foo }}")
assert_template_result("false", "{% liquid assign foo = false <\n%}{{ foo }}")
assert_template_result("false", "{% liquid assign foo = false =\n%}{{ foo }}")
assert_template_result("false", "{% liquid assign foo = false !\n%}{{ foo }}")
assert_template_result("false", "{% liquid assign foo = false 1\n%}{{ foo }}")
assert_template_result("false", "{% liquid assign foo = false a\n%}{{ foo }}")
end
end
end # ParsingQuirksTest
+1 -1
View File
@@ -33,7 +33,7 @@ class ProfilerTest < Minitest::Test
end
def setup
Liquid::Template.file_system = ProfilingFileSystem.new
Liquid::Environment.default.file_system = ProfilingFileSystem.new
end
def test_template_allows_flagging_profiling
+29 -22
View File
@@ -44,33 +44,40 @@ class SecurityTest < Minitest::Test
end
def test_does_not_permanently_add_filters_to_symbol_table
current_symbols = Symbol.all_symbols
assert_no_new_symbols do
# MRI imprecisely marks objects found on the C stack, which can result
# in uninitialized memory being marked. This can even result in the test failing
# deterministically for a given compilation of ruby. Using a separate thread will
# keep these writes of the symbol pointer on a separate stack that will be garbage
# collected after Thread#join.
Thread.new do
test = %( {{ "some_string" | a_bad_filter }} )
Template.parse(test).render!
nil
end.join
# MRI imprecisely marks objects found on the C stack, which can result
# in uninitialized memory being marked. This can even result in the test failing
# deterministically for a given compilation of ruby. Using a separate thread will
# keep these writes of the symbol pointer on a separate stack that will be garbage
# collected after Thread#join.
Thread.new do
test = %( {{ "some_string" | a_bad_filter }} )
Template.parse(test).render!
nil
end.join
GC.start
assert_equal([], (Symbol.all_symbols - current_symbols))
GC.start
end
end
def test_does_not_add_drop_methods_to_symbol_table
assert_no_new_symbols do
assigns = { 'drop' => Drop.new }
assert_equal("", Template.parse("{{ drop.custom_method_1 }}", assigns).render!)
assert_equal("", Template.parse("{{ drop.custom_method_2 }}", assigns).render!)
assert_equal("", Template.parse("{{ drop.custom_method_3 }}", assigns).render!)
end
end
def assert_no_new_symbols
# Run once to trigger any first-time initialization which might create some symbols,
# for example autoload or lazy method parsing might create symbols on first execution.
yield
# Ensure no new symbols for further runs, i.e. the code does not leak symbols
current_symbols = Symbol.all_symbols
assigns = { 'drop' => Drop.new }
assert_equal("", Template.parse("{{ drop.custom_method_1 }}", assigns).render!)
assert_equal("", Template.parse("{{ drop.custom_method_2 }}", assigns).render!)
assert_equal("", Template.parse("{{ drop.custom_method_3 }}", assigns).render!)
assert_equal([], (Symbol.all_symbols - current_symbols))
yield
assert_equal([], Symbol.all_symbols - current_symbols)
end
def test_max_depth_nested_blocks_does_not_raise_exception
+377 -11
View File
@@ -32,7 +32,7 @@ class TestDrop < Liquid::Drop
attr_reader :value
def registers
{ @value => @context.registers[@value] }
"{#{@value.inspect}=>#{@context.registers[@value].inspect}}"
end
end
@@ -116,7 +116,7 @@ class StandardFiltersTest < Minitest::Test
end
def test_slice_on_arrays
input = 'foobar'.split(//)
input = 'foobar'.split('')
assert_equal(%w(o o b), @filters.slice(input, 1, 3))
assert_equal(%w(o o b a r), @filters.slice(input, 1, 1000))
assert_equal(%w(), @filters.slice(input, 1, 0))
@@ -133,6 +133,18 @@ class StandardFiltersTest < Minitest::Test
assert_equal([], @filters.slice(input, -(1 << 63), 6))
end
def test_find_on_empty_array
assert_nil(@filters.find([], 'foo', 'bar'))
end
def test_find_index_on_empty_array
assert_nil(@filters.find_index([], 'foo', 'bar'))
end
def test_has_on_empty_array
refute(@filters.has([], 'foo', 'bar'))
end
def test_truncate
assert_equal('1234...', @filters.truncate('1234567890', 7))
assert_equal('1234567890', @filters.truncate('1234567890', 20))
@@ -152,6 +164,13 @@ class StandardFiltersTest < Minitest::Test
assert_equal(['A', 'Z'], @filters.split('A1Z', 1))
end
def test_squish_filter
assert_equal("foo bar boo", Liquid::Template.parse(%({{ " foo bar
\t boo " | squish }})).render)
assert_equal("", Liquid::Template.parse('{{ nil | squish }}').render)
assert_equal("", Liquid::Template.parse('{{ " " | squish }}').render)
end
def test_escape
assert_equal('&lt;strong&gt;', @filters.escape('<strong>'))
assert_equal('1', @filters.escape(1))
@@ -176,7 +195,17 @@ class StandardFiltersTest < Minitest::Test
end
def test_base64_decode
assert_equal('one two three', @filters.base64_decode('b25lIHR3byB0aHJlZQ=='))
decoded = @filters.base64_decode('b25lIHR3byB0aHJlZQ==')
assert_equal('one two three', decoded)
assert_equal(Encoding::UTF_8, decoded.encoding)
decoded = @filters.base64_decode('4pyF')
assert_equal('✅', decoded)
assert_equal(Encoding::UTF_8, decoded.encoding)
decoded = @filters.base64_decode("/w==")
assert_equal(Encoding::ASCII_8BIT, decoded.encoding)
assert_equal((+"\xFF").force_encoding(Encoding::ASCII_8BIT), decoded)
exception = assert_raises(Liquid::ArgumentError) do
@filters.base64_decode("invalidbase64")
@@ -194,10 +223,21 @@ class StandardFiltersTest < Minitest::Test
end
def test_base64_url_safe_decode
decoded = @filters.base64_url_safe_decode('YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXogQUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVogMTIzNDU2Nzg5MCAhQCMkJV4mKigpLT1fKy8_Ljo7W117fVx8')
assert_equal(
'abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ 1234567890 !@#$%^&*()-=_+/?.:;[]{}\|',
@filters.base64_url_safe_decode('YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXogQUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVogMTIzNDU2Nzg5MCAhQCMkJV4mKigpLT1fKy8_Ljo7W117fVx8'),
decoded,
)
assert_equal(Encoding::UTF_8, decoded.encoding)
decoded = @filters.base64_url_safe_decode('4pyF')
assert_equal('✅', decoded)
assert_equal(Encoding::UTF_8, decoded.encoding)
decoded = @filters.base64_url_safe_decode("_w==")
assert_equal(Encoding::ASCII_8BIT, decoded.encoding)
assert_equal((+"\xFF").force_encoding(Encoding::ASCII_8BIT), decoded)
exception = assert_raises(Liquid::ArgumentError) do
@filters.base64_url_safe_decode("invalidbase64")
end
@@ -260,6 +300,10 @@ class StandardFiltersTest < Minitest::Test
assert_equal('1121314', @filters.join([1, 2, 3, 4], 1))
end
def test_join_calls_to_liquid_on_each_element
assert_equal('i did it, i did it', @filters.join([CustomToLiquidDrop.new('i did it'), CustomToLiquidDrop.new('i did it')], ", "))
end
def test_sort
assert_equal([1, 2, 3, 4], @filters.sort([4, 3, 2, 1]))
assert_equal([{ "a" => 1 }, { "a" => 2 }, { "a" => 3 }, { "a" => 4 }], @filters.sort([{ "a" => 4 }, { "a" => 3 }, { "a" => 1 }, { "a" => 2 }], "a"))
@@ -310,8 +354,8 @@ class StandardFiltersTest < Minitest::Test
{ "price" => "1", "handle" => "gamma" },
{ "price" => 2, "handle" => "epsilon" },
{ "price" => "4", "handle" => "alpha" },
{ "handle" => "delta" },
{ "handle" => "beta" },
{ "handle" => "delta" },
]
assert_equal(expectation, @filters.sort_natural(input, "price"))
end
@@ -517,12 +561,23 @@ class StandardFiltersTest < Minitest::Test
end
end
def test_map_returns_empty_with_no_property
def test_map_with_value_property
array = [
{ "handle" => "alpha", "value" => "A" },
{ "handle" => "beta", "value" => "B" },
{ "handle" => "gamma", "value" => "C" }
]
assert_template_result("A B C", "{{ array | map: 'value' | join: ' ' }}", { "array" => array })
end
def test_map_returns_input_with_no_property
foo = [
[1],
[2],
[3],
]
assert_raises(Liquid::ArgumentError) do
@filters.map(foo, nil)
end
@@ -579,6 +634,40 @@ class StandardFiltersTest < Minitest::Test
assert_nil(@filters.last([]))
end
def test_first_last_on_strings
# Ruby's String class does not have first/last methods by default.
# ActiveSupport adds String#first and String#last to return the first/last character.
# Liquid must work without ActiveSupport, so the first/last filters handle strings specially.
#
# This enables template patterns like:
# {{ product.title | first }} => "S" (for "Snowboard")
# {{ customer.name | last }} => "h" (for "Smith")
#
# Note: ActiveSupport returns "" for empty strings, not nil.
assert_equal('f', @filters.first('foo'))
assert_equal('o', @filters.last('foo'))
assert_equal('', @filters.first(''))
assert_equal('', @filters.last(''))
end
def test_first_last_on_unicode_strings
# Unicode strings should return the first/last grapheme cluster (character),
# not the first/last byte. Ruby's String#[] handles this correctly with index 0/-1.
# This ensures international text works properly:
# {{ korean_name | first }} => "고" (not a partial byte sequence)
assert_equal('고', @filters.first('고스트빈'))
assert_equal('빈', @filters.last('고스트빈'))
end
def test_first_last_on_strings_via_template
# Integration test to verify the filter works end-to-end in templates.
# Empty strings return empty output (nil renders as empty string).
assert_template_result('f', '{{ name | first }}', { 'name' => 'foo' })
assert_template_result('o', '{{ name | last }}', { 'name' => 'foo' })
assert_template_result('', '{{ name | first }}', { 'name' => '' })
assert_template_result('', '{{ name | last }}', { 'name' => '' })
end
def test_replace
assert_equal('b b b b', @filters.replace('a a a a', 'a', 'b'))
assert_equal('2 2 2 2', @filters.replace('1 1 1 1', 1, 2))
@@ -806,21 +895,233 @@ class StandardFiltersTest < Minitest::Test
assert_template_result('abc', "{{ 'abc' | date: '%D' }}")
end
def test_where
input = [
def test_reject
array = [
{ "handle" => "alpha", "ok" => true },
{ "handle" => "beta", "ok" => false },
{ "handle" => "gamma", "ok" => false },
{ "handle" => "delta", "ok" => true },
]
expectation = [
template = "{{ array | reject: 'ok' | map: 'handle' | join: ' ' }}"
expected_output = "beta gamma"
assert_template_result(expected_output, template, { "array" => array })
end
def test_reject_with_value
array = [
{ "handle" => "alpha", "ok" => true },
{ "handle" => "beta", "ok" => false },
{ "handle" => "gamma", "ok" => false },
{ "handle" => "delta", "ok" => true },
]
assert_equal(expectation, @filters.where(input, "ok", true))
assert_equal(expectation, @filters.where(input, "ok"))
template = "{{ array | reject: 'ok', true | map: 'handle' | join: ' ' }}"
expected_output = "beta gamma"
assert_template_result(expected_output, template, { "array" => array })
end
def test_reject_with_false_value
array = [
{ "handle" => "alpha", "ok" => true },
{ "handle" => "beta", "ok" => false },
{ "handle" => "gamma", "ok" => false },
{ "handle" => "delta", "ok" => true },
]
template = "{{ array | reject: 'ok', false | map: 'handle' | join: ' ' }}"
expected_output = "alpha delta"
assert_template_result(expected_output, template, { "array" => array })
end
def test_has
array = [
{ "handle" => "alpha", "ok" => true },
{ "handle" => "beta", "ok" => false },
{ "handle" => "gamma", "ok" => false },
{ "handle" => "delta", "ok" => false },
]
expected_output = "true"
assert_template_result(expected_output, "{{ array | has: 'ok' }}", { "array" => array })
assert_template_result(expected_output, "{{ array | has: 'ok', true }}", { "array" => array })
end
def test_has_when_does_not_have_it
array = [
{ "handle" => "alpha", "ok" => false },
{ "handle" => "beta", "ok" => false },
{ "handle" => "gamma", "ok" => false },
{ "handle" => "delta", "ok" => false },
]
expected_output = "false"
assert_template_result(expected_output, "{{ array | has: 'ok' }}", { "array" => array })
assert_template_result(expected_output, "{{ array | has: 'ok', true }}", { "array" => array })
end
def test_has_with_empty_arrays
template = <<~LIQUID
{%- assign has_product = products | has: 'title.content', 'Not found' -%}
{%- unless has_product -%}
Product not found.
{%- endunless -%}
LIQUID
expected_output = "Product not found."
assert_template_result(expected_output, template, { "products" => [] })
end
def test_has_with_false_value
array = [
{ "handle" => "alpha", "ok" => true },
{ "handle" => "beta", "ok" => false },
{ "handle" => "gamma", "ok" => false },
{ "handle" => "delta", "ok" => true },
]
template = "{{ array | has: 'ok', false }}"
expected_output = "true"
assert_template_result(expected_output, template, { "array" => array })
end
def test_has_with_false_value_when_does_not_have_it
array = [
{ "handle" => "alpha", "ok" => true },
{ "handle" => "beta", "ok" => true },
{ "handle" => "gamma", "ok" => true },
{ "handle" => "delta", "ok" => true },
]
template = "{{ array | has: 'ok', false }}"
expected_output = "false"
assert_template_result(expected_output, template, { "array" => array })
end
def test_find_with_value
products = [
{ "title" => "Pro goggles", "price" => 1299 },
{ "title" => "Thermal gloves", "price" => 1499 },
{ "title" => "Alpine jacket", "price" => 3999 },
{ "title" => "Mountain boots", "price" => 3899 },
{ "title" => "Safety helmet", "price" => 1999 }
]
template = <<~LIQUID
{%- assign product = products | find: 'price', 3999 -%}
{{- product.title -}}
LIQUID
expected_output = "Alpine jacket"
assert_template_result(expected_output, template, { "products" => products })
end
def test_find_with_empty_arrays
template = <<~LIQUID
{%- assign product = products | find: 'title.content', 'Not found' -%}
{%- unless product -%}
Product not found.
{%- endunless -%}
LIQUID
expected_output = "Product not found."
assert_template_result(expected_output, template, { "products" => [] })
end
def test_find_index_with_value
products = [
{ "title" => "Pro goggles", "price" => 1299 },
{ "title" => "Thermal gloves", "price" => 1499 },
{ "title" => "Alpine jacket", "price" => 3999 },
{ "title" => "Mountain boots", "price" => 3899 },
{ "title" => "Safety helmet", "price" => 1999 }
]
template = <<~LIQUID
{%- assign index = products | find_index: 'price', 3999 -%}
{{- index -}}
LIQUID
expected_output = "2"
assert_template_result(expected_output, template, { "products" => products })
end
def test_find_index_with_empty_arrays
template = <<~LIQUID
{%- assign index = products | find_index: 'title.content', 'Not found' -%}
{%- unless index -%}
Index not found.
{%- endunless -%}
LIQUID
expected_output = "Index not found."
assert_template_result(expected_output, template, { "products" => [] })
end
def test_where
array = [
{ "handle" => "alpha", "ok" => true },
{ "handle" => "beta", "ok" => false },
{ "handle" => "gamma", "ok" => false },
{ "handle" => "delta", "ok" => true },
]
template = "{{ array | where: 'ok' | map: 'handle' | join: ' ' }}"
expected_output = "alpha delta"
assert_template_result(expected_output, template, { "array" => array })
end
def test_where_with_empty_string_is_a_no_op
environment = { "array" => ["alpha", "beta", "gamma"] }
expected_output = "alpha beta gamma"
template = "{{ array | where: '' | join: ' ' }}"
assert_template_result(expected_output, template, environment)
end
def test_where_with_nil_is_a_no_op
environment = { "array" => ["alpha", "beta", "gamma"] }
template = "{{ array | where: nil | join: ' ' }}"
assert_raises(Liquid::ArgumentError) do
assert_template_result("alpha beta gamma", template, environment)
end
end
def test_where_with_value
array = [
{ "handle" => "alpha", "ok" => true },
{ "handle" => "beta", "ok" => false },
{ "handle" => "gamma", "ok" => false },
{ "handle" => "delta", "ok" => true },
]
template = "{{ array | where: 'ok', true | map: 'handle' | join: ' ' }}"
expected_output = "alpha delta"
assert_template_result(expected_output, template, { "array" => array })
end
def test_where_with_false_value
array = [
{ "handle" => "alpha", "ok" => true },
{ "handle" => "beta", "ok" => false },
{ "handle" => "gamma", "ok" => false },
{ "handle" => "delta", "ok" => true },
]
template = "{{ array | where: 'ok', false | map: 'handle' | join: ' ' }}"
expected_output = "beta gamma"
assert_template_result(expected_output, template, { "array" => array })
end
def test_where_string_keys
@@ -880,6 +1181,8 @@ class StandardFiltersTest < Minitest::Test
end
def test_all_filters_never_raise_non_liquid_exception
skip("too slow on non-CRuby due to many exceptions") unless RUBY_ENGINE == 'ruby'
test_drop = TestDrop.new(value: "test")
test_drop.context = Context.new
test_enum = TestEnumerable.new
@@ -994,6 +1297,69 @@ class StandardFiltersTest < Minitest::Test
assert(t.foo > 0)
end
def test_sum_of_floats
input = [0.1, 0.2, 0.3]
assert_equal(0.6, @filters.sum(input))
assert_template_result("0.6", "{{ input | sum }}", { "input" => input })
end
def test_sum_of_negative_floats
input = [0.1, 0.2, -0.3]
assert_equal(0.0, @filters.sum(input))
assert_template_result("0.0", "{{ input | sum }}", { "input" => input })
end
def test_sum_with_float_strings
input = [0.1, "0.2", "0.3"]
assert_equal(0.6, @filters.sum(input))
assert_template_result("0.6", "{{ input | sum }}", { "input" => input })
end
def test_sum_resulting_in_negative_float
input = [0.1, -0.2, -0.3]
assert_equal(-0.4, @filters.sum(input))
assert_template_result("-0.4", "{{ input | sum }}", { "input" => input })
end
def test_sum_with_floats_and_indexable_map_values
input = [{ "quantity" => 1 }, { "quantity" => 0.2, "weight" => -0.3 }, { "weight" => 0.4 }]
assert_equal(0.0, @filters.sum(input))
assert_equal(1.2, @filters.sum(input, "quantity"))
assert_equal(0.1, @filters.sum(input, "weight"))
assert_equal(0.0, @filters.sum(input, "subtotal"))
assert_template_result("0", "{{ input | sum }}", { "input" => input })
assert_template_result("1.2", "{{ input | sum: 'quantity' }}", { "input" => input })
assert_template_result("0.1", "{{ input | sum: 'weight' }}", { "input" => input })
assert_template_result("0", "{{ input | sum: 'subtotal' }}", { "input" => input })
end
def test_sum_with_non_string_property
input = [{ true => 1 }, { 1.0 => 0.2, 1 => -0.3 }, { 1..5 => 0.4 }]
assert_equal(1, @filters.sum(input, true))
assert_equal(0.2, @filters.sum(input, 1.0))
assert_equal(-0.3, @filters.sum(input, 1))
assert_equal(0.4, @filters.sum(input, 1..5))
assert_equal(0, @filters.sum(input, nil))
assert_equal(0, @filters.sum(input, ""))
end
def test_uniq_with_to_liquid_value
input = [StringDrop.new("foo"), StringDrop.new("bar"), "foo"]
expected = [StringDrop.new("foo"), StringDrop.new("bar")]
result = @filters.uniq(input)
assert_equal(expected, result)
end
def test_uniq_with_to_liquid_value_pick_correct_classes
input = ["foo", StringDrop.new("foo"), StringDrop.new("bar")]
expected = [String, StringDrop]
result = @filters.uniq(input).map(&:class)
assert_equal(expected, result)
end
private
def with_timezone(tz)
+182
View File
@@ -0,0 +1,182 @@
# frozen_string_literal: true
require 'test_helper'
class CycleTagTest < Minitest::Test
def test_simple_cycle_inside_for_loop
template = <<~LIQUID
{%- for i in (1..3) -%}
{%- cycle '1', '2', '3' -%}
{%- endfor -%}
LIQUID
assert_template_result("123", template)
end
def test_cycle_with_variables_inside_for_loop
template = <<~LIQUID
{%- assign a = 1 -%}
{%- assign b = 2 -%}
{%- assign c = 3 -%}
{%- for i in (1..3) -%}
{% cycle a, b, c %}
{%- endfor -%}
LIQUID
assert_template_result("123", template)
end
def test_cycle_named_groups_string
template = <<~LIQUID
{%- for i in (1..3) -%}
{%- cycle 'placeholder1': 1, 2, 3 -%}
{%- cycle 'placeholder2': 1, 2, 3 -%}
{%- endfor -%}
LIQUID
assert_template_result("112233", template)
end
def test_cycle_named_groups_vlookup
template = <<~LIQUID
{%- assign placeholder1 = 'placeholder1' -%}
{%- assign placeholder2 = 'placeholder2' -%}
{%- for i in (1..3) -%}
{%- cycle placeholder1: 1, 2, 3 -%}
{%- cycle placeholder2: 1, 2, 3 -%}
{%- endfor -%}
LIQUID
assert_template_result("112233", template)
end
def test_unnamed_cycle_have_independent_counters_when_used_with_lookups
template = <<~LIQUID
{%- assign a = "1" -%}
{%- for i in (1..3) -%}
{%- cycle a, "2" -%}
{%- cycle a, "2" -%}
{%- endfor -%}
LIQUID
assert_template_result("112211", template)
end
def test_unnamed_cycle_dependent_counter_when_used_with_literal_values
template = <<~LIQUID
{%- cycle "1", "2" -%}
{%- cycle "1", "2" -%}
{%- cycle "1", "2" -%}
LIQUID
assert_template_result("121", template)
end
def test_optional_trailing_comma
template = <<~LIQUID
{%- cycle "1", "2", -%}
{%- cycle "1", "2", -%}
{%- cycle "1", "2", -%}
{%- cycle "1", -%}
LIQUID
assert_template_result("1211", template)
end
def test_cycle_tag_without_arguments
error = assert_raises(Liquid::SyntaxError) do
Template.parse("{% cycle %}")
end
assert_match(/Syntax Error in 'cycle' - Valid syntax: cycle \[name :\] var/, error.message)
end
def test_cycle_tag_with_error_mode
# QuotedFragment is more permissive than what Parser#expression allows.
template1 = "{% assign 5 = 'b' %}{% cycle .5, .4 %}"
template2 = "{% cycle .5: 'a', 'b' %}"
with_error_modes(:lax, :strict) do
assert_template_result("b", template1)
assert_template_result("a", template2)
end
with_error_modes(:strict2) do
error1 = assert_raises(Liquid::SyntaxError) { Template.parse(template1) }
error2 = assert_raises(Liquid::SyntaxError) { Template.parse(template2) }
expected_error = /Liquid syntax error: \[:dot, "."\] is not a valid expression/
assert_match(expected_error, error1.message)
assert_match(expected_error, error2.message)
end
end
def test_cycle_with_trailing_elements
assignments = "{% assign a = 'A' %}{% assign n = 'N' %}"
template1 = "#{assignments}{% cycle 'a' 'b', 'c' %}"
template2 = "#{assignments}{% cycle name: 'a' 'b', 'c' %}"
template3 = "#{assignments}{% cycle name: 'a', 'b' 'c' %}"
template4 = "#{assignments}{% cycle n e: 'a', 'b', 'c' %}"
template5 = "#{assignments}{% cycle n e 'a', 'b', 'c' %}"
with_error_modes(:lax, :strict) do
assert_template_result("a", template1)
assert_template_result("a", template2)
assert_template_result("a", template3)
assert_template_result("N", template4)
assert_template_result("N", template5)
end
with_error_modes(:strict2) do
error1 = assert_raises(Liquid::SyntaxError) { Template.parse(template1) }
error2 = assert_raises(Liquid::SyntaxError) { Template.parse(template2) }
error3 = assert_raises(Liquid::SyntaxError) { Template.parse(template3) }
error4 = assert_raises(Liquid::SyntaxError) { Template.parse(template4) }
error5 = assert_raises(Liquid::SyntaxError) { Template.parse(template5) }
expected_error = /Expected end_of_string but found/
assert_match(expected_error, error1.message)
assert_match(expected_error, error2.message)
assert_match(expected_error, error3.message)
assert_match(expected_error, error4.message)
assert_match(expected_error, error5.message)
end
end
def test_cycle_name_with_invalid_expression
template = <<~LIQUID
{% for i in (1..3) %}
{% cycle foo=>bar: "a", "b" %}
{% endfor %}
LIQUID
with_error_modes(:lax, :strict) do
refute_nil(Template.parse(template))
end
with_error_modes(:strict2) do
error = assert_raises(Liquid::SyntaxError) { Template.parse(template) }
assert_match(/Unexpected character =/, error.message)
end
end
def test_cycle_variable_with_invalid_expression
template = <<~LIQUID
{% for i in (1..3) %}
{% cycle foo=>bar, "a", "b" %}
{% endfor %}
LIQUID
with_error_modes(:lax, :strict) do
refute_nil(Template.parse(template))
end
with_error_modes(:strict2) do
error = assert_raises(Liquid::SyntaxError) { Template.parse(template) }
assert_match(/Unexpected character =/, error.message)
end
end
end
+84 -22
View File
@@ -49,14 +49,6 @@ end
class IncludeTagTest < Minitest::Test
include Liquid
def setup
@default_file_system = Liquid::Template.file_system
end
def teardown
Liquid::Template.file_system = @default_file_system
end
def test_include_tag_looks_for_file_system_in_registers_first
assert_equal(
'from OtherFileSystem',
@@ -182,10 +174,10 @@ class IncludeTagTest < Minitest::Test
end
end
Liquid::Template.file_system = infinite_file_system.new
env = Liquid::Environment.build(file_system: infinite_file_system.new)
assert_raises(Liquid::StackLevelError) do
Template.parse("{% include 'loop' %}").render!
Template.parse("{% include 'loop' %}", environment: env).render!
end
end
@@ -212,11 +204,38 @@ class IncludeTagTest < Minitest::Test
)
end
def test_strict2_parsing_errors
with_error_modes(:lax, :strict) do
assert_template_result(
'hello value1 value2',
'{% include "snippet" !!! arg1: "value1" ~~~ arg2: "value2" %}',
partials: { 'snippet' => 'hello {{ arg1 }} {{ arg2 }}' },
)
end
with_error_modes(:strict2) do
assert_syntax_error(
'{% include "snippet" !!! arg1: "value1" ~~~ arg2: "value2" %}',
)
assert_syntax_error(
'{% include "snippet" | filter %}',
)
end
end
def test_optional_commas
partials = { 'snippet' => 'hello {{ arg1 }} {{ arg2 }}' }
assert_template_result('hello value1 value2', '{% include "snippet", arg1: "value1", arg2: "value2" %}', partials: partials)
assert_template_result('hello value1 value2', '{% include "snippet" arg1: "value1", arg2: "value2" %}', partials: partials)
assert_template_result('hello value1 value2', '{% include "snippet" arg1: "value1" arg2: "value2" %}', partials: partials)
end
def test_include_tag_caches_second_read_of_same_partial
file_system = CountingFileSystem.new
environment = Liquid::Environment.build(file_system: file_system)
assert_equal(
'from CountingFileSystemfrom CountingFileSystem',
Template.parse("{% include 'pick_a_source' %}{% include 'pick_a_source' %}").render!({}, registers: { file_system: file_system }),
Template.parse("{% include 'pick_a_source' %}{% include 'pick_a_source' %}", environment: environment).render!({}, registers: { file_system: file_system }),
)
assert_equal(1, file_system.count)
end
@@ -271,26 +290,27 @@ class IncludeTagTest < Minitest::Test
end
def test_does_not_add_error_in_strict_mode_for_missing_variable
Liquid::Template.file_system = TestFileSystem.new
env = Liquid::Environment.build(file_system: TestFileSystem.new)
a = Liquid::Template.parse(' {% include "nested_template" %}')
a = Liquid::Template.parse(' {% include "nested_template" %}', environment: env)
a.render!
assert_empty(a.errors)
end
def test_passing_options_to_included_templates
Liquid::Template.file_system = TestFileSystem.new
env = Liquid::Environment.build(file_system: TestFileSystem.new)
assert_raises(Liquid::SyntaxError) do
Template.parse("{% include template %}", error_mode: :strict).render!("template" => '{{ "X" || downcase }}')
Template.parse("{% include template %}", error_mode: :strict, environment: env).render!("template" => '{{ "X" || downcase }}')
end
with_error_mode(:lax) do
assert_equal('x', Template.parse("{% include template %}", error_mode: :strict, include_options_blacklist: true).render!("template" => '{{ "X" || downcase }}'))
with_error_modes(:lax) do
assert_equal('x', Template.parse("{% include template %}", error_mode: :strict, include_options_blacklist: true, environment: env).render!("template" => '{{ "X" || downcase }}'))
end
assert_raises(Liquid::SyntaxError) do
Template.parse("{% include template %}", error_mode: :strict, include_options_blacklist: [:locale]).render!("template" => '{{ "X" || downcase }}')
Template.parse("{% include template %}", error_mode: :strict, include_options_blacklist: [:locale], environment: env).render!("template" => '{{ "X" || downcase }}')
end
with_error_mode(:lax) do
assert_equal('x', Template.parse("{% include template %}", error_mode: :strict, include_options_blacklist: [:error_mode]).render!("template" => '{{ "X" || downcase }}'))
with_error_modes(:lax) do
assert_equal('x', Template.parse("{% include template %}", error_mode: :strict, include_options_blacklist: [:error_mode], environment: env).render!("template" => '{{ "X" || downcase }}'))
end
end
@@ -341,8 +361,11 @@ class IncludeTagTest < Minitest::Test
end
def test_including_with_strict_variables
Liquid::Template.file_system = StubFileSystem.new({ "simple" => "simple" })
template = Liquid::Template.parse("{% include 'simple' %}", error_mode: :warn)
env = Liquid::Environment.build(
file_system: StubFileSystem.new('simple' => 'simple'),
)
template = Liquid::Template.parse("{% include 'simple' %}", error_mode: :warn, environment: env)
template.render(nil, strict_variables: true)
assert_equal([], template.errors)
@@ -377,4 +400,43 @@ class IncludeTagTest < Minitest::Test
render_errors: true,
)
end
def test_include_template_with_invalid_expression
template = "{% include foo=>bar %}"
with_error_modes(:lax, :strict) do
refute_nil(Template.parse(template))
end
with_error_modes(:strict2) do
error = assert_raises(Liquid::SyntaxError) { Template.parse(template) }
assert_match(/Unexpected character =/, error.message)
end
end
def test_include_with_invalid_expression
template = '{% include "snippet" with foo=>bar %}'
with_error_modes(:lax, :strict) do
refute_nil(Template.parse(template))
end
with_error_modes(:strict2) do
error = assert_raises(Liquid::SyntaxError) { Template.parse(template) }
assert_match(/Unexpected character =/, error.message)
end
end
def test_include_attribute_with_invalid_expression
template = '{% include "snippet", key: foo=>bar %}'
with_error_modes(:lax, :strict) do
refute_nil(Template.parse(template))
end
with_error_modes(:strict2) do
error = assert_raises(Liquid::SyntaxError) { Template.parse(template) }
assert_match(/Unexpected character =/, error.message)
end
end
end # IncludeTagTest
+1
View File
@@ -16,6 +16,7 @@ class RawTagTest < Minitest::Test
assert_template_result('>{{ test }}<', '> {%- raw -%}{{ test }}{%- endraw -%} <')
assert_template_result("> inner <", "> {%- raw -%} inner {%- endraw %} <")
assert_template_result("> inner <", "> {%- raw -%} inner {%- endraw -%} <")
assert_template_result("{Hello}", "{% raw %}{{% endraw %}Hello{% raw %}}{% endraw %}")
end
def test_open_tag_in_raw
+61 -6
View File
@@ -82,19 +82,22 @@ class RenderTagTest < Minitest::Test
end
def test_recursively_rendered_template_does_not_produce_endless_loop
Liquid::Template.file_system = StubFileSystem.new('loop' => '{% render "loop" %}')
env = Liquid::Environment.build(
file_system: StubFileSystem.new('loop' => '{% render "loop" %}'),
)
assert_raises(Liquid::StackLevelError) do
Template.parse('{% render "loop" %}').render!
Template.parse('{% render "loop" %}', environment: env).render!
end
end
def test_sub_contexts_count_towards_the_same_recursion_limit
Liquid::Template.file_system = StubFileSystem.new(
'loop_render' => '{% render "loop_render" %}',
env = Liquid::Environment.build(
file_system: StubFileSystem.new('loop_render' => '{% render "loop_render" %}'),
)
assert_raises(Liquid::StackLevelError) do
Template.parse('{% render "loop_render" %}').render!
Template.parse('{% render "loop_render" %}', environment: env).render!
end
end
@@ -102,7 +105,33 @@ class RenderTagTest < Minitest::Test
assert_syntax_error("{% assign name = 'snippet' %}{% render name %}")
end
def test_include_tag_caches_second_read_of_same_partial
def test_strict2_parsing_errors
with_error_modes(:lax, :strict) do
assert_template_result(
'hello value1 value2',
'{% render "snippet" !!! arg1: "value1" ~~~ arg2: "value2" %}',
partials: { 'snippet' => 'hello {{ arg1 }} {{ arg2 }}' },
)
end
with_error_modes(:strict2) do
assert_syntax_error(
'{% render "snippet" !!! arg1: "value1" ~~~ arg2: "value2" %}',
)
assert_syntax_error(
'{% render "snippet" | filter %}',
)
end
end
def test_optional_commas
partials = { 'snippet' => 'hello {{ arg1 }} {{ arg2 }}' }
assert_template_result('hello value1 value2', '{% render "snippet", arg1: "value1", arg2: "value2" %}', partials: partials)
assert_template_result('hello value1 value2', '{% render "snippet" arg1: "value1", arg2: "value2" %}', partials: partials)
assert_template_result('hello value1 value2', '{% render "snippet" arg1: "value1" arg2: "value2" %}', partials: partials)
end
def test_render_tag_caches_second_read_of_same_partial
file_system = StubFileSystem.new('snippet' => 'echo')
assert_equal(
'echoecho',
@@ -285,4 +314,30 @@ class RenderTagTest < Minitest::Test
render_errors: true,
)
end
def test_render_with_invalid_expression
template = '{% render "snippet" with foo=>bar %}'
with_error_modes(:lax, :strict) do
refute_nil(Template.parse(template))
end
with_error_modes(:strict2) do
error = assert_raises(Liquid::SyntaxError) { Template.parse(template) }
assert_match(/Unexpected character =/, error.message)
end
end
def test_render_attribute_with_invalid_expression
template = '{% render "snippet", key: foo=>bar %}'
with_error_modes(:lax, :strict) do
refute_nil(Template.parse(template))
end
with_error_modes(:strict2) do
error = assert_raises(Liquid::SyntaxError) { Template.parse(template) }
assert_match(/Unexpected character =/, error.message)
end
end
end
+1 -1
View File
@@ -117,7 +117,7 @@ class StandardTagTest < Minitest::Test
assigns = { 'condition' => "bad string here" }
assert_template_result(
'',
'{% case condition %}{% when "string here" %} hit {% endcase %}',\
'{% case condition %}{% when "string here" %} hit {% endcase %}',
assigns,
)
end
+259 -1
View File
@@ -138,7 +138,7 @@ class TableRowTest < Minitest::Test
def test_tablerow_loop_drop_attributes
template = <<~LIQUID.chomp
{% tablerow i in (1...2) %}
{% tablerow i in (1..2) %}
col: {{ tablerowloop.col }}
col0: {{ tablerowloop.col0 }}
col_first: {{ tablerowloop.col_first }}
@@ -192,12 +192,14 @@ class TableRowTest < Minitest::Test
assert_template_result(
"Liquid error (line 1): invalid integer",
'{% tablerow n in (1...10) limit:true %} {{n}} {% endtablerow %}',
error_mode: :warn,
render_errors: true,
)
assert_template_result(
"Liquid error (line 1): invalid integer",
'{% tablerow n in (1...10) offset:true %} {{n}} {% endtablerow %}',
error_mode: :warn,
render_errors: true,
)
@@ -205,6 +207,262 @@ class TableRowTest < Minitest::Test
"Liquid error (line 1): invalid integer",
'{% tablerow n in (1...10) cols:true %} {{n}} {% endtablerow %}',
render_errors: true,
error_mode: :warn,
)
end
def test_table_row_handles_interrupts
assert_template_result(
"<tr class=\"row1\">\n<td class=\"col1\"> 1 </td></tr>\n",
'{% tablerow n in (1..3) cols:2 %} {{n}} {% break %} {{n}} {% endtablerow %}',
)
assert_template_result(
"<tr class=\"row1\">\n<td class=\"col1\"> 1 </td><td class=\"col2\"> 2 </td></tr>\n<tr class=\"row2\"><td class=\"col1\"> 3 </td></tr>\n",
'{% tablerow n in (1..3) cols:2 %} {{n}} {% continue %} {{n}} {% endtablerow %}',
)
end
def test_table_row_does_not_leak_interrupts
template = <<~LIQUID
{% for i in (1..2) -%}
{% for j in (1..2) -%}
{% tablerow k in (1..3) %}{% break %}{% endtablerow -%}
loop j={{ j }}
{% endfor -%}
loop i={{ i }}
{% endfor -%}
after loop
LIQUID
expected = <<~STR
<tr class="row1">
<td class="col1"></td></tr>
loop j=1
<tr class="row1">
<td class="col1"></td></tr>
loop j=2
loop i=1
<tr class="row1">
<td class="col1"></td></tr>
loop j=1
<tr class="row1">
<td class="col1"></td></tr>
loop j=2
loop i=2
after loop
STR
assert_template_result(
expected,
template,
)
end
def test_tablerow_with_cols_attribute_in_strict2_mode
template = <<~LIQUID.chomp
{% tablerow i in (1..6) cols: 3 %}{{ i }}{% endtablerow %}
LIQUID
expected = <<~OUTPUT
<tr class="row1">
<td class="col1">1</td><td class="col2">2</td><td class="col3">3</td></tr>
<tr class="row2"><td class="col1">4</td><td class="col2">5</td><td class="col3">6</td></tr>
OUTPUT
with_error_modes(:strict2) do
assert_template_result(expected, template)
end
end
def test_tablerow_with_limit_attribute_in_strict2_mode
template = <<~LIQUID.chomp
{% tablerow i in (1..10) limit: 3 %}{{ i }}{% endtablerow %}
LIQUID
expected = <<~OUTPUT
<tr class="row1">
<td class="col1">1</td><td class="col2">2</td><td class="col3">3</td></tr>
OUTPUT
with_error_modes(:strict2) do
assert_template_result(expected, template)
end
end
def test_tablerow_with_offset_attribute_in_strict2_mode
template = <<~LIQUID.chomp
{% tablerow i in (1..5) offset: 2 %}{{ i }}{% endtablerow %}
LIQUID
expected = <<~OUTPUT
<tr class="row1">
<td class="col1">3</td><td class="col2">4</td><td class="col3">5</td></tr>
OUTPUT
with_error_modes(:strict2) do
assert_template_result(expected, template)
end
end
def test_tablerow_with_range_attribute_in_strict2_mode
template = <<~LIQUID.chomp
{% tablerow i in (1..3) range: (1..10) %}{{ i }}{% endtablerow %}
LIQUID
expected = <<~OUTPUT
<tr class="row1">
<td class="col1">1</td><td class="col2">2</td><td class="col3">3</td></tr>
OUTPUT
with_error_modes(:strict2) do
assert_template_result(expected, template)
end
end
def test_tablerow_with_multiple_attributes_in_strict2_mode
template = <<~LIQUID.chomp
{% tablerow i in (1..10) cols: 2, limit: 4, offset: 1 %}{{ i }}{% endtablerow %}
LIQUID
expected = <<~OUTPUT
<tr class="row1">
<td class="col1">2</td><td class="col2">3</td></tr>
<tr class="row2"><td class="col1">4</td><td class="col2">5</td></tr>
OUTPUT
with_error_modes(:strict2) do
assert_template_result(expected, template)
end
end
def test_tablerow_with_variable_collection_in_strict2_mode
template = <<~LIQUID.chomp
{% tablerow n in numbers cols: 2 %}{{ n }}{% endtablerow %}
LIQUID
expected = <<~OUTPUT
<tr class="row1">
<td class="col1">1</td><td class="col2">2</td></tr>
<tr class="row2"><td class="col1">3</td><td class="col2">4</td></tr>
OUTPUT
with_error_modes(:strict2) do
assert_template_result(expected, template, { 'numbers' => [1, 2, 3, 4] })
end
end
def test_tablerow_with_dotted_access_in_strict2_mode
template = <<~LIQUID.chomp
{% tablerow n in obj.numbers cols: 2 %}{{ n }}{% endtablerow %}
LIQUID
expected = <<~OUTPUT
<tr class="row1">
<td class="col1">1</td><td class="col2">2</td></tr>
<tr class="row2"><td class="col1">3</td><td class="col2">4</td></tr>
OUTPUT
with_error_modes(:strict2) do
assert_template_result(expected, template, { 'obj' => { 'numbers' => [1, 2, 3, 4] } })
end
end
def test_tablerow_with_bracketed_access_in_strict2_mode
template = <<~LIQUID.chomp
{% tablerow n in obj["numbers"] cols: 2 %}{{ n }}{% endtablerow %}
LIQUID
expected = <<~OUTPUT
<tr class="row1">
<td class="col1">10</td><td class="col2">20</td></tr>
OUTPUT
with_error_modes(:strict2) do
assert_template_result(expected, template, { 'obj' => { 'numbers' => [10, 20] } })
end
end
def test_tablerow_without_attributes_in_strict2_mode
template = <<~LIQUID.chomp
{% tablerow i in (1..3) %}{{ i }}{% endtablerow %}
LIQUID
expected = <<~OUTPUT
<tr class="row1">
<td class="col1">1</td><td class="col2">2</td><td class="col3">3</td></tr>
OUTPUT
with_error_modes(:strict2) do
assert_template_result(expected, template)
end
end
def test_tablerow_without_in_keyword_in_strict2_mode
template = '{% tablerow i (1..10) %}{{ i }}{% endtablerow %}'
with_error_modes(:strict2) do
error = assert_raises(SyntaxError) { Template.parse(template) }
assert_equal("Liquid syntax error: For loops require an 'in' clause in \"i (1..10)\"", error.message)
end
end
def test_tablerow_with_multiple_invalid_attributes_reports_first_in_strict2_mode
template = '{% tablerow i in (1..10) invalid1: 5, invalid2: 10 %}{{ i }}{% endtablerow %}'
with_error_modes(:strict2) do
error = assert_raises(SyntaxError) { Template.parse(template) }
assert_equal("Liquid syntax error: Invalid attribute 'invalid1' in tablerow loop. Valid attributes are cols, limit, offset, and range in \"i in (1..10) invalid1: 5, invalid2: 10\"", error.message)
end
end
def test_tablerow_with_empty_collection_in_strict2_mode
template = <<~LIQUID.chomp
{% tablerow i in empty_array cols: 2 %}{{ i }}{% endtablerow %}
LIQUID
expected = <<~OUTPUT
<tr class="row1">
</tr>
OUTPUT
with_error_modes(:strict2) do
assert_template_result(expected, template, { 'empty_array' => [] })
end
end
def test_tablerow_with_invalid_attribute_strict_vs_strict2
template = '{% tablerow i in (1..5) invalid_attr: 10 %}{{ i }}{% endtablerow %}'
expected = <<~OUTPUT
<tr class="row1">
<td class="col1">1</td><td class="col2">2</td><td class="col3">3</td><td class="col4">4</td><td class="col5">5</td></tr>
OUTPUT
with_error_modes(:lax, :strict) do
assert_template_result(expected, template)
end
with_error_modes(:strict2) do
error = assert_raises(SyntaxError) { Template.parse(template) }
assert_match(/Invalid attribute 'invalid_attr'/, error.message)
end
end
def test_tablerow_with_invalid_expression_strict_vs_strict2
template = '{% tablerow i in (1..5) limit: foo=>bar %}{{ i }}{% endtablerow %}'
with_error_modes(:lax, :strict) do
expected = <<~OUTPUT
<tr class="row1">
</tr>
OUTPUT
assert_template_result(expected, template)
end
with_error_modes(:strict2) do
error = assert_raises(SyntaxError) { Template.parse(template) }
assert_match(/Unexpected character =/, error.message)
end
end
end
+99 -1
View File
@@ -133,7 +133,7 @@ class TemplateTest < Minitest::Test
assert(t.resource_limits.reached?)
t.resource_limits.render_score_limit = 200
assert_equal((" foo " * 100), t.render!)
assert_equal(" foo " * 100, t.render!)
refute_nil(t.resource_limits.render_score)
end
@@ -179,6 +179,86 @@ class TemplateTest < Minitest::Test
assert_equal("すごい", t.render)
end
def test_cumulative_render_score_limit_across_render_tags
file_system = StubFileSystem.new(
'loop' => '{% for a in (1..10) %} foo {% endfor %}',
)
environment = Liquid::Environment.build(file_system: file_system)
# Without cumulative limit, all 5 partials render successfully
t = Template.parse(
'{% render "loop" %}{% render "loop" %}{% render "loop" %}{% render "loop" %}{% render "loop" %}',
environment: environment,
)
unlimited_output = t.render!
total_cumulative = t.resource_limits.cumulative_render_score
# With cumulative limit set below the total, rendering stops early
t2 = Template.parse(
'{% render "loop" %}{% render "loop" %}{% render "loop" %}{% render "loop" %}{% render "loop" %}',
environment: environment,
)
t2.resource_limits.cumulative_render_score_limit = total_cumulative / 2
limited_output = t2.render
assert(t2.resource_limits.reached?)
assert_operator(limited_output.length, :<, unlimited_output.length)
end
def test_cumulative_render_score_limit_raises_on_render_bang
file_system = StubFileSystem.new(
'loop' => '{% for a in (1..10) %} foo {% endfor %}',
)
environment = Liquid::Environment.build(file_system: file_system)
t = Template.parse(
'{% render "loop" %}{% render "loop" %}{% render "loop" %}{% render "loop" %}{% render "loop" %}',
environment: environment,
)
t.resource_limits.cumulative_render_score_limit = 20
assert_raises(Liquid::MemoryError) do
t.render!
end
end
def test_cumulative_assign_score_limit_across_include_tags
file_system = StubFileSystem.new(
'assign_partial' => '{% assign x = "a long string value here" %}',
)
environment = Liquid::Environment.build(file_system: file_system)
# Without cumulative limit, all 5 partials render
t = Template.parse(
'{% include "assign_partial" %}{% include "assign_partial" %}{% include "assign_partial" %}{% include "assign_partial" %}{% include "assign_partial" %}',
environment: environment,
)
t.render!
total_cumulative = t.resource_limits.cumulative_assign_score
# With cumulative limit set below the total, rendering stops early
t2 = Template.parse(
'{% include "assign_partial" %}{% include "assign_partial" %}{% include "assign_partial" %}{% include "assign_partial" %}{% include "assign_partial" %}',
environment: environment,
)
t2.resource_limits.cumulative_assign_score_limit = total_cumulative / 2
t2.render
assert(t2.resource_limits.reached?)
end
def test_cumulative_render_score_tracks_across_partials_without_limit
file_system = StubFileSystem.new(
'loop' => '{% for a in (1..10) %} foo {% endfor %}',
)
environment = Liquid::Environment.build(file_system: file_system)
t = Template.parse(
'{% render "loop" %}{% render "loop" %}{% render "loop" %}',
environment: environment,
)
t.render!
assert(
t.resource_limits.cumulative_render_score > t.resource_limits.render_score,
"cumulative should exceed per-template score after multiple partials",
)
end
def test_default_resource_limits_unaffected_by_render_with_context
context = Context.new
t = Template.parse("{% for a in (1..100) %}x{% assign foo = 1 %} {% endfor %}")
@@ -337,4 +417,22 @@ class TemplateTest < Minitest::Test
assert_equal("x=2", output)
assert_instance_of(String, output)
end
def test_raises_error_with_invalid_utf8
e = assert_raises(TemplateEncodingError) do
Template.parse(<<~LIQUID)
{% comment %}
\xC0
{% endcomment %}
LIQUID
end
assert_equal('Liquid error: Invalid template encoding', e.message)
end
def test_allows_non_string_values_as_source
assert_equal('', Template.parse(nil).render)
assert_equal('1', Template.parse(1).render)
assert_equal('true', Template.parse(true).render)
end
end
+69
View File
@@ -130,6 +130,10 @@ class VariableTest < Minitest::Test
assert_template_result('bar', '{{ foo }}', { 'foo' => :bar })
end
def test_nested_array
assert_template_result('', '{{ foo }}', { 'foo' => [[nil]] })
end
def test_dynamic_find_var
assert_template_result('bar', '{{ [key] }}', { 'key' => 'foo', 'foo' => 'bar' })
end
@@ -205,4 +209,69 @@ class VariableTest < Minitest::Test
end
end
end
def test_filter_with_single_trailing_comma
template = '{{ "hello" | append: "world", }}'
with_error_modes(:strict) do
error = assert_raises(Liquid::SyntaxError) { Template.parse(template) }
assert_match(/is not a valid expression/, error.message)
end
with_error_modes(:strict2) do
assert_template_result('helloworld', template)
end
end
def test_multiple_filters_with_trailing_commas
template = '{{ "hello" | append: "1", | append: "2", }}'
with_error_modes(:strict) do
error = assert_raises(Liquid::SyntaxError) { Template.parse(template) }
assert_match(/is not a valid expression/, error.message)
end
with_error_modes(:strict2) do
assert_template_result('hello12', template)
end
end
def test_filter_with_colon_but_no_arguments
template = '{{ "test" | upcase: }}'
with_error_modes(:strict) do
error = assert_raises(Liquid::SyntaxError) { Template.parse(template) }
assert_match(/is not a valid expression/, error.message)
end
with_error_modes(:strict2) do
assert_template_result('TEST', template)
end
end
def test_filter_chain_with_colon_no_args
template = '{{ "test" | append: "x" | upcase: }}'
with_error_modes(:strict) do
error = assert_raises(Liquid::SyntaxError) { Template.parse(template) }
assert_match(/is not a valid expression/, error.message)
end
with_error_modes(:strict2) do
assert_template_result('TESTX', template)
end
end
def test_combining_trailing_comma_and_empty_args
template = '{{ "test" | append: "x", | upcase: }}'
with_error_modes(:strict) do
error = assert_raises(Liquid::SyntaxError) { Template.parse(template) }
assert_match(/is not a valid expression/, error.message)
end
with_error_modes(:strict2) do
assert_template_result('TESTX', template)
end
end
end
+71 -41
View File
@@ -13,12 +13,7 @@ if (env_mode = ENV['LIQUID_PARSER_MODE'])
puts "-- #{env_mode.upcase} ERROR MODE"
mode = env_mode.to_sym
end
Liquid::Template.error_mode = mode
if ENV['LIQUID_C'] == '1'
puts "-- LIQUID C"
require 'liquid/c'
end
Liquid::Environment.default.error_mode = mode
if Minitest.const_defined?('Test')
# We're on Minitest 5+. Nothing to do here.
@@ -39,13 +34,14 @@ module Minitest
def assert_template_result(
expected, template, assigns = {},
message: nil, partials: nil, error_mode: nil, render_errors: false,
message: nil, partials: nil, error_mode: Liquid::Environment.default.error_mode, render_errors: false,
template_factory: nil
)
template = Liquid::Template.parse(template, line_numbers: true, error_mode: error_mode&.to_sym)
file_system = StubFileSystem.new(partials || {})
environment = Liquid::Environment.build(file_system: file_system)
template = Liquid::Template.parse(template, line_numbers: true, error_mode: error_mode&.to_sym, environment: environment)
registers = Liquid::Registers.new(file_system: file_system, template_factory: template_factory)
context = Liquid::Context.build(static_environments: assigns, rethrow_errors: !render_errors, registers: registers)
context = Liquid::Context.build(static_environments: assigns, rethrow_errors: !render_errors, registers: registers, environment: environment)
output = template.render(context)
assert_equal(expected, output, message)
end
@@ -78,44 +74,29 @@ module Minitest
assert_equal(times, calls, "Number of calls to Usage.increment with #{name.inspect}")
end
def with_global_filter(*globals)
original_global_cache = Liquid::StrainerFactory::GlobalCache
Liquid::StrainerFactory.send(:remove_const, :GlobalCache)
Liquid::StrainerFactory.const_set(:GlobalCache, Class.new(Liquid::StrainerTemplate))
def with_global_filter(*globals, &blk)
environment = Liquid::Environment.build do |w|
w.register_filters(globals)
end
globals.each do |global|
Liquid::Template.register_filter(global)
end
Liquid::StrainerFactory.send(:strainer_class_cache).clear
begin
yield
ensure
Liquid::StrainerFactory.send(:remove_const, :GlobalCache)
Liquid::StrainerFactory.const_set(:GlobalCache, original_global_cache)
Liquid::StrainerFactory.send(:strainer_class_cache).clear
end
Environment.dangerously_override(environment, &blk)
end
def with_error_mode(mode)
old_mode = Liquid::Template.error_mode
Liquid::Template.error_mode = mode
yield
def with_error_modes(*modes)
old_mode = Liquid::Environment.default.error_mode
modes.each do |mode|
Liquid::Environment.default.error_mode = mode
yield
end
ensure
Liquid::Template.error_mode = old_mode
Liquid::Environment.default.error_mode = old_mode
end
def with_custom_tag(tag_name, tag_class)
old_tag = Liquid::Template.tags[tag_name]
begin
Liquid::Template.register_tag(tag_name, tag_class)
yield
ensure
if old_tag
Liquid::Template.tags[tag_name] = old_tag
else
Liquid::Template.tags.delete(tag_name)
end
end
def with_custom_tag(tag_name, tag_class, &block)
environment = Liquid::Environment.default.dup
environment.register_tag(tag_name, tag_class)
Environment.dangerously_override(environment, &block)
end
end
end
@@ -167,6 +148,35 @@ class BooleanDrop < Liquid::Drop
end
end
class StringDrop < Liquid::Drop
include Comparable
def initialize(value)
super()
@value = value
end
def to_liquid_value
@value
end
def to_s
@value
end
def to_str
@value
end
def inspect
"#<StringDrop @value=#{@value.inspect}>"
end
def <=>(other)
to_liquid_value <=> Liquid::Utils.to_liquid_value(other)
end
end
class ErrorDrop < Liquid::Drop
def standard_error
raise Liquid::StandardError, 'standard error'
@@ -189,6 +199,26 @@ class ErrorDrop < Liquid::Drop
end
end
class CustomToLiquidDrop < Liquid::Drop
def initialize(value)
@value = value
super()
end
def to_liquid
@value
end
end
class HashWithCustomToS < Hash
def to_s
"kewl"
end
end
class HashWithoutCustomToS < Hash
end
class StubFileSystem
attr_reader :file_read_count
+13 -1
View File
@@ -32,6 +32,12 @@ class BlockUnitTest < Minitest::Test
assert_equal(String, template.root.nodelist[2].class)
end
def test_variable_with_multibyte_character
template = Liquid::Template.parse("{{ '❤️' }}")
assert_equal(1, template.root.nodelist.size)
assert_equal(Variable, template.root.nodelist[0].class)
end
def test_variable_many_embedded_fragments
template = Liquid::Template.parse(" {{funk}} {{so}} {{brother}} ")
assert_equal(7, template.root.nodelist.size)
@@ -41,12 +47,18 @@ class BlockUnitTest < Minitest::Test
)
end
def test_with_block
def test_comment_tag_with_block
template = Liquid::Template.parse(" {% comment %} {% endcomment %} ")
assert_equal([String, Comment, String], block_types(template.root.nodelist))
assert_equal(3, template.root.nodelist.size)
end
def test_doc_tag_with_block
template = Liquid::Template.parse(" {% doc %} {% enddoc %} ")
assert_equal([String, Doc, String], block_types(template.root.nodelist))
assert_equal(3, template.root.nodelist.size)
end
private
def block_types(nodelist)

Some files were not shown because too many files have changed in this diff Show More