Compare commits

..
Author SHA1 Message Date
Tobi Lutke 3182b7c1b3 update autoresearch experiment log 2026-03-12 17:38:05 -04:00
Tobi Lutke b195d09212 Replace @blocks.each with while loop in If render — avoids block proc allocation per render\n\nResult: {"status":"keep","combined_µs":3496,"parse_µs":2356,"render_µs":1140,"allocations":24530} 2026-03-12 17:31:41 -04:00
Tobi Lutke db348e0dac Inline to_liquid_value in If render — avoids one method dispatch per condition evaluation\n\nResult: {"status":"keep","combined_µs":3459,"parse_µs":2318,"render_µs":1141,"allocations":24647} 2026-03-12 17:31:20 -04:00
Tobi Lutke 99454a9be2 Replace simple_lookup? byte scan with match? regex — 8x faster per call, cleaner code\n\nResult: {"status":"keep","combined_µs":3489,"parse_µs":2353,"render_µs":1136,"allocations":24647} 2026-03-12 17:27:17 -04:00
Tobi Lutke ca327b01b1 Condition#evaluate: skip loop block for simple conditions (no child_relation) — saves 235 allocs\n\nResult: {"status":"keep","combined_µs":3445,"parse_µs":2284,"render_µs":1161,"allocations":24647} 2026-03-12 17:24:51 -04:00
Tobi Lutke ae9a2e26b0 Clean confirmation run: 3,314µs (-55% from main), stable\n\nResult: {"status":"keep","combined_µs":3314,"parse_µs":2203,"render_µs":1111,"allocations":24882} 2026-03-12 17:22:24 -04:00
Tobi Lutke 46927b9e90 update autoresearch docs with current progress 2026-03-12 17:21:28 -04:00
Tobi Lutke f6baeaed1e parse_tag_token without StringScanner: pure byte ops avoid reset(token) overhead, -12% combined\n\nResult: {"status":"keep","combined_µs":3350,"parse_µs":2212,"render_µs":1138,"allocations":24882} 2026-03-12 17:17:10 -04:00
Tobi Lutke b37fa98c91 Clean up tokenizer: remove unused StringScanner setup and regex constants\n\nResult: {"status":"keep","combined_µs":3490,"parse_µs":2331,"render_µs":1159,"allocations":24882} 2026-03-12 17:12:08 -04:00
Tobi Lutke e25f2f1d52 Confirmation run: byteindex tokenizer consistently 3,400-3,600µs\n\nResult: {"status":"keep","combined_µs":3464,"parse_µs":2335,"render_µs":1129,"allocations":24882} 2026-03-12 17:11:29 -04:00
Tobi Lutke b7ae55f7a9 Replace StringScanner tokenizer with String#byteindex — 12% faster parse, no regex overhead for delimiter finding\n\nResult: {"status":"keep","combined_µs":3556,"parse_µs":2388,"render_µs":1168,"allocations":24882} 2026-03-12 17:11:15 -04:00
Tobi Lutke c09e722f9b Baseline: 3,818µs combined, 24,881 allocs\n\nResult: {"status":"keep","combined_µs":3818,"parse_µs":2722,"render_µs":1096,"allocations":24881} 2026-03-12 16:48:10 -04:00
Tobi Lutke 8f2f0ee035 Skip context.evaluate for String lookup keys in VariableLookup — avoids respond_to? dispatch\n\nResult: {"status":"keep","combined_µs":4103,"parse_µs":2881,"render_µs":1222,"allocations":24881} 2026-03-11 10:52:07 -04:00
Tobi Lutke 38d8055c3b update autoresearch.md with current progress 2026-03-11 10:51:39 -04:00
Tobi Lutke 228ecdb6a2 Cache no-arg filter tuples [name, EMPTY_ARRAY] — reuse frozen tuples across templates\n\nResult: {"status":"keep","combined_µs":4147,"parse_µs":2992,"render_µs":1155,"allocations":24881} 2026-03-11 10:49:44 -04:00
Tobi Lutke 76afdf154f Replace manual blank_string? with regex match — cleaner code\n\nResult: {"status":"keep","combined_µs":4196,"parse_µs":3042,"render_µs":1154,"allocations":25535} 2026-03-11 10:44:43 -04:00
Tobi Lutke 22b5ff1587 Skip to_liquid_value for String/Integer keys in VariableLookup — avoids respond_to? dispatch\n\nResult: {"status":"keep","combined_µs":4131,"parse_µs":2893,"render_µs":1238,"allocations":25535} 2026-03-11 10:44:05 -04:00
Tobi Lutke 1a019151eb Minor cleanup: optimize expect_id with while loop and early return\n\nResult: {"status":"keep","combined_µs":4184,"parse_µs":2921,"render_µs":1263,"allocations":25535} 2026-03-11 10:43:20 -04:00
Tobi Lutke 71e22e6a84 Replace manual scan_dotted_id with regex\n\nResult: {"status":"keep","combined_µs":4121,"parse_µs":2812,"render_µs":1309,"allocations":25535} 2026-03-11 10:42:07 -04:00
Tobi Lutke fd4a7af290 Replace manual scan_quoted_string with regex capture groups\n\nResult: {"status":"keep","combined_µs":4102,"parse_µs":2849,"render_µs":1253,"allocations":25535} 2026-03-11 10:41:44 -04:00
Tobi Lutke e15b163f1a Replace manual rest_blank? with regex skip + eos? check\n\nResult: {"status":"keep","combined_µs":4047,"parse_µs":2795,"render_µs":1252,"allocations":25535} 2026-03-11 10:41:20 -04:00
Tobi Lutke 11c22eb75d Replace manual scan_comparison_op with regex — cleaner and avoids byteslice allocation for op strings\n\nResult: {"status":"keep","combined_µs":4007,"parse_µs":2808,"render_µs":1199,"allocations":25535} 2026-03-11 10:40:47 -04:00
Tobi Lutke f8b08b5b64 Replace manual scan_fragment/scan_quoted_string_raw/skip_fragment with regex — cleaner, same/better perf\n\nResult: {"status":"keep","combined_µs":4132,"parse_µs":2890,"render_µs":1242,"allocations":25535} 2026-03-11 10:40:10 -04:00
Tobi Lutke 6db20e908b Replace manual byte-level scan_number with regex — cleaner code, same performance\n\nResult: {"status":"keep","combined_µs":4184,"parse_µs":2931,"render_µs":1253,"allocations":25535} 2026-03-11 10:39:14 -04:00
Tobi Lutke ecc23184ce Replace manual byte-level scan_id/skip_id with regex — C-level StringScanner.scan is faster than Ruby-level byte scanning\n\nResult: {"status":"keep","combined_µs":4185,"parse_µs":2943,"render_µs":1242,"allocations":25535} 2026-03-11 10:37:09 -04:00
Tobi Lutke 4df608a12f Fast-path Hash lookups in VariableLookup#evaluate — skip respond_to? checks for Hash objects\n\nResult: {"status":"keep","combined_µs":4110,"parse_µs":2922,"render_µs":1188,"allocations":25535} 2026-03-11 10:35:29 -04:00
Tobi Lutke b058f79cbf Skip to_liquid/context= for primitives in VariableLookup#evaluate\n\nResult: {"status":"keep","combined_µs":4334,"parse_µs":3062,"render_µs":1272,"allocations":25535} 2026-03-11 10:32:37 -04:00
Tobi Lutke 94562eae32 Fast return for primitive types in find_variable — skip to_liquid and respond_to?(:context=)\n\nResult: {"status":"keep","combined_µs":4225,"parse_µs":3009,"render_µs":1216,"allocations":25535} 2026-03-11 10:31:59 -04:00
Tobi Lutke 0e84955531 Skip find_index when only one scope in find_variable — go straight to environments\n\nResult: {"status":"keep","combined_µs":4323,"parse_µs":3055,"render_µs":1268,"allocations":25535} 2026-03-11 10:31:35 -04:00
Tobi Lutke c4593ceeb6 Skip respond_to?(:context=) for primitive types in find_variable — avoids method lookup overhead\n\nResult: {"status":"keep","combined_µs":4207,"parse_µs":2943,"render_µs":1264,"allocations":25535} 2026-03-11 10:30:41 -04:00
Tobi Lutke 9e2937945b Use EMPTY_ARRAY for empty static_environments in Context — avoids 60 array allocs per render cycle\n\nResult: {"status":"keep","combined_µs":4262,"parse_µs":3079,"render_µs":1183,"allocations":25535} 2026-03-11 10:26:30 -04:00
Tobi Lutke cd308b8b01 Lazy @changes hash in Registers — only allocate when a register is actually written\n\nResult: {"status":"keep","combined_µs":4287,"parse_µs":3059,"render_µs":1228,"allocations":25595} 2026-03-11 10:25:29 -04:00
Tobi Lutke e3fc735de7 Cache block_delimiter strings per tag name — avoids repeated string interpolation\n\nResult: {"status":"keep","combined_µs":4372,"parse_µs":3127,"render_µs":1245,"allocations":25605} 2026-03-11 10:23:56 -04:00
Tobi Lutke 9af3ba3aa7 Lazy Context init: defer StringScanner and @interrupts array allocation until needed\n\nResult: {"status":"keep","combined_µs":4299,"parse_µs":3057,"render_µs":1242,"allocations":26015} 2026-03-11 10:22:29 -04:00
Tobi Lutke 99e55c2eb5 Cache small integer to_s (0-999): avoids 267 Integer#to_s allocations per render cycle\n\nResult: {"status":"keep","combined_µs":4158,"parse_µs":2920,"render_µs":1238,"allocations":26128} 2026-03-11 10:20:45 -04:00
Tobi Lutke b48615f4a7 Replace split+join in truncatewords with manual word scan — avoids array + string allocations\n\nResult: {"status":"keep","combined_µs":4280,"parse_µs":3009,"render_µs":1271,"allocations":26395} 2026-03-11 10:20:11 -04:00
Tobi Lutke 6723d4fa15 Extend fast-path filter parsing to handle comma-separated multi-arg filters (e.g. pluralize: 'item', 'items')\n\nResult: {"status":"keep","combined_µs":4266,"parse_µs":3032,"render_µs":1234,"allocations":26480} 2026-03-11 10:16:39 -04:00
Tobi Lutke c252d50aa0 Avoid expr_markup byteslice when name is entire markup string (no whitespace, no filters)\n\nResult: {"status":"keep","combined_µs":4277,"parse_µs":3057,"render_µs":1220,"allocations":27026} 2026-03-11 10:12:26 -04:00
Tobi Lutke a249010cef Fast-path single-arg filter parsing: handle quoted strings, numbers, identifiers without Lexer/Parser\n\nResult: {"status":"keep","combined_µs":4427,"parse_µs":3181,"render_µs":1246,"allocations":27235} 2026-03-11 10:09:59 -04:00
Tobi Lutke 18a72db820 fix rubocop offenses: autocorrect style/layout violations 2026-03-11 09:53:44 -04:00
Tobi Lutke 1f59732aee update autoresearch.md with full progress log 2026-03-11 09:24:26 -04:00
Tobi Lutke cdc34388e3 For tag: migrate lax_parse to Cursor with zero-alloc skip_id/expect_id 2026-03-11 09:19:59 -04:00
Tobi Lutke bf1f5cb62d Cursor: add skip_id, expect_id, skip_fragment for zero-alloc scanning 2026-03-11 09:19:27 -04:00
Tobi Lutke 0596591fdf REVERTED: Cursor for For tag adds 148 allocs from scan_id/scan_fragment string creation\n\nResult: {"status":"discard","combined_µs":5049,"parse_us":3765,"render_us":1284,"allocations":29793,"parse_µs":3765,"render_µs":1284} 2026-03-11 09:18:35 -04:00
Tobi Lutke dd4a100346 remove dead BlockBody.parse_tag_token and If SIMPLE_CONDITION - now in Cursor 2026-03-11 09:17:29 -04:00
Tobi Lutke 9de1527099 introduce Cursor class: centralize byte-level scanning for tag/variable/condition parsing 2026-03-11 09:15:21 -04:00
Tobi Lutke 091534f981 add parse_simple to skip simple_lookup? check when caller validates 2026-03-11 09:10:05 -04:00
Tobi Lutke 0b07487e0c fast-path VariableLookup init: skip scan_variable for simple identifier chains 2026-03-11 09:08:44 -04:00
Tobi Lutke 3799d4c488 avoid allocating seen={} hash in Utils.to_s/inspect when not needed 2026-03-11 09:02:42 -04:00
Tobi Lutke c4186a16fc update autoresearch.sh: 3-run best-of, skip liquid-spec for speed 2026-03-11 09:01:57 -04:00
Tobi Lutke b90d7f0a08 optimize Context init: avoid unnecessary array wrapping for environments 2026-03-11 08:56:45 -04:00
Tobi Lutke 405e3dca48 use frozen EMPTY_ARRAY/EMPTY_HASH for Context @filters/@disabled_tags 2026-03-11 08:55:55 -04:00
Tobi Lutke 69430e9a88 replace INTEGER_REGEX/FLOAT_REGEX with byte-level parse_number 2026-03-11 08:54:31 -04:00
Tobi Lutke 79840b1eaa replace SIMPLE_CONDITION regex with manual byte parser in if/elsif lax_parse 2026-03-11 08:52:49 -04:00
Tobi Lutke 4cda1a578c fast-path slice_collection: skip copy for full Array without offset/limit 2026-03-11 08:50:34 -04:00
Tobi Lutke d574f193dc add invoke_two fast path for single-arg filter invocation, avoids splat chain 2026-03-11 08:49:37 -04:00
Tobi Lutke 76ae8f13e9 fast-path find_variable: check top scope first before find_index 2026-03-11 08:48:16 -04:00
Tobi Lutke 526af22574 add invoke_single fast path for no-arg filter invocation, avoids splat alloc 2026-03-11 08:47:21 -04:00
Tobi Lutke 03a1977ffe fast-path simple if truthiness: use byte scanner before SIMPLE_CONDITION regex 2026-03-11 08:36:28 -04:00
Tobi Lutke b03adefb1c update autoresearch.md progress log 2026-03-11 08:35:43 -04:00
Tobi Lutke 2e207e6844 replace WhitespaceOrNothing regex with byte-level blank_string? check 2026-03-11 08:34:15 -04:00
Tobi Lutke e5933fc6e4 avoid array allocation in parse_tag_token: return tag_name, store markup/newlines as class ivars 2026-03-11 08:33:19 -04:00
Tobi Lutke 1882edb1e1 clean up filter parsing: Lexer fallback for args, no-arg fast scan stays 2026-03-11 08:31:30 -04:00
Tobi Lutke 83037f978b autoresearch.md: add strategic direction toward single-pass scanner architecture 2026-03-11 08:29:12 -04:00
Tobi Lutke ad98d1f329 add security constraint to autoresearch.md, fix strict mode gate 2026-03-11 08:26:40 -04:00
Tobi Lutke 9fd7cec564 split filter parsing: scan no-arg filters directly, only invoke Lexer when args present 2026-03-11 08:26:04 -04:00
Tobi Lutke 2543fdc1a1 autoresearch: add autoresearch.md/sh, increase benchmark warmup to 20 iterations 2026-03-11 08:23:51 -04:00
Tobi Lutke 17daac92da extend fast-path to handle quoted string literal variables (262 more fast-pathed) 2026-03-11 08:20:39 -04:00
Tobi Lutke 283961d6c9 skip filter arg splat for no-arg filters, trim render loop comments 2026-03-11 08:06:23 -04:00
Tobi Lutke db434923d0 hoist write score check out of render loop: skip increment_write_score when no limits active 2026-03-11 08:04:40 -04:00
Tobi Lutke b86143eb0e use frozen EMPTY_ARRAY for disabled_tags in Variable 2026-03-11 07:47:51 -04:00
Tobi Lutke 58d2514521 return [tag_name, markup, newlines] from parse_tag_token: avoid 2 whitespace string allocs 2026-03-11 07:45:15 -04:00
Tobi Lutke 82407092cc use getbyte dispatch instead of start_with? in parse_for_document 2026-03-11 07:44:12 -04:00
Tobi Lutke 544d8f1c17 avoid empty array allocation in evaluate_filter_expressions for no-arg filters 2026-03-11 07:43:09 -04:00
Tobi Lutke cfa0dfe3ca replace For tag Syntax regex with manual byte-level parser 2026-03-11 07:39:36 -04:00
Tobi Lutke 2d3b856b36 expose expression_cache/string_scanner via attr_reader, skip regex in filter args without colon 2026-03-11 07:38:16 -04:00
Tobi Lutke 8a92a4e451 unified fast-path Variable parsing: handle both plain lookups and filter chains without full Lexer pass for name 2026-03-11 07:36:29 -04:00
Tobi Lutke f8b015646a fast-path render for filter-less variables: skip render method overhead 2026-03-11 07:33:18 -04:00
Tobi Lutke 6bcc2936a2 skip TagAttributes scan in for tag when no colon present 2026-03-11 07:32:39 -04:00
Tobi Lutke fe7a2f5aa8 fast-path simple if conditions: skip ExpressionsAndOperators scan for single conditions 2026-03-11 07:31:26 -04:00
Tobi Lutke 3939d74531 replace SIMPLE_VARIABLE regex with byte-level scanner to avoid MatchData 2026-03-11 07:30:34 -04:00
Tobi Lutke 25f9224c85 fast-path simple variable parsing: skip Lexer/Parser for plain dot-separated lookups 2026-03-11 07:29:47 -04:00
Tobi Lutke 5da223275a use frozen EMPTY_ARRAY for Variable filters when no filters present 2026-03-11 07:28:04 -04:00
Tobi Lutke 1a79cf6266 fast-path variable_lookups: skip mutable string alloc when no dot/bracket follows 2026-03-11 07:26:40 -04:00
Tobi Lutke c1113ad2f8 fast-path String in render_obj_to_output, avoid Utils.to_s dispatch for common case 2026-03-11 07:24:30 -04:00
Tobi Lutke fa412245f7 short-circuit parse_number with first-byte check before regex 2026-03-11 07:23:41 -04:00
Tobi Lutke d79b9fa254 avoid unnecessary strip allocation in Expression.parse, use byteslice for string literals 2026-03-11 07:23:08 -04:00
Tobi Lutke d291e63006 use equal? for frozen array comparison in Lexer, skip whitespace with \s+ 2026-03-11 07:21:55 -04:00
Tobi Lutke 2b78e4bf72 use getbyte instead of string indexing in whitespace_handler and create_variable 2026-03-11 07:19:26 -04:00
Tobi Lutke 7aded8e61f add auto/bench.sh: unit tests + liquid-spec + perf benchmark 2026-03-11 07:17:01 -04:00
Tobi Lutke 97e6893c1a replace VariableParser regex scan with manual byte parser in VariableLookup 2026-03-11 07:14:14 -04:00
Tobi Lutke 3329b09dd4 replace FullToken regex with manual byte parsing in parse_for_document 2026-03-11 07:12:38 -04:00
Tobi Lutke 4ea835ae04 add quick benchmark script for autoresearch 2026-03-11 07:10:28 -04:00
Gray GilmoreandGitHub 5fa36267aa Merge pull request #2054 from Shopify/gg-fix-rubocop-offenses
Fix rubocop offenses in test file
2026-03-06 13:28:54 -08:00
Gray Gilmore a72b604680 Fix rubocop offenses in test file 2026-03-06 13:27:05 -08:00
Gray GilmoreandGitHub 3e76244cd2 Merge pull request #2050 from bakura10/squish-filter
Add squish filter
2026-03-06 13:22:18 -08:00
Michaël Gallego d589c51697 Add squish filter 2026-02-19 10:03:52 +09:00
CP ClermontandGitHub d897899f66 Merge pull request #2036 from Shopify/cp-fix-rubocop
Update the specs to new signature and fix CI
2026-01-14 09:14:01 -05:00
Charles-P. ClermontandClaude Opus 4.5 aa817c4cfd Update liquid-spec adapters for new ctx-based API
liquid-spec main changed the adapter API:
- compile block now receives (ctx, source, options) and should store
  the template in ctx[:template]
- render block now receives (ctx, assigns, options) and retrieves
  the template from ctx[:template]

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-01-13 13:06:55 -05:00
Charles-P. Clermont 7d90b524ea Remove on pull_request trigger. It's redundant. 2026-01-12 13:27:53 -05:00
Charles-P. Clermont bbcf8d6ad8 Better matrix CI check names 2026-01-12 13:27:52 -05:00
Charles-P. Clermont 51ff08db7b Fix CI 2026-01-12 13:06:13 -05:00
Tobi LutkeandClaude Opus 4.5 eaa9f215bf Add lax and YJIT liquid-spec adapters
- ruby_liquid_lax.rb: Tests lax parsing mode with :lax_parsing feature
- ruby_liquid_yjit.rb: Tests YJIT + strict mode + ActiveSupport

Matrix results: 4483 matched, 18 different (lax edge cases), 61 skipped

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-01-05 15:11:29 -10:00
Tobi LutkeandClaude Opus 4.5 ccd05e869c Make blank/empty comparisons invariant to ActiveSupport
- Implement liquid_blank? and liquid_empty? methods in Condition
  to emulate ActiveSupport's behavior when it's not loaded
- This ensures templates like `{% if x == blank %}` work identically
  whether ActiveSupport is loaded or not
- Update liquid-spec adapters for new API (ctx parameter)
- Add rake spec task for running liquid-spec matrix

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-01-05 14:54:43 -10:00
Tobias LütkeandGitHub a4a29f3e08 Merge pull request #2034 from Shopify/fix-liquid-spec-without-activesupport
Require liquid-spec to be run on commit automatically and related fixes
2026-01-01 22:32:38 -05:00
Tobi Lutke 0058e4322b Add fail-fast: false to prevent job cancellation 2026-01-01 22:30:14 -05:00
Tobi Lutke 50e1789537 Use liquid-spec feature branch until PR is merged 2026-01-01 22:29:32 -05:00
Tobi Lutke 79a2e042ff Use liquid-spec main branch 2026-01-01 22:24:00 -05:00
Tobi Lutke ddee08fb95 Add activesupport feature to with_active_support adapter
Both adapters now pass with 0 failures:
- ruby_liquid.rb: 4194 passed (skips activesupport and shopify_error_handling specs)
- ruby_liquid_with_active_support.rb: 4203 passed (skips shopify_error_handling specs)
2026-01-01 22:21:47 -05:00
Tobi Lutke 2988f1a500 Update liquid-spec to branch with per-spec required_features support 2026-01-01 22:15:38 -05:00
Tobi Lutke ef13b2dfd5 Fix empty? semantics and string first/last for empty strings
- nil is NOT empty (but IS blank) - matches Shopify production
- String first/last returns '' for empty strings, not nil - matches ActiveSupport
- Add test for nil not being empty
2026-01-01 22:06:22 -05:00
Tobi Lutke b0fb0ad83f Run liquid-spec for all adapters in spec/*.rb 2026-01-01 22:02:42 -05:00
Tobi Lutke ae26cb29ac Disable auto-require for activesupport gem 2026-01-01 22:00:49 -05:00
Tobi Lutke 608a877053 Add spec adapter with ActiveSupport for comparison testing 2026-01-01 22:00:38 -05:00
Tobi Lutke d321adae77 Fix spec adapter for liquid-spec API (template, assigns, options) 2026-01-01 21:59:16 -05:00
Tobi Lutke 53641e19ce Pin liquid-spec to minimum required commit 3d1b492 2026-01-01 21:57:34 -05:00
Tobi Lutke ccd10a986a Pin liquid-spec to main branch 2026-01-01 21:56:39 -05:00
Tobi Lutke f4890de9d5 hm 2026-01-01 21:54:08 -05:00
Tobi Lutke 7e3ccbc188 test 2026-01-01 21:49:33 -05:00
Tobi Lutke e0b46049af Add Ruby 3.4 yjit, 4.0 zjit, and head zjit to CI matrix 2026-01-01 21:44:28 -05:00
Tobi Lutke 19528a9b3f Update CI matrix: remove Ruby 3.0-3.2, add Ruby 4.0 2026-01-01 21:42:47 -05:00
Tobi Lutke 34c274d314 Fix rubocop: rename ruby-liquid.rb to ruby_liquid.rb and add trailing comma 2026-01-01 21:42:11 -05:00
Tobi Lutke 533d470723 Fix spec job to include :spec bundle group 2026-01-01 21:35:13 -05:00
Tobi Lutke 05f9c2a030 Add liquid-spec for conformance testing
- Add liquid-spec gem from GitHub to :spec group
- Create spec/ruby-liquid.rb adapter for the reference implementation
- Add spec job to CI workflow to run liquid-spec tests
2026-01-01 21:34:10 -05:00
Tobi Lutke af58800c16 Update rubocop-shopify to 2.18.0 and fix new offenses 2026-01-01 20:22:19 -05:00
Tobi Lutke 361d1d52b1 Fix rubocop offenses from 1.82 upgrade 2026-01-01 20:20:50 -05:00
Tobi Lutke 391c0df57a Update rubocop to 1.82.0 for Ruby 4.0 support 2026-01-01 20:19:21 -05:00
Tobi Lutke 0ed29760c0 Add benchmark gem for Ruby 4.0 compatibility 2026-01-01 20:18:35 -05:00
Tobi Lutke 0e3548d39e Remove redundant else-clause 2026-01-01 20:16:24 -05:00
Tobi Lutke 33bac87a5c Address liquid-spec issues without ActiveSupport loaded
Implement ActiveSupport-compatible behaviors internally so Liquid works
correctly without ActiveSupport being loaded:

1. String first/last via property access (name.first, name.last)
   - VariableLookup now handles string[0] and string[-1] for first/last

2. String first/last via filters (name | first, name | last)
   - StandardFilters#first and #last now handle strings

3. blank?/empty? comparisons for types without these methods
   - Condition now implements liquid_blank? and liquid_empty? internally
   - blank? matches ActiveSupport: nil, false, empty/whitespace strings,
     empty arrays/hashes are all blank
   - empty? checks length == 0 only (whitespace is NOT empty)

This fixes spec failures for templates like:
- {{ name.first }} / {{ name | first }} on strings
- {% if x == blank %} for whitespace strings, empty hashes/arrays
- {% case ' ' %}{% when blank %} matching whitespace
2026-01-01 20:14:21 -05:00
CP ClermontandGitHub a60a6c0d93 Merge pull request #2027 from Shopify/cp-make-new-tests-serializable-to-liquid-spec
Make new tests serializable to liquid-spec
2025-12-18 15:41:04 -05:00
Charles-P. Clermont bad29caaae Fixup GH action 2025-12-18 15:39:13 -05:00
Charles-P. Clermont cbeff64708 Make new tests serializable to liquid-spec
(No anonymous classes)
2025-12-18 13:56:02 -05:00
Ian Ker-SeymerandGitHub 22e979a6fa Use floating-point format for BigDecimal stringification (#2022)
## Summary

- BigDecimal values now stringify using `to_s("F")` format instead of the default `to_s`

## Why

Ruby's default `BigDecimal#to_s` produces engineering/scientific notation for certain values:

```ruby
BigDecimal("0.00001").to_s      # => "0.1E-4"
BigDecimal("12345678.9").to_s   # => "0.123456789E8"
```

This is rarely the desired output in templates. Using `to_s("F")` produces the expected floating-point format:

```ruby
BigDecimal("0.00001").to_s("F")      # => "0.00001"
BigDecimal("12345678.9").to_s("F")   # => "12345678.9"
```
2025-12-05 15:54:52 -05:00
JuliaandGitHub 735d551168 Merge pull request #2016 from Shopify/jb-method-literals
Preserve literal semantics in strict2 case/when
2025-12-04 09:11:58 -07:00
Julia Boutin fa27bfe6e0 Preserve literal semantics in strict2 case/when
Previously, strict2 case/when used `safe_parse_expression`
to parse when expressions causing `blank`/`empty` to be
treated as string literals (Expression::LITERALS maps 'empty' => ''),
rather than method literals

This caused unexpected behavior:

```
{%- case empty_obj -%}
{%- when empty -%}
  previously: doesn't render (empty_obj == '' is false)
  now: renders (empty_obj.empty? is true)
{%- endcase -%}
```

This commit instead calls `Condition.parse_expression`
with `safe: true`, which will correctly handle `blank`
and `empty`
2025-11-28 15:51:48 -07:00
Guilherme CarreiroandGitHub 32b50ecafe Bump Liquid to 5.11.0 (#2012)
This commit reverts the Inline Snippets tag (#2001) and bumps
Liquid to 5.11. For now, the inclusion of Inline Snippets
in the latest Liquid release is being treated as a bug.

While #2001 does implement the scope contained in RFC#1916,
we need to take a step back to make sure we’re setting our
sights high enough with this feature, and that we’re truly
supporting theme developers in the ways they need.

If you have any feedback, please leave a comment on RFC#1916.

- Liquid Developer Tools
2025-11-19 18:03:23 +01:00
46 changed files with 2162 additions and 338 deletions
+37 -17
View File
@@ -1,5 +1,5 @@
name: Liquid
on: [push, pull_request]
on: [push]
env:
BUNDLE_JOBS: 4
@@ -9,30 +9,33 @@ jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
entry:
- { ruby: 3.0, allowed-failure: false } # minimum supported
- { ruby: 3.2, allowed-failure: false }
- { ruby: 3.3, allowed-failure: false }
- { ruby: 3.3, allowed-failure: false }
- { ruby: 3.4, allowed-failure: false } # latest
- {
ruby: 3.4,
allowed-failure: false,
rubyopt: "--enable-frozen-string-literal",
}
- { ruby: 3.3, allowed-failure: false } # minimum supported
- { ruby: 3.4, allowed-failure: false, rubyopt: "--yjit" }
- { ruby: ruby-head, allowed-failure: false }
- { ruby: 4.0, allowed-failure: false } # latest stable
- {
ruby: ruby-head,
ruby: 4.0,
allowed-failure: false,
rubyopt: "--enable-frozen-string-literal",
}
- { ruby: ruby-head, allowed-failure: false, rubyopt: "--yjit" }
name: Test Ruby ${{ matrix.entry.ruby }}
- { ruby: 4.0, allowed-failure: false, rubyopt: "--yjit" }
- { ruby: 4.0, allowed-failure: false, rubyopt: "--zjit" }
# Head can have failures due to being in development
- { ruby: head, allowed-failure: true }
- {
ruby: head,
allowed-failure: true,
rubyopt: "--enable-frozen-string-literal",
}
- { ruby: head, allowed-failure: true, rubyopt: "--yjit" }
- { ruby: head, allowed-failure: true, rubyopt: "--zjit" }
name: Test Ruby ${{ matrix.entry.ruby }} ${{ matrix.entry.rubyopt }} --${{ matrix.entry.allowed-failure && 'allowed-failure' || 'strict' }}
steps:
- uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # v3.6.0
- uses: ruby/setup-ruby@dffc446db9ba5a0c4446edb5bca1c5c473a806c5 # v1.235.0
- uses: ruby/setup-ruby@a25f1e45f0e65a92fcb1e95e8847f78fb0a7197a # v1.273.0
with:
ruby-version: ${{ matrix.entry.ruby }}
bundler-cache: true
@@ -42,11 +45,28 @@ jobs:
env:
RUBYOPT: ${{ matrix.entry.rubyopt }}
spec:
runs-on: ubuntu-latest
env:
BUNDLE_WITH: spec
steps:
- uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # v3.6.0
- uses: ruby/setup-ruby@a25f1e45f0e65a92fcb1e95e8847f78fb0a7197a # v1.273.0
with:
bundler-cache: true
bundler: latest
- name: Run liquid-spec for all adapters
run: |
for adapter in spec/*.rb; do
echo "=== Running $adapter ==="
bundle exec liquid-spec run "$adapter" --no-max-failures
done
memory_profile:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # v3.6.0
- uses: ruby/setup-ruby@dffc446db9ba5a0c4446edb5bca1c5c473a806c5 # v1.235.0
- uses: ruby/setup-ruby@a25f1e45f0e65a92fcb1e95e8847f78fb0a7197a # v1.273.0
with:
bundler-cache: true
- run: bundle exec rake memory_profile:run
+10 -1
View File
@@ -174,7 +174,16 @@ Style/WordArray:
# Offense count: 117
# This cop supports safe auto-correction (--auto-correct).
# Configuration parameters: AllowHeredoc, AllowURI, URISchemes, IgnoreCopDirectives, AllowedPatterns, IgnoredPatterns.
# Configuration parameters: AllowHeredoc, AllowURI, URISchemes, AllowCopDirectives, AllowedPatterns.
# URISchemes: http, https
Layout/LineLength:
Max: 260
Naming/PredicatePrefix:
Enabled: false
# Offense count: 1
# This is intentional - early return from begin/rescue in assignment context
Lint/NoReturnInBeginEndBlocks:
Exclude:
- 'lib/liquid/standardfilters.rb'
+8 -2
View File
@@ -25,7 +25,13 @@ group :development do
end
group :test do
gem 'rubocop', '~> 1.61.0'
gem 'rubocop-shopify', '~> 2.12.0', require: false
gem 'benchmark'
gem 'rubocop', '~> 1.82.0'
gem 'rubocop-shopify', '~> 2.18.0', require: false
gem 'rubocop-performance', require: false
end
group :spec do
gem 'liquid-spec', github: 'Shopify/liquid-spec', branch: 'main'
gem 'activesupport', require: false
end
+6
View File
@@ -148,3 +148,9 @@ end
task :console do
exec 'irb -I lib -r liquid'
end
desc('run liquid-spec suite across all adapters')
task :spec do
adapters = Dir['./spec/*.rb'].join(',')
sh "bundle exec liquid-spec matrix --adapters=#{adapters} --reference=ruby_liquid"
end
+30
View File
@@ -0,0 +1,30 @@
# Autoresearch Ideas
## Dead Ends (tried and failed)
- **Tag name interning** (skip+byte dispatch): saves 878 allocs but verification loop overhead kills speed
- **String dedup (-@)** for filter names: no alloc savings, creates temp strings anyway
- **Split-based tokenizer**: 2.5x faster C-level split but can't handle {{ followed by %} nesting
- **Streaming tokenizer**: needs own StringScanner (+alloc), per-shift overhead worse than eager array
- **Merge simple_lookup? into initialize**: logic overhead offsets saved index call
- **Cursor for filter scanning**: cursor.reset overhead worse than inline byte loops
- **Direct strainer call**: YJIT already inlines context.invoke_single well
- **TruthyCondition subclass**: YJIT polymorphism at evaluate call site hurts more than 115 saved allocs
- **Index loop for filters**: YJIT optimizes each+destructure MUCH better than manual filter[0]/filter[1]
## Key Insights
- YJIT monomorphism > allocation reduction at this scale
- C-level StringScanner.scan/skip > Ruby-level byte loops (already applied)
- String#split is 2.5x faster than manual tokenization, but Liquid's grammar is too complex for regex
- 74% of total CPU time is GC — alloc reduction is the highest-leverage optimization
- But YJIT-deoptimization from polymorphism costs more than the GC savings
## Remaining Ideas
- **Tokenizer: use String#index + byteslice instead of StringScanner**: avoid the StringScanner overhead entirely for the simple case of finding {%/{{ delimiters
- **Pre-freeze all Condition operator lambdas**: reduce alloc in Condition initialization
- **Avoid `@blocks = []` in If with single-element optimization**: use `@block` ivar for single condition, only create array for elsif
- **Reduce ForloopDrop allocation**: reuse ForloopDrop objects across iterations or use a lighter-weight object
- **VariableLookup: single-segment optimization**: for "product.title" (1 lookup), use an ivar instead of 1-element Array
+109
View File
@@ -0,0 +1,109 @@
# Autoresearch: Liquid Parse+Render Performance
## Objective
Optimize the Shopify Liquid template engine's parse and render performance.
The workload is the ThemeRunner benchmark which parses and renders real Shopify
theme templates (dropify, ripen, tribble, vogue) with realistic data from
`performance/shopify/database.rb`. We measure parse time, render time, and
object allocations. The optimization target is combined parse+render time (µs).
## How to Run
Run `./auto/autoresearch.sh` — it runs unit tests, liquid-spec conformance,
then the performance benchmark, outputting metrics in parseable format.
## Metrics
- **Primary (optimization target)**: `combined_µs` (µs, lower is better) — sum of parse + render time
- **Secondary (tradeoff monitoring)**:
- `parse_µs` — time to parse all theme templates (Liquid::Template#parse)
- `render_µs` — time to render all pre-compiled templates
- `allocations` — total object allocations for one parse+render cycle
Parse dominates (~70-75% of combined). Allocations correlate with GC pressure.
## Files in Scope
- `lib/liquid/*.rb` — core Liquid library (parser, lexer, context, expression, etc.)
- `lib/liquid/tags/*.rb` — tag implementations (for, if, assign, etc.)
- `performance/bench_quick.rb` — benchmark script
## Off Limits
- `test/` — tests must continue to pass unchanged
- `performance/tests/` — benchmark templates, do not modify
- `performance/shopify/` — benchmark data/filters, do not modify
## Constraints
- All unit tests must pass (`bundle exec rake base_test`)
- liquid-spec failures must not increase beyond 2 (pre-existing UTF-8 edge cases)
- No new gem dependencies
- Semantic correctness must be preserved — templates must render identical output
- **Security**: Liquid runs untrusted user code. See Strategic Direction for details.
## Strategic Direction
The long-term goal is to converge toward a **single-pass, forward-only parsing
architecture** using one shared StringScanner instance. The current system has
multiple redundant passes: Tokenizer → BlockBody → Lexer → Parser → Expression
→ VariableLookup, each re-scanning portions of the source. A unified scanner
approach would:
1. **One StringScanner** flows through the entire parse — no intermediate token
arrays, no re-lexing filter chains, no string reconstruction in Parser#expression.
2. **Emit a lightweight IL or normalized AST** during the single forward pass,
decoupling strictness checking from the hot parse path. The LiquidIL project
(`~/src/tries/2026-01-05-liquid-il`) demonstrated this: a recursive-descent
parser emitting IL directly achieved significant speedups.
3. **Minimal backtracking** — the scanner advances forward, byte-checking as it
goes. liquid-c (`~/src/tries/2026-01-16-Shopify-liquid-c`) showed that a
C-level cursor-based tokenizer eliminates most allocation overhead.
Current fast-path optimizations (byte-level tag/variable/for/if parsing) are
steps toward this goal. Each one replaces a regex+MatchData pattern with
forward-only byte scanning. The remaining Lexer→Parser path for filter args
is the next target for elimination.
**Security note**: Liquid executes untrusted user templates. All parsing must
use explicit byte-range checks. Never use eval, send on user input, dynamic
method dispatch, const_get, or any pattern that lets template authors escape
the sandbox.
## Baseline
- **Commit**: 4ea835a (original, before any optimizations)
- **combined_µs**: 7,374
- **parse_µs**: 5,928
- **render_µs**: 1,446
- **allocations**: 62,620
## Progress Log
- 3329b09: Replace FullToken regex with manual byte parsing → combined 7,262 (-1.5%)
- 97e6893: Replace VariableParser regex with manual byte scanner → combined 6,945 (-5.8%), allocs 58,009
- 2b78e4b: getbyte instead of string indexing in whitespace_handler/create_variable → allocs 51,477
- d291e63: Lexer equal? for frozen arrays, \s+ whitespace skip → combined ~6,331
- d79b9fa: Avoid strip alloc in Expression.parse, byteslice for strings → allocs 49,151
- fa41224: Short-circuit parse_number with first-byte check → allocs 48,240
- c1113ad: Fast-path String in render_obj_to_output → combined ~6,071
- 25f9224: Fast-path simple variable parsing (skip Lexer/Parser) → combined ~5,860, allocs 45,202
- 3939d74: Replace SIMPLE_VARIABLE regex with byte scanner → combined ~5,717, allocs 42,763
- fe7a2f5: Fast-path simple if conditions → combined ~5,444, allocs 41,490
- cfa0dfe: Replace For tag Syntax regex with manual byte parser → combined ~4,974, allocs 39,847
- 8a92a4e: Unified fast-path Variable: parse name directly, only lex filter chain → combined ~5,060, allocs 40,520
- 58d2514: parse_tag_token returns [tag_name, markup, newlines] → combined ~4,815, allocs 37,355
- db43492: Hoist write score check out of render loop → render ~1,345
- 17daac9: Extend fast-path to quoted string literal variables → all 1,197 variables fast-pathed
- 9fd7cec: Split filter parsing: no-arg filters scanned directly, Lexer only for args → combined ~4,595, allocs 35,159
- e5933fc: Avoid array alloc in parse_tag_token via class ivars → allocs 34,281
- 2e207e6: Replace WhitespaceOrNothing regex with byte-level blank_string? → combined ~4,800
- 526af22: invoke_single fast path for no-arg filter invocation → allocs 32,621
- 76ae8f1: find_variable top-scope fast path → combined ~4,740
- 4cda1a5: slice_collection: skip copy for full Array → allocs 32,004
- 79840b1: Replace SIMPLE_CONDITION regex with manual byte parser → combined ~4,663, allocs 31,465
- 69430e9: Replace INTEGER_REGEX/FLOAT_REGEX with byte-level parse_number → allocs 31,129
- 405e3dc: Frozen EMPTY_ARRAY/EMPTY_HASH for Context @filters/@disabled_tags → allocs 31,009
- b90d7f0: Avoid unnecessary array wrapping for Context environments → allocs 30,709
- 3799d4c: Lazy seen={} hash in Utils.to_s/inspect → allocs 30,169
- 0b07487: Fast-path VariableLookup: skip scan_variable for simple identifiers → allocs 29,711
- 9de1527: Introduce Cursor class for centralized byte-level scanning
- dd4a100: Remove dead parse_tag_token/SIMPLE_CONDITION (now in Cursor)
- cdc3438: For tag: migrate lax_parse to Cursor with zero-alloc scanning → allocs 29,620
## Current Best
- **combined_µs**: ~3,400 (-54% from original 7,374 baseline)
- **parse_µs**: ~2,300
- **render_µs**: ~1,100
- **allocations**: 24,882 (-60% from original 62,620 baseline)
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
# Autoresearch benchmark runner for Liquid performance optimization
# Runs: unit tests → performance benchmark (3 runs, takes best)
# Outputs METRIC lines for the agent to parse
# Exit code 0 = all good, non-zero = broken
set -euo pipefail
cd "$(dirname "$0")/.."
# ── Step 1: Unit tests (fast gate) ──────────────────────────────────
echo "=== Unit Tests ==="
TEST_OUT=$(bundle exec rake base_test 2>&1)
TEST_RESULT=$(echo "$TEST_OUT" | tail -1)
if echo "$TEST_OUT" | grep -q 'failures\|errors' && ! echo "$TEST_RESULT" | grep -q '0 failures, 0 errors'; then
echo "$TEST_OUT" | grep -E 'Failure|Error|failures|errors' | head -20
echo "FATAL: unit tests failed"
exit 1
fi
echo "$TEST_RESULT"
# ── Step 2: Performance benchmark (3 runs, take best) ──────────────
echo ""
echo "=== Performance Benchmark (3 runs) ==="
BEST_COMBINED=999999
BEST_PARSE=0
BEST_RENDER=0
BEST_ALLOC=0
for i in 1 2 3; do
OUT=$(bundle exec ruby performance/bench_quick.rb 2>&1)
P=$(echo "$OUT" | grep '^parse_us=' | cut -d= -f2)
R=$(echo "$OUT" | grep '^render_us=' | cut -d= -f2)
C=$(echo "$OUT" | grep '^combined_us=' | cut -d= -f2)
A=$(echo "$OUT" | grep '^allocations=' | cut -d= -f2)
echo " run $i: combined=${C}µs (parse=${P} render=${R}) allocs=${A}"
if [ "$C" -lt "$BEST_COMBINED" ]; then
BEST_COMBINED=$C
BEST_PARSE=$P
BEST_RENDER=$R
BEST_ALLOC=$A
fi
done
echo ""
echo "METRIC combined_us=$BEST_COMBINED"
echo "METRIC parse_us=$BEST_PARSE"
echo "METRIC render_us=$BEST_RENDER"
echo "METRIC allocations=$BEST_ALLOC"
Executable
+40
View File
@@ -0,0 +1,40 @@
#!/usr/bin/env bash
# Auto-research benchmark script for Liquid
# Runs: unit tests → liquid-spec → performance benchmark
# Outputs machine-readable metrics on success
# Exit code 0 = all good, non-zero = broken
set -euo pipefail
cd "$(dirname "$0")/.."
# ── Step 1: Unit tests (fast gate) ──────────────────────────────────
echo "=== Unit Tests ==="
if ! bundle exec rake base_test 2>&1; then
echo "FATAL: unit tests failed"
exit 1
fi
# ── Step 2: liquid-spec (correctness gate) ──────────────────────────
echo ""
echo "=== Liquid Spec ==="
SPEC_OUTPUT=$(bundle exec liquid-spec run spec/ruby_liquid.rb 2>&1 || true)
echo "$SPEC_OUTPUT" | tail -3
# Extract failure count from "Total: N passed, N failed, N errors" line
# Allow known pre-existing failures (≤2)
TOTAL_LINE=$(echo "$SPEC_OUTPUT" | grep "^Total:" || echo "Total: 0 passed, 0 failed, 0 errors")
FAILURES=$(echo "$TOTAL_LINE" | sed -n 's/.*\([0-9][0-9]*\) failed.*/\1/p')
ERRORS=$(echo "$TOTAL_LINE" | sed -n 's/.*\([0-9][0-9]*\) error.*/\1/p')
FAILURES=${FAILURES:-0}
ERRORS=${ERRORS:-0}
TOTAL_BAD=$((FAILURES + ERRORS))
if [ "$TOTAL_BAD" -gt 2 ]; then
echo "FATAL: liquid-spec has $FAILURES failures and $ERRORS errors (threshold: 2)"
exit 1
fi
# ── Step 3: Performance benchmark ──────────────────────────────────
echo ""
echo "=== Performance Benchmark ==="
bundle exec ruby performance/bench_quick.rb 2>&1
+30
View File
@@ -0,0 +1,30 @@
{"type":"config","name":"Liquid parse+render performance (tenderlove-inspired)","metricName":"combined_µs","metricUnit":"µs","bestDirection":"lower"}
{"run":1,"commit":"c09e722","metric":3818,"metrics":{"parse_µs":2722,"render_µs":1096,"allocations":24881},"status":"keep","description":"Baseline: 3,818µs combined, 24,881 allocs","timestamp":1773348490227}
{"run":2,"commit":"c09e722","metric":4063,"metrics":{"parse_µs":2901,"render_µs":1162,"allocations":24003},"status":"discard","description":"Tag name interning via skip+byte dispatch: saves 878 allocs but verification loop slower than scan","timestamp":1773348738557,"segment":0}
{"run":3,"commit":"c09e722","metric":3881,"metrics":{"parse_µs":2720,"render_µs":1161,"allocations":24881},"status":"discard","description":"String dedup (-@) for filter names: no alloc savings, no speed benefit","timestamp":1773348781481,"segment":0}
{"run":4,"commit":"c09e722","metric":3970,"metrics":{"parse_µs":2829,"render_µs":1141,"allocations":24881},"status":"discard","description":"Streaming tokenizer: needs own StringScanner (+1 alloc), per-shift overhead worse than saved array","timestamp":1773348883093,"segment":0}
{"run":5,"commit":"c09e722","metric":0,"metrics":{"parse_µs":0,"render_µs":0,"allocations":0},"status":"crash","description":"REVERTED: split-based tokenizer — regex can't handle unclosed tags inside raw blocks","timestamp":1773349089230,"segment":0}
{"run":6,"commit":"c09e722","metric":0,"metrics":{"parse_µs":0,"render_µs":0,"allocations":0},"status":"crash","description":"REVERTED: split regex tokenizer v2 — can't handle {{ followed by %} (variable-becomes-tag nesting)","timestamp":1773349248313,"segment":0}
{"run":7,"commit":"c09e722","metric":3861,"metrics":{"parse_µs":2744,"render_µs":1117,"allocations":24881},"status":"discard","description":"Merge simple_lookup? dot position into initialize — logic overhead offsets saved index call","timestamp":1773349376707,"segment":0}
{"run":8,"commit":"c09e722","metric":4048,"metrics":{"parse_µs":2929,"render_µs":1119,"allocations":24881},"status":"discard","description":"Use Cursor regex for filter name scanning — cursor.reset + method dispatch overhead worse than inline bytes","timestamp":1773349447172,"segment":0}
{"run":9,"commit":"c09e722","metric":3872,"metrics":{"parse_µs":2744,"render_µs":1128,"allocations":24881},"status":"discard","description":"Direct strainer call in Variable#render — YJIT already inlines context.invoke_single well","timestamp":1773349497593,"segment":0}
{"run":10,"commit":"c09e722","metric":3839,"metrics":{"parse_µs":2732,"render_µs":1107,"allocations":24879},"status":"discard","description":"Array#[] fast path for slice_collection with limit/offset — only 2 alloc savings, not meaningful","timestamp":1773349555348,"segment":0}
{"run":11,"commit":"c09e722","metric":3889,"metrics":{"parse_µs":2770,"render_µs":1119,"allocations":24766},"status":"discard","description":"TruthyCondition for simple if checks: -115 allocs but YJIT polymorphism at evaluate call site hurts speed","timestamp":1773349649377,"segment":0}
{"run":12,"commit":"c09e722","metric":4150,"metrics":{"parse_µs":2769,"render_µs":1381,"allocations":24881},"status":"discard","description":"Index loop for filters: YJIT optimizes each+destructure better than manual indexing","timestamp":1773349699285,"segment":0}
{"run":13,"commit":"b7ae55f","metric":3556,"metrics":{"parse_µs":2388,"render_µs":1168,"allocations":24882},"status":"keep","description":"Replace StringScanner tokenizer with String#byteindex — 12% faster parse, no regex overhead for delimiter finding","timestamp":1773349875890,"segment":0}
{"run":14,"commit":"e25f2f1","metric":3464,"metrics":{"parse_µs":2335,"render_µs":1129,"allocations":24882},"status":"keep","description":"Confirmation run: byteindex tokenizer consistently 3,400-3,600µs","timestamp":1773349889465,"segment":0}
{"run":15,"commit":"b37fa98","metric":3490,"metrics":{"parse_µs":2331,"render_µs":1159,"allocations":24882},"status":"keep","description":"Clean up tokenizer: remove unused StringScanner setup and regex constants","timestamp":1773349928672,"segment":0}
{"run":16,"commit":"b37fa98","metric":3638,"metrics":{"parse_µs":2460,"render_µs":1178,"allocations":24882},"status":"discard","description":"Single-char byteindex for %} search: Ruby loop overhead worse for nearby targets","timestamp":1773349985509,"segment":0}
{"run":17,"commit":"b37fa98","metric":3553,"metrics":{"parse_µs":2431,"render_µs":1122,"allocations":25256},"status":"discard","description":"Regex simple_variable_markup: MatchData creates 374 extra allocs, offsetting speed gain","timestamp":1773350066627,"segment":0}
{"run":18,"commit":"b37fa98","metric":3629,"metrics":{"parse_µs":2455,"render_µs":1174,"allocations":25002},"status":"discard","description":"String.new(capacity: 4096) for output buffer: allocates more objects, not fewer","timestamp":1773350101852,"segment":0}
{"run":19,"commit":"f6baeae","metric":3350,"metrics":{"parse_µs":2212,"render_µs":1138,"allocations":24882},"status":"keep","description":"parse_tag_token without StringScanner: pure byte ops avoid reset(token) overhead, -12% combined","timestamp":1773350230252,"segment":0}
{"run":20,"commit":"f6baead","metric":0,"metrics":{"parse_µs":0,"render_µs":0,"allocations":0},"status":"crash","description":"REVERTED: regex ultra-fast path for Variable — name pattern too broad, matches invalid trailing dots","timestamp":1773350472859,"segment":0}
{"run":21,"commit":"ae9a2e2","metric":3314,"metrics":{"parse_µs":2203,"render_µs":1111,"allocations":24882},"status":"keep","description":"Clean confirmation run: 3,314µs (-55% from main), stable","timestamp":1773350544354,"segment":0}
{"run":22,"commit":"ae9a2e2","metric":3497,"metrics":{"parse_µs":2336,"render_µs":1161,"allocations":24882},"status":"discard","description":"Regex fast path for no-filter variables: include? + match? overhead exceeds byte scan savings","timestamp":1773350641375,"segment":0}
{"run":23,"commit":"ca327b0","metric":3445,"metrics":{"parse_µs":2284,"render_µs":1161,"allocations":24647},"status":"keep","description":"Condition#evaluate: skip loop block for simple conditions (no child_relation) — saves 235 allocs","timestamp":1773350691752,"segment":0}
{"run":24,"commit":"99454a9","metric":3489,"metrics":{"parse_µs":2353,"render_µs":1136,"allocations":24647},"status":"keep","description":"Replace simple_lookup? byte scan with match? regex — 8x faster per call, cleaner code","timestamp":1773350837721,"segment":0}
{"run":25,"commit":"99454a9","metric":3797,"metrics":{"parse_µs":2636,"render_µs":1161,"allocations":29627},"status":"discard","description":"Regex name extraction in try_fast_parse: MatchData creates 5K extra allocs, much worse","timestamp":1773351048938,"segment":0}
{"run":26,"commit":"db348e0","metric":3459,"metrics":{"parse_µs":2318,"render_µs":1141,"allocations":24647},"status":"keep","description":"Inline to_liquid_value in If render — avoids one method dispatch per condition evaluation","timestamp":1773351080001,"segment":0}
{"run":27,"commit":"b195d09","metric":3496,"metrics":{"parse_µs":2356,"render_µs":1140,"allocations":24530},"status":"keep","description":"Replace @blocks.each with while loop in If render — avoids block proc allocation per render","timestamp":1773351101134,"segment":0}
{"run":28,"commit":"b195d09","metric":3648,"metrics":{"parse_µs":2457,"render_µs":1191,"allocations":24530},"status":"discard","description":"While loop in For render: YJIT optimizes each well for hot loops with many iterations","timestamp":1773351142275,"segment":0}
{"run":29,"commit":"b195d09","metric":3966,"metrics":{"parse_µs":2641,"render_µs":1325,"allocations":24060},"status":"discard","description":"While loop for environment search: -470 allocs but YJIT deopt makes render 16% slower","timestamp":1773351193863,"segment":0}
+1
View File
@@ -83,6 +83,7 @@ require 'liquid/expression'
require 'liquid/template'
require 'liquid/condition'
require 'liquid/utils'
require 'liquid/cursor'
require 'liquid/tokenizer'
require 'liquid/parse_context'
require 'liquid/partial_cache'
+4 -1
View File
@@ -60,8 +60,11 @@ module Liquid
@tag_name
end
# Cache block delimiters per tag name to avoid repeated string allocation
BLOCK_DELIMITER_CACHE = Hash.new { |h, k| h[k] = "end#{k}".freeze }
def block_delimiter
@block_delimiter ||= "end#{block_name}"
@block_delimiter ||= BLOCK_DELIMITER_CACHE[block_name]
end
private
+68 -46
View File
@@ -38,7 +38,7 @@ module Liquid
private def parse_for_liquid_tag(tokenizer, parse_context)
while (token = tokenizer.shift)
unless token.empty? || token.match?(WhitespaceOrNothing)
unless token.empty? || BlockBody.blank_string?(token)
unless token =~ LiquidTagToken
# line isn't empty but didn't match tag syntax, yield and let the
# caller raise a syntax error
@@ -124,48 +124,70 @@ module Liquid
end
end
OPEN_CURLEY_BYTE = 123 # '{'.ord
PERCENT_BYTE = 37 # '%'.ord
# Fast check if string is whitespace-only (replaces WhitespaceOrNothing regex)
BLANK_STRING_REGEX = /\A\s*\z/
def self.blank_string?(str)
str.match?(BLANK_STRING_REGEX)
end
private def parse_for_document(tokenizer, parse_context, &block)
while (token = tokenizer.shift)
next if token.empty?
case
when token.start_with?(TAGSTART)
whitespace_handler(token, parse_context)
unless token =~ FullToken
return handle_invalid_tag_token(token, parse_context, &block)
end
tag_name = Regexp.last_match(2)
markup = Regexp.last_match(4)
if parse_context.line_number
# newlines inside the tag should increase the line number,
# particularly important for multiline {% liquid %} tags
parse_context.line_number += Regexp.last_match(1).count("\n") + Regexp.last_match(3).count("\n")
end
first_byte = token.getbyte(0)
if first_byte == OPEN_CURLEY_BYTE
second_byte = token.getbyte(1)
if second_byte == PERCENT_BYTE
whitespace_handler(token, parse_context)
cursor = parse_context.cursor
tag_name = cursor.parse_tag_token(token)
unless tag_name
return handle_invalid_tag_token(token, parse_context, &block)
end
markup = cursor.tag_markup
if tag_name == 'liquid'
parse_liquid_tag(markup, parse_context)
next
end
if parse_context.line_number
newlines = cursor.tag_newlines
parse_context.line_number += newlines if newlines > 0
end
unless (tag = parse_context.environment.tag_for_name(tag_name))
# end parsing if we reach an unknown tag and let the caller decide
# determine how to proceed
return yield tag_name, markup
if tag_name == 'liquid'
parse_liquid_tag(markup, parse_context)
next
end
unless (tag = parse_context.environment.tag_for_name(tag_name))
# end parsing if we reach an unknown tag and let the caller decide
# determine how to proceed
return yield tag_name, markup
end
new_tag = tag.parse(tag_name, markup, tokenizer, parse_context)
@blank &&= new_tag.blank?
@nodelist << new_tag
elsif second_byte == OPEN_CURLEY_BYTE
whitespace_handler(token, parse_context)
@nodelist << create_variable(token, parse_context)
@blank = false
else
# Fallback: text token starting with '{'
if parse_context.trim_whitespace
token.lstrip!
end
parse_context.trim_whitespace = false
@nodelist << token
@blank &&= BlockBody.blank_string?(token)
end
new_tag = tag.parse(tag_name, markup, tokenizer, parse_context)
@blank &&= new_tag.blank?
@nodelist << new_tag
when token.start_with?(VARSTART)
whitespace_handler(token, parse_context)
@nodelist << create_variable(token, parse_context)
@blank = false
else
if parse_context.trim_whitespace
token.lstrip!
end
parse_context.trim_whitespace = false
@nodelist << token
@blank &&= token.match?(WhitespaceOrNothing)
@blank &&= BlockBody.blank_string?(token)
end
parse_context.line_number = tokenizer.line_number
end
@@ -173,8 +195,10 @@ module Liquid
yield nil, nil
end
DASH_BYTE = 45 # '-'.ord
def whitespace_handler(token, parse_context)
if token[2] == WhitespaceControl
if token.getbyte(2) == DASH_BYTE
previous_token = @nodelist.last
if previous_token.is_a?(String)
first_byte = previous_token.getbyte(0)
@@ -184,7 +208,7 @@ module Liquid
end
end
end
parse_context.trim_whitespace = (token[-3] == WhitespaceControl)
parse_context.trim_whitespace = (token.getbyte(token.bytesize - 3) == DASH_BYTE)
end
def blank?
@@ -218,7 +242,11 @@ module Liquid
def render_to_output_buffer(context, output)
freeze unless frozen?
context.resource_limits.increment_render_score(@nodelist.length)
resource_limits = context.resource_limits
resource_limits.increment_render_score(@nodelist.length)
# Check if we need per-node write score tracking
check_write = resource_limits.render_length_limit || resource_limits.last_capture_length
idx = 0
while (node = @nodelist[idx])
@@ -226,14 +254,11 @@ module Liquid
output << node
else
render_node(context, output, node)
# If we get an Interrupt that means the block must stop processing. An
# Interrupt is any command that stops block execution such as {% break %}
# or {% continue %}. These tags may also occur through Block or Include tags.
break if context.interrupt? # might have happened in a for-block
break if context.interrupt?
end
idx += 1
context.resource_limits.increment_write_score(output)
resource_limits.increment_write_score(output) if check_write
end
output
@@ -245,15 +270,12 @@ module Liquid
BlockBody.render_node(context, output, node)
end
def create_variable(token, parse_context)
if token.end_with?("}}")
i = 2
i = 3 if token[i] == "-"
parse_end = token.length - 3
parse_end -= 1 if token[parse_end] == "-"
markup_end = parse_end - i + 1
markup = markup_end <= 0 ? "" : token.slice(i, markup_end)
CLOSE_CURLEY_BYTE = 125 # '}'.ord
def create_variable(token, parse_context)
len = token.bytesize
if len >= 4 && token.getbyte(len - 1) == CLOSE_CURLEY_BYTE && token.getbyte(len - 2) == CLOSE_CURLEY_BYTE
markup = parse_context.cursor.parse_variable_token(token)
return Variable.new(markup, parse_context)
end
+62 -16
View File
@@ -65,11 +65,13 @@ module Liquid
end
def evaluate(context = deprecated_default_context)
condition = self
result = nil
loop do
result = interpret_condition(condition.left, condition.right, condition.operator, context)
result = interpret_condition(@left, @right, @operator, context)
# Fast path: no child conditions (most common)
return result unless @child_relation
condition = self
loop do
case condition.child_relation
when :or
break if Liquid::Utils.to_liquid_value(result)
@@ -79,6 +81,7 @@ module Liquid
break
end
condition = condition.child_condition
result = interpret_condition(condition.left, condition.right, condition.operator, context)
end
result
end
@@ -113,24 +116,67 @@ module Liquid
def equal_variables(left, right)
if left.is_a?(MethodLiteral)
if right.respond_to?(left.method_name)
return right.send(left.method_name)
else
return nil
end
return call_method_literal(left, right)
end
if right.is_a?(MethodLiteral)
if left.respond_to?(right.method_name)
return left.send(right.method_name)
else
return nil
end
return call_method_literal(right, left)
end
left == right
end
def call_method_literal(literal, value)
method_name = literal.method_name
# If the object responds to the method (e.g., ActiveSupport is loaded), use it
if value.respond_to?(method_name)
value.send(method_name)
else
# Emulate ActiveSupport's blank?/empty? to make Liquid invariant
# to whether ActiveSupport is loaded or not
case method_name
when :blank?
liquid_blank?(value)
when :empty?
liquid_empty?(value)
else
false
end
end
end
# Implement blank? semantics matching ActiveSupport
# blank? returns true for nil, false, empty strings, whitespace-only strings,
# empty arrays, and empty hashes
def liquid_blank?(value)
case value
when NilClass, FalseClass
true
when TrueClass, Numeric
false
when String
# Blank if empty or whitespace only (matches ActiveSupport)
value.empty? || value.match?(/\A\s*\z/)
when Array, Hash
value.empty?
else
# Fall back to empty? if available, otherwise false
value.respond_to?(:empty?) ? value.empty? : false
end
end
# Implement empty? semantics
# Note: nil is NOT empty. empty? checks if a collection has zero elements.
def liquid_empty?(value)
case value
when String, Array, Hash
value.empty?
else
value.respond_to?(:empty?) ? value.empty? : false
end
end
def interpret_condition(left, right, op, context)
# If the operator is empty this means that the decision statement is just
# a single variable. We can just poll this variable from the context and
@@ -154,8 +200,8 @@ module Liquid
end
def deprecated_default_context
warn("DEPRECATION WARNING: Condition#evaluate without a context argument is deprecated" \
" and will be removed from Liquid 6.0.0.")
warn("DEPRECATION WARNING: Condition#evaluate without a context argument is deprecated " \
"and will be removed from Liquid 6.0.0.")
Context.new
end
+54 -19
View File
@@ -24,10 +24,15 @@ module Liquid
def initialize(environments = {}, outer_scope = {}, registers = {}, rethrow_errors = false, resource_limits = nil, static_environments = {}, environment = Environment.default)
@environment = environment
@environments = [environments]
@environments.flatten!
@environments = environments.is_a?(Array) ? environments : [environments]
@static_environments = [static_environments].flatten(1).freeze
@static_environments = if static_environments.is_a?(Array)
static_environments.frozen? ? static_environments : static_environments.freeze
elsif static_environments.empty?
Const::EMPTY_ARRAY
else
[static_environments].freeze
end
@scopes = [outer_scope || {}]
@registers = registers.is_a?(Registers) ? registers : Registers.new(registers)
@errors = []
@@ -35,14 +40,13 @@ module Liquid
@strict_variables = false
@resource_limits = resource_limits || ResourceLimits.new(environment.default_resource_limits)
@base_scope_depth = 0
@interrupts = []
@filters = []
@interrupts = Const::EMPTY_ARRAY
@filters = Const::EMPTY_ARRAY
@global_filter = nil
@disabled_tags = {}
@disabled_tags = Const::EMPTY_HASH
# Instead of constructing new StringScanner objects for each Expression parse,
# we recycle the same one.
@string_scanner = StringScanner.new("")
# Lazy-init StringScanner — only needed if Context#[] is called during render
@string_scanner = nil
@registers.static[:cached_partials] ||= {}
@registers.static[:file_system] ||= environment.file_system
@@ -84,11 +88,12 @@ module Liquid
# are there any not handled interrupts?
def interrupt?
!@interrupts.empty?
!@interrupts.frozen? && !@interrupts.empty?
end
# push an interrupt to the stack. this interrupt is considered not handled.
def push_interrupt(e)
@interrupts = [] if @interrupts.frozen?
@interrupts.push(e)
end
@@ -109,6 +114,17 @@ module Liquid
strainer.invoke(method, *args).to_liquid
end
# Fast path for single-argument filter invocation (the most common case:
# {{ value | filter }}) — avoids *args splat allocation.
def invoke_single(method, input)
strainer.invoke_single(method, input).to_liquid
end
# Fast path for two-argument filter invocation (e.g. {{ value | default: 'x' }})
def invoke_two(method, input, arg1)
strainer.invoke_two(method, input, arg1).to_liquid
end
# Push new local scope on the stack. use <tt>Context#stack</tt> instead
def push(new_scope = {})
@scopes.unshift(new_scope)
@@ -180,7 +196,7 @@ module Liquid
# Example:
# products == empty #=> products.empty?
def [](expression)
evaluate(Expression.parse(expression, @string_scanner))
evaluate(Expression.parse(expression, @string_scanner ||= StringScanner.new("")))
end
def key?(key)
@@ -193,22 +209,40 @@ module Liquid
# Fetches an object starting at the local scope and then moving up the hierachy
def find_variable(key, raise_on_not_found: true)
# This was changed from find() to find_index() because this is a very hot
# path and find_index() is optimized in MRI to reduce object allocation
index = @scopes.find_index { |s| s.key?(key) }
variable = if index
lookup_and_evaluate(@scopes[index], key, raise_on_not_found: raise_on_not_found)
# Fast path: check top scope first (most common in for loops)
scope = @scopes[0]
if scope.key?(key)
variable = lookup_and_evaluate(scope, key, raise_on_not_found: raise_on_not_found)
elsif @scopes.length == 1
# Only one scope and key not found — go straight to environments
variable = try_variable_find_in_environments(key, raise_on_not_found: raise_on_not_found)
else
try_variable_find_in_environments(key, raise_on_not_found: raise_on_not_found)
# Multiple scopes — search through all of them
index = @scopes.find_index { |s| s.key?(key) }
variable = if index
lookup_and_evaluate(@scopes[index], key, raise_on_not_found: raise_on_not_found)
else
try_variable_find_in_environments(key, raise_on_not_found: raise_on_not_found)
end
end
# update variable's context before invoking #to_liquid
# Fast path: primitive types don't need context= or to_liquid conversion
case variable
when String, Integer, Float, NilClass, TrueClass, FalseClass
return variable
when Array, Hash, Time
return variable
end
variable.context = self if variable.respond_to?(:context=)
liquid_variable = variable.to_liquid
liquid_variable.context = self if variable != liquid_variable && liquid_variable.respond_to?(:context=)
if variable != liquid_variable
liquid_variable.context = self if liquid_variable.respond_to?(:context=)
end
liquid_variable
end
@@ -228,6 +262,7 @@ module Liquid
end
def with_disabled_tags(tag_names)
@disabled_tags = {} if @disabled_tags.frozen?
tag_names.each do |name|
@disabled_tags[name] = @disabled_tags.fetch(name, 0) + 1
end
+362
View File
@@ -0,0 +1,362 @@
# frozen_string_literal: true
require "strscan"
module Liquid
# Single-pass forward-only scanner for Liquid parsing.
# Wraps StringScanner with higher-level methods for common Liquid constructs.
# One Cursor per template parse — threaded through all parsing code.
class Cursor
# Byte constants
SPACE = 32
TAB = 9
NL = 10
CR = 13
FF = 12
DASH = 45 # '-'
DOT = 46 # '.'
COLON = 58 # ':'
PIPE = 124 # '|'
QUOTE_S = 39 # "'"
QUOTE_D = 34 # '"'
LBRACK = 91 # '['
RBRACK = 93 # ']'
LPAREN = 40 # '('
RPAREN = 41 # ')'
QMARK = 63 # '?'
HASH = 35 # '#'
USCORE = 95 # '_'
COMMA = 44
ZERO = 48
NINE = 57
PCT = 37 # '%'
LCURLY = 123 # '{'
RCURLY = 125 # '}'
attr_reader :ss
def initialize(source)
@source = source
@ss = StringScanner.new(source)
end
# ── Position ────────────────────────────────────────────────────
def pos = @ss.pos
def pos=(n)
@ss.pos = n
end
def eos? = @ss.eos?
def peek_byte = @ss.peek_byte
def scan_byte = @ss.scan_byte
# Reset scanner to a new string (for reuse on sub-markup)
def reset(source)
@source = source
@ss.string = source
end
# Extract a slice from the source (deferred allocation)
def slice(start, len)
@source.byteslice(start, len)
end
# ── Whitespace ──────────────────────────────────────────────────
# Skip spaces/tabs/newlines/cr, return count of newlines skipped
def skip_ws
nl = 0
while (b = @ss.peek_byte)
case b
when SPACE, TAB, CR, FF then @ss.scan_byte
when NL then @ss.scan_byte
nl += 1
else break
end
end
nl
end
# Check if remaining bytes are all whitespace
def rest_blank?
saved = @ss.pos
@ss.skip(/\s*/)
result = @ss.eos?
@ss.pos = saved
result
end
# Regex for identifier: [a-zA-Z_][\w-]*\??
ID_REGEX = /[a-zA-Z_][\w-]*\??/
# ── Identifiers ─────────────────────────────────────────────────
# Skip an identifier without allocating a string. Returns length skipped, or 0.
def skip_id
@ss.skip(ID_REGEX) || 0
end
# Check if next id matches expected string, consume if so. No allocation.
def expect_id(expected)
start = @ss.pos
len = @ss.skip(ID_REGEX)
if len == expected.bytesize
# Compare bytes directly without allocating a string
i = 0
while i < len
unless @source.getbyte(start + i) == expected.getbyte(i)
@ss.pos = start
return false
end
i += 1
end
return true
end
@ss.pos = start if len
false
end
# Scan a single identifier: [a-zA-Z_][\w-]*\??
# Returns the string or nil if not at an identifier
def scan_id
@ss.scan(ID_REGEX)
end
# Scan a tag name: '#' or \w+
def scan_tag_name
if @ss.peek_byte == HASH
@ss.scan_byte
"#"
else
scan_id
end
end
# Regex for numbers: -?\d+(\.\d+)?
FLOAT_REGEX = /-?\d+\.\d+/
INT_REGEX = /-?\d+/
# ── Numbers ─────────────────────────────────────────────────────
# Try to scan an integer or float. Returns the number or nil.
def scan_number
if (s = @ss.scan(FLOAT_REGEX))
s.to_f
elsif (s = @ss.scan(INT_REGEX))
s.to_i
end
end
# Regex for quoted string content (without quotes)
SINGLE_QUOTED_CONTENT = /'([^']*)'/
DOUBLE_QUOTED_CONTENT = /"([^"]*)"/
# ── Strings ─────────────────────────────────────────────────────
# Scan a quoted string ('...' or "..."). Returns the content without quotes, or nil.
def scan_quoted_string
if @ss.scan(SINGLE_QUOTED_CONTENT) || @ss.scan(DOUBLE_QUOTED_CONTENT)
@ss[1]
end
end
# Regex for quoted strings (single or double quoted, including quotes)
QUOTED_STRING_RAW = /"[^"]*"|'[^']*'/
# Scan a quoted string including quotes. Returns the full "..." or '...' string, or nil.
def scan_quoted_string_raw
@ss.scan(QUOTED_STRING_RAW)
end
# Regex for dotted identifier: name(.name)*
DOTTED_ID_REGEX = /[a-zA-Z_][\w-]*\??(?:\.[a-zA-Z_][\w-]*\??)*/
# ── Expressions ─────────────────────────────────────────────────
# Scan a simple variable lookup: name(.name)* — no brackets, no filters
# Returns the string or nil
def scan_dotted_id
@ss.scan(DOTTED_ID_REGEX)
end
# Skip a fragment without allocating. Returns length skipped, or 0.
def skip_fragment
@ss.skip(QUOTED_STRING_RAW) || @ss.skip(UNQUOTED_FRAGMENT) || 0
end
# Regex for unquoted fragment: non-whitespace/comma/pipe sequence
UNQUOTED_FRAGMENT = /[^\s,|]+/
# Scan a "QuotedFragment" — a quoted string or non-whitespace/comma/pipe run
def scan_fragment
@ss.scan(QUOTED_STRING_RAW) || @ss.scan(UNQUOTED_FRAGMENT)
end
# ── Comparison operators ────────────────────────────────────────
COMPARISON_OPS = {
'==' => '==',
'!=' => '!=',
'<>' => '<>',
'<=' => '<=',
'>=' => '>=',
'<' => '<',
'>' => '>',
'contains' => 'contains',
}.freeze
# Scan a comparison operator. Returns frozen string or nil.
# Regex for comparison operators
COMPARISON_OP_REGEX = /==|!=|<>|<=|>=|<|>|contains(?!\w)/
def scan_comparison_op
if (op = @ss.scan(COMPARISON_OP_REGEX))
COMPARISON_OPS[op]
end
end
# ── Tag parsing helpers ─────────────────────────────────────────
# Results from last parse_tag_token call (avoids array allocation)
attr_reader :tag_markup, :tag_newlines
# Parse the interior of a tag token: "{%[-] tag_name markup [-]%}"
# Pure byte operations — avoids StringScanner reset overhead.
# Returns tag_name string or nil. Sets tag_markup and tag_newlines.
def parse_tag_token(token)
len = token.bytesize
pos = 2 # skip "{%"
pos += 1 if token.getbyte(pos) == DASH # skip '-'
nl = 0
# Skip whitespace, count newlines
while pos < len
b = token.getbyte(pos)
case b
when SPACE, TAB, CR, FF then pos += 1
when NL then pos += 1; nl += 1
else break
end
end
# Scan tag name: '#' or [a-zA-Z_][\w-]*
name_start = pos
b = token.getbyte(pos)
if b == HASH
pos += 1
elsif b && ((b >= 97 && b <= 122) || (b >= 65 && b <= 90) || b == USCORE)
pos += 1
while pos < len
b = token.getbyte(pos)
break unless (b >= 97 && b <= 122) || (b >= 65 && b <= 90) || (b >= 48 && b <= 57) || b == USCORE || b == DASH
pos += 1
end
pos += 1 if pos < len && token.getbyte(pos) == QMARK
else
return
end
tag_name = token.byteslice(name_start, pos - name_start)
# Skip whitespace after tag name, count newlines
while pos < len
b = token.getbyte(pos)
case b
when SPACE, TAB, CR, FF then pos += 1
when NL then pos += 1; nl += 1
else break
end
end
# markup is everything up to optional '-' before '%}'
markup_end = len - 2
markup_end -= 1 if markup_end > pos && token.getbyte(markup_end - 1) == DASH
@tag_markup = pos >= markup_end ? "" : token.byteslice(pos, markup_end - pos)
@tag_newlines = nl
tag_name
end
# Parse variable token interior: extract markup from "{{[-] ... [-]}}"
def parse_variable_token(token)
len = token.bytesize
return if len < 4
i = 2
i = 3 if token.getbyte(i) == DASH
parse_end = len - 3
parse_end -= 1 if token.getbyte(parse_end) == DASH
markup_len = parse_end - i + 1
markup_len <= 0 ? "" : token.byteslice(i, markup_len)
end
# ── Simple condition parser ─────────────────────────────────────
# Results from last parse_simple_condition call
attr_reader :cond_left, :cond_op, :cond_right
# Parse "expr [op expr]" from current position to end.
# Returns true on success, nil on failure. Sets cond_left, cond_op, cond_right.
def parse_simple_condition
skip_ws
@cond_left = scan_fragment
return unless @cond_left
skip_ws
if eos?
@cond_op = nil
@cond_right = nil
return true
end
@cond_op = scan_comparison_op
return unless @cond_op
skip_ws
@cond_right = scan_fragment
return unless @cond_right
skip_ws
return unless eos? # trailing junk
true
end
# ── For tag parser ────────────────────────────────────────────────
# Results from parse_for_markup
attr_reader :for_var, :for_collection, :for_reversed
# Parse "var in collection [reversed] [limit:N] [offset:N]"
# Returns true on success, nil on failure.
def parse_for_markup
skip_ws
@for_var = scan_id
return unless @for_var
skip_ws
# expect "in"
return unless scan_id == "in"
skip_ws
# Collection: parenthesized range or fragment
if peek_byte == LPAREN
start = @ss.pos
depth = 1
@ss.scan_byte
while !@ss.eos? && depth > 0
b = @ss.scan_byte
depth += 1 if b == LPAREN
depth -= 1 if b == RPAREN
end
@for_collection = @source.byteslice(start, @ss.pos - start)
else
@for_collection = scan_fragment
return unless @for_collection
end
skip_ws
# Check for 'reversed'
saved = @ss.pos
word = scan_id
if word == "reversed"
@for_reversed = true
else
@for_reversed = false
@ss.pos = saved if word # rewind if we consumed a non-'reversed' word
end
true
end
end
end
+1 -1
View File
@@ -31,7 +31,7 @@ module Liquid
# Catch all for the method
def liquid_method_missing(method)
return nil unless @context&.strict_variables
return unless @context&.strict_variables
raise Liquid::UndefinedDropMethod, "undefined method #{method}"
end
+72 -44
View File
@@ -35,11 +35,18 @@ module Liquid
def parse(markup, ss = StringScanner.new(""), cache = nil)
return unless markup
markup = markup.strip # markup can be a frozen string
# Only strip if there's leading/trailing whitespace (avoids allocation)
first_byte = markup.getbyte(0)
if first_byte == 32 || first_byte == 9 || first_byte == 10 || first_byte == 13 # space, tab, \n, \r
markup = markup.strip
else
last_byte = markup.getbyte(markup.bytesize - 1)
markup = markup.strip if last_byte == 32 || last_byte == 9 || last_byte == 10 || last_byte == 13
end
if (markup.start_with?('"') && markup.end_with?('"')) ||
(markup.start_with?("'") && markup.end_with?("'"))
return markup[1..-2]
return markup.byteslice(1, markup.bytesize - 2)
elsif LITERALS.key?(markup)
return LITERALS[markup]
end
@@ -55,7 +62,7 @@ module Liquid
end
def inner_parse(markup, ss, cache)
if (markup.start_with?("(") && markup.end_with?(")")) && markup =~ RANGES_REGEX
if markup.start_with?("(") && markup.end_with?(")") && markup =~ RANGES_REGEX
return RangeLookup.parse(
Regexp.last_match(1),
Regexp.last_match(2),
@@ -71,57 +78,78 @@ module Liquid
end
end
def parse_number(markup, ss)
# check if the markup is simple integer or float
case markup
when INTEGER_REGEX
def parse_number(markup, _ss = nil)
len = markup.bytesize
return false if len == 0
# Quick reject: first byte must be digit or dash
pos = 0
first = markup.getbyte(pos)
if first == DASH
pos += 1
return false if pos >= len
b = markup.getbyte(pos)
return false if b < ZERO || b > NINE
pos += 1
elsif first >= ZERO && first <= NINE
pos += 1
else
return false
end
# Scan digits
while pos < len
b = markup.getbyte(pos)
break if b < ZERO || b > NINE
pos += 1
end
# If we consumed everything, it's a simple integer
if pos == len
return Integer(markup, 10)
when FLOAT_REGEX
return markup.to_f
end
ss.string = markup
# the first byte must be a digit or a dash
byte = ss.scan_byte
# Check for dot (float)
if markup.getbyte(pos) == DOT
dot_pos = pos
pos += 1
# Must have at least one digit after dot
digit_after_dot = pos
while pos < len
b = markup.getbyte(pos)
break if b < ZERO || b > NINE
return false if byte != DASH && (byte < ZERO || byte > NINE)
pos += 1
end
if byte == DASH
peek_byte = ss.peek_byte
if pos > digit_after_dot && pos == len
# Simple float like "123.456"
return markup.to_f
elsif pos > digit_after_dot
# Float followed by more dots or other chars: "1.2.3.4"
# Return the float portion up to second dot
while pos < len
b = markup.getbyte(pos)
if b == DOT
return markup.byteslice(0, pos).to_f
elsif b < ZERO || b > NINE
return false
end
# if it starts with a dash, the next byte must be a digit
return false if peek_byte.nil? || !(peek_byte >= ZERO && peek_byte <= NINE)
end
# The markup could be a float with multiple dots
first_dot_pos = nil
num_end_pos = nil
while (byte = ss.scan_byte)
return false if byte != DOT && (byte < ZERO || byte > NINE)
# we found our number and now we are just scanning the rest of the string
next if num_end_pos
if byte == DOT
if first_dot_pos.nil?
first_dot_pos = ss.pos
else
# we found another dot, so we know that the number ends here
num_end_pos = ss.pos - 1
pos += 1
end
return markup.byteslice(0, pos).to_f
else
# dot at end: "123."
return markup.byteslice(0, dot_pos).to_f
end
end
num_end_pos = markup.length if ss.eos?
if num_end_pos
# number ends with a number "123.123"
markup.byteslice(0, num_end_pos).to_f
else
# number ends with a dot "123."
markup.byteslice(0, first_dot_pos).to_f
end
# Not a number (has non-digit, non-dot characters)
false
end
end
end
+1 -1
View File
@@ -28,7 +28,7 @@ module Liquid
def interpolate(name, vars)
name.gsub(/%\{(\w+)\}/) do
# raise TranslationError, "Undefined key #{$1} for interpolation in translation #{name}" unless vars[$1.to_sym]
(vars[Regexp.last_match(1).to_sym]).to_s
vars[Regexp.last_match(1).to_sym].to_s
end
end
+4 -3
View File
@@ -29,6 +29,7 @@ module Liquid
RUBY_WHITESPACE = [" ", "\t", "\r", "\n", "\f"].freeze
SINGLE_STRING_LITERAL = /'[^\']*'/
WHITESPACE_OR_NOTHING = /\s*/
WHITESPACE = /\s+/
SINGLE_COMPARISON_TOKENS = [].tap do |table|
table["<".ord] = COMPARISON_LESS_THAN
@@ -104,7 +105,7 @@ module Liquid
output = []
until ss.eos?
ss.skip(WHITESPACE_OR_NOTHING)
ss.skip(WHITESPACE)
break if ss.eos?
@@ -114,10 +115,10 @@ module Liquid
if (special = SPECIAL_TABLE[peeked])
ss.scan_byte
# Special case for ".."
if special == DOT && ss.peek_byte == DOT_ORD
if special.equal?(DOT) && ss.peek_byte == DOT_ORD
ss.scan_byte
output << DOTDOT
elsif special == DASH
elsif special.equal?(DASH)
# Special case for negative numbers
if (peeked_byte = ss.peek_byte) && NUMBER_TABLE[peeked_byte]
ss.pos -= 1
+3 -1
View File
@@ -3,7 +3,7 @@
module Liquid
class ParseContext
attr_accessor :locale, :line_number, :trim_whitespace, :depth
attr_reader :partial, :warnings, :error_mode, :environment
attr_reader :partial, :warnings, :error_mode, :environment, :expression_cache, :string_scanner, :cursor
def initialize(options = Const::EMPTY_HASH)
@environment = options.fetch(:environment, Environment.default)
@@ -24,6 +24,8 @@ module Liquid
{}
end
@cursor = Cursor.new("")
self.depth = 0
self.partial = false
end
+3
View File
@@ -83,6 +83,9 @@ module Liquid
end
def variable_lookups
# Fast path: no lookups at all (most common case for simple identifiers)
return "" unless look(:dot) || look(:open_square)
str = +""
loop do
if look(:open_square)
+6 -6
View File
@@ -6,15 +6,15 @@ module Liquid
def initialize(registers = {})
@static = registers.is_a?(Registers) ? registers.static : registers
@changes = {}
@changes = nil
end
def []=(key, value)
@changes[key] = value
(@changes ||= {})[key] = value
end
def [](key)
if @changes.key?(key)
if @changes&.key?(key)
@changes[key]
else
@static[key]
@@ -22,13 +22,13 @@ module Liquid
end
def delete(key)
@changes.delete(key)
@changes&.delete(key)
end
UNDEFINED = Object.new
def fetch(key, default = UNDEFINED, &block)
if @changes.key?(key)
if @changes&.key?(key)
@changes.fetch(key)
elsif default != UNDEFINED
if block_given?
@@ -42,7 +42,7 @@ module Liquid
end
def key?(key)
@changes.key?(key) || @static.key?(key)
@changes&.key?(key) || @static.key?(key)
end
end
+1 -1
View File
@@ -3,7 +3,7 @@
module Liquid
class ResourceLimits
attr_accessor :render_length_limit, :render_score_limit, :assign_score_limit
attr_reader :render_score, :assign_score
attr_reader :render_score, :assign_score, :last_capture_length
def initialize(limits)
@render_length_limit = limits[:render_length_limit]
+64 -11
View File
@@ -266,18 +266,54 @@ module Liquid
words = Utils.to_integer(words)
words = 1 if words <= 0
wordlist = begin
input.split(" ", words + 1)
rescue RangeError
# integer too big for String#split, but we can semantically assume no truncation is needed
return input if words + 1 > MAX_I32
raise # unexpected error
end
return input if wordlist.length <= words
return input if words + 1 > MAX_I32
wordlist.pop
truncate_string = Utils.to_s(truncate_string)
wordlist.join(" ").concat(truncate_string)
# Build result incrementally — avoids split() array + string allocations
len = input.bytesize
pos = 0
word_count = 0
result = nil
# Skip leading whitespace
while pos < len
b = input.getbyte(pos)
break unless b == 32 || b == 9 || b == 10 || b == 13 || b == 12
pos += 1
end
while pos < len
word_start = pos
word_count += 1
# Skip non-whitespace chars (word body)
while pos < len
b = input.getbyte(pos)
break if b == 32 || b == 9 || b == 10 || b == 13 || b == 12
pos += 1
end
if word_count > words
# Truncate — result already has the first N words
truncate_string = Utils.to_s(truncate_string)
return result.concat(truncate_string)
end
# Append word to result (only allocate result when we know truncation is possible)
if result
result << " " << input.byteslice(word_start, pos - word_start)
else
result = +input.byteslice(word_start, pos - word_start)
end
# Skip whitespace between words
while pos < len
b = input.getbyte(pos)
break unless b == 32 || b == 9 || b == 10 || b == 13 || b == 12
pos += 1
end
end
input
end
# @liquid_public_docs
@@ -293,6 +329,19 @@ module Liquid
input.split(pattern)
end
# @liquid_public_docs
# @liquid_type filter
# @liquid_category string
# @liquid_summary
# Removes leading and trailing whitespace and collapses consecutive whitespace to a single space.
# @liquid_syntax string | squish
# @liquid_return [string]
def squish(input)
return if input.nil?
Utils.to_s(input).strip.gsub(/\s+/, ' ')
end
# @liquid_public_docs
# @liquid_type filter
# @liquid_category string
@@ -768,6 +817,8 @@ module Liquid
# @liquid_syntax array | first
# @liquid_return [untyped]
def first(array)
# ActiveSupport returns "" for empty strings, not nil
return array[0] || "" if array.is_a?(String)
array.first if array.respond_to?(:first)
end
@@ -779,6 +830,8 @@ module Liquid
# @liquid_syntax array | last
# @liquid_return [untyped]
def last(array)
# ActiveSupport returns "" for empty strings, not nil
return array[-1] || "" if array.is_a?(String)
array.last if array.respond_to?(:last)
end
+27
View File
@@ -58,5 +58,32 @@ module Liquid
rescue ::ArgumentError => e
raise Liquid::ArgumentError, e.message, e.backtrace
end
# Fast path for single-argument (no extra args) filter invocation.
# Avoids *args splat allocation for the common {{ value | filter }} case.
def invoke_single(method, input)
if self.class.invokable?(method)
send(method, input)
elsif @context.strict_filters
raise Liquid::UndefinedFilter, "undefined filter #{method}"
else
input
end
rescue ::ArgumentError => e
raise Liquid::ArgumentError, e.message, e.backtrace
end
# Fast path for two-argument filter invocation (input + one arg).
def invoke_two(method, input, arg1)
if self.class.invokable?(method)
send(method, input, arg1)
elsif @context.strict_filters
raise Liquid::UndefinedFilter, "undefined filter #{method}"
else
input
end
rescue ::ArgumentError => e
raise Liquid::ArgumentError, e.message, e.backtrace
end
end
end
+1 -1
View File
@@ -118,7 +118,7 @@ module Liquid
parser = @parse_context.new_parser(markup)
loop do
expr = safe_parse_expression(parser)
expr = Condition.parse_expression(parse_context, parser.expression, safe: true)
block = Condition.new(@left, '==', expr)
block.attach(body)
@blocks << block
+45 -9
View File
@@ -72,18 +72,54 @@ module Liquid
protected
# Fast byte-level parser for "var in collection [reversed] [limit:N] [offset:N]"
REVERSED_BYTES = "reversed".bytes.freeze
def lax_parse(markup)
if markup =~ Syntax
@variable_name = Regexp.last_match(1)
collection_name = Regexp.last_match(2)
@reversed = !!Regexp.last_match(3)
@name = "#{@variable_name}-#{collection_name}"
@collection_name = parse_expression(collection_name)
markup.scan(TagAttributes) do |key, value|
set_attribute(key, value)
c = @parse_context.cursor
c.reset(markup)
c.skip_ws
# Parse variable name
var_start = c.pos
var_len = c.skip_id
raise SyntaxError, options[:locale].t("errors.syntax.for") if var_len == 0
@variable_name = c.slice(var_start, var_len)
# Expect "in"
c.skip_ws
raise SyntaxError, options[:locale].t("errors.syntax.for") unless c.expect_id("in")
c.skip_ws
# Parse collection name
col_start = c.pos
if c.peek_byte == Cursor::LPAREN
# Parenthesized range: (1..10)
depth = 1
c.scan_byte
while !c.eos? && depth > 0
b = c.scan_byte
depth += 1 if b == Cursor::LPAREN
depth -= 1 if b == Cursor::RPAREN
end
else
raise SyntaxError, options[:locale].t("errors.syntax.for")
c.skip_fragment
end
collection_name = c.slice(col_start, c.pos - col_start)
@name = "#{@variable_name}-#{collection_name}"
@collection_name = parse_expression(collection_name)
c.skip_ws
@reversed = c.expect_id("reversed")
c.skip_ws
# Parse limit:/offset: if present
if !c.eos? && markup.include?(':')
rest = c.slice(c.pos, markup.bytesize - c.pos)
rest.scan(TagAttributes) do |key, value|
set_attribute(key, value)
end
end
end
+25 -4
View File
@@ -51,14 +51,17 @@ module Liquid
end
def render_to_output_buffer(context, output)
@blocks.each do |block|
result = Liquid::Utils.to_liquid_value(
block.evaluate(context),
)
idx = 0
blocks = @blocks
while idx < blocks.length
block = blocks[idx]
result = block.evaluate(context)
result = result.to_liquid_value if result.respond_to?(:to_liquid_value)
if result
return block.attachment.render_to_output_buffer(context, output)
end
idx += 1
end
output
@@ -86,6 +89,24 @@ module Liquid
end
def lax_parse(markup)
# Fastest path: simple identifier truthiness like "product.available" or "forloop.first"
if (simple = Variable.simple_variable_markup(markup))
return Condition.new(parse_expression(simple))
end
# Fast path: simple condition without and/or — use Cursor
if !markup.include?(' and ') && !markup.include?(' or ')
cursor = @parse_context.cursor
cursor.reset(markup)
if cursor.parse_simple_condition
return Condition.new(
parse_expression(cursor.cond_left),
cursor.cond_op,
cursor.cond_right ? parse_expression(cursor.cond_right) : nil,
)
end
end
expressions = markup.scan(ExpressionsAndOperators)
raise SyntaxError, options[:locale].t("errors.syntax.if") unless expressions.pop =~ Syntax
+106 -95
View File
@@ -6,10 +6,6 @@ module Liquid
class Tokenizer
attr_reader :line_number, :for_liquid_tag
TAG_END = /%\}/
TAG_OR_VARIABLE_START = /\{[\{\%]/
NEWLINE = /\n/
OPEN_CURLEY = "{".ord
CLOSE_CURLEY = "}".ord
PERCENTAGE = "%".ord
@@ -27,11 +23,7 @@ module Liquid
@offset = 0
@tokens = []
if @source
@ss = string_scanner
@ss.string = @source
tokenize
end
tokenize if @source
end
def shift
@@ -54,108 +46,127 @@ module Liquid
if @for_liquid_tag
@tokens = @source.split("\n")
else
@tokens << shift_normal until @ss.eos?
tokenize_fast
end
@source = nil
@ss = nil
end
def shift_normal
token = next_token
# Fast tokenizer using String#index instead of StringScanner regex.
# String#index is ~40% faster for finding { delimiters.
def tokenize_fast
src = @source
unless src.valid_encoding?
raise SyntaxError, "Invalid byte sequence in #{src.encoding}"
end
return unless token
len = src.bytesize
pos = 0
token
end
while pos < len
# Find next { which could start a tag or variable
idx = src.byteindex('{', pos)
def next_token
# possible states: :text, :tag, :variable
byte_a = @ss.peek_byte
if byte_a == OPEN_CURLEY
@ss.scan_byte
byte_b = @ss.peek_byte
if byte_b == PERCENTAGE
@ss.scan_byte
return next_tag_token
elsif byte_b == OPEN_CURLEY
@ss.scan_byte
return next_variable_token
unless idx
# No more tags/variables — rest is text
@tokens << src.byteslice(pos, len - pos) if pos < len
break
end
@ss.pos -= 1
end
next_byte = idx + 1 < len ? src.getbyte(idx + 1) : nil
next_text_token
end
if next_byte == PERCENTAGE # {%
# Emit text before tag
@tokens << src.byteslice(pos, idx - pos) if idx > pos
def next_text_token
start = @ss.pos
unless @ss.skip_until(TAG_OR_VARIABLE_START)
token = @ss.rest
@ss.terminate
return token
end
pos = @ss.pos -= 2
@source.byteslice(start, pos - start)
rescue ::ArgumentError => e
if e.message == "invalid byte sequence in #{@ss.string.encoding}"
raise SyntaxError, "Invalid byte sequence in #{@ss.string.encoding}"
else
raise
end
end
def next_variable_token
start = @ss.pos - 2
byte_a = byte_b = @ss.scan_byte
while byte_b
byte_a = @ss.scan_byte while byte_a && (byte_a != CLOSE_CURLEY && byte_a != OPEN_CURLEY)
break unless byte_a
if @ss.eos?
return byte_a == CLOSE_CURLEY ? @source.byteslice(start, @ss.pos - start) : "{{"
end
byte_b = @ss.scan_byte
if byte_a == CLOSE_CURLEY
if byte_b == CLOSE_CURLEY
return @source.byteslice(start, @ss.pos - start)
elsif byte_b != CLOSE_CURLEY
@ss.pos -= 1
return @source.byteslice(start, @ss.pos - start)
# Find %} to close the tag
close = src.byteindex('%}', idx + 2)
if close
@tokens << src.byteslice(idx, close + 2 - idx)
pos = close + 2
else
@tokens << "{%"
pos = idx + 2
end
elsif next_byte == OPEN_CURLEY # {{
# Emit text before variable
@tokens << src.byteslice(pos, idx - pos) if idx > pos
# Scan variable token — matches original tokenizer's byte-by-byte logic:
# Find } or {, then check next byte for }}/{% nesting
scan_pos = idx + 2
found = false
while scan_pos < len
b = src.getbyte(scan_pos)
if b == CLOSE_CURLEY # }
if scan_pos + 1 >= len
# } at end of string — emit token up to here
@tokens << src.byteslice(idx, scan_pos + 1 - idx)
pos = scan_pos + 1
found = true
break
end
b2 = src.getbyte(scan_pos + 1)
if b2 == CLOSE_CURLEY
# Found }} — close variable
@tokens << src.byteslice(idx, scan_pos + 2 - idx)
pos = scan_pos + 2
found = true
break
else
# } followed by non-} — emit token up to here (matches original: @ss.pos -= 1)
@tokens << src.byteslice(idx, scan_pos + 1 - idx)
pos = scan_pos + 1
found = true
break
end
elsif b == OPEN_CURLEY
if scan_pos + 1 < len && src.getbyte(scan_pos + 1) == PERCENTAGE
# Found {% inside {{ — scan to %} and emit as one token
close = src.byteindex('%}', scan_pos + 2)
if close
@tokens << src.byteslice(idx, close + 2 - idx)
pos = close + 2
else
@tokens << src.byteslice(idx, len - idx)
pos = len
end
found = true
break
end
scan_pos += 1
else
scan_pos += 1
end
end
unless found
@tokens << "{{"
pos = idx + 2
end
else
# { followed by something else — it's text
# Keep scanning from after this {
# Find next { that could be {% or {{
next_open = idx + 1
while next_open < len
ni = src.byteindex('{', next_open)
unless ni
@tokens << src.byteslice(pos, len - pos)
pos = len
break
end
nb = ni + 1 < len ? src.getbyte(ni + 1) : nil
if nb == PERCENTAGE || nb == OPEN_CURLEY
@tokens << src.byteslice(pos, ni - pos)
pos = ni
break
end
next_open = ni + 1
end
elsif byte_a == OPEN_CURLEY && byte_b == PERCENTAGE
return next_tag_token_with_start(start)
end
byte_a = byte_b
end
"{{"
end
def next_tag_token
start = @ss.pos - 2
if (len = @ss.skip_until(TAG_END))
@source.byteslice(start, len + 2)
else
"{%"
end
end
def next_tag_token_with_start(start)
@ss.skip_until(TAG_END)
@source.byteslice(start, @ss.pos - start)
end
end
end
+18 -7
View File
@@ -8,6 +8,9 @@ module Liquid
def self.slice_collection(collection, from, to)
if (from != 0 || !to.nil?) && collection.respond_to?(:load_slice)
collection.load_slice(from, to)
elsif from == 0 && to.nil? && collection.is_a?(Array)
# Fast path: no offset/limit on an Array — return as-is (avoid copy)
collection
else
slice_collection_using_each(collection, from, to)
end
@@ -69,7 +72,7 @@ module Liquid
return obj if obj.respond_to?(:strftime)
if obj.is_a?(String)
return nil if obj.empty?
return if obj.empty?
obj = obj.downcase
end
@@ -93,37 +96,45 @@ module Liquid
obj
end
def self.to_s(obj, seen = {})
# Cached string representations for common small integers (0-999)
# Avoids repeated Integer#to_s allocations during rendering
SMALL_INT_STRINGS = Array.new(1000) { |i| i.to_s.freeze }.freeze
def self.to_s(obj, seen = nil)
case obj
when Integer
return (obj >= 0 && obj < 1000) ? SMALL_INT_STRINGS[obj] : obj.to_s
when BigDecimal
obj.to_s("F")
when Hash
# If the custom hash implementation overrides `#to_s`, use their
# custom implementation. Otherwise we use Liquid's default
# implementation.
if obj.class.instance_method(:to_s) == HASH_TO_S_METHOD
hash_inspect(obj, seen)
hash_inspect(obj, seen || {})
else
obj.to_s
end
when Array
array_inspect(obj, seen)
array_inspect(obj, seen || {})
else
obj.to_s
end
end
def self.inspect(obj, seen = {})
def self.inspect(obj, seen = nil)
case obj
when Hash
# If the custom hash implementation overrides `#inspect`, use their
# custom implementation. Otherwise we use Liquid's default
# implementation.
if obj.class.instance_method(:inspect) == HASH_INSPECT_METHOD
hash_inspect(obj, seen)
hash_inspect(obj, seen || {})
else
obj.inspect
end
when Array
array_inspect(obj, seen)
array_inspect(obj, seen || {})
else
obj.inspect
end
+318 -15
View File
@@ -12,6 +12,68 @@ module Liquid
# {{ user | link }}
#
class Variable
# Checks if markup is a simple "name.lookup.chain" with no filters/brackets/quotes.
# Returns the trimmed markup string, or nil if not simple.
# Avoids regex MatchData allocation.
def self.simple_variable_markup(markup)
len = markup.bytesize
return if len == 0
# Skip leading whitespace
pos = 0
while pos < len
b = markup.getbyte(pos)
break unless b == 32 || b == 9 || b == 10 || b == 13
pos += 1
end
return if pos >= len
start = pos
# First char must be [a-zA-Z_]
b = markup.getbyte(pos)
return unless (b >= 97 && b <= 122) || (b >= 65 && b <= 90) || b == 95
pos += 1
# Scan segments: [\w-]* (. [\w-]*)*
while pos < len
b = markup.getbyte(pos)
if (b >= 97 && b <= 122) || (b >= 65 && b <= 90) || (b >= 48 && b <= 57) || b == 95 || b == 45
pos += 1
elsif b == 46 # '.'
pos += 1
# After dot, must have [a-zA-Z_]
return if pos >= len
b = markup.getbyte(pos)
return unless (b >= 97 && b <= 122) || (b >= 65 && b <= 90) || b == 95
pos += 1
else
break
end
end
content_end = pos
# Skip trailing whitespace
while pos < len
b = markup.getbyte(pos)
return unless b == 32 || b == 9 || b == 10 || b == 13
pos += 1
end
# Must have consumed everything
return unless pos == len
if start == 0 && content_end == len
markup
else
markup.byteslice(start, content_end - start)
end
end
# Cache for [filtername, EMPTY_ARRAY] tuples — avoids repeated array creation
NO_ARG_FILTER_CACHE = Hash.new { |h, k| h[k] = [k, Const::EMPTY_ARRAY].freeze }
FilterMarkupRegex = /#{FilterSeparator}\s*(.*)/om
FilterParser = /(?:\s+|#{QuotedFragment}|#{ArgumentSeparator})+/o
FilterArgsRegex = /(?:#{FilterArgumentSeparator}|#{ArgumentSeparator})\s*((?:\w+\s*\:\s*)?#{QuotedFragment})/o
@@ -30,7 +92,225 @@ module Liquid
@parse_context = parse_context
@line_number = parse_context.line_number
strict_parse_with_error_mode_fallback(markup)
# Fast path: try to parse without going through Lexer → Parser
# Skip for strict2/rigid modes which require different parsing
# Fast path only for lax/warn modes — strict modes need full error checking
error_mode = parse_context.error_mode
if error_mode == :strict2 || error_mode == :rigid || error_mode == :strict || !try_fast_parse(markup, parse_context)
strict_parse_with_error_mode_fallback(markup)
end
end
private def try_fast_parse(markup, parse_context)
len = markup.bytesize
return false if len == 0
# Skip leading whitespace
pos = 0
while pos < len
b = markup.getbyte(pos)
break unless b == 32 || b == 9 || b == 10 || b == 13
pos += 1
end
return false if pos >= len
b = markup.getbyte(pos)
if b == 39 || b == 34 # single or double quote
# Quoted string literal: scan to matching close quote
quote = b
name_start = pos
pos += 1
pos += 1 while pos < len && markup.getbyte(pos) != quote
pos += 1 if pos < len # skip closing quote
name_end = pos
elsif (b >= 97 && b <= 122) || (b >= 65 && b <= 90) || b == 95
# Identifier: scan [\w-]*(\.[\w-]*)*
name_start = pos
pos += 1
while pos < len
b = markup.getbyte(pos)
if (b >= 97 && b <= 122) || (b >= 65 && b <= 90) || (b >= 48 && b <= 57) || b == 95 || b == 45
pos += 1
elsif b == 46 # '.'
pos += 1
return false if pos >= len
b = markup.getbyte(pos)
return false unless (b >= 97 && b <= 122) || (b >= 65 && b <= 90) || b == 95
pos += 1
else
break
end
end
name_end = pos
else
return false
end
# Skip whitespace after name
while pos < len
b = markup.getbyte(pos)
break unless b == 32 || b == 9 || b == 10 || b == 13
pos += 1
end
# Resolve the name expression — avoid byteslice when markup is already the name
expr_markup = if name_start == 0 && name_end == len
markup # no whitespace, no filters — reuse the string
else
markup.byteslice(name_start, name_end - name_start)
end
cache = parse_context.expression_cache
ss = parse_context.string_scanner
first_byte = expr_markup.getbyte(0)
@name = if first_byte == 39 || first_byte == 34 # quoted string
# Strip quotes for string literal
expr_markup.byteslice(1, expr_markup.bytesize - 2)
elsif Expression::LITERALS.key?(expr_markup)
Expression::LITERALS[expr_markup]
elsif cache
cache[expr_markup] || (cache[expr_markup] = VariableLookup.parse_simple(expr_markup, ss, cache).freeze)
else
VariableLookup.parse_simple(expr_markup, ss || StringScanner.new(""), nil).freeze
end
# End of markup? No filters.
if pos >= len
@filters = Const::EMPTY_ARRAY
return true
end
# Must be a pipe for filters
return false unless markup.getbyte(pos) == 124 # '|'
# Try fast filter scanning first — handles no-arg and simple-arg filters
# Falls through to Lexer-based parsing for complex cases
@filters = []
filter_pos = pos
while filter_pos < len && markup.getbyte(filter_pos) == 124 # '|'
filter_pos += 1
# Skip whitespace
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) == 32
# Scan filter name
fname_start = filter_pos
b = filter_pos < len ? markup.getbyte(filter_pos) : nil
break unless b && ((b >= 97 && b <= 122) || (b >= 65 && b <= 90) || b == 95)
filter_pos += 1
while filter_pos < len
b = markup.getbyte(filter_pos)
break unless (b >= 97 && b <= 122) || (b >= 65 && b <= 90) || (b >= 48 && b <= 57) || b == 95 || b == 45
filter_pos += 1
end
filtername = markup.byteslice(fname_start, filter_pos - fname_start)
# Skip whitespace
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) == 32
# Has arguments — try fast scanning for positional args
if filter_pos < len && markup.getbyte(filter_pos) == 58 # ':'
filter_pos += 1 # skip ':'
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) == 32
filter_args = []
fall_to_lexer = false
loop do
arg_start = filter_pos
b = filter_pos < len ? markup.getbyte(filter_pos) : nil
if b == 39 || b == 34 # quoted string
quote = b
filter_pos += 1
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) != quote
filter_pos += 1 if filter_pos < len # skip closing quote
filter_args << markup.byteslice(arg_start + 1, filter_pos - arg_start - 2)
elsif b && ((b >= 48 && b <= 57) || (b == 45 && filter_pos + 1 < len && markup.getbyte(filter_pos + 1) >= 48 && markup.getbyte(filter_pos + 1) <= 57))
# Number
filter_pos += 1 if b == 45
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) >= 48 && markup.getbyte(filter_pos) <= 57
if filter_pos < len && markup.getbyte(filter_pos) == 46 # float
filter_pos += 1
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) >= 48 && markup.getbyte(filter_pos) <= 57
end
num_str = markup.byteslice(arg_start, filter_pos - arg_start)
filter_args << (num_str.include?('.') ? num_str.to_f : num_str.to_i)
elsif b && ((b >= 97 && b <= 122) || (b >= 65 && b <= 90) || b == 95)
# Identifier
id_start = filter_pos
filter_pos += 1
while filter_pos < len
b2 = markup.getbyte(filter_pos)
break unless (b2 >= 97 && b2 <= 122) || (b2 >= 65 && b2 <= 90) || (b2 >= 48 && b2 <= 57) || b2 == 95 || b2 == 45 || b2 == 46
filter_pos += 1
end
filter_pos += 1 if filter_pos < len && markup.getbyte(filter_pos) == 63
# Check if keyword arg (id followed by ':')
kw_check = filter_pos
kw_check += 1 while kw_check < len && markup.getbyte(kw_check) == 32
if kw_check < len && markup.getbyte(kw_check) == 58
fall_to_lexer = true
break
end
id_markup = markup.byteslice(id_start, filter_pos - id_start)
filter_args << Expression.parse(id_markup, parse_context.string_scanner, parse_context.expression_cache)
else
fall_to_lexer = true
break
end
# Skip whitespace after arg
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) == 32
# Comma = more args; pipe/end = done
if filter_pos < len && markup.getbyte(filter_pos) == 44
filter_pos += 1
filter_pos += 1 while filter_pos < len && markup.getbyte(filter_pos) == 32
else
break
end
end
if fall_to_lexer
# Complex filter — fall to Lexer for this and remaining filters
rest_start = fname_start
rest_start -= 1 while rest_start > pos && markup.getbyte(rest_start) != 124
rest_markup = markup.byteslice(rest_start, len - rest_start)
p = parse_context.new_parser(rest_markup)
while p.consume?(:pipe)
fn = p.consume(:id)
fa = p.consume?(:colon) ? parse_filterargs(p) : Const::EMPTY_ARRAY
@filters << lax_parse_filter_expressions(fn, fa)
end
p.consume(:end_of_string)
@filters = Const::EMPTY_ARRAY if @filters.empty?
return true
end
@filters << [filtername, filter_args]
else
# No args — add as simple filter
@filters << NO_ARG_FILTER_CACHE[filtername]
end
# Skip whitespace between filters
filter_pos += 1 while filter_pos < len && (markup.getbyte(filter_pos) == 32 || markup.getbyte(filter_pos) == 9 || markup.getbyte(filter_pos) == 10 || markup.getbyte(filter_pos) == 13)
end
# Must have consumed everything
return false if filter_pos < len
@filters = Const::EMPTY_ARRAY if @filters.empty?
true
rescue SyntaxError
# If fast parse fails, fall back to full parse
@name = nil
@filters = nil
false
end
def raw
@@ -42,7 +322,7 @@ module Liquid
end
def lax_parse(markup)
@filters = []
@filters = Const::EMPTY_ARRAY
return unless markup =~ MarkupWithQuotedFragment
name_markup = Regexp.last_match(1)
@@ -54,19 +334,21 @@ module Liquid
next unless f =~ /\w+/
filtername = Regexp.last_match(0)
filterargs = f.scan(FilterArgsRegex).flatten
@filters = [] if @filters.frozen?
@filters << lax_parse_filter_expressions(filtername, filterargs)
end
end
end
def strict_parse(markup)
@filters = []
@filters = Const::EMPTY_ARRAY
p = @parse_context.new_parser(markup)
return if p.look(:end_of_string)
@name = parse_context.safe_parse_expression(p)
while p.consume?(:pipe)
@filters = [] if @filters.frozen?
filtername = p.consume(:id)
filterargs = p.consume?(:colon) ? parse_filterargs(p) : Const::EMPTY_ARRAY
@filters << lax_parse_filter_expressions(filtername, filterargs)
@@ -75,13 +357,16 @@ module Liquid
end
def strict2_parse(markup)
@filters = []
@filters = Const::EMPTY_ARRAY
p = @parse_context.new_parser(markup)
return if p.look(:end_of_string)
@name = parse_context.safe_parse_expression(p)
@filters << strict2_parse_filter_expressions(p) while p.consume?(:pipe)
while p.consume?(:pipe)
@filters = [] if @filters.frozen?
@filters << strict2_parse_filter_expressions(p)
end
p.consume(:end_of_string)
end
@@ -97,24 +382,37 @@ module Liquid
obj = context.evaluate(@name)
@filters.each do |filter_name, filter_args, filter_kwargs|
filter_args = evaluate_filter_expressions(context, filter_args, filter_kwargs)
obj = context.invoke(filter_name, obj, *filter_args)
if filter_args.empty? && !filter_kwargs
obj = context.invoke_single(filter_name, obj)
elsif !filter_kwargs && filter_args.length == 1
# Single positional arg — most common after no-arg
obj = context.invoke_two(filter_name, obj, context.evaluate(filter_args[0]))
else
filter_args = evaluate_filter_expressions(context, filter_args, filter_kwargs)
obj = context.invoke(filter_name, obj, *filter_args)
end
end
context.apply_global_filter(obj)
end
def render_to_output_buffer(context, output)
obj = render(context)
# Fast path: no filters and no global filter
obj = if @filters.empty? && context.global_filter.nil?
context.evaluate(@name)
else
render(context)
end
render_obj_to_output(obj, output)
output
end
def render_obj_to_output(obj, output)
case obj
when NilClass
if obj.instance_of?(String)
output << obj
elsif obj.nil?
# Do nothing
when Array
elsif obj.instance_of?(Array)
obj.each do |o|
render_obj_to_output(o, output)
end
@@ -128,7 +426,7 @@ module Liquid
end
def disabled_tags
[]
Const::EMPTY_ARRAY
end
private
@@ -137,7 +435,8 @@ module Liquid
filter_args = []
keyword_args = nil
unparsed_args.each do |a|
if (matches = a.match(JustTagAttributes))
# Fast check: keyword args must contain ':'
if a.include?(':') && (matches = a.match(JustTagAttributes))
keyword_args ||= {}
keyword_args[matches[1]] = parse_context.parse_expression(matches[2])
else
@@ -190,15 +489,19 @@ module Liquid
end
def evaluate_filter_expressions(context, filter_args, filter_kwargs)
parsed_args = filter_args.map { |expr| context.evaluate(expr) }
if filter_kwargs
parsed_args = filter_args.map { |expr| context.evaluate(expr) }
parsed_kwargs = {}
filter_kwargs.each do |key, expr|
parsed_kwargs[key] = context.evaluate(expr)
end
parsed_args << parsed_kwargs
parsed_args
elsif filter_args.empty?
Const::EMPTY_ARRAY
else
filter_args.map { |expr| context.evaluate(expr) }
end
parsed_args
end
class ParseTreeVisitor < Liquid::ParseTreeVisitor
+129 -13
View File
@@ -10,8 +10,108 @@ module Liquid
new(markup, string_scanner, cache)
end
def initialize(markup, string_scanner = StringScanner.new(""), cache = nil)
lookups = markup.scan(VariableParser)
# Fast parse that skips simple_lookup? check — caller guarantees simple identifier chain
def self.parse_simple(markup, string_scanner = nil, cache = nil)
new(markup, string_scanner, cache, true)
end
# Fast manual scanner replacing markup.scan(VariableParser)
# VariableParser = /\[(?>[^\[\]]+|\g<0>)*\]|[\w-]+\??/
# Splits "product.variants[0].title" into ["product", "variants", "[0]", "title"]
def self.scan_variable(markup)
result = []
pos = 0
len = markup.bytesize
while pos < len
byte = markup.getbyte(pos)
if byte == 91 # '['
# Scan balanced brackets
depth = 1
start = pos
pos += 1
while pos < len && depth > 0
b = markup.getbyte(pos)
if b == 91
depth += 1
elsif b == 93
depth -= 1
end
pos += 1
end
if depth == 0
result << markup.byteslice(start, pos - start)
else
# Unbalanced bracket - skip '[' and continue
pos = start + 1
end
elsif byte == 46 # '.'
pos += 1
elsif (byte >= 97 && byte <= 122) || (byte >= 65 && byte <= 90) || (byte >= 48 && byte <= 57) || byte == 95 || byte == 45 # \w or -
start = pos
pos += 1
while pos < len
b = markup.getbyte(pos)
break unless (b >= 97 && b <= 122) || (b >= 65 && b <= 90) || (b >= 48 && b <= 57) || b == 95 || b == 45
pos += 1
end
# Check trailing '?'
if pos < len && markup.getbyte(pos) == 63
pos += 1
end
result << markup.byteslice(start, pos - start)
else
pos += 1
end
end
result
end
# Check if markup is a simple identifier chain: [\w-]+\??(.[\w-]+\??)*
# Uses C-level match? — 8x faster than Ruby byte scanning
SIMPLE_LOOKUP_RE = /\A[\w-]+\??(?:\.[\w-]+\??)*\z/
def self.simple_lookup?(markup)
markup.bytesize > 0 && markup.match?(SIMPLE_LOOKUP_RE)
end
def initialize(markup, string_scanner = StringScanner.new(""), cache = nil, simple = false)
# Fast path: simple identifier chain without brackets
if simple || self.class.simple_lookup?(markup)
dot_pos = markup.index('.')
if dot_pos.nil?
@name = markup
@lookups = Const::EMPTY_ARRAY
@command_flags = 0
return
end
@name = markup.byteslice(0, dot_pos)
# Build lookups array from remaining dot-separated segments
lookups = []
@command_flags = 0
pos = dot_pos + 1
len = markup.bytesize
while pos < len
seg_start = pos
while pos < len
b = markup.getbyte(pos)
break if b == 46 # '.'
pos += 1
end
seg = markup.byteslice(seg_start, pos - seg_start)
if COMMAND_METHODS.include?(seg)
@command_flags |= 1 << lookups.length
end
lookups << seg
pos += 1 # skip dot
end
@lookups = lookups
return
end
lookups = self.class.scan_variable(markup)
name = lookups.shift
if name&.start_with?('[') && name&.end_with?(']')
@@ -49,26 +149,45 @@ module Liquid
object = context.find_variable(name)
@lookups.each_index do |i|
key = context.evaluate(@lookups[i])
lookup = @lookups[i]
key = lookup.instance_of?(String) ? lookup : context.evaluate(lookup)
# Cast "key" to its liquid value to enable it to act as a primitive value
key = Liquid::Utils.to_liquid_value(key)
# Fast path: strings and integers (most common key types) don't need conversion
unless key.instance_of?(String) || key.instance_of?(Integer)
key = Liquid::Utils.to_liquid_value(key)
end
# If object is a hash- or array-like object we look for the
# presence of the key and if its available we return it
if object.respond_to?(:[]) &&
((object.respond_to?(:key?) && object.key?(key)) ||
(object.respond_to?(:fetch) && key.is_a?(Integer)))
if object.instance_of?(Hash) ? object.key?(key) :
(object.respond_to?(:[]) &&
((object.respond_to?(:key?) && object.key?(key)) ||
(object.respond_to?(:fetch) && key.is_a?(Integer))))
# if its a proc we will replace the entry with the proc
res = context.lookup_and_evaluate(object, key)
object = res.to_liquid
object = context.lookup_and_evaluate(object, key)
# Skip to_liquid for common primitive types (they return self)
unless object.instance_of?(String) || object.instance_of?(Integer) || object.instance_of?(Float) ||
object.instance_of?(Array) || object.instance_of?(Hash) || object.nil?
object = object.to_liquid
object.context = context if object.respond_to?(:context=)
end
# Some special cases. If the part wasn't in square brackets and
# no key with the same name was found we interpret following calls
# as commands and call them on the current object
elsif lookup_command?(i) && object.respond_to?(key)
object = object.send(key).to_liquid
object = object.send(key)
unless object.instance_of?(String) || object.instance_of?(Integer) || object.instance_of?(Array) || object.nil?
object = object.to_liquid
object.context = context if object.respond_to?(:context=)
end
# Handle string first/last like ActiveSupport does (returns first/last character)
# ActiveSupport returns "" for empty strings, not nil
elsif lookup_command?(i) && object.is_a?(String) && (key == "first" || key == "last")
object = key == "first" ? (object[0] || "") : (object[-1] || "")
# No key was present with the desired value and it wasn't one of the directly supported
# keywords either. The only thing we got left is to return nil or
@@ -77,9 +196,6 @@ module Liquid
return nil unless context.strict_variables
raise Liquid::UndefinedVariable, "undefined variable #{key}"
end
# If we are dealing with a drop here we have to
object.context = context if object.respond_to?(:context=)
end
object
+62
View File
@@ -0,0 +1,62 @@
# frozen_string_literal: true
# Quick benchmark for autoresearch: measures parse µs, render µs, and object allocations
# Outputs machine-readable metrics to stdout
require_relative 'theme_runner'
RubyVM::YJIT.enable if defined?(RubyVM::YJIT)
runner = ThemeRunner.new
# Warmup — enough iterations for YJIT to fully optimize hot paths
20.times { runner.compile }
20.times { runner.render }
GC.start
GC.compact if GC.respond_to?(:compact)
# Measure parse
parse_times = []
10.times do
GC.disable
t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
runner.compile
t1 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
GC.enable
GC.start
parse_times << (t1 - t0) * 1_000_000 # µs
end
# Measure render
render_times = []
10.times do
GC.disable
t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
runner.render
t1 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
GC.enable
GC.start
render_times << (t1 - t0) * 1_000_000 # µs
end
# Measure object allocations for one parse+render cycle
require 'objspace'
GC.start
GC.disable
before = ObjectSpace.count_objects.values_at(:TOTAL).first - ObjectSpace.count_objects.values_at(:FREE).first
runner.compile
runner.render
after = ObjectSpace.count_objects.values_at(:TOTAL).first - ObjectSpace.count_objects.values_at(:FREE).first
GC.enable
allocations = after - before
parse_us = parse_times.min.round(0)
render_us = render_times.min.round(0)
combined_us = parse_us + render_us
puts "RESULTS"
puts "parse_us=#{parse_us}"
puts "render_us=#{render_us}"
puts "combined_us=#{combined_us}"
puts "allocations=#{allocations}"
+36
View File
@@ -0,0 +1,36 @@
# frozen_string_literal: true
# Liquid Spec Adapter for Shopify/liquid (Ruby reference implementation)
#
# Run with: bundle exec liquid-spec run spec/ruby_liquid.rb
$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
require 'liquid'
LiquidSpec.configure do |config|
# Run core Liquid specs
config.features = [:core]
end
# Compile a template string into a Liquid::Template
LiquidSpec.compile do |ctx, source, options|
ctx[:template] = Liquid::Template.parse(source, **options)
end
# Render a compiled template with the given context
# @param ctx [Hash] adapter context containing :template
# @param assigns [Hash] environment variables
# @param options [Hash] :registers, :strict_errors, :exception_renderer
LiquidSpec.render do |ctx, assigns, options|
registers = Liquid::Registers.new(options[:registers] || {})
context = Liquid::Context.build(
static_environments: assigns,
registers: registers,
rethrow_errors: options[:strict_errors],
)
context.exception_renderer = options[:exception_renderer] if options[:exception_renderer]
ctx[:template].render(context)
end
+34
View File
@@ -0,0 +1,34 @@
# frozen_string_literal: true
# Liquid Spec Adapter for Shopify/liquid with lax parsing mode
#
# Run with: bundle exec liquid-spec run spec/ruby_liquid_lax.rb
$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
require 'liquid'
LiquidSpec.configure do |config|
config.features = [:core, :lax_parsing]
end
# Compile a template string into a Liquid::Template
LiquidSpec.compile do |ctx, source, options|
# Force lax mode
options = options.merge(error_mode: :lax)
ctx[:template] = Liquid::Template.parse(source, **options)
end
# Render a compiled template with the given context
LiquidSpec.render do |ctx, assigns, options|
registers = Liquid::Registers.new(options[:registers] || {})
context = Liquid::Context.build(
static_environments: assigns,
registers: registers,
rethrow_errors: options[:strict_errors],
)
context.exception_renderer = options[:exception_renderer] if options[:exception_renderer]
ctx[:template].render(context)
end
+37
View File
@@ -0,0 +1,37 @@
# frozen_string_literal: true
# Liquid Spec Adapter for Shopify/liquid with ActiveSupport loaded
#
# Run with: bundle exec liquid-spec run spec/ruby_liquid_with_active_support.rb
$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
require 'active_support/all'
require 'liquid'
LiquidSpec.configure do |config|
# Run core Liquid specs plus ActiveSupport SafeBuffer tests
config.features = [:core, :activesupport]
end
# Compile a template string into a Liquid::Template
LiquidSpec.compile do |ctx, source, options|
ctx[:template] = Liquid::Template.parse(source, **options)
end
# Render a compiled template with the given context
# @param ctx [Hash] adapter context containing :template
# @param assigns [Hash] environment variables
# @param options [Hash] :registers, :strict_errors, :exception_renderer
LiquidSpec.render do |ctx, assigns, options|
registers = Liquid::Registers.new(options[:registers] || {})
context = Liquid::Context.build(
static_environments: assigns,
registers: registers,
rethrow_errors: options[:strict_errors],
)
context.exception_renderer = options[:exception_renderer] if options[:exception_renderer]
ctx[:template].render(context)
end
+41
View File
@@ -0,0 +1,41 @@
# frozen_string_literal: true
# Liquid Spec Adapter for Shopify/liquid with YJIT + strict mode + ActiveSupport
#
# Run with: bundle exec liquid-spec run spec/ruby_liquid_yjit.rb
$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
# Enable YJIT if available
if defined?(RubyVM::YJIT) && RubyVM::YJIT.respond_to?(:enable)
RubyVM::YJIT.enable
end
require 'active_support/all'
require 'liquid'
LiquidSpec.configure do |config|
config.features = [:core, :activesupport]
end
# Compile a template string into a Liquid::Template
LiquidSpec.compile do |ctx, source, options|
# Force strict mode
options = { error_mode: :strict }.merge(options)
ctx[:template] = Liquid::Template.parse(source, **options)
end
# Render a compiled template with the given context
LiquidSpec.render do |ctx, assigns, options|
registers = Liquid::Registers.new(options[:registers] || {})
context = Liquid::Context.build(
static_environments: assigns,
registers: registers,
rethrow_errors: options[:strict_errors],
)
context.exception_renderer = options[:exception_renderer] if options[:exception_renderer]
ctx[:template].render(context)
end
+2 -8
View File
@@ -88,17 +88,11 @@ class HashRenderingTest < Minitest::Test
end
def test_rendering_hash_with_custom_to_s_method_uses_custom_to_s
my_hash = Class.new(Hash) do
def to_s
"kewl"
end
end.new
assert_template_result("kewl", "{{ my_hash }}", { "my_hash" => my_hash })
assert_template_result("kewl", "{{ my_hash }}", { "my_hash" => HashWithCustomToS.new })
end
def test_rendering_hash_without_custom_to_s_uses_default_inspect
my_hash = Class.new(Hash).new
my_hash = HashWithoutCustomToS.new
my_hash[:foo] = :bar
assert_template_result("{:foo=>:bar}", "{{ my_hash }}", { "my_hash" => my_hash })
+2 -2
View File
@@ -59,7 +59,7 @@ class SecurityTest < Minitest::Test
GC.start
assert_equal([], (Symbol.all_symbols - current_symbols))
assert_equal([], Symbol.all_symbols - current_symbols)
end
def test_does_not_add_drop_methods_to_symbol_table
@@ -70,7 +70,7 @@ class SecurityTest < Minitest::Test
assert_equal("", Template.parse("{{ drop.custom_method_2 }}", assigns).render!)
assert_equal("", Template.parse("{{ drop.custom_method_3 }}", assigns).render!)
assert_equal([], (Symbol.all_symbols - current_symbols))
assert_equal([], Symbol.all_symbols - current_symbols)
end
def test_max_depth_nested_blocks_does_not_raise_exception
+44 -9
View File
@@ -116,7 +116,7 @@ class StandardFiltersTest < Minitest::Test
end
def test_slice_on_arrays
input = 'foobar'.split(//)
input = 'foobar'.split('')
assert_equal(%w(o o b), @filters.slice(input, 1, 3))
assert_equal(%w(o o b a r), @filters.slice(input, 1, 1000))
assert_equal(%w(), @filters.slice(input, 1, 0))
@@ -164,6 +164,13 @@ class StandardFiltersTest < Minitest::Test
assert_equal(['A', 'Z'], @filters.split('A1Z', 1))
end
def test_squish_filter
assert_equal("foo bar boo", Liquid::Template.parse(%({{ " foo bar
\t boo " | squish }})).render)
assert_equal("", Liquid::Template.parse('{{ nil | squish }}').render)
assert_equal("", Liquid::Template.parse('{{ " " | squish }}').render)
end
def test_escape
assert_equal('&lt;strong&gt;', @filters.escape('<strong>'))
assert_equal('1', @filters.escape(1))
@@ -294,13 +301,7 @@ class StandardFiltersTest < Minitest::Test
end
def test_join_calls_to_liquid_on_each_element
drop = Class.new(Liquid::Drop) do
def to_liquid
'i did it'
end
end
assert_equal('i did it, i did it', @filters.join([drop.new, drop.new], ", "))
assert_equal('i did it, i did it', @filters.join([CustomToLiquidDrop.new('i did it'), CustomToLiquidDrop.new('i did it')], ", "))
end
def test_sort
@@ -633,6 +634,40 @@ class StandardFiltersTest < Minitest::Test
assert_nil(@filters.last([]))
end
def test_first_last_on_strings
# Ruby's String class does not have first/last methods by default.
# ActiveSupport adds String#first and String#last to return the first/last character.
# Liquid must work without ActiveSupport, so the first/last filters handle strings specially.
#
# This enables template patterns like:
# {{ product.title | first }} => "S" (for "Snowboard")
# {{ customer.name | last }} => "h" (for "Smith")
#
# Note: ActiveSupport returns "" for empty strings, not nil.
assert_equal('f', @filters.first('foo'))
assert_equal('o', @filters.last('foo'))
assert_equal('', @filters.first(''))
assert_equal('', @filters.last(''))
end
def test_first_last_on_unicode_strings
# Unicode strings should return the first/last grapheme cluster (character),
# not the first/last byte. Ruby's String#[] handles this correctly with index 0/-1.
# This ensures international text works properly:
# {{ korean_name | first }} => "고" (not a partial byte sequence)
assert_equal('고', @filters.first('고스트빈'))
assert_equal('빈', @filters.last('고스트빈'))
end
def test_first_last_on_strings_via_template
# Integration test to verify the filter works end-to-end in templates.
# Empty strings return empty output (nil renders as empty string).
assert_template_result('f', '{{ name | first }}', { 'name' => 'foo' })
assert_template_result('o', '{{ name | last }}', { 'name' => 'foo' })
assert_template_result('', '{{ name | first }}', { 'name' => '' })
assert_template_result('', '{{ name | last }}', { 'name' => '' })
end
def test_replace
assert_equal('b b b b', @filters.replace('a a a a', 'a', 'b'))
assert_equal('2 2 2 2', @filters.replace('1 1 1 1', 1, 2))
@@ -1302,7 +1337,7 @@ class StandardFiltersTest < Minitest::Test
assert_equal(1, @filters.sum(input, true))
assert_equal(0.2, @filters.sum(input, 1.0))
assert_equal(-0.3, @filters.sum(input, 1))
assert_equal(0.4, @filters.sum(input, (1..5)))
assert_equal(0.4, @filters.sum(input, 1..5))
assert_equal(0, @filters.sum(input, nil))
assert_equal(0, @filters.sum(input, ""))
end
+1 -1
View File
@@ -117,7 +117,7 @@ class StandardTagTest < Minitest::Test
assigns = { 'condition' => "bad string here" }
assert_template_result(
'',
'{% case condition %}{% when "string here" %} hit {% endcase %}',\
'{% case condition %}{% when "string here" %} hit {% endcase %}',
assigns,
)
end
+1 -1
View File
@@ -133,7 +133,7 @@ class TemplateTest < Minitest::Test
assert(t.resource_limits.reached?)
t.resource_limits.render_score_limit = 200
assert_equal((" foo " * 100), t.render!)
assert_equal(" foo " * 100, t.render!)
refute_nil(t.resource_limits.render_score)
end
+20
View File
@@ -199,6 +199,26 @@ class ErrorDrop < Liquid::Drop
end
end
class CustomToLiquidDrop < Liquid::Drop
def initialize(value)
@value = value
super()
end
def to_liquid
@value
end
end
class HashWithCustomToS < Hash
def to_s
"kewl"
end
end
class HashWithoutCustomToS < Hash
end
class StubFileSystem
attr_reader :file_read_count
+168 -2
View File
@@ -161,8 +161,8 @@ class ConditionUnitTest < Minitest::Test
assert_equal(true, Condition.new(1, '==', 1).evaluate)
end
expected = "DEPRECATION WARNING: Condition#evaluate without a context argument is deprecated" \
" and will be removed from Liquid 6.0.0."
expected = "DEPRECATION WARNING: Condition#evaluate without a context argument is deprecated " \
"and will be removed from Liquid 6.0.0."
assert_includes(err.lines.map(&:strip), expected)
end
@@ -197,6 +197,172 @@ class ConditionUnitTest < Minitest::Test
assert_equal(['title'], result.lookups)
end
# Tests for blank? comparison without ActiveSupport
#
# Ruby's standard library does not include blank? on String, Array, Hash, etc.
# ActiveSupport adds blank? but Liquid must work without it. These tests verify
# that Liquid implements blank? semantics internally for use in templates like:
# {% if x == blank %}...{% endif %}
#
# The blank? semantics match ActiveSupport's behavior:
# - nil and false are blank
# - Strings are blank if empty or contain only whitespace
# - Arrays and Hashes are blank if empty
# - true and numbers are never blank
def test_blank_with_whitespace_string
# Template authors expect " " to be blank since it has no visible content.
# This matches ActiveSupport's String#blank? which returns true for whitespace-only strings.
@context['whitespace'] = ' '
blank_literal = Condition.class_variable_get(:@@method_literals)['blank']
assert_evaluates_true(VariableLookup.new('whitespace'), '==', blank_literal)
end
def test_blank_with_empty_string
# An empty string has no content, so it should be considered blank.
# This is the most basic case of a blank string.
@context['empty_string'] = ''
blank_literal = Condition.class_variable_get(:@@method_literals)['blank']
assert_evaluates_true(VariableLookup.new('empty_string'), '==', blank_literal)
end
def test_blank_with_empty_array
# Empty arrays have no elements, so they are blank.
# Useful for checking if a collection has items: {% if products == blank %}
@context['empty_array'] = []
blank_literal = Condition.class_variable_get(:@@method_literals)['blank']
assert_evaluates_true(VariableLookup.new('empty_array'), '==', blank_literal)
end
def test_blank_with_empty_hash
# Empty hashes have no key-value pairs, so they are blank.
# Useful for checking if settings/options exist: {% if settings == blank %}
@context['empty_hash'] = {}
blank_literal = Condition.class_variable_get(:@@method_literals)['blank']
assert_evaluates_true(VariableLookup.new('empty_hash'), '==', blank_literal)
end
def test_blank_with_nil
# nil represents "nothing" and is the canonical blank value.
# Unassigned variables resolve to nil, so this enables: {% if missing_var == blank %}
@context['nil_value'] = nil
blank_literal = Condition.class_variable_get(:@@method_literals)['blank']
assert_evaluates_true(VariableLookup.new('nil_value'), '==', blank_literal)
end
def test_blank_with_false
# false is considered blank to match ActiveSupport semantics.
# This allows {% if some_flag == blank %} to work when flag is false.
@context['false_value'] = false
blank_literal = Condition.class_variable_get(:@@method_literals)['blank']
assert_evaluates_true(VariableLookup.new('false_value'), '==', blank_literal)
end
def test_not_blank_with_true
# true is a definite value, not blank.
# Ensures {% if flag == blank %} works correctly for boolean flags.
@context['true_value'] = true
blank_literal = Condition.class_variable_get(:@@method_literals)['blank']
assert_evaluates_false(VariableLookup.new('true_value'), '==', blank_literal)
end
def test_not_blank_with_number
# Numbers (including zero) are never blank - they represent actual values.
# 0 is a valid quantity, not the absence of a value.
@context['number'] = 42
blank_literal = Condition.class_variable_get(:@@method_literals)['blank']
assert_evaluates_false(VariableLookup.new('number'), '==', blank_literal)
end
def test_not_blank_with_string_content
# A string with actual content is not blank.
# This is the expected behavior for most template string comparisons.
@context['string'] = 'hello'
blank_literal = Condition.class_variable_get(:@@method_literals)['blank']
assert_evaluates_false(VariableLookup.new('string'), '==', blank_literal)
end
def test_not_blank_with_non_empty_array
# An array with elements has content, so it's not blank.
# Enables patterns like {% unless products == blank %}Show products{% endunless %}
@context['array'] = [1, 2, 3]
blank_literal = Condition.class_variable_get(:@@method_literals)['blank']
assert_evaluates_false(VariableLookup.new('array'), '==', blank_literal)
end
def test_not_blank_with_non_empty_hash
# A hash with key-value pairs has content, so it's not blank.
# Useful for checking if configuration exists: {% if config != blank %}
@context['hash'] = { 'a' => 1 }
blank_literal = Condition.class_variable_get(:@@method_literals)['blank']
assert_evaluates_false(VariableLookup.new('hash'), '==', blank_literal)
end
# Tests for empty? comparison without ActiveSupport
#
# empty? is distinct from blank? - it only checks if a collection has zero elements.
# For strings, empty? checks length == 0, NOT whitespace content.
# Ruby's standard library has empty? on String, Array, and Hash, but Liquid
# provides a fallback implementation for consistency.
def test_empty_with_empty_string
# An empty string ("") has length 0, so it's empty.
# Different from blank - empty is a stricter check.
@context['empty_string'] = ''
empty_literal = Condition.class_variable_get(:@@method_literals)['empty']
assert_evaluates_true(VariableLookup.new('empty_string'), '==', empty_literal)
end
def test_empty_with_whitespace_string_not_empty
# Whitespace strings have length > 0, so they are NOT empty.
# This is the key difference between empty and blank:
# " ".empty? => false, but " ".blank? => true
@context['whitespace'] = ' '
empty_literal = Condition.class_variable_get(:@@method_literals)['empty']
assert_evaluates_false(VariableLookup.new('whitespace'), '==', empty_literal)
end
def test_empty_with_empty_array
# An array with no elements is empty.
# [].empty? => true
@context['empty_array'] = []
empty_literal = Condition.class_variable_get(:@@method_literals)['empty']
assert_evaluates_true(VariableLookup.new('empty_array'), '==', empty_literal)
end
def test_empty_with_empty_hash
# A hash with no key-value pairs is empty.
# {}.empty? => true
@context['empty_hash'] = {}
empty_literal = Condition.class_variable_get(:@@method_literals)['empty']
assert_evaluates_true(VariableLookup.new('empty_hash'), '==', empty_literal)
end
def test_nil_is_not_empty
# nil is NOT empty - empty? checks if a collection has zero elements.
# nil is not a collection, so it cannot be empty.
# This differs from blank: nil IS blank, but nil is NOT empty.
@context['nil_value'] = nil
empty_literal = Condition.class_variable_get(:@@method_literals)['empty']
assert_evaluates_false(VariableLookup.new('nil_value'), '==', empty_literal)
end
private
def assert_evaluates_true(left, op, right)
+1 -1
View File
@@ -8,7 +8,7 @@ class StrainerTemplateUnitTest < Minitest::Test
def test_add_filter_when_wrong_filter_class
c = Context.new
s = c.strainer
wrong_filter = ->(v) { v.reverse }
wrong_filter = lambda(&:reverse)
exception = assert_raises(TypeError) do
s.class.add_filter(wrong_filter)
+20
View File
@@ -82,6 +82,26 @@ class CaseTagUnitTest < Minitest::Test
end
end
def test_case_when_empty
template = <<~LIQUID
{%- case x -%}
{%- when 2 or empty -%}
2 or empty
{%- else -%}
not 2 or empty
{%- endcase -%}
LIQUID
with_error_modes(:lax, :strict, :strict2) do
assert_template_result("2 or empty", template, { 'x' => 2 })
assert_template_result("2 or empty", template, { 'x' => {} })
assert_template_result("2 or empty", template, { 'x' => [] })
assert_template_result("not 2 or empty", template, { 'x' => { 'a' => 'b' } })
assert_template_result("not 2 or empty", template, { 'x' => ['a'] })
assert_template_result("not 2 or empty", template, { 'x' => 4 })
end
end
def test_case_with_invalid_expression
template = <<~LIQUID
{%- case foo=>bar -%}