Chris Pak
731f64deef
Improves performance on hot parse and render paths
...
Splits the render loop in block_body.rb on the loop-invariant check_write
condition. The common case (no resource limits) now pays zero branch cost
per node.
Rewrites truncatewords in standardfilters.rb to scan word positions into a
flat int array and builds the result string only when truncation is confirmed.
No string allocation in the common no-truncation case beyond the array itself.
Simplifies rest_blank? in cursor.rb: replaces manual save/skip/restore of
StringScanner position with [email protected] ?(/\S/). exist? does not advance
position; returns nil when no non-whitespace remains; handles EOS correctly.
Removes the nl newline counter from skip_ws in cursor.rb -- all callers
discarded the return value. NL now handled in the same when-branch as the
other whitespace bytes.
2026-04-04 22:14:19 -07:00
Chris Pak
84779f8a28
Removes dead code, tightens idioms, adds clarifying comments
...
Dead code removed:
cursor.rb: parse_for_markup + attr_reader :for_var/collection/reversed
(zero callers in lib/; method also incomplete -- no limit/offset)
for.rb: Syntax regex, REVERSED_BYTES constant (both unreferenced by
lax_parse and strict_parse)
block_body.rb: BLANK_STRING_REGEX (exact duplicate of WhitespaceOrNothing)
Idioms:
expression.rb: parse_number returns nil on failure, not false
(Ruby convention for 'no result'; callers use if (num = parse_number))
block_body.rb: freeze (idempotent by spec); whitespace_handler marked private;
render_node gets rationale comment; redundant comments collapsed
variable_lookup.rb: COMMAND_METHODS -> %w[]; initialize loop uses each_with_index;
removes redundant &. on second clause of &&
for.rb: strict2_parse -> alias_method; nil-guard if/else -> ternary;
render_else -> ternary; include? checks unconsumed rest only
condition.rb: loop/break chain -> while condition.child_relation
utils.rb: tightens slice_collection_using_each loop
Comments:
variable.rb: backward pipe-walk algorithm; SPACE-only whitespace asymmetry
cursor.rb: COMPARISON_OPS identity-map exists for frozen string interning
if.rb: include? pre-check is both correctness guard and perf gate
for.rb: cursor->regex fallback for limit:/offset: attributes
strainer_template.rb: __LINE__+1 limitation in module_eval loop
2026-04-04 22:13:43 -07:00
Chris Pak
ba11b85875
Decomposes, extracts, and names the important concepts
...
Decomposes the 211-line try_fast_parse monolith in variable.rb into four
named private methods -- fast_scan_name, fast_resolve_name, fast_scan_filters,
fast_scan_filter_args -- and simplifies simple_variable_markup from a 55-line
byte scanner to a 10-line regex with fast pre-checks.
Generates invoke_single/invoke_two via module_eval in strainer_template.rb
and context.rb instead of duplicating near-identical method bodies. The two
arity variants differ only in parameter lists; generating them makes the
pattern explicit and eliminates copy-paste drift.
Extracts identical 4-line text-token handling block in block_body.rb into
private append_text_token(token, parse_context). The block appeared in
both the stray-{ fallback and the plain-text branch of parse_for_document.
Extracts accessible?(object, key) predicate from the 4-line inline ternary
in variable_lookup.rb that checked hash/array key presence; extracts
liquidize(object, context) from the duplicated to_liquid + context= wiring
that appeared in both the key-found and command-method branches.
Extracts find_in_envs(envs, key, raise_on_not_found:) from
try_variable_find_in_environments in context.rb, which looped over
@environments then @static_environments with identical loop bodies.
Adds 28 fast-path equivalence tests for Variable (variable_fast_parse_test.rb).
2026-04-04 22:13:43 -07:00
Chris Pak
03e5e29b0b
Adds ByteTables, moves cursor load order, consolidates byte constants
...
Adds ByteTables (lib/liquid/byte_tables.rb): four frozen 256-entry boolean
lookup arrays replacing inline byte-range comparisons throughout:
IDENT_START, IDENT_CONT, DIGIT, WHITESPACE
Moves require 'liquid/cursor' to immediately after byte_tables in liquid.rb.
Cursor has zero Liquid dependencies; loading it early lets every subsequent
file reference Cursor:: constants directly.
Removes all local byte-constant definitions that duplicated Cursor::
tokenizer.rb OPEN_CURLEY / CLOSE_CURLEY / PERCENTAGE
block_body.rb OPEN_CURLEY_BYTE / PERCENT_BYTE / DASH_BYTE / CLOSE_CURLEY_BYTE
expression.rb DOT / DASH / ZERO / NINE / INTEGER_REGEX / FLOAT_REGEX
Replaces inline byte-range comparisons with ByteTables lookups in:
cursor.rb, variable_lookup.rb, expression.rb, standardfilters.rb
Removes incidental dead code alongside the constant consolidation:
tokenizer.rb: require 'strscan', unused string_scanner: param,
@ss = nil, .to_s.to_str, 'tokenize if @source' guard
block_body.rb: require 'English'
2026-04-04 22:09:14 -07:00
Chris Pak
d9c42fd2eb
Fixes infinite loop in tokenizer on trailing stray '{'
...
The stray-{ else branch in tokenize_fast had a nested while loop that could
exit via its condition (when next_open >= len) without advancing pos.
The outer while pos < len loop would then find the same { again forever.
Reproduced by: Liquid::Template.parse('a{') -- hangs indefinitely.
Replaces the nested scan loop with two String#byteindex calls to find the
next '{%' and '{{' directly, then takes the minimum. Always O(n), eliminates
the nested loop entirely, and impossible to leave pos stranded.
Adds regression tests covering the three inputs that previously hung
plus adjacent stray-brace cases.
2026-04-04 21:25:59 -07:00
Tobi Lutke and Chris Pak
bc60deb671
update autoresearch experiment log
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
17b691ee32
Replace @blocks.each with while loop in If render — avoids block proc allocation per render\n\nResult: {"status":"keep","combined_µs":3496,"parse_µs":2356,"render_µs":1140,"allocations":24530}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
de95af5414
Inline to_liquid_value in If render — avoids one method dispatch per condition evaluation\n\nResult: {"status":"keep","combined_µs":3459,"parse_µs":2318,"render_µs":1141,"allocations":24647}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
6faa62697b
Replace simple_lookup? byte scan with match? regex — 8x faster per call, cleaner code\n\nResult: {"status":"keep","combined_µs":3489,"parse_µs":2353,"render_µs":1136,"allocations":24647}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
f39fad8934
Condition#evaluate: skip loop block for simple conditions (no child_relation) — saves 235 allocs\n\nResult: {"status":"keep","combined_µs":3445,"parse_µs":2284,"render_µs":1161,"allocations":24647}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
1aa5854f44
Clean confirmation run: 3,314µs (-55% from main), stable\n\nResult: {"status":"keep","combined_µs":3314,"parse_µs":2203,"render_µs":1111,"allocations":24882}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
feb7036664
update autoresearch docs with current progress
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
111aeed8f1
parse_tag_token without StringScanner: pure byte ops avoid reset(token) overhead, -12% combined\n\nResult: {"status":"keep","combined_µs":3350,"parse_µs":2212,"render_µs":1138,"allocations":24882}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
9cbfae22b0
Clean up tokenizer: remove unused StringScanner setup and regex constants\n\nResult: {"status":"keep","combined_µs":3490,"parse_µs":2331,"render_µs":1159,"allocations":24882}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
89076dbf3a
Confirmation run: byteindex tokenizer consistently 3,400-3,600µs\n\nResult: {"status":"keep","combined_µs":3464,"parse_µs":2335,"render_µs":1129,"allocations":24882}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
f2c0fbfa0b
Replace StringScanner tokenizer with String#byteindex — 12% faster parse, no regex overhead for delimiter finding\n\nResult: {"status":"keep","combined_µs":3556,"parse_µs":2388,"render_µs":1168,"allocations":24882}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
78550c0e0d
Baseline: 3,818µs combined, 24,881 allocs\n\nResult: {"status":"keep","combined_µs":3818,"parse_µs":2722,"render_µs":1096,"allocations":24881}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
68d697d1c5
Skip context.evaluate for String lookup keys in VariableLookup — avoids respond_to? dispatch\n\nResult: {"status":"keep","combined_µs":4103,"parse_µs":2881,"render_µs":1222,"allocations":24881}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
a206932c4a
update autoresearch.md with current progress
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
8a39c3aa33
Cache no-arg filter tuples [name, EMPTY_ARRAY] — reuse frozen tuples across templates\n\nResult: {"status":"keep","combined_µs":4147,"parse_µs":2992,"render_µs":1155,"allocations":24881}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
89fcea371e
Replace manual blank_string? with regex match — cleaner code\n\nResult: {"status":"keep","combined_µs":4196,"parse_µs":3042,"render_µs":1154,"allocations":25535}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
de8675f8fb
Skip to_liquid_value for String/Integer keys in VariableLookup — avoids respond_to? dispatch\n\nResult: {"status":"keep","combined_µs":4131,"parse_µs":2893,"render_µs":1238,"allocations":25535}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
e3ba4aa958
Minor cleanup: optimize expect_id with while loop and early return\n\nResult: {"status":"keep","combined_µs":4184,"parse_µs":2921,"render_µs":1263,"allocations":25535}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
c3e55e92d9
Replace manual scan_dotted_id with regex\n\nResult: {"status":"keep","combined_µs":4121,"parse_µs":2812,"render_µs":1309,"allocations":25535}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
0924e59840
Replace manual scan_quoted_string with regex capture groups\n\nResult: {"status":"keep","combined_µs":4102,"parse_µs":2849,"render_µs":1253,"allocations":25535}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
8d1f030825
Replace manual rest_blank? with regex skip + eos? check\n\nResult: {"status":"keep","combined_µs":4047,"parse_µs":2795,"render_µs":1252,"allocations":25535}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
4896d6d497
Replace manual scan_comparison_op with regex — cleaner and avoids byteslice allocation for op strings\n\nResult: {"status":"keep","combined_µs":4007,"parse_µs":2808,"render_µs":1199,"allocations":25535}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
edd8fabb3a
Replace manual scan_fragment/scan_quoted_string_raw/skip_fragment with regex — cleaner, same/better perf\n\nResult: {"status":"keep","combined_µs":4132,"parse_µs":2890,"render_µs":1242,"allocations":25535}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
28ede9c539
Replace manual byte-level scan_number with regex — cleaner code, same performance\n\nResult: {"status":"keep","combined_µs":4184,"parse_µs":2931,"render_µs":1253,"allocations":25535}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
14ac591403
Replace manual byte-level scan_id/skip_id with regex — C-level StringScanner.scan is faster than Ruby-level byte scanning\n\nResult: {"status":"keep","combined_µs":4185,"parse_µs":2943,"render_µs":1242,"allocations":25535}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
3f9b8916b2
Fast-path Hash lookups in VariableLookup#evaluate — skip respond_to? checks for Hash objects\n\nResult: {"status":"keep","combined_µs":4110,"parse_µs":2922,"render_µs":1188,"allocations":25535}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
41de814045
Skip to_liquid/context= for primitives in VariableLookup#evaluate\n\nResult: {"status":"keep","combined_µs":4334,"parse_µs":3062,"render_µs":1272,"allocations":25535}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
91d9a509f9
Fast return for primitive types in find_variable — skip to_liquid and respond_to?(:context=)\n\nResult: {"status":"keep","combined_µs":4225,"parse_µs":3009,"render_µs":1216,"allocations":25535}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
37a4c19405
Skip find_index when only one scope in find_variable — go straight to environments\n\nResult: {"status":"keep","combined_µs":4323,"parse_µs":3055,"render_µs":1268,"allocations":25535}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
dc35b765e8
Skip respond_to?(:context=) for primitive types in find_variable — avoids method lookup overhead\n\nResult: {"status":"keep","combined_µs":4207,"parse_µs":2943,"render_µs":1264,"allocations":25535}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
1dfdce82dc
Use EMPTY_ARRAY for empty static_environments in Context — avoids 60 array allocs per render cycle\n\nResult: {"status":"keep","combined_µs":4262,"parse_µs":3079,"render_µs":1183,"allocations":25535}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
bf16d291c9
Lazy @changes hash in Registers — only allocate when a register is actually written\n\nResult: {"status":"keep","combined_µs":4287,"parse_µs":3059,"render_µs":1228,"allocations":25595}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
2da3ae330a
Cache block_delimiter strings per tag name — avoids repeated string interpolation\n\nResult: {"status":"keep","combined_µs":4372,"parse_µs":3127,"render_µs":1245,"allocations":25605}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
1800cffd3b
Lazy Context init: defer StringScanner and @interrupts array allocation until needed\n\nResult: {"status":"keep","combined_µs":4299,"parse_µs":3057,"render_µs":1242,"allocations":26015}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
02764d28f7
Cache small integer to_s (0-999): avoids 267 Integer#to_s allocations per render cycle\n\nResult: {"status":"keep","combined_µs":4158,"parse_µs":2920,"render_µs":1238,"allocations":26128}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
f08fe63042
Replace split+join in truncatewords with manual word scan — avoids array + string allocations\n\nResult: {"status":"keep","combined_µs":4280,"parse_µs":3009,"render_µs":1271,"allocations":26395}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
588966fd1c
Extend fast-path filter parsing to handle comma-separated multi-arg filters (e.g. pluralize: 'item', 'items')\n\nResult: {"status":"keep","combined_µs":4266,"parse_µs":3032,"render_µs":1234,"allocations":26480}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
0e5edcc9da
Avoid expr_markup byteslice when name is entire markup string (no whitespace, no filters)\n\nResult: {"status":"keep","combined_µs":4277,"parse_µs":3057,"render_µs":1220,"allocations":27026}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
3f10ac702c
Fast-path single-arg filter parsing: handle quoted strings, numbers, identifiers without Lexer/Parser\n\nResult: {"status":"keep","combined_µs":4427,"parse_µs":3181,"render_µs":1246,"allocations":27235}
2026-04-04 17:42:33 -07:00
Tobi Lutke and Chris Pak
f31930555e
fix rubocop offenses: autocorrect style/layout violations
2026-04-04 17:42:32 -07:00
Tobi Lutke and Chris Pak
f0ac941ac6
update autoresearch.md with full progress log
2026-04-04 17:42:32 -07:00
Tobi Lutke and Chris Pak
19bf49c116
For tag: migrate lax_parse to Cursor with zero-alloc skip_id/expect_id
2026-04-04 17:42:32 -07:00
Tobi Lutke and Chris Pak
5204033588
Cursor: add skip_id, expect_id, skip_fragment for zero-alloc scanning
2026-04-04 17:42:32 -07:00
Tobi Lutke and Chris Pak
b0c9f576af
REVERTED: Cursor for For tag adds 148 allocs from scan_id/scan_fragment string creation\n\nResult: {"status":"discard","combined_µs":5049,"parse_us":3765,"render_us":1284,"allocations":29793,"parse_µs":3765,"render_µs":1284}
2026-04-04 17:42:32 -07:00
Tobi Lutke and Chris Pak
343ae1da56
remove dead BlockBody.parse_tag_token and If SIMPLE_CONDITION - now in Cursor
2026-04-04 17:42:32 -07:00
Chris Pak