Fixes infinite loop in tokenizer on trailing stray '{'

The stray-{ else branch in tokenize_fast had a nested while loop that could
exit via its condition (when next_open >= len) without advancing pos.
The outer while pos < len loop would then find the same { again forever.

Reproduced by: Liquid::Template.parse('a{') -- hangs indefinitely.

Replaces the nested scan loop with two String#byteindex calls to find the
next '{%' and '{{' directly, then takes the minimum. Always O(n), eliminates
the nested loop entirely, and impossible to leave pos stranded.

Adds regression tests covering the three inputs that previously hung
plus adjacent stray-brace cases.
This commit is contained in:
Chris Pak
2026-04-04 21:25:59 -07:00
parent bc60deb671
commit d9c42fd2eb
2 changed files with 39 additions and 18 deletions
+12 -18
View File
@@ -146,24 +146,18 @@ module Liquid
pos = idx + 2
end
else
# { followed by something else — it's text
# Keep scanning from after this {
# Find next { that could be {% or {{
next_open = idx + 1
while next_open < len
ni = src.byteindex('{', next_open)
unless ni
@tokens << src.byteslice(pos, len - pos)
pos = len
break
end
nb = ni + 1 < len ? src.getbyte(ni + 1) : nil
if nb == PERCENTAGE || nb == OPEN_CURLEY
@tokens << src.byteslice(pos, ni - pos)
pos = ni
break
end
next_open = ni + 1
# Lone '{' — not the start of a tag or variable.
# Find the next '{{' or '{%' to know where this text token ends.
# Using two byteindex calls avoids a nested loop and is always O(n).
tag_start = src.byteindex('{%', idx + 1)
var_start = src.byteindex('{{', idx + 1)
next_token = [tag_start, var_start].compact.min
if next_token
@tokens << src.byteslice(pos, next_token - pos)
pos = next_token
else
@tokens << src.byteslice(pos, len - pos)
pos = len
end
end
end