Compare commits

..
Author SHA1 Message Date
Chris Pak e7373023d6 Add targeted tests for byte-walking fast paths
36 tests covering the three optimization sites:

Expression.parse_number (13 tests):
  - Simple integers, negatives, floats, trailing dots
  - Multi-dot truncation (1.2.3 → 1.2)
  - Rejection of non-numeric input and trailing alpha (1.2.3a)

Expression.parse strip guard (7 tests):
  - Leading, trailing, both-sides whitespace
  - No-strip-needed case (no allocation)
  - Null byte stripping (matches String#strip behavior)

VariableLookup.simple_lookup? (8 tests):
  - Accepts: single names, dotted chains, question marks, hyphens
  - Rejects: brackets, empty, leading/trailing dots, double dots

VariableLookup fast path equivalence (7 tests):
  - name/lookups/command_flags match for simple and deep chains
  - Bracket inputs fall through to regex path correctly

BlockBody.try_parse_tag_token (10 tests):
  - Simple tags, whitespace control variants ({%-, -%}, both)
  - No-markup tags, hash comments, newline counting
  - Hyphenated names stop at hyphen (matching TagName = /\w+/)
  - Malformed tokens return nil (fallback to FullToken regex)
2026-04-05 21:01:31 -07:00
Chris Pak d3e39525f3 BlockBody: byte-walk tag tokens instead of FullToken regex
Add try_parse_tag_token that parses {%...%} tag tokens using
getbyte/byteslice + ByteTables lookup arrays instead of the
FullToken regex with 4 capture groups. Allocates only the 2
strings needed (tag_name, markup) vs 4+ from regex captures.

Uses ByteTables::WORD (no hyphen) for tag name scanning,
matching TagName = /#|\w+/ exactly. Falls back to FullToken
regex when the fast path returns nil.
2026-04-05 20:53:53 -07:00
Chris Pak fd853a3593 VariableLookup: fast path for simple identifier chains
Skip the expensive recursive VariableParser regex for simple
lookups like 'product.title' (~90% of real-world cases).

SIMPLE_LOOKUP_RE validates the input is a plain a.b.c chain
(no brackets, no quotes). On match, byte-walks on dots to split
segments instead of invoking the regex engine. Falls through to
the original VariableParser scan for complex inputs.
2026-04-05 20:53:53 -07:00
Chris Pak aaabfc5017 Expression: byte-walk parse_number and guard strip allocation
parse_number: replace INTEGER_REGEX/FLOAT_REGEX matching and
StringScanner loop with a single byte-walking pass using
ByteTables::DIGIT. Avoids MatchData allocation and StringScanner
reset on every call.

Expression.parse: only call String#strip when leading/trailing
whitespace is actually present (checked via ByteTables::WHITESPACE).
Avoids allocating a new String on ~4,464 calls per compile.
2026-04-05 20:53:53 -07:00
Chris Pak 7cd8df6fa8 Add ByteTables module and bench_quick.rb benchmark harness
ByteTables provides pre-computed 256-entry boolean lookup arrays for
byte classification (IDENT_START, IDENT_CONT, WORD, DIGIT, WHITESPACE)
and named constants for delimiter bytes (NEWLINE, DASH, DOT, HASH).

bench_quick.rb measures parse µs, render µs, and object allocations
for the theme benchmark suite.
2026-04-05 20:53:53 -07:00
45 changed files with 615 additions and 1776 deletions
+1 -1
View File
@@ -1 +1 @@
4.0.2
3.4.1
+1 -1
View File
@@ -32,6 +32,6 @@ group :test do
end
group :spec do
gem 'liquid-spec', github: 'Shopify/liquid-spec'
gem 'liquid-spec', github: 'Shopify/liquid-spec', branch: 'main'
gem 'activesupport', require: false
end
-12
View File
@@ -1,17 +1,5 @@
# Liquid Change Log
## 5.13.0
* Add TruffleRuby in CI [Benoit Daloze]
* Skip slow test raising many exceptions on non-CRuby [Benoit Daloze]
* Reject bare-bracket syntax in strict2 and introduce `self` keyword by [Alok Swamy]
* Add strict2_parse to assign and capture tags by [Alok Swamy]
* Add strict2_parse to increment and decrement tags by [Alok Swamy]
* Update liquid-spec adapters for `missing_features` [Ian Ker-Seymer]
* Prevent `SelfDrop` context mutation across render boundaries [Guilherme Carreiro]
* Fix `SelfDrop` equality [Guilherme Carreiro]
* Let environment `self` shadow `SelfDrop` [Ian Ker-Seymer]
## 5.11.0
* Revert the Inline Snippets tag (#2001), treat its inclusion in the latest Liquid release as a bug, and allow for feedback on RFC#1916 to better support Liquid developers [Guilherme Carreiro]
* Rename the `:rigid` error mode to `:strict2` and display a warning when users attempt to use the `:rigid` mode [Guilherme Carreiro]
+2 -4
View File
@@ -151,8 +151,6 @@ end
desc('run liquid-spec suite across all adapters')
task :spec do
Dir['./spec/*.rb'].sort.each do |adapter|
puts "=== Running #{adapter} ==="
sh 'bundle', 'exec', 'liquid-spec', 'run', adapter, '--no-max-failures'
end
adapters = Dir['./spec/*.rb'].join(',')
sh "bundle exec liquid-spec matrix --adapters=#{adapters} --reference=ruby_liquid"
end
-72
View File
@@ -1,72 +0,0 @@
# Recording and replaying renders
`Liquid::TemplateRecorder` captures successful template renders so they can be
replayed without the application's file system or Drop implementations.
Recording does not wrap or replace assigns, so the recorded render has the same
semantics as a normal render.
```ruby
Liquid::TemplateRecorder.record("render.json") do
template = Liquid::Template.parse(source)
template.render!(assigns)
end
replayer = Liquid::TemplateRecorder.replay_from("render.json", mode: :verify)
replayer.render # raises if the output changed
```
A recording contains the root template, every parsed partial, partial contents,
plain Hash/Array values resolved by the template, properties actually read from `Liquid::Drop` objects,
filter-call diagnostics, engine options, and the rendered output. Drop instance
variables are never inspected. An unsupported Ruby object raises
`Liquid::TemplateRecorder::SerializationError` rather than silently producing a
recording that cannot be replayed.
## Storage formats
A `.json` destination is written atomically after the recording block succeeds.
It contains a session with every render performed by the block.
A `.jsonl` destination is append-only. Each successful top-level render is one
compact, self-contained JSON line. This is the recommended format for production
sampling: a process failure can lose at most the render being written, writers
are serialized with `flock`, and a recording can be replayed by index.
A destination may instead be any writer object responding to `write(record)`. The
writer receives one self-contained recording Hash per successful render. Liquid
does not own or close injected writers, so applications can publish records to
Kafka, object storage, or another transport without coupling that transport to
the recorder.
Pass `on_error:` to keep serialization or sink failures out of the render path;
the callback receives the error and should not raise.
```ruby
Liquid::TemplateRecorder.record(kafka_writer) do
template.render!(assigns)
end
```
```ruby
Liquid::TemplateRecorder.replay_from("renders.jsonl") # last render
Liquid::TemplateRecorder.replay_from("renders.jsonl", index: 0) # first render
Liquid::TemplateRecorder.records("renders.jsonl") # inspect all
```
Compression is intentionally separate from the schema. In particular, one
long-lived compressed stream makes appending, recovery, and selecting a render
harder. Compress rotated `.jsonl` files with the storage system of your choice;
a future compressed writer can use one independent frame per record without a
schema change.
Recording sessions are thread-local. Nested sessions in the same thread are
rejected. Existing application register names and the one-argument
`FileSystem#read_template_file` API remain unchanged.
## Replay modes
* `:compute` runs filters normally. Pass application filters with
`replayer.render(filters: MyFilters)`.
* `:strict` returns each exact recorded filter result and rejects a changed
filter sequence. This can replay application-specific or nondeterministic
filters without loading their implementations.
* `:verify` computes normally and raises when the final output differs.
+1 -2
View File
@@ -52,6 +52,7 @@ end
require "liquid/version"
require "liquid/deprecations"
require "liquid/const"
require "liquid/byte_tables"
require 'liquid/standardfilters'
require 'liquid/file_system'
require 'liquid/parser_switching'
@@ -65,7 +66,6 @@ require 'liquid/lexer'
require 'liquid/parser'
require 'liquid/i18n'
require 'liquid/drop'
require 'liquid/self_drop'
require 'liquid/tablerowloop_drop'
require 'liquid/forloop_drop'
require 'liquid/extensions'
@@ -90,4 +90,3 @@ require 'liquid/partial_cache'
require 'liquid/usage'
require 'liquid/registers'
require 'liquid/template_factory'
require "liquid/template_recorder"
+84 -15
View File
@@ -84,15 +84,10 @@ module Liquid
# @api private
def self.render_node(context, output, node)
recorder = TemplateRecorder.current
tag_call = recorder&.begin_tag_render(node, context)
output_start = output.length
node.render_to_output_buffer(context, output)
rescue => exc
blank_tag = !node.instance_of?(Variable) && node.blank?
rescue_render_node(context, output, node.line_number, exc, blank_tag)
ensure
recorder&.finish_tag_render(tag_call, output[output_start..]) if output_start
end
# @api private
@@ -104,9 +99,7 @@ module Liquid
context.handle_error(exc, line_number)
else
error_message = context.handle_error(exc, line_number)
error_mode = context.registers.static[:template_error_mode]
suppress_error_text = blank_tag && error_mode != :strict2 && error_mode != :rigid
unless suppress_error_text # blank-tag suppression is kept for backwards compatibility outside strict2
unless blank_tag # conditional for backwards compatibility
output << error_message
end
end
@@ -137,16 +130,21 @@ module Liquid
case
when token.start_with?(TAGSTART)
whitespace_handler(token, parse_context)
unless token =~ FullToken
# rubocop:disable Metrics/BlockNesting
fast = try_parse_tag_token(token)
if fast
tag_name, markup, newlines = fast
elsif token =~ FullToken
tag_name = Regexp.last_match(2)
markup = Regexp.last_match(4)
newlines = parse_context.line_number ? Regexp.last_match(1).count("\n") + Regexp.last_match(3).count("\n") : 0
else
return handle_invalid_tag_token(token, parse_context, &block)
end
tag_name = Regexp.last_match(2)
markup = Regexp.last_match(4)
# rubocop:enable Metrics/BlockNesting
if parse_context.line_number
# newlines inside the tag should increase the line number,
# particularly important for multiline {% liquid %} tags
parse_context.line_number += Regexp.last_match(1).count("\n") + Regexp.last_match(3).count("\n")
if parse_context.line_number && newlines > 0
parse_context.line_number += newlines
end
if tag_name == 'liquid'
@@ -267,6 +265,77 @@ module Liquid
BlockBody.raise_missing_variable_terminator(token, parse_context)
end
# Fast path for parsing "{%[-] tag_name markup [-]%}" tag tokens.
# Returns [tag_name, markup, newline_count] or nil.
#
# Accepts tokens where:
# - Tag name is '#' or starts with [a-zA-Z_] followed by \w chars
# (matching TagName = /#|\w+/ exactly — no hyphens, no '?' suffix)
# - Whitespace is spaces, tabs, newlines, \r, \f, \v
# - Whitespace control dashes are at positions 2 and len-3
# Rejects (returns nil → caller falls back to FullToken regex):
# - Tokens shorter than "{%x%}" (4 bytes)
# - Tag names starting with a digit (valid in FullToken but rare)
# - Any structure the byte-walk can't confidently parse
# Fallback: nil return triggers the original `token =~ FullToken` regex
# match in parse_for_document, preserving identical behavior for any
# input the fast path doesn't handle.
def try_parse_tag_token(token)
len = token.bytesize
pos = 2 # skip "{%"
return if pos >= len
pos += 1 if token.getbyte(pos) == ByteTables::DASH
newline_count = 0
# Skip whitespace before tag name, count newlines
while pos < len
b = token.getbyte(pos)
if b == ByteTables::NEWLINE
pos += 1
newline_count += 1
elsif ByteTables::WHITESPACE[b]
pos += 1
else
break
end
end
return if pos >= len
# Scan tag name: '#' or \w+ (matching TagName = /#|\w+/)
name_start = pos
b = token.getbyte(pos)
if b == ByteTables::HASH
pos += 1
elsif ByteTables::IDENT_START[b]
pos += 1
pos += 1 while pos < len && ByteTables::WORD[token.getbyte(pos)]
else
return
end
tag_name = token.byteslice(name_start, pos - name_start)
# Skip whitespace after tag name, count newlines
while pos < len
b = token.getbyte(pos)
if b == ByteTables::NEWLINE
pos += 1
newline_count += 1
elsif ByteTables::WHITESPACE[b]
pos += 1
else
break
end
end
# Markup: everything up to optional '-' before '%}'
markup_end = len - 2 # skip '%}'
markup_end -= 1 if markup_end > pos && token.getbyte(markup_end - 1) == ByteTables::DASH
markup = pos >= markup_end ? "" : token.byteslice(pos, markup_end - pos)
[tag_name, markup, newline_count]
end
# @deprecated Use {.raise_missing_tag_terminator} instead
def raise_missing_tag_terminator(token, parse_context)
BlockBody.raise_missing_tag_terminator(token, parse_context)
+48
View File
@@ -0,0 +1,48 @@
# frozen_string_literal: true
module Liquid
# Pre-computed 256-entry boolean lookup tables for byte classification.
# Built once at load time; used as TABLE[byte] — a single array index
# instead of 3-5 comparison operators per check.
module ByteTables
# [a-zA-Z_] — valid first byte of an identifier
IDENT_START = Array.new(256, false).tap do |t|
(97..122).each { |b| t[b] = true } # a-z
(65..90).each { |b| t[b] = true } # A-Z
t[95] = true # _
end.freeze
# [a-zA-Z0-9_-] — valid continuation byte of an identifier
IDENT_CONT = Array.new(256, false).tap do |t|
(97..122).each { |b| t[b] = true } # a-z
(65..90).each { |b| t[b] = true } # A-Z
(48..57).each { |b| t[b] = true } # 0-9
t[95] = true # _
t[45] = true # -
end.freeze
# [a-zA-Z0-9_] — \w equivalent (no hyphen), for tag name scanning
WORD = Array.new(256, false).tap do |t|
(97..122).each { |b| t[b] = true } # a-z
(65..90).each { |b| t[b] = true } # A-Z
(48..57).each { |b| t[b] = true } # 0-9
t[95] = true # _
end.freeze
# [0-9] — ASCII digit
DIGIT = Array.new(256, false).tap do |t|
(48..57).each { |b| t[b] = true }
end.freeze
# Matches bytes removed by Ruby's String#strip: \x00, \t, \n, \v, \f, \r, space
WHITESPACE = Array.new(256, false).tap do |t|
[0, 9, 10, 11, 12, 13, 32].each { |b| t[b] = true }
end.freeze
# Byte constants for delimiters and punctuation
NEWLINE = 10
DASH = 45 # '-'
DOT = 46 # '.'
HASH = 35 # '#'
end
end
+1 -21
View File
@@ -187,15 +187,6 @@ module Liquid
find_variable(key, raise_on_not_found: false) != nil
end
# Checks whether a variable is defined in any scope, including nil-valued keys.
# Unlike #key?, this uses Hash#key? so that variables explicitly set to nil
# are still considered defined.
def variable_defined?(key)
@scopes.any? { |s| s.key?(key) } ||
@environments.any? { |e| e.key?(key) } ||
@static_environments.any? { |e| e.key?(key) }
end
def evaluate(object)
object.respond_to?(:evaluate) ? object.evaluate(self) : object
end
@@ -206,21 +197,12 @@ module Liquid
# path and find_index() is optimized in MRI to reduce object allocation
index = @scopes.find_index { |s| s.key?(key) }
fallback_to_self_drop = key == Expression::SELF && index.nil?
variable = if index
lookup_and_evaluate(@scopes[index], key, raise_on_not_found: raise_on_not_found)
else
try_variable_find_in_environments(
key,
raise_on_not_found: raise_on_not_found && !fallback_to_self_drop,
)
try_variable_find_in_environments(key, raise_on_not_found: raise_on_not_found)
end
# `self` resolves to a SelfDrop (enabling `self['var']` lookups),
# but only after the normal environment lookup doesn't find a value.
return @self_drop ||= SelfDrop.new(self) if fallback_to_self_drop && variable.nil?
# update variable's context before invoking #to_liquid
variable.context = self if variable.respond_to?(:context=)
@@ -228,8 +210,6 @@ module Liquid
liquid_variable.context = self if variable != liquid_variable && liquid_variable.respond_to?(:context=)
recorder = @registers[TemplateRecorder::REGISTER_KEY] if defined?(TemplateRecorder)
recorder&.emit_variable_read(key, liquid_variable)
liquid_variable
end
+1 -5
View File
@@ -37,15 +37,11 @@ module Liquid
# called by liquid to invoke a drop
def invoke_drop(method_or_key)
result = if self.class.invokable?(method_or_key)
if self.class.invokable?(method_or_key)
send(method_or_key)
else
liquid_method_missing(method_or_key)
end
recorder = @context&.registers&.[](TemplateRecorder::REGISTER_KEY) if defined?(TemplateRecorder)
recorder&.emit_drop_read(self, method_or_key, result)
result
end
def key?(_name)
+73 -56
View File
@@ -2,8 +2,6 @@
module Liquid
class Expression
SELF = 'self'
LITERALS = {
nil => nil,
'nil' => nil,
@@ -18,16 +16,9 @@ module Liquid
'-' => VariableLookup.parse("-", nil).freeze,
}.freeze
DOT = ".".ord
ZERO = "0".ord
NINE = "9".ord
DASH = "-".ord
# Use an atomic group (?>...) to avoid pathological backtracing from
# malicious input as described in https://github.com/Shopify/liquid/issues/1357
RANGES_REGEX = /\A\(\s*(?>(\S+)\s*\.\.)\s*(\S+)\s*\)\z/
INTEGER_REGEX = /\A(-?\d+)\z/
FLOAT_REGEX = /\A(-?\d+)\.\d+\z/
class << self
def safe_parse(parser, ss = StringScanner.new(""), cache = nil)
@@ -37,7 +28,15 @@ module Liquid
def parse(markup, ss = StringScanner.new(""), cache = nil)
return unless markup
markup = markup.strip # markup can be a frozen string
# Guard: only call .strip when the first or last byte is whitespace.
# String#strip always allocates a new String, even when there's nothing
# to strip. ByteTables::WHITESPACE matches the same bytes that strip
# removes (space, \t, \n, \v, \f, \r, \x00). When neither end has
# whitespace, we skip the call and avoid ~4,464 allocations per compile.
first = markup.getbyte(0)
if first && (ByteTables::WHITESPACE[first] || ByteTables::WHITESPACE[markup.getbyte(markup.bytesize - 1)])
markup = markup.strip
end
if (markup.start_with?('"') && markup.end_with?('"')) ||
(markup.start_with?("'") && markup.end_with?("'"))
@@ -73,56 +72,74 @@ module Liquid
end
end
def parse_number(markup, ss)
# check if the markup is simple integer or float
case markup
when INTEGER_REGEX
return Integer(markup, 10)
when FLOAT_REGEX
return markup.to_f
end
# Fast path for number parsing. Accepts:
# - Simple integers: "42", "-7"
# - Simple floats: "3.14", "-0.5"
# - Multi-dot floats (truncated at second dot): "1.2.3" → 1.2
# - Trailing-dot floats: "123." → 123.0
# Rejects (returns nil → caller treats as VariableLookup):
# - Non-numeric input: "hello", ""
# - Inputs with non-digit/non-dot bytes after the number: "1.2.3a"
# Fallback: nil return causes caller to fall through to VariableLookup.parse,
# which is the same path the old regex-based code took on non-match.
def parse_number(markup, _ss = nil)
len = markup.bytesize
return if len == 0
ss.string = markup
# the first byte must be a digit or a dash
byte = ss.scan_byte
pos = 0
first = markup.getbyte(pos)
return false if byte != DASH && (byte < ZERO || byte > NINE)
if first == ByteTables::DASH
pos += 1
return if pos >= len
return unless ByteTables::DIGIT[markup.getbyte(pos)]
if byte == DASH
peek_byte = ss.peek_byte
# if it starts with a dash, the next byte must be a digit
return false if peek_byte.nil? || !(peek_byte >= ZERO && peek_byte <= NINE)
end
# The markup could be a float with multiple dots
first_dot_pos = nil
num_end_pos = nil
while (byte = ss.scan_byte)
return false if byte != DOT && (byte < ZERO || byte > NINE)
# we found our number and now we are just scanning the rest of the string
next if num_end_pos
if byte == DOT
if first_dot_pos.nil?
first_dot_pos = ss.pos
else
# we found another dot, so we know that the number ends here
num_end_pos = ss.pos - 1
end
end
end
num_end_pos = markup.length if ss.eos?
if num_end_pos
# number ends with a number "123.123"
markup.byteslice(0, num_end_pos).to_f
pos += 1
elsif ByteTables::DIGIT[first]
pos += 1
else
# number ends with a dot "123."
markup.byteslice(0, first_dot_pos).to_f
return
end
# Scan digits
pos += 1 while pos < len && ByteTables::DIGIT[markup.getbyte(pos)]
# Consumed everything = simple integer
return Integer(markup, 10) if pos == len
# Check for dot — three float cases:
# 1. Simple float: "123.456" → markup.to_f
# 2. Multi-dot: "1.2.3.4" → truncate at second dot → 1.2
# 3. Trailing dot: "123." → truncate before dot → 123.0
return unless markup.getbyte(pos) == ByteTables::DOT
dot_pos = pos
pos += 1
digit_start = pos
pos += 1 while pos < len && ByteTables::DIGIT[markup.getbyte(pos)]
if pos > digit_start && pos == len
# Case 1: simple float like "123.456"
markup.to_f
elsif pos > digit_start
# Case 2: multi-dot like "1.2.3.4" — find where the numeric
# portion ends. Reject if any non-digit, non-dot byte is found
# (e.g. "1.2.3a" → nil, matching the old regex-based behavior).
num_end = nil
check = pos
while check < len
b = markup.getbyte(check)
if b == ByteTables::DOT
num_end ||= check
elsif !ByteTables::DIGIT[b]
return
end
check += 1
end
markup.byteslice(0, num_end || len).to_f
else
# Case 3: trailing dot like "123."
markup.byteslice(0, dot_pos).to_f
end
end
end
+1 -1
View File
@@ -38,7 +38,7 @@ module Liquid
def new_parser(input)
@string_scanner.string = input
Parser.new(@string_scanner, reject_bare_brackets: @error_mode == :strict2 || @error_mode == :rigid)
Parser.new(@string_scanner)
end
def new_tokenizer(source, start_line_number: nil, for_liquid_tag: false)
+1 -5
View File
@@ -2,11 +2,10 @@
module Liquid
class Parser
def initialize(input, reject_bare_brackets: false)
def initialize(input)
ss = input.is_a?(StringScanner) ? input : StringScanner.new(input)
@tokens = Lexer.tokenize(ss)
@p = 0 # pointer to current location
@reject_bare_brackets = reject_bare_brackets
end
def jump(point)
@@ -54,9 +53,6 @@ module Liquid
str = consume
str << variable_lookups
when :open_square
if @reject_bare_brackets
raise SyntaxError, "Bare bracket access is not allowed. Use #{Expression::SELF}['...'] instead"
end
str = consume.dup
str << expression
str << consume(:close_square)
-2
View File
@@ -10,8 +10,6 @@ module Liquid
file_system = context.registers[:file_system]
source = file_system.read_template_file(template_name)
recorder = context.registers[TemplateRecorder::REGISTER_KEY] if defined?(TemplateRecorder)
recorder&.emit_file_read(template_name, source)
parse_context.partial = true
-54
View File
@@ -1,54 +0,0 @@
# frozen_string_literal: true
module Liquid
# @liquid_public_docs
# @liquid_type object
# @liquid_name self
# @liquid_summary
# Provides access to variables through the current scope chain.
# @liquid_description
# The `self` object resolves variables through the normal lookup hierarchy
# (local > file > global) without exposing filters, interrupts, errors,
# or other context internals. It's used when bare bracket notation
# (`['variable']`) needs to be replaced with an explicit variable lookup.
#
# If `self` is explicitly assigned as a local variable (e.g. `{% assign self = 'value' %}`),
# then the local value takes precedence over the `self` object.
# @liquid_access global
class SelfDrop < Drop
def initialize(self_context)
super()
@self_context = self_context
end
def [](key)
@self_context.find_variable(key)
rescue UndefinedVariable
nil
end
def key?(key)
@self_context.variable_defined?(key)
end
def to_liquid
self
end
def ==(other)
other.is_a?(SelfDrop) && other.self_context.equal?(@self_context)
end
alias_method :eql?, :==
def hash
@self_context.object_id.hash
end
protected
attr_reader :self_context
undef context=
end
end
+1 -5
View File
@@ -48,17 +48,13 @@ module Liquid
end
def invoke(method, *args)
result = if self.class.invokable?(method)
if self.class.invokable?(method)
send(method, *args)
elsif @context.strict_filters
raise Liquid::UndefinedFilter, "undefined filter #{method}"
else
args.first
end
recorder = @context&.registers&.[](TemplateRecorder::REGISTER_KEY) if defined?(TemplateRecorder)
recorder&.emit_filter_call(method, args.first, args.drop(1), result)
result
rescue ::ArgumentError => e
raise Liquid::ArgumentError, e.message, e.backtrace
end
-25
View File
@@ -18,8 +18,6 @@ module Liquid
# @liquid_syntax_keyword variable_name The name of the variable being created.
# @liquid_syntax_keyword value The value you want to assign to the variable.
class Assign < Tag
include ParserSwitching
Syntax = /(#{VariableSignature}+)\s*=\s*(.*)\s*/om
# @api private
@@ -31,10 +29,6 @@ module Liquid
def initialize(tag_name, markup, parse_context)
super
parse_with_selected_parser(markup)
end
def lax_parse(markup)
if markup =~ Syntax
@to = Regexp.last_match(1)
@from = Variable.new(Regexp.last_match(2), parse_context)
@@ -43,25 +37,6 @@ module Liquid
end
end
def strict_parse(markup)
lax_parse(markup)
end
def strict2_parse(markup)
unless markup =~ Syntax
self.class.raise_syntax_error(parse_context)
end
lhs = Regexp.last_match(1).strip
rhs = Regexp.last_match(2)
p = @parse_context.new_parser(lhs)
@to = p.consume(:id)
p.consume(:end_of_string)
@from = Variable.new(rhs, parse_context)
end
def render_to_output_buffer(context, output)
val = @from.render(context)
context.scopes.last[@to] = val
-18
View File
@@ -20,18 +20,10 @@ module Liquid
# @liquid_syntax_keyword variable The name of the variable being created.
# @liquid_syntax_keyword value The value you want to assign to the variable.
class Capture < Block
include ParserSwitching
Syntax = /(#{VariableSignature}+)/o
attr_reader :to
def initialize(tag_name, markup, options)
super
parse_with_selected_parser(markup)
end
def lax_parse(markup)
if markup =~ Syntax
@to = Regexp.last_match(1)
else
@@ -39,16 +31,6 @@ module Liquid
end
end
def strict_parse(markup)
lax_parse(markup)
end
def strict2_parse(markup)
p = @parse_context.new_parser(markup.strip)
@to = p.consume(:id)
p.consume(:end_of_string)
end
def render_to_output_buffer(context, output)
context.resource_limits.with_capture do
capture_output = render(context)
-16
View File
@@ -23,29 +23,13 @@ module Liquid
# {% decrement variable_name %}
# @liquid_syntax_keyword variable_name The name of the variable being decremented.
class Decrement < Tag
include ParserSwitching
attr_reader :variable_name
def initialize(tag_name, markup, options)
super
parse_with_selected_parser(markup)
end
def lax_parse(markup)
@variable_name = markup.strip
end
def strict_parse(markup)
lax_parse(markup)
end
def strict2_parse(markup)
p = @parse_context.new_parser(markup.strip)
@variable_name = p.consume(:id)
p.consume(:end_of_string)
end
def render_to_output_buffer(context, output)
counter_environment = context.environments.first
value = counter_environment[@variable_name] || 0
+4 -13
View File
@@ -20,8 +20,7 @@ module Liquid
class Include < Tag
prepend Tag::Disableable
FOR = 'for'
SYNTAX = /(#{QuotedFragment}+)(\s+(with|#{FOR})\s+(#{QuotedFragment}+))?(\s+(?:as)\s+(#{VariableSegment}+))?/o
SYNTAX = /(#{QuotedFragment}+)(\s+(?:with|for)\s+(#{QuotedFragment}+))?(\s+(?:as)\s+(#{VariableSegment}+))?/o
Syntax = SYNTAX
attr_reader :template_name_expr, :variable_name_expr, :attributes
@@ -85,18 +84,12 @@ module Liquid
alias_method :parse_context, :options
private :parse_context
def for_loop?
@is_for_loop
end
def strict2_parse(markup)
p = @parse_context.new_parser(markup)
@template_name_expr = safe_parse_expression(p)
with_or_for = p.id?("for") || p.id?("with")
@variable_name_expr = safe_parse_expression(p) if with_or_for
@variable_name_expr = safe_parse_expression(p) if p.id?("for") || p.id?("with")
@alias_name = p.consume(:id) if p.id?("as")
@is_for_loop = (with_or_for == FOR)
p.consume?(:comma)
@@ -118,13 +111,11 @@ module Liquid
def lax_parse(markup)
if markup =~ SYNTAX
template_name = Regexp.last_match(1)
with_or_for = Regexp.last_match(3)
variable_name = Regexp.last_match(4)
variable_name = Regexp.last_match(3)
@alias_name = Regexp.last_match(6)
@alias_name = Regexp.last_match(5)
@variable_name_expr = variable_name ? parse_expression(variable_name) : nil
@template_name_expr = parse_expression(template_name)
@is_for_loop = (with_or_for == FOR)
@attributes = {}
markup.scan(TagAttributes) do |key, value|
-16
View File
@@ -23,29 +23,13 @@ module Liquid
# {% increment variable_name %}
# @liquid_syntax_keyword variable_name The name of the variable being incremented.
class Increment < Tag
include ParserSwitching
attr_reader :variable_name
def initialize(tag_name, markup, options)
super
parse_with_selected_parser(markup)
end
def lax_parse(markup)
@variable_name = markup.strip
end
def strict_parse(markup)
lax_parse(markup)
end
def strict2_parse(markup)
p = @parse_context.new_parser(markup.strip)
@variable_name = p.consume(:id)
p.consume(:end_of_string)
end
def render_to_output_buffer(context, output)
counter_environment = context.environments.first
value = counter_environment[@variable_name] || 0
+4 -39
View File
@@ -105,7 +105,6 @@ module Liquid
tokenizer = parse_context.new_tokenizer(source, start_line_number: @line_numbers && 1)
@root = Document.parse(tokenizer, parse_context)
@template_recorder_source = source.dup.freeze if defined?(TemplateRecorder) && TemplateRecorder.current
self
end
@@ -142,8 +141,6 @@ module Liquid
def render(*args)
return '' if @root.nil?
recording_session = TemplateRecorder.current if defined?(TemplateRecorder)
recording_assigns = args.first
context = case args.first
when Liquid::Context
c = args.shift
@@ -154,10 +151,8 @@ module Liquid
c
when Liquid::Drop
drop = args.shift
c = Context.new([drop, assigns], instance_assigns, registers, @rethrow_errors, @resource_limits, {}, @environment)
drop.context = c if drop.respond_to?(:context=)
c
drop = args.shift
drop.context = Context.new([drop, assigns], instance_assigns, registers, @rethrow_errors, @resource_limits, {}, @environment)
when Hash
Context.new([args.shift, assigns], instance_assigns, registers, @rethrow_errors, @resource_limits, {}, @environment)
when nil
@@ -183,13 +178,6 @@ module Liquid
context.add_filters(args.pop)
end
recording = recording_session&.begin_render(self, recording_assigns, context)
if recording
recorder_registers = context.registers.static
previous_recorder = recorder_registers[TemplateRecorder::REGISTER_KEY]
recorder_registers[TemplateRecorder::REGISTER_KEY] = recording
end
# Retrying a render resets resource usage
context.resource_limits.reset
@@ -199,34 +187,12 @@ module Liquid
context.template_name ||= name
previous_error_mode = context.registers.static[:template_error_mode]
context.registers.static[:template_error_mode] = @error_mode
rendered_output = nil
render_succeeded = false
begin
# render the nodelist.
rendered_output = @root.render_to_output_buffer(context, output || +'')
render_succeeded = true
rendered_output
@root.render_to_output_buffer(context, output || +'')
rescue Liquid::MemoryError => e
rendered_output = context.handle_error(e)
render_succeeded = true
rendered_output
context.handle_error(e)
ensure
if previous_error_mode
context.registers.static[:template_error_mode] = previous_error_mode
else
context.registers.static.delete(:template_error_mode)
end
if recording
if previous_recorder
recorder_registers[TemplateRecorder::REGISTER_KEY] = previous_recorder
else
recorder_registers.delete(TemplateRecorder::REGISTER_KEY)
end
recording_session.finish_render(recording, rendered_output, context, success: render_succeeded)
end
@errors = context.errors
end
end
@@ -258,7 +224,6 @@ module Liquid
end
@warnings = parse_context.warnings
@error_mode = parse_context.error_mode
parse_context
end
-550
View File
@@ -1,550 +0,0 @@
# frozen_string_literal: true
require 'English'
require "digest/sha2"
require "json"
require "securerandom"
require "tempfile"
require "time"
module Liquid
# Records complete Liquid renders without changing the objects being rendered.
# A .json file contains one session; a .jsonl file is an append-only sequence
# of independently replayable renders.
class TemplateRecorder
FORMAT = "liquid-render"
SCHEMA_VERSION = 1
REGISTER_KEY = :__liquid_template_recorder
REPLAYER_REGISTER_KEY = :__liquid_template_recorder_replayer
class Error < StandardError; end
class ReplayError < Error; end
class SerializationError < Error; end
class << self
def record(destination, on_error: nil)
raise ArgumentError, "a block is required" unless block_given?
previous_session = current
raise Error, "nested recording sessions are not supported" if previous_session
session = Session.new(destination, on_error: on_error)
Thread.current[thread_key] = session
yield
ensure
if session
Thread.current[thread_key] = previous_session
session.close if $ERROR_INFO.nil?
end
end
def current
Thread.current[thread_key]
end
def replay_from(path, mode: :compute, index: -1)
records = Store.read(path)
raise ReplayError, "recording contains no renders" if records.empty?
record = records.fetch(index)
Replayer.new(record, mode: mode)
rescue IndexError
raise ReplayError, "render index #{index} does not exist"
end
def records(path)
Store.read(path)
end
private
def thread_key
:__liquid_template_recorder_session
end
end
class Session
def initialize(destination, on_error:)
@path = destination.to_s if destination.is_a?(String) || destination.respond_to?(:to_path)
@writer = destination unless @path
@on_error = on_error
@records = []
@active = nil
@pending_files = {}
end
def begin_render(template, _assigns, context)
if @active
@active.add_template(template)
@active.nesting += 1
return @active
end
@active = Render.new(template, context)
@pending_files.each { |path, source| @active.emit_file_read(path, source) }
@pending_files.clear
@active
end
def emit_file_read(path, source)
if @active
@active.emit_file_read(path, source)
else
@pending_files[path.to_s] = source
end
end
def emit_variable_output(output)
@active&.emit_variable_output(output)
end
def begin_tag_render(node, context)
@active&.begin_tag_render(node, context)
end
def finish_tag_render(call, output)
@active&.finish_tag_render(call, output)
end
def finish_render(render, output, context, success:)
return unless render.equal?(@active)
if render.nesting.positive?
render.nesting -= 1
return
end
if success
begin
record = render.finish(output, context)
if @writer
@writer.write(record)
elsif Store.jsonl?(@path)
Store.append(@path, record)
else
@records << record
end
rescue => error
handle_error(error)
end
end
@active = nil
end
def close
return if @writer || Store.jsonl?(@path)
Store.write_session(@path, @records)
rescue => error
handle_error(error)
end
private
def handle_error(error)
raise error unless @on_error
@on_error.call(error)
end
end
class Render
attr_accessor :nesting
def initialize(template, _context)
@nesting = 0
@templates = []
@files = {}
@filter_calls = []
@tag_calls = []
@tag_render_depth = 0
@variables = {}
@variable_outputs = []
@drop_values = {}
@bindings = {}.compare_by_identity
@root_template = template
add_template(template)
end
def add_template(template)
source = template.instance_variable_get(:@template_recorder_source)
return unless source
entrypoint = template.name
digest = Digest::SHA256.hexdigest(source)
return if @templates.any? { |item| item["sha256"] == digest && item["entrypoint"] == entrypoint }
@templates << { "source" => source, "entrypoint" => entrypoint, "sha256" => digest }
end
def emit_variable_output(output)
return if @tag_render_depth.positive?
@variable_outputs << output
end
def begin_tag_render(node, context)
name = context.environment.tags.key(node.class)
return unless name
@tag_render_depth += 1
return :nested if @tag_render_depth > 1
call = { "name" => name.to_s, "output" => nil }
@tag_calls << call
call
end
def finish_tag_render(call, output)
return unless call
@tag_render_depth -= 1
call["output"] = output unless call == :nested
end
def emit_variable_read(name, value)
path = [name.to_s]
@variables[name.to_s] = serialize(value, path, bind: true)
rescue SerializationError
# Unsupported values must not affect the render being observed.
end
def emit_drop_read(drop, key, value)
base = @bindings[drop]
return unless base
path = base + [key.to_s]
set_path(@drop_values, path, serialize(value, path, bind: true))
rescue SerializationError
# Unsupported values must not affect the render being observed.
end
def emit_file_read(path, source)
@files[path.to_s] = source.to_s
end
def emit_filter_call(name, input, arguments, output)
return if @tag_render_depth.positive?
@filter_calls << {
"name" => name.to_s,
"input" => serialize(input, ["filters", @filter_calls.length, "input"]),
"arguments" => serialize(arguments, ["filters", @filter_calls.length, "arguments"]),
"output" => serialize(output, ["filters", @filter_calls.length, "output"]),
}
rescue SerializationError
# Filter diagnostics must never make an otherwise replayable render fail.
end
def finish(output, context)
variables = deep_merge(@variables, @drop_values)
source = @root_template.instance_variable_get(:@template_recorder_source)
raise Error, "the rendered template was parsed outside the recording block" unless source
{
"format" => FORMAT,
"schema_version" => SCHEMA_VERSION,
"id" => SecureRandom.uuid,
"recorded_at" => Time.now.utc.iso8601,
"engine" => {
"liquid_version" => Liquid::VERSION,
"ruby_version" => RUBY_VERSION,
"strict_variables" => !!context.strict_variables,
"strict_filters" => !!context.strict_filters,
},
"template" => @templates.first,
"templates" => @templates,
"assigns" => variables,
"variable_outputs" => @variable_outputs,
"file_system" => @files,
"filter_calls" => @filter_calls,
"tag_calls" => @tag_calls,
"output" => output.to_s,
}
end
private
def serialize(value, path, seen = {}.compare_by_identity, bind: false)
case value
when nil, true, false, String, Integer, Float
value
when Symbol
value.to_s
when Liquid::Drop
@bindings[value] ||= path if bind
existing = value_at(@drop_values, @bindings[value])
existing || {}
when Hash
raise SerializationError, "circular value at #{format_path(path)}" if seen.key?(value)
seen[value] = true
result = value.each_with_object({}) do |(key, child), hash|
string_key = key.to_s
hash[string_key] = serialize(child, path + [string_key], seen, bind: bind)
end
seen.delete(value)
result
when Array
raise SerializationError, "circular value at #{format_path(path)}" if seen.key?(value)
seen[value] = true
result = value.each_with_index.map { |child, index| serialize(child, path + [index], seen, bind: bind) }
seen.delete(value)
result
else
raise SerializationError, "cannot record #{value.class} at #{format_path(path)}"
end
end
def set_path(root, path, value)
return root.replace(value) if path.empty? && value.is_a?(Hash)
cursor = root
path.each_with_index do |segment, index|
last = index == path.length - 1
if segment.is_a?(Integer)
break unless cursor.is_a?(Array)
end
cursor[segment] = last ? value : (cursor[segment] ||= container_for(path[index + 1]))
cursor = cursor[segment] unless last
end
end
def value_at(root, path)
return unless path
path.reduce(root) { |value, segment| value.respond_to?(:[]) ? value[segment] : nil }
end
def container_for(segment)
segment.is_a?(Integer) ? [] : {}
end
def deep_merge(left, right)
return right unless left.is_a?(Hash) && right.is_a?(Hash)
left.merge(right) { |_key, a, b| deep_merge(a, b) }
end
def format_path(path)
path.empty? ? "<root>" : path.join(".")
end
end
class Store
class << self
def jsonl?(path)
path.end_with?(".jsonl")
end
def append(path, record)
line = JSON.generate(record) << "\n"
File.open(path, File::WRONLY | File::CREAT | File::APPEND, 0o600) do |file|
file.flock(File::LOCK_EX)
file.write(line)
file.flush
end
end
def write_session(path, records)
payload = JSON.pretty_generate(
"format" => "liquid-recording-session",
"schema_version" => SCHEMA_VERSION,
"renders" => records,
) << "\n"
directory = File.dirname(File.expand_path(path))
Tempfile.create([".liquid-recording", ".tmp"], directory, mode: File::RDWR, perm: 0o600) do |file|
file.write(payload)
file.flush
file.fsync
File.rename(file.path, path)
end
end
def read(path)
content = File.binread(path)
records = if jsonl?(path)
read_jsonl(content)
else
parsed = JSON.parse(content)
parsed["renders"] || [parsed]
end
records.each { |record| validate!(record) }
records
rescue Errno::ENOENT
raise ReplayError, "recording file not found: #{path}"
rescue JSON::ParserError => error
raise ReplayError, "invalid recording JSON: #{error.message}"
end
def read_jsonl(content)
lines = content.lines
lines.filter_map.with_index do |line, index|
next if line.strip.empty?
JSON.parse(line)
rescue JSON::ParserError
last_truncated_line = index == lines.length - 1 && !content.end_with?("\n")
raise unless last_truncated_line
end
end
def validate!(record)
raise ReplayError, "unsupported recording format" unless record["format"] == FORMAT
raise ReplayError, "unsupported schema version #{record["schema_version"].inspect}" unless record["schema_version"] == SCHEMA_VERSION
['template', 'assigns', 'file_system', 'output'].each do |key|
raise ReplayError, "recording is missing #{key}" unless record.key?(key)
end
template = record["template"]
expected = Digest::SHA256.hexdigest(template.fetch("source"))
raise ReplayError, "template checksum does not match" unless template["sha256"] == expected
rescue KeyError, TypeError => error
raise ReplayError, "invalid recording schema: #{error.message}"
end
end
end
class MemoryFileSystem
def initialize(files)
@files = files
end
def read_template_file(path)
@files.fetch(path.to_s) { raise FileSystemError, "No such template '#{path}'" }
end
end
class Replayer
def initialize(record, mode: :compute, environment: nil)
@record = record
@mode = mode.to_sym
@environment = environment
unless [:compute, :strict, :verify].include?(@mode)
raise ReplayError, "mode must be :compute, :strict, or :verify"
end
end
def render(to: nil, filters: nil)
@filter_index = 0
@tag_index = 0
@variable_index = 0
parse_options = {}
parse_options[:environment] = strict_environment if @mode == :strict
template = Liquid::Template.parse(@record.dig("template", "source"), parse_options)
registers = { file_system: MemoryFileSystem.new(@record["file_system"]) }
registers[REPLAYER_REGISTER_KEY] = self if @mode == :strict && @record.key?("variable_outputs")
options = {
registers: registers,
strict_variables: @record.dig("engine", "strict_variables"),
strict_filters: @record.dig("engine", "strict_filters"),
}
options[:filters] = filters if filters
output = template.render!(@record["assigns"], options)
if @mode == :strict
verify_filter_count! unless @record.key?("variable_outputs")
verify_tag_count!
verify_variable_count!
end
if [:strict, :verify].include?(@mode) && output != @record["output"]
raise ReplayError, "replayed output does not match the recording"
end
File.binwrite(to, output) if to
output
end
def replay_filter(name)
call = @record["filter_calls"].fetch(@filter_index) do
raise ReplayError, "unexpected filter call #{name}"
end
if call["name"] != name.to_s
raise ReplayError, "expected filter #{call["name"]}, got #{name}"
end
@filter_index += 1
JSON.parse(JSON.generate(call["output"]))
end
def replay_variable
value = @record.fetch("variable_outputs").fetch(@variable_index) do
raise ReplayError, "unexpected variable render"
end
@variable_index += 1
value
end
def replay_tag(name)
call = @record.fetch("tag_calls", []).fetch(@tag_index) do
raise ReplayError, "unexpected tag call #{name}"
end
if call["name"] != name.to_s
raise ReplayError, "expected tag #{call["name"]}, got #{name}"
end
@tag_index += 1
call["output"]
end
def recorded_output
@record["output"]
end
def templates
@record["templates"]
end
private
def strict_environment
replayer = self
strainer = Class.new(Liquid::StrainerTemplate) do
define_method(:invoke) do |name, *_args|
replayer.replay_filter(name)
end
end
tags = (@environment || Liquid::Environment.default).tags.dup
tags&.each do |name, tag_class|
tags[name] = replay_tag_class(tag_class, name)
end
Liquid::Environment.build(tags: tags) do |environment|
environment.strainer_template = strainer
end
end
def replay_tag_class(tag_class, name)
replayer = self
Class.new(tag_class) do
define_method(:render_to_output_buffer) do |_context, output|
output << replayer.replay_tag(name)
end
end
end
def verify_variable_count!
return unless @record.key?("variable_outputs")
expected = @record["variable_outputs"].length
return if @variable_index == expected
raise ReplayError, "expected #{expected} variable renders, got #{@variable_index}"
end
def verify_tag_count!
expected = @record.fetch("tag_calls", []).length
return if @tag_index == expected
raise ReplayError, "expected #{expected} tag calls, got #{@tag_index}"
end
def verify_filter_count!
expected = @record["filter_calls"].length
return if @filter_index == expected
raise ReplayError, "expected #{expected} filter calls, got #{@filter_index}"
end
end
end
end
-10
View File
@@ -37,10 +37,6 @@ module Liquid
@markup
end
def ==(other)
self.class == other.class && name == other.name && filters == other.filters
end
def markup_context(markup)
"in \"{{#{markup}}}\""
end
@@ -109,14 +105,8 @@ module Liquid
end
def render_to_output_buffer(context, output)
if (replayer = context.registers[TemplateRecorder::REPLAYER_REGISTER_KEY])
return output << replayer.replay_variable
end
output_start = output.length
obj = render(context)
render_obj_to_output(obj, output)
TemplateRecorder.current&.emit_variable_output(output[output_start..])
output
end
+44
View File
@@ -4,6 +4,25 @@ module Liquid
class VariableLookup
COMMAND_METHODS = ['size', 'first', 'last'].freeze
# Matches simple identifier chains: name(.name)* with no brackets/quotes
SIMPLE_LOOKUP_RE = /\A[\w-]+\??(?:\.[\w-]+\??)*\z/
# Returns true when markup is a simple dotted identifier chain that the
# fast path in initialize can handle. Accepts:
# - Single names: "product", "item"
# - Dotted chains: "product.title", "cart.items.first"
# - Question-mark suffixes: "product.available?"
# - Hyphens in names: "my-var.some-field"
# Rejects (falls through to VariableParser regex):
# - Bracket lookups: "product[0]", "hash['key']"
# - Quoted strings, empty input, leading/trailing dots
# Fallback: when this returns false, initialize uses the original
# markup.scan(VariableParser) path — behavior is identical to
# the pre-optimization code for any input the fast path rejects.
def self.simple_lookup?(markup)
markup.bytesize > 0 && markup.match?(SIMPLE_LOOKUP_RE)
end
attr_reader :name, :lookups
def self.parse(markup, string_scanner = StringScanner.new(""), cache = nil)
@@ -11,6 +30,31 @@ module Liquid
end
def initialize(markup, string_scanner = StringScanner.new(""), cache = nil)
if self.class.simple_lookup?(markup)
dot_pos = markup.index('.')
if dot_pos.nil?
@name = markup
@lookups = Const::EMPTY_ARRAY
@command_flags = 0
return
end
@name = markup.byteslice(0, dot_pos)
@lookups = []
@command_flags = 0
pos = dot_pos + 1
len = markup.bytesize
while pos < len
seg_start = pos
pos += 1 while pos < len && markup.getbyte(pos) != ByteTables::DOT
seg = markup.byteslice(seg_start, pos - seg_start)
@command_flags |= 1 << @lookups.length if COMMAND_METHODS.include?(seg)
@lookups << seg
pos += 1 # skip dot
end
return
end
lookups = markup.scan(VariableParser)
name = lookups.shift
+1 -1
View File
@@ -2,5 +2,5 @@
# frozen_string_literal: true
module Liquid
VERSION = "5.13.0"
VERSION = "5.12.0"
end
+62
View File
@@ -0,0 +1,62 @@
# frozen_string_literal: true
# Quick benchmark: measures parse µs, render µs, and object allocations
# Outputs machine-readable metrics to stdout
require_relative 'theme_runner'
RubyVM::YJIT.enable if defined?(RubyVM::YJIT)
runner = ThemeRunner.new
# Warmup — enough iterations for YJIT to fully optimize hot paths
20.times { runner.compile }
20.times { runner.render }
GC.start
GC.compact if GC.respond_to?(:compact)
# Measure parse
parse_times = []
10.times do
GC.disable
t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
runner.compile
t1 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
GC.enable
GC.start
parse_times << (t1 - t0) * 1_000_000 # µs
end
# Measure render
render_times = []
10.times do
GC.disable
t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
runner.render
t1 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
GC.enable
GC.start
render_times << (t1 - t0) * 1_000_000 # µs
end
# Measure object allocations for one parse+render cycle
require 'objspace'
GC.start
GC.disable
before = ObjectSpace.count_objects.values_at(:TOTAL).first - ObjectSpace.count_objects.values_at(:FREE).first
runner.compile
runner.render
after = ObjectSpace.count_objects.values_at(:TOTAL).first - ObjectSpace.count_objects.values_at(:FREE).first
GC.enable
allocations = after - before
parse_us = parse_times.min.round(0)
render_us = render_times.min.round(0)
combined_us = parse_us + render_us
puts "RESULTS"
puts "parse_us=#{parse_us}"
puts "render_us=#{render_us}"
puts "combined_us=#{combined_us}"
puts "allocations=#{allocations}"
+3 -16
View File
@@ -6,24 +6,14 @@
$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
require 'liquid'
require_relative 'support/liquid_spec_adapter_helper'
LiquidSpec.configure do |config|
config.missing_features = [
:activesupport,
:lax_parsing,
:shopify_filters,
:shopify_includes,
:shopify_blank,
:shopify_error_handling,
:shopify_error_format,
:shopify_string_access,
]
# Run core Liquid specs
config.features = [:core]
end
# Compile a template string into a Liquid::Template
LiquidSpec.compile do |ctx, source, options|
options[:error_mode] ||= :strict
ctx[:template] = Liquid::Template.parse(source, **options)
end
@@ -38,12 +28,9 @@ LiquidSpec.render do |ctx, assigns, options|
static_environments: assigns,
registers: registers,
rethrow_errors: options[:strict_errors],
resource_limits: LiquidSpecAdapterHelper.resource_limits(options),
)
context.exception_renderer = options[:exception_renderer] if options[:exception_renderer]
LiquidSpecAdapterHelper.with_frozen_time do
ctx[:template].render(context)
end
ctx[:template].render(context)
end
+4 -16
View File
@@ -6,24 +6,15 @@
$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
require 'liquid'
require_relative 'support/liquid_spec_adapter_helper'
LiquidSpec.configure do |config|
config.missing_features = [
:activesupport,
:shopify_filters,
:shopify_includes,
:shopify_blank,
:shopify_error_handling,
:shopify_error_format,
:shopify_string_access,
]
config.features = [:core, :lax_parsing]
end
# Compile a template string into a Liquid::Template
LiquidSpec.compile do |ctx, source, options|
# Default to lax mode while still honoring specs that explicitly set error_mode.
options = { error_mode: :lax }.merge(options)
# Force lax mode
options = options.merge(error_mode: :lax)
ctx[:template] = Liquid::Template.parse(source, **options)
end
@@ -35,12 +26,9 @@ LiquidSpec.render do |ctx, assigns, options|
static_environments: assigns,
registers: registers,
rethrow_errors: options[:strict_errors],
resource_limits: LiquidSpecAdapterHelper.resource_limits(options),
)
context.exception_renderer = options[:exception_renderer] if options[:exception_renderer]
LiquidSpecAdapterHelper.with_frozen_time do
ctx[:template].render(context)
end
ctx[:template].render(context)
end
+3 -15
View File
@@ -7,23 +7,14 @@
$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
require 'active_support/all'
require 'liquid'
require_relative 'support/liquid_spec_adapter_helper'
LiquidSpec.configure do |config|
config.missing_features = [
:lax_parsing,
:shopify_filters,
:shopify_includes,
:shopify_blank,
:shopify_error_handling,
:shopify_error_format,
:shopify_string_access,
]
# Run core Liquid specs plus ActiveSupport SafeBuffer tests
config.features = [:core, :activesupport]
end
# Compile a template string into a Liquid::Template
LiquidSpec.compile do |ctx, source, options|
options[:error_mode] ||= :strict
ctx[:template] = Liquid::Template.parse(source, **options)
end
@@ -38,12 +29,9 @@ LiquidSpec.render do |ctx, assigns, options|
static_environments: assigns,
registers: registers,
rethrow_errors: options[:strict_errors],
resource_limits: LiquidSpecAdapterHelper.resource_limits(options),
)
context.exception_renderer = options[:exception_renderer] if options[:exception_renderer]
LiquidSpecAdapterHelper.with_frozen_time do
ctx[:template].render(context)
end
ctx[:template].render(context)
end
+2 -14
View File
@@ -13,18 +13,9 @@ end
require 'active_support/all'
require 'liquid'
require_relative 'support/liquid_spec_adapter_helper'
LiquidSpec.configure do |config|
config.missing_features = [
:lax_parsing,
:shopify_filters,
:shopify_includes,
:shopify_blank,
:shopify_error_handling,
:shopify_error_format,
:shopify_string_access,
]
config.features = [:core, :activesupport]
end
# Compile a template string into a Liquid::Template
@@ -42,12 +33,9 @@ LiquidSpec.render do |ctx, assigns, options|
static_environments: assigns,
registers: registers,
rethrow_errors: options[:strict_errors],
resource_limits: LiquidSpecAdapterHelper.resource_limits(options),
)
context.exception_renderer = options[:exception_renderer] if options[:exception_renderer]
LiquidSpecAdapterHelper.with_frozen_time do
ctx[:template].render(context)
end
ctx[:template].render(context)
end
@@ -1,24 +0,0 @@
# frozen_string_literal: true
module LiquidSpecAdapterHelper
extend self
def resource_limits(render_options)
return unless render_options[:resource_limits]
Liquid::ResourceLimits.new({}).tap do |limits|
render_options[:resource_limits].each do |key, value|
limits.public_send(:"#{key}=", value)
end
end
end
def with_frozen_time(&block)
original_tz = ENV['TZ']
ENV['TZ'] = 'UTC'
Liquid::Spec::TimeFreezer.freeze(Liquid::Spec::AdapterRunner::TEST_TIME, &block)
ensure
ENV['TZ'] = original_tz
end
end
-40
View File
@@ -97,46 +97,6 @@ class AssignTest < Minitest::Test
assert_equal(12, assign_score_of('int' => 123, 'str' => 'abcd'))
end
def test_assign_with_valid_identifier_in_strict2
assert_template_result("hello", "{% assign my_var = 'hello' %}{{ my_var }}", error_mode: :strict2)
end
def test_assign_with_hyphen_in_strict2
assert_template_result("hello", "{% assign my-var = 'hello' %}{{ my-var }}", error_mode: :strict2)
end
def test_assign_rejects_parentheses_in_variable_name_in_strict2
assert_raises(Liquid::SyntaxError) do
Liquid::Template.parse("{% assign (a(b(c) = 1234 %}", error_mode: :strict2)
end
end
def test_assign_rejects_brackets_in_variable_name_in_strict2
assert_raises(Liquid::SyntaxError) do
Liquid::Template.parse("{% assign [x.y] = 'hello' %}", error_mode: :strict2)
end
end
def test_assign_rejects_dot_in_variable_name_in_strict2
assert_raises(Liquid::SyntaxError) do
Liquid::Template.parse("{% assign a.b = 'hello' %}", error_mode: :strict2)
end
end
def test_assign_rejects_numeric_variable_name_in_strict2
assert_raises(Liquid::SyntaxError) do
Liquid::Template.parse("{% assign 1abc = 'hello' %}", error_mode: :strict2)
end
end
def test_assign_allows_invalid_names_in_lax
assert_template_result("1234", "{% assign (a(b(c) = 1234 %}{{ self['(a(b(c)'] }}", error_mode: :lax)
end
def test_assign_with_filter_in_strict2
assert_template_result("HELLO", "{% assign my_var = 'hello' | upcase %}{{ my_var }}", error_mode: :strict2)
end
private
class ObjectWrapperDrop < Liquid::Drop
@@ -1,85 +0,0 @@
# frozen_string_literal: true
require 'test_helper'
class BlankBodyErrorHandlingTest < Minitest::Test
COMPARISON_ERROR = 'Liquid error (line 1): comparison of Integer with String failed'
INVALID_INTEGER_ERROR = 'Liquid error (line 1): invalid integer'
def render_inline(source, error_mode:, assigns: {})
Liquid::Template.parse(source, line_numbers: true, error_mode: error_mode).render(assigns, render_errors: true)
end
def assert_render_raises(source, error_mode:, assigns: {}, message: nil)
error = assert_raises(Liquid::ArgumentError) do
Liquid::Template.parse(source, line_numbers: true, error_mode: error_mode).render!(assigns)
end
assert_includes(error.message, message) if message
end
def test_blank_if_body_suppresses_inline_error_text_in_lax_and_strict
[:lax, :strict].each do |mode|
assert_equal('', render_inline('{% if 5 > "x" %}{% endif %}', error_mode: mode))
end
end
def test_blank_unless_body_suppresses_inline_error_text_in_lax_and_strict
[:lax, :strict].each do |mode|
assert_equal('', render_inline('{% unless 5 > "x" %} {% endunless %}', error_mode: mode))
end
end
def test_blank_for_body_suppresses_inline_error_text_in_lax_and_strict
[:lax, :strict].each do |mode|
assert_equal('', render_inline('{% for i in (1..3) offset: xs %}{% endfor %}', error_mode: mode, assigns: { 'xs' => 'bad' }))
end
end
def test_strict2_blank_if_body_shows_inline_error_text
assert_equal(COMPARISON_ERROR, render_inline('{% if 5 > "x" %}{% endif %}', error_mode: :strict2))
end
def test_strict2_whitespace_if_body_shows_inline_error_text
assert_equal(COMPARISON_ERROR, render_inline('{% if 5 > "x" %} {% endif %}', error_mode: :strict2))
end
def test_strict2_assign_if_body_shows_inline_error_text
assert_equal(COMPARISON_ERROR, render_inline('{% if 5 > "x" %}{% assign a = 1 %}{% endif %}', error_mode: :strict2))
end
def test_strict2_comment_if_body_shows_inline_error_text
assert_equal(COMPARISON_ERROR, render_inline('{% if 5 > "x" %}{% comment %}c{% endcomment %}{% endif %}', error_mode: :strict2))
end
def test_strict2_capture_if_body_shows_inline_error_text
assert_equal(COMPARISON_ERROR, render_inline('{% if 5 > "x" %}{% capture c %}text{% endcapture %}{% endif %}', error_mode: :strict2))
end
def test_strict2_blank_unless_body_shows_inline_error_text
assert_equal(COMPARISON_ERROR, render_inline('{% unless 5 > "x" %} {% endunless %}', error_mode: :strict2))
end
def test_strict2_blank_for_body_shows_inline_error_text
assert_equal(INVALID_INTEGER_ERROR, render_inline('{% for i in (1..3) offset: xs %}{% endfor %}', error_mode: :strict2, assigns: { 'xs' => 'bad' }))
end
def test_nonblank_bodies_show_inline_error_text_in_all_modes
[:lax, :strict, :strict2].each do |mode|
assert_equal(COMPARISON_ERROR, render_inline('{% if 5 > "x" %}{% echo 1 %}{% endif %}', error_mode: mode))
assert_equal(COMPARISON_ERROR, render_inline('{% if 5 > "x" %}{{ "" }}{% endif %}', error_mode: mode))
assert_equal(COMPARISON_ERROR, render_inline('{% if 5 > "x" %}{% else %}E{% endif %}', error_mode: mode))
end
end
def test_raised_errors_are_not_swallowed_by_blank_if_body
[:lax, :strict, :strict2].each do |mode|
assert_render_raises('{% if 5 > "x" %}{% endif %}', error_mode: mode, message: 'comparison of Integer with String failed')
end
end
def test_raised_errors_are_not_swallowed_by_blank_for_body
[:lax, :strict, :strict2].each do |mode|
assert_render_raises('{% for i in (1..3) offset: xs %}{% endfor %}', error_mode: mode, assigns: { 'xs' => 'bad' }, message: 'invalid integer')
end
end
end
+1 -36
View File
@@ -6,11 +6,7 @@ class CaptureTest < Minitest::Test
include Liquid
def test_captures_block_content_in_variable
assert_template_result("test string", "{% capture var %}test string{% endcapture %}{{var}}", {})
end
def test_captures_block_content_in_quoted_variable_in_lax
assert_template_result("test string", "{% capture 'var' %}test string{% endcapture %}{{var}}", {}, error_mode: :lax)
assert_template_result("test string", "{% capture 'var' %}test string{% endcapture %}{{var}}", {})
end
def test_capture_with_hyphen_in_variable_name
@@ -53,35 +49,4 @@ class CaptureTest < Minitest::Test
t.render!
assert_equal(9, t.resource_limits.assign_score)
end
def test_capture_with_valid_identifier_in_strict2
assert_template_result("hello", "{% capture my_var %}hello{% endcapture %}{{ my_var }}", error_mode: :strict2)
end
def test_capture_with_hyphen_in_strict2
assert_template_result("hello", "{% capture my-var %}hello{% endcapture %}{{ my-var }}", error_mode: :strict2)
end
def test_capture_rejects_parentheses_in_variable_name_in_strict2
assert_raises(Liquid::SyntaxError) do
Liquid::Template.parse("{% capture (x[y %}hello{% endcapture %}", error_mode: :strict2)
end
end
def test_capture_rejects_dot_in_variable_name_in_strict2
assert_raises(Liquid::SyntaxError) do
Liquid::Template.parse("{% capture a.b %}hello{% endcapture %}", error_mode: :strict2)
end
end
def test_capture_rejects_numeric_variable_name_in_strict2
assert_raises(Liquid::SyntaxError) do
Liquid::Template.parse("{% capture 1abc %}hello{% endcapture %}", error_mode: :strict2)
end
end
def test_capture_allows_invalid_names_in_lax
t = Liquid::Template.parse("{% capture (x[y %}hello{% endcapture %}", error_mode: :lax)
assert_equal("(x[y", t.root.nodelist.first.to)
end
end
+2 -2
View File
@@ -296,8 +296,8 @@ class ContextTest < Minitest::Test
end
def test_access_variable_with_hash_notation
assert_template_result('baz', '{{ foo }}', { "foo" => "baz" })
assert_template_result('baz', '{{ self[bar] }}', { 'foo' => 'baz', 'bar' => 'foo' })
assert_template_result('baz', '{{ ["foo"] }}', { "foo" => "baz" })
assert_template_result('baz', '{{ [bar] }}', { 'foo' => 'baz', 'bar' => 'foo' })
end
def test_access_hashes_with_hash_access_variables
+1 -2
View File
@@ -269,8 +269,7 @@ class ErrorHandlingTest < Minitest::Test
assert_equal("Liquid error: comparison of Integer with String failed0", output)
output = Liquid::Template.parse("{% assign x = 0 %}{% if 1 < '2' %}{% assign x = 3 %}{% endif %}{{ x }}").render
expected = ENV["LIQUID_PARSER_MODE"] == "strict2" ? "Liquid error: comparison of Integer with String failed0" : "0"
assert_equal(expected, output)
assert_equal("0", output)
end
def test_syntax_error_is_raised_with_template_name
-120
View File
@@ -1,120 +0,0 @@
# frozen_string_literal: true
require 'test_helper'
class SelfDropContextTest < Minitest::Test
include Liquid
def test_self_drop_passed_as_render_param_preserves_original_scope
source = <<~LIQUID
{%- assign var = 42 -%}
{%- assign s = self -%}
{%- render "snippet1", other_self: s -%}
LIQUID
partials = {
'snippet1' => <<~LIQUID,
{%- assign var = 43 -%}
{{- other_self.var }}|{{ self.var -}}
LIQUID
}
assert_template_result('42|43', source, partials: partials)
end
def test_self_drop_in_render_without_passing_resolves_inner_scope
source = <<~LIQUID
{%- assign var = 42 -%}
{%- render "snippet1" -%}
LIQUID
partials = {
'snippet1' => <<~LIQUID,
{%- assign var = 99 -%}
{{- self.var -}}
LIQUID
}
assert_template_result('99', source, partials: partials)
end
def test_self_drop_passed_to_nested_renders_preserves_each_level
source = <<~LIQUID
{%- assign a = 1 -%}
{%- assign s1 = self -%}
{%- render "snippet1", outer: s1 -%}
LIQUID
partials = {
'snippet1' => <<~LIQUID,
{%- assign a = 2 -%}
{%- assign s2 = self -%}
{%- render "snippet2", outer: outer, middle: s2 -%}
LIQUID
'snippet2' => <<~LIQUID,
{%- assign a = 3 -%}
{{- outer.a }}|{{ middle.a }}|{{ self.a -}}
LIQUID
}
assert_template_result('1|2|3', source, partials: partials)
end
def test_self_drop_reflects_variables_assigned_after_creation
source = <<~LIQUID
{%- assign s = self -%}
{%- assign x = 42 %}{{ s.x -}}
LIQUID
assert_template_result('42', source)
end
def test_self_drop_context_setter_is_undefined
context = Context.new
drop = SelfDrop.new(context)
refute(drop.respond_to?(:context=))
assert_template_result('42', '{{ self.x }}', { 'x' => 42 })
end
def test_self_drop_repeated_lookups_compare_equal_for_same_context
context = Context.new
drop = context.find_variable("self")
cached_drop = context.find_variable("self")
assert_same(drop, cached_drop)
assert_equal(drop.object_id, cached_drop.object_id)
assert_equal(drop, cached_drop)
end
def test_assigned_self_drop_compares_equal_to_itself
assert_template_result('T', '{% assign s = self %}{% if s == s %}T{% else %}F{% endif %}')
end
def test_distinct_self_assignments_compare_equal_for_same_context
assert_template_result('T', '{% assign a = self %}{% assign b = self %}{% if a == b %}T{% else %}F{% endif %}')
end
def test_bare_self_compares_equal_to_bare_self
assert_template_result('T', '{% if self == self %}T{% else %}F{% endif %}')
end
def test_self_drop_with_strict_variables_does_not_raise_for_defined_var
t = Template.parse('{{ self.x }}')
result = t.render({ 'x' => 42 }, strict_variables: true)
assert_equal('42', result)
end
def test_self_drop_with_strict_variables_returns_nil_for_undefined_var
t = Template.parse('{{ self.x }}')
result = t.render({}, strict_variables: true)
assert_equal('', result)
end
def test_self_drop_can_be_passed_as_bare_drop_to_render
t = Template.parse('{{ self.x }}')
drop = SelfDrop.new(Context.new({ 'x' => 42 }))
result = t.render(drop)
assert_equal('42', result)
end
end
+4 -2
View File
@@ -105,8 +105,10 @@ class CycleTagTest < Minitest::Test
error1 = assert_raises(Liquid::SyntaxError) { Template.parse(template1) }
error2 = assert_raises(Liquid::SyntaxError) { Template.parse(template2) }
assert_match(/Liquid syntax error:/, error1.message)
assert_match(/Liquid syntax error: \[:dot, "."\] is not a valid expression/, error2.message)
expected_error = /Liquid syntax error: \[:dot, "."\] is not a valid expression/
assert_match(expected_error, error1.message)
assert_match(expected_error, error2.message)
end
end
-45
View File
@@ -439,49 +439,4 @@ class IncludeTagTest < Minitest::Test
assert_match(/Unexpected character =/, error.message)
end
end
def test_include_for_loop_true_with_for_keyword
with_error_modes(:lax, :strict, :strict2) do
template = Template.parse("{% include 'product' for products %}")
include_node = template.root.nodelist.first
assert(include_node.for_loop?, "Expected for_loop? to be true for 'for' keyword")
end
end
def test_include_for_loop_false_with_with_keyword
with_error_modes(:lax, :strict, :strict2) do
template = Template.parse("{% include 'product' with product %}")
include_node = template.root.nodelist.first
refute(include_node.for_loop?, "Expected for_loop? to be false for 'with' keyword")
end
end
def test_include_for_loop_false_without_keyword
with_error_modes(:lax, :strict, :strict2) do
template = Template.parse("{% include 'header' %}")
include_node = template.root.nodelist.first
refute(include_node.for_loop?, "Expected for_loop? to be false when no keyword")
end
end
def test_include_for_loop_with_alias
with_error_modes(:lax, :strict, :strict2) do
template = Template.parse("{% include 'product' for products as item %}")
include_node = template.root.nodelist.first
assert(include_node.for_loop?, "Expected for_loop? to be true for 'for' with alias")
end
end
def test_include_with_keyword_and_alias
with_error_modes(:lax, :strict, :strict2) do
template = Template.parse("{% include 'product' with products[0] as item %}")
include_node = template.root.nodelist.first
refute(include_node.for_loop?, "Expected for_loop? to be false for 'with' with alias")
end
end
end # IncludeTagTest
@@ -27,50 +27,4 @@ class IncrementTagTest < Minitest::Test
'{%decrement starboard %}',
)
end
def test_increment_strict2_rejects_invalid_variable_name
assert_raises(Liquid::SyntaxError) do
Template.parse('{% increment foo bar %}', error_mode: :strict2)
end
end
def test_increment_strict2_rejects_variable_starting_with_number
assert_raises(Liquid::SyntaxError) do
Template.parse('{% increment 11aa %}', error_mode: :strict2)
end
end
def test_increment_strict2_accepts_valid_variable_name
template = Template.parse('{% increment my-var %}', error_mode: :strict2)
assert_equal('0', template.render)
end
def test_decrement_strict2_rejects_invalid_variable_name
assert_raises(Liquid::SyntaxError) do
Template.parse('{% decrement foo bar %}', error_mode: :strict2)
end
end
def test_decrement_strict2_rejects_variable_starting_with_number
assert_raises(Liquid::SyntaxError) do
Template.parse('{% decrement 11aa %}', error_mode: :strict2)
end
end
def test_decrement_strict2_accepts_valid_variable_name
template = Template.parse('{% decrement my-var %}', error_mode: :strict2)
assert_equal('-1', template.render)
end
def test_increment_strict2_rejects_empty_variable_name
assert_raises(Liquid::SyntaxError) do
Template.parse('{% increment %}', error_mode: :strict2)
end
end
def test_decrement_strict2_rejects_empty_variable_name
assert_raises(Liquid::SyntaxError) do
Template.parse('{% decrement %}', error_mode: :strict2)
end
end
end
+4 -4
View File
@@ -174,16 +174,16 @@ class RenderTagTest < Minitest::Test
def test_increment_is_isolated_between_renders
assert_template_result(
'010',
'{% increment port %}{% increment port %}{% render "incr" %}',
partials: { 'incr' => '{% increment port %}' },
'{% increment %}{% increment %}{% render "incr" %}',
partials: { 'incr' => '{% increment %}' },
)
end
def test_decrement_is_isolated_between_renders
assert_template_result(
'-1-2-1',
'{% decrement port %}{% decrement port %}{% render "decr" %}',
partials: { 'decr' => '{% decrement port %}' },
'{% decrement %}{% decrement %}{% render "decr" %}',
partials: { 'decr' => '{% decrement %}' },
)
end
-360
View File
@@ -1,360 +0,0 @@
# frozen_string_literal: true
require_relative "../test_helper"
require "tmpdir"
class TemplateRecorderTest < Minitest::Test
class ProductDrop < Liquid::Drop
def initialize(title, secret)
super()
@title = title
@secret = secret
end
attr_reader :title
def details
DetailsDrop.new
end
end
class DetailsDrop < Liquid::Drop
def count
3
end
end
class LegacyFileSystem
attr_reader :reads
def initialize
@reads = []
end
def read_template_file(name)
@reads << name
"partial={{ product.title }}"
end
end
class UnsupportedLiquidValue
def to_liquid
self
end
def to_s
"unsupported"
end
end
class WrapperTag < Liquid::Block
end
class MarkerTag < Liquid::Tag
def render(_context)
"custom"
end
end
class CollectingWriter
attr_reader :records
def initialize
@records = []
end
def write(record)
@records << record
end
end
def setup
@directory = Dir.mktmpdir
end
def teardown
FileUtils.remove_entry(@directory)
end
def path(name = "recording.json")
File.join(@directory, name)
end
def test_records_and_verifies_a_render_without_changing_drop_behavior
product = ProductDrop.new("Computed", "must not be recorded")
template_source = "{{ product.title }} ({{ product.details.count }})"
output = Liquid::TemplateRecorder.record(path) do
Liquid::Template.parse(template_source).render!("product" => product)
end
assert_equal("Computed (3)", output)
record = Liquid::TemplateRecorder.records(path).first
assert_equal({ "title" => "Computed", "details" => { "count" => 3 } }, record.dig("assigns", "product"))
refute_includes(File.read(path), "must not be recorded")
assert_equal(output, Liquid::TemplateRecorder.replay_from(path, mode: :verify).render)
end
def test_preserves_the_legacy_one_argument_file_system_contract
file_system = LegacyFileSystem.new
environment = Liquid::Environment.build { |env| env.file_system = file_system }
output = Liquid::TemplateRecorder.record(path) do
Liquid::Template.parse("before {% include 'card' %}", environment: environment)
.render!("product" => ProductDrop.new("Hat", "secret"))
end
assert_equal("before partial=Hat", output)
assert_equal(["card"], file_system.reads)
record = Liquid::TemplateRecorder.records(path).first
assert_equal({ "card" => "partial={{ product.title }}" }, record["file_system"])
assert_equal(2, record["templates"].length)
assert_equal(output, Liquid::TemplateRecorder.replay_from(path, mode: :verify).render)
end
def test_jsonl_appends_one_self_contained_record_per_render
recording = path("renders.jsonl")
2.times do |sequence|
Liquid::TemplateRecorder.record(recording) do
Liquid::Template.parse("value={{ value }}").render!("value" => sequence)
end
end
assert_equal(2, File.readlines(recording).length)
assert_equal(["value=0", "value=1"], Liquid::TemplateRecorder.records(recording).map { |item| item["output"] })
assert_equal("value=0", Liquid::TemplateRecorder.replay_from(recording, index: 0).render)
assert_equal("value=1", Liquid::TemplateRecorder.replay_from(recording).render)
end
def test_json_session_supports_multiple_renders
Liquid::TemplateRecorder.record(path) do
Liquid::Template.parse("one={{ value }}").render!("value" => 1)
Liquid::Template.new.parse("two={{ value }}").render!("value" => 2)
end
records = Liquid::TemplateRecorder.records(path)
assert_equal(["one=1", "two=2"], records.map { |item| item["output"] })
assert_equal("two=2", Liquid::TemplateRecorder.replay_from(path).render)
end
def test_failed_recording_does_not_delete_an_existing_json_file
File.write(path, "existing")
assert_raises(RuntimeError) do
Liquid::TemplateRecorder.record(path) { raise "boom" }
end
assert_equal("existing", File.read(path))
end
def test_render_failure_is_not_written_to_jsonl
recording = path("renders.jsonl")
assert_raises(Liquid::UndefinedVariable) do
Liquid::TemplateRecorder.record(recording) do
Liquid::Template.parse("{{ missing }}").render!(nil, strict_variables: true)
end
end
refute_path_exists(recording)
end
def test_tampered_template_is_rejected
Liquid::TemplateRecorder.record(path) { Liquid::Template.parse("safe").render! }
session = JSON.parse(File.read(path))
session["renders"][0]["template"]["source"] = "changed"
File.write(path, JSON.generate(session))
error = assert_raises(Liquid::TemplateRecorder::ReplayError) do
Liquid::TemplateRecorder.replay_from(path)
end
assert_match(/checksum/, error.message)
end
def test_recordings_are_thread_local
paths = [path("a.json"), path("b.json")]
ready = Queue.new
release = Queue.new
threads = Array.new(2) do |index|
Thread.new do
Liquid::TemplateRecorder.record(paths[index]) do
ready << true
release.pop
Liquid::Template.parse("thread={{ value }}").render!("value" => index)
end
end
end
2.times { ready.pop }
2.times { release << true }
threads.each(&:join)
assert_equal("thread=0", Liquid::TemplateRecorder.records(paths[0]).first["output"])
assert_equal("thread=1", Liquid::TemplateRecorder.records(paths[1]).first["output"])
end
def test_nested_sessions_fail_without_corrupting_outer_session
error = nil
Liquid::TemplateRecorder.record(path) do
error = assert_raises(Liquid::TemplateRecorder::Error) do
Liquid::TemplateRecorder.record(path("inner.json")) { flunk }
end
Liquid::Template.parse("outer").render!
end
assert_match(/nested/, error.message)
assert_equal("outer", Liquid::TemplateRecorder.replay_from(path).render)
end
def test_supported_render_argument_forms_keep_working
filter = Module.new do
def decorate(input)
"[#{input}]"
end
end
template = nil
context = Liquid::Context.new([{ "value" => "context" }])
Liquid::TemplateRecorder.record(path) do
template = Liquid::Template.parse("{{ value | decorate }}")
assert_equal("[hash]", template.render({ "value" => "hash" }, filter))
context.add_filters(filter)
assert_equal("[context]", template.render(context))
end
assert_equal(2, Liquid::TemplateRecorder.records(path).length)
end
def test_strict_replay_uses_exact_recorded_filter_outputs
filter = Module.new do
def external_lookup(_input)
"x" * 150
end
end
output = Liquid::TemplateRecorder.record(path) do
Liquid::Template.parse("{{ key | external_lookup }}").render!({ "key" => "a" }, filter)
end
assert_equal("x" * 150, output)
assert_equal(output, Liquid::TemplateRecorder.replay_from(path, mode: :strict).render)
assert_equal("a", Liquid::TemplateRecorder.replay_from(path, mode: :compute).render)
end
def test_jsonl_reader_ignores_only_a_truncated_final_record
recording = path("renders.jsonl")
Liquid::TemplateRecorder.record(recording) { Liquid::Template.parse("complete").render! }
File.open(recording, "ab") { |file| file.write('{"format":') }
assert_equal(["complete"], Liquid::TemplateRecorder.records(recording).map { |item| item["output"] })
end
def test_accepts_a_pluggable_writer
writer = CollectingWriter.new
output = Liquid::TemplateRecorder.record(writer) do
Liquid::Template.parse("Hello {{ name }}").render!("name" => "Shopify")
end
assert_equal("Hello Shopify", output)
assert_equal(["Hello Shopify"], writer.records.map { |record| record["output"] })
end
def test_records_only_variables_resolved_by_the_template
unused = Object.new
assigns = { "visible" => "yes", "unused" => unused }
Liquid::TemplateRecorder.record(path) do
Liquid::Template.parse("{{ visible }}").render!(assigns)
end
assert_equal({ "visible" => "yes" }, Liquid::TemplateRecorder.records(path).first["assigns"])
end
def test_recording_scope_is_fiber_local
writer = CollectingWriter.new
ordinary_output = nil
Liquid::TemplateRecorder.record(writer) do
Fiber.new do
ordinary_output = Liquid::Template.parse("ordinary").render!
end.resume
Liquid::Template.parse("recorded").render!
end
assert_equal("ordinary", ordinary_output)
assert_equal(["recorded"], writer.records.map { |record| record["output"] })
end
def test_on_error_keeps_recording_failures_out_of_the_render_path
writer = Object.new
writer.define_singleton_method(:write) { |_record| raise "sink unavailable" }
errors = []
output = Liquid::TemplateRecorder.record(writer, on_error: errors.method(:<<)) do
Liquid::Template.parse("still rendered").render!
end
assert_equal("still rendered", output)
assert_equal(["sink unavailable"], errors.map(&:message))
end
def test_unsupported_accessed_values_do_not_affect_the_render
value = UnsupportedLiquidValue.new
writer = CollectingWriter.new
output = Liquid::TemplateRecorder.record(writer) do
Liquid::Template.parse("{{ value }}").render!("value" => value)
end
assert_equal(value.to_s, output)
assert_equal({}, writer.records.first["assigns"])
end
def test_strict_replay_accepts_the_application_environment
environment = Liquid::Environment.build(
tags: Liquid::Environment.default.tags.merge("marker" => MarkerTag),
)
writer = CollectingWriter.new
Liquid::TemplateRecorder.record(writer) do
Liquid::Template.parse("{% marker %}", environment: environment).render!
end
replay = Liquid::TemplateRecorder::Replayer.new(
writer.records.first,
mode: :strict,
environment: environment,
)
assert_equal("custom", replay.render)
end
def test_captures_file_reads_that_happen_before_the_template_starts_rendering
writer = CollectingWriter.new
Liquid::TemplateRecorder.record(writer) do
Liquid::TemplateRecorder.current.emit_file_read("card", "Card")
file_system = LegacyFileSystem.new
Liquid::Template.parse("{% render 'card' %}").render!({}, registers: { file_system: file_system })
end
replay = Liquid::TemplateRecorder::Replayer.new(writer.records.first, mode: :strict)
assert_equal("partial=", replay.render)
end
def test_strict_replay_skips_nested_custom_tag_calls
environment = Liquid::Environment.build(
tags: Liquid::Environment.default.tags.merge("wrapper" => WrapperTag, "marker" => MarkerTag),
)
writer = CollectingWriter.new
Liquid::TemplateRecorder.record(writer) do
Liquid::Template.parse("{% wrapper %}{% marker %}{% endwrapper %}", environment: environment).render!
end
record = writer.records.first
replay = Liquid::TemplateRecorder::Replayer.new(record, mode: :strict, environment: environment)
assert_equal(["wrapper"], record["tag_calls"].map { |call| call["name"] })
assert_equal("custom", replay.render)
end
end
+5 -5
View File
@@ -55,7 +55,7 @@ class VariableTest < Minitest::Test
def test_expression_with_whitespace_in_square_brackets
assert_template_result('result', "{{ a[ 'b' ] }}", { 'a' => { 'b' => 'result' } })
assert_template_result('result', "{{ a[ self[ 'b' ] ] }}", { 'b' => 'c', 'a' => { 'c' => 'result' } })
assert_template_result('result', "{{ a[ [ 'b' ] ] }}", { 'b' => 'c', 'a' => { 'c' => 'result' } })
end
def test_ignore_unknown
@@ -135,17 +135,17 @@ class VariableTest < Minitest::Test
end
def test_dynamic_find_var
assert_template_result('bar', '{{ self[key] }}', { 'key' => 'foo', 'foo' => 'bar' })
assert_template_result('bar', '{{ [key] }}', { 'key' => 'foo', 'foo' => 'bar' })
end
def test_raw_value_variable
assert_template_result('bar', '{{ self[key] }}', { 'key' => 'foo', 'foo' => 'bar' })
assert_template_result('bar', '{{ [key] }}', { 'key' => 'foo', 'foo' => 'bar' })
end
def test_dynamic_find_var_with_drop
assert_template_result(
'bar',
'{{ self[list[settings.zero]] }}',
'{{ [list[settings.zero]] }}',
{
'list' => ['foo'],
'settings' => SettingsDrop.new("zero" => 0),
@@ -155,7 +155,7 @@ class VariableTest < Minitest::Test
assert_template_result(
'foo',
'{{ self[list[settings.zero]["foo"]] }}',
'{{ [list[settings.zero]["foo"]] }}',
{
'list' => [{ 'foo' => 'bar' }],
'settings' => SettingsDrop.new("zero" => 0),
+256
View File
@@ -0,0 +1,256 @@
# frozen_string_literal: true
require 'test_helper'
# Tests for the byte-walking fast paths introduced in the ByteTables optimization.
# Each fast path is tested for equivalence with the original regex-based code path
# it replaces, covering accepted inputs, rejected inputs, and boundary cases.
class FastPathUnitTest < Minitest::Test
# ── Expression.parse_number ────────────────────────────────────────
# Fast path: byte-walk with ByteTables::DIGIT
# Replaces: INTEGER_REGEX, FLOAT_REGEX, StringScanner loop
def test_parse_number_simple_integers
assert_equal(42, Liquid::Expression.parse_number("42"))
assert_equal(0, Liquid::Expression.parse_number("0"))
assert_equal(999, Liquid::Expression.parse_number("999"))
end
def test_parse_number_negative_integers
assert_equal(-7, Liquid::Expression.parse_number("-7"))
assert_equal(-0, Liquid::Expression.parse_number("-0"))
assert_equal(-123, Liquid::Expression.parse_number("-123"))
end
def test_parse_number_simple_floats
assert_equal(3.14, Liquid::Expression.parse_number("3.14"))
assert_equal(0.5, Liquid::Expression.parse_number("0.5"))
assert_equal(-0.5, Liquid::Expression.parse_number("-0.5"))
assert_equal(100.0, Liquid::Expression.parse_number("100.0"))
end
def test_parse_number_trailing_dot
# "123." → 123.0 (truncate before dot)
assert_equal(123.0, Liquid::Expression.parse_number("123."))
assert_equal(0.0, Liquid::Expression.parse_number("0."))
end
def test_parse_number_multi_dot_floats
# "1.2.3" → 1.2 (truncate at second dot)
assert_equal(1.2, Liquid::Expression.parse_number("1.2.3"))
assert_equal(1.2, Liquid::Expression.parse_number("1.2.3.4"))
assert_equal(0.0, Liquid::Expression.parse_number("0.0.0"))
end
def test_parse_number_rejects_non_numeric
assert_nil(Liquid::Expression.parse_number("hello"))
assert_nil(Liquid::Expression.parse_number(""))
assert_nil(Liquid::Expression.parse_number("abc123"))
assert_nil(Liquid::Expression.parse_number(".5"))
assert_nil(Liquid::Expression.parse_number("-.5"))
end
def test_parse_number_rejects_trailing_alpha_after_multi_dot
# "1.2.3a" must be nil, not 1.2 — these are not valid numbers
assert_nil(Liquid::Expression.parse_number("1.2.3a"))
assert_nil(Liquid::Expression.parse_number("1.2.3.4a"))
assert_nil(Liquid::Expression.parse_number("1.2.34a"))
assert_nil(Liquid::Expression.parse_number("-1.2.3a"))
end
def test_parse_number_rejects_bare_dash
assert_nil(Liquid::Expression.parse_number("-"))
assert_nil(Liquid::Expression.parse_number("-a"))
end
# ── Expression.parse strip guard ───────────────────────────────────
# Fast path: skip String#strip when no leading/trailing whitespace
# Must produce identical results to unconditional .strip
def test_parse_strips_leading_whitespace
assert_equal(42, Liquid::Expression.parse(" 42"))
assert_equal(42, Liquid::Expression.parse("\t42"))
assert_equal(42, Liquid::Expression.parse("\n42"))
end
def test_parse_strips_trailing_whitespace
assert_equal(42, Liquid::Expression.parse("42 "))
assert_equal(42, Liquid::Expression.parse("42\t"))
assert_equal(42, Liquid::Expression.parse("42\n"))
end
def test_parse_strips_both_sides
assert_equal(42, Liquid::Expression.parse(" 42 "))
assert_equal("hello", Liquid::Expression.parse(" 'hello' "))
end
def test_parse_no_strip_needed
assert_equal(42, Liquid::Expression.parse("42"))
assert_equal("hello", Liquid::Expression.parse("'hello'"))
assert_equal(true, Liquid::Expression.parse("true"))
end
def test_parse_strips_null_bytes
# String#strip removes \x00 — the WHITESPACE table must match
assert_equal(true, Liquid::Expression.parse("\x00true"))
assert_equal(true, Liquid::Expression.parse("true\x00"))
end
# ── VariableLookup.simple_lookup? ──────────────────────────────────
# Fast path: regex gate for simple a.b.c chains
# Must accept only inputs the byte-walk can handle correctly
def test_simple_lookup_accepts_single_names
assert(Liquid::VariableLookup.simple_lookup?("product"))
assert(Liquid::VariableLookup.simple_lookup?("x"))
assert(Liquid::VariableLookup.simple_lookup?("_private"))
end
def test_simple_lookup_accepts_dotted_chains
assert(Liquid::VariableLookup.simple_lookup?("product.title"))
assert(Liquid::VariableLookup.simple_lookup?("a.b.c.d"))
end
def test_simple_lookup_accepts_question_marks
assert(Liquid::VariableLookup.simple_lookup?("product.available?"))
assert(Liquid::VariableLookup.simple_lookup?("empty?"))
end
def test_simple_lookup_accepts_hyphens
assert(Liquid::VariableLookup.simple_lookup?("my-var"))
assert(Liquid::VariableLookup.simple_lookup?("my-var.some-field"))
end
def test_simple_lookup_rejects_brackets
refute(Liquid::VariableLookup.simple_lookup?("product[0]"))
refute(Liquid::VariableLookup.simple_lookup?("hash['key']"))
refute(Liquid::VariableLookup.simple_lookup?("[0]"))
end
def test_simple_lookup_rejects_empty_and_malformed
refute(Liquid::VariableLookup.simple_lookup?(""))
refute(Liquid::VariableLookup.simple_lookup?(".leading"))
refute(Liquid::VariableLookup.simple_lookup?("trailing."))
refute(Liquid::VariableLookup.simple_lookup?("a..b"))
end
# ── VariableLookup fast path equivalence ───────────────────────────
# The fast path must produce identical name, lookups, and command_flags
# to the original VariableParser regex path
def test_fast_path_simple_name
vl = Liquid::VariableLookup.new("product")
assert_equal("product", vl.name)
assert_equal([], vl.lookups)
end
def test_fast_path_dotted_chain
vl = Liquid::VariableLookup.new("product.title")
assert_equal("product", vl.name)
assert_equal(["title"], vl.lookups)
end
def test_fast_path_deep_chain
vl = Liquid::VariableLookup.new("a.b.c.d")
assert_equal("a", vl.name)
assert_equal(["b", "c", "d"], vl.lookups)
end
def test_fast_path_command_methods
vl = Liquid::VariableLookup.new("items.size")
assert_equal("items", vl.name)
assert_equal(["size"], vl.lookups)
assert(vl.lookup_command?(0))
vl2 = Liquid::VariableLookup.new("items.first")
assert(vl2.lookup_command?(0))
vl3 = Liquid::VariableLookup.new("items.last")
assert(vl3.lookup_command?(0))
end
def test_fast_path_non_command_lookups
vl = Liquid::VariableLookup.new("product.title")
refute(vl.lookup_command?(0))
end
def test_fast_path_question_mark
vl = Liquid::VariableLookup.new("product.available?")
assert_equal("product", vl.name)
assert_equal(["available?"], vl.lookups)
end
def test_bracket_lookup_falls_to_regex_path
vl = Liquid::VariableLookup.new("product[0]")
assert_equal("product", vl.name)
assert_equal([0], vl.lookups)
end
# ── BlockBody.try_parse_tag_token ──────────────────────────────────
# Fast path: byte-walk tag tokens instead of FullToken regex
# Must produce identical [tag_name, markup, newline_count] or nil
def test_tag_token_simple
body = Liquid::BlockBody.new
result = body.send(:try_parse_tag_token, "{% if x %}")
assert_equal(["if", "x ", 0], result)
end
def test_tag_token_whitespace_control_leading
body = Liquid::BlockBody.new
result = body.send(:try_parse_tag_token, "{%- if x %}")
assert_equal(["if", "x ", 0], result)
end
def test_tag_token_whitespace_control_trailing
body = Liquid::BlockBody.new
result = body.send(:try_parse_tag_token, "{% if x -%}")
assert_equal(["if", "x ", 0], result)
end
def test_tag_token_whitespace_control_both
body = Liquid::BlockBody.new
result = body.send(:try_parse_tag_token, "{%- if x -%}")
assert_equal(["if", "x ", 0], result)
end
def test_tag_token_no_markup
body = Liquid::BlockBody.new
result = body.send(:try_parse_tag_token, "{% endif %}")
assert_equal(["endif", "", 0], result)
end
def test_tag_token_hash_comment
body = Liquid::BlockBody.new
result = body.send(:try_parse_tag_token, "{% # this is a comment %}")
assert_equal(["#", "this is a comment ", 0], result)
end
def test_tag_token_with_newlines
body = Liquid::BlockBody.new
result = body.send(:try_parse_tag_token, "{% \n if \n x %}")
assert_equal(["if", "x ", 2], result)
end
def test_tag_token_hyphenated_name_stops_at_hyphen
# TagName = /\w+/ does not include hyphens
body = Liquid::BlockBody.new
result = body.send(:try_parse_tag_token, "{% my-tag markup %}")
assert_equal("my", result[0])
end
def test_tag_token_complex_markup
body = Liquid::BlockBody.new
result = body.send(:try_parse_tag_token, "{% for item in collection reversed %}")
assert_equal("for", result[0])
assert_equal("item in collection reversed ", result[1])
end
def test_tag_token_malformed_returns_nil
body = Liquid::BlockBody.new
# Token too short
assert_nil(body.send(:try_parse_tag_token, "{%"))
# No valid tag name start (digit)
assert_nil(body.send(:try_parse_tag_token, "{% 123 %}"))
end
end