Compare commits

...
Author SHA1 Message Date
Michael Go fb6ac72520 use byteslice to create Variable BlockBody 2024-10-25 15:41:35 -03:00
Michael Go cb16219552 faster BlockBody variable matching 2024-10-25 15:22:56 -03:00
Ian Ker-SeymerandGitHub b3553787c8 Speed up the lexer for Ruby 3.4+ (#1832)
* Speed up lexing

* Bump msrv to 3.0 (from 2.7)

* Normalize test for ruby-head compat

* Fix bug when parsing negative numbers
2024-10-23 14:15:33 -04:00
Ian Ker-SeymerandGitHub b233b3d081 Bump to v5.6.0.alpha (#1819) 2024-08-15 16:12:16 -04:00
Alex CocoandGitHub ac91d31268 Merge pull request #1818 from Shopify/tablerow-interrupt
Handle interrupts in table row
2024-08-15 14:04:35 -04:00
Alex Coco 9067e5167a Handle interrupts in table row 2024-08-15 11:56:29 -04:00
Ian Ker-SeymerandGitHub fb6634f454 Add concept of Liquid::Environment (#1815)
* Add concept of `Liquid::World`

* Rename `World` to `Environment`
2024-08-07 15:00:44 -04:00
Ian Ker-SeymerandIan Ker-Seymer a0411e0927 Bump to v5.5.1 2024-07-22 23:25:56 -04:00
Ian Ker-SeymerandGitHub ed421202e2 Merge pull request #1811 from Shopify/marco-cycle-update
Add named? method to Cycle class for checking if the cycle is named
2024-07-16 15:58:21 -04:00
Ian Ker-Seymer d6ca569e8a Require base64 2024-07-16 15:56:47 -04:00
Marco Concetto RudilossoandIan Ker-Seymer d36937d17f Add named? method to Cycle class for checking if the cycle is named 2024-07-16 15:52:25 -04:00
Peter ZhuandGitHub 77bc56a1c2 Merge pull request #1792 from Shopify/centralize_ruby_version
Centralize Ruby Version to `.ruby-version`
2024-04-11 11:12:08 -04:00
Jenny Shen 88d013c8da Use latest compatible Bundler of each Ruby version run 2024-04-05 10:33:27 -04:00
Jenny Shen 36c7fc8e07 Remove ruby version definition in CI/CD
it will be read from .ruby-version
2024-04-05 10:33:06 -04:00
Jenny Shen 2b4810006b Remove TargetRubyVersion in RuboCop config
reads from required_ruby_version in RuboCop 1.61+
2024-04-05 10:23:39 -04:00
Jenny Shen fc4f19471e Update rubocop to 1.61.0 2024-04-05 10:23:37 -04:00
Jenny Shen 8596bb2e38 Commit a .ruby-version 2024-04-05 10:23:05 -04:00
53 changed files with 785 additions and 242 deletions
+3 -3
View File
@@ -11,10 +11,10 @@ jobs:
strategy:
matrix:
entry:
- { ruby: 2.7, allowed-failure: false } # minimum supported
- { ruby: 3.0, allowed-failure: false } # minimum supported
- { ruby: 3.2, allowed-failure: false }
- { ruby: 3.3, allowed-failure: false } # latest
- { ruby: ruby-head, allowed-failure: true }
- { ruby: ruby-head, allowed-failure: false }
name: Test Ruby ${{ matrix.entry.ruby }}
steps:
- uses: actions/checkout@v3
@@ -22,6 +22,7 @@ jobs:
with:
ruby-version: ${{ matrix.entry.ruby }}
bundler-cache: true
bundler: latest
- run: bundle exec rake
continue-on-error: ${{ matrix.entry.allowed-failure }}
@@ -31,6 +32,5 @@ jobs:
- uses: actions/checkout@v3
- uses: ruby/setup-ruby@v1
with:
ruby-version: 3.3
bundler-cache: true
- run: bundle exec rake memory_profile:run
+1 -2
View File
@@ -4,7 +4,6 @@
pkg
*.rbc
.rvmrc
.ruby-version
Gemfile.lock
.bundle
.byebug_history
Gemfile.lock
-1
View File
@@ -10,7 +10,6 @@ Performance:
Enabled: true
AllCops:
TargetRubyVersion: 2.7
NewCops: disable
SuggestExtensions: false
Exclude:
+1
View File
@@ -0,0 +1 @@
3.3.4
+3 -1
View File
@@ -7,6 +7,8 @@ end
gemspec
gem "base64"
group :benchmark, :test do
gem 'benchmark-ips'
gem 'memory_profiler'
@@ -18,7 +20,7 @@ group :benchmark, :test do
end
group :test do
gem 'rubocop', '~> 1.44.0'
gem 'rubocop', '~> 1.61.0'
gem 'rubocop-shopify', '~> 2.12.0', require: false
gem 'rubocop-performance', require: false
+45 -3
View File
@@ -52,6 +52,47 @@ For standard use you can just pass it the content of a file and call render with
@template.render('name' => 'tobi') # => "hi tobi"
```
### Concept of Environments
In Liquid, a "Environment" is a scoped environment that encapsulates custom tags, filters, and other configurations. This allows you to define and isolate different sets of functionality for different contexts, avoiding global overrides that can lead to conflicts and unexpected behavior.
By using environments, you can:
1. **Encapsulate Logic**: Keep the logic for different parts of your application separate.
2. **Avoid Conflicts**: Prevent custom tags and filters from clashing with each other.
3. **Improve Maintainability**: Make it easier to manage and understand the scope of customizations.
4. **Enhance Security**: Limit the availability of certain tags and filters to specific contexts.
We encourage the use of Environments over globally overriding things because it promotes better software design principles such as modularity, encapsulation, and separation of concerns.
Here's an example of how you can define and use Environments in Liquid:
```ruby
user_environment = Liquid::Environment.build do |environment|
environment.register_tag("renderobj", RenderObjTag)
end
Liquid::Template.parse(<<~LIQUID, environment: user_environment)
{% renderobj src: "path/to/model.obj" %}
LIQUID
```
In this example, `RenderObjTag` is a custom tag that is only available within the `user_environment`.
Similarly, you can define another environment for a different context, such as email templates:
```ruby
email_environment = Liquid::Environment.build do |environment|
environment.register_tag("unsubscribe_footer", UnsubscribeFooter)
end
Liquid::Template.parse(<<~LIQUID, environment: email_environment)
{% unsubscribe_footer %}
LIQUID
```
By using Environments, you ensure that custom tags and filters are only available in the contexts where they are needed, making your Liquid templates more robust and easier to manage.
### Error Modes
Setting the error mode of Liquid lets you specify how strictly you want your templates to be interpreted.
@@ -62,9 +103,10 @@ Liquid also comes with a stricter parser that can be used when editing templates
when templates are invalid. You can enable this new parser like this:
```ruby
Liquid::Template.error_mode = :strict # Raises a SyntaxError when invalid syntax is used
Liquid::Template.error_mode = :warn # Adds strict errors to template.errors but continues as normal
Liquid::Template.error_mode = :lax # The default mode, accepts almost anything.
Liquid::Environment.default.error_mode = :strict
Liquid::Environment.default.error_mode = :strict # Raises a SyntaxError when invalid syntax is used
Liquid::Environment.default.error_mode = :warn # Adds strict errors to template.errors but continues as normal
Liquid::Environment.default.error_mode = :lax # The default mode, accepts almost anything.
```
If you want to set the error mode only on specific templates you can pass `:error_mode` as an option to `parse`:
+8
View File
@@ -81,6 +81,14 @@ namespace :benchmark do
task :strict do
ruby "./performance/benchmark.rb strict"
end
desc "Run unit benchmarks"
task :unit do
Dir["./performance/unit/*_benchmark.rb"].each do |file|
puts "🧪 Running #{file}"
ruby file
end
end
end
namespace :profile do
+11 -10
View File
@@ -50,7 +50,18 @@ module Liquid
end
require "liquid/version"
require "liquid/deprecations"
require "liquid/const"
require "liquid/template/tag_registry"
require 'liquid/standardfilters'
require 'liquid/file_system'
require 'liquid/parser_switching'
require 'liquid/tag'
require 'liquid/block'
require 'liquid/parse_tree_visitor'
require 'liquid/interrupts'
require 'liquid/tags'
require "liquid/environment"
require 'liquid/lexer'
require 'liquid/parser'
require 'liquid/i18n'
@@ -64,20 +75,14 @@ require 'liquid/strainer_template'
require 'liquid/strainer_factory'
require 'liquid/expression'
require 'liquid/context'
require 'liquid/parser_switching'
require 'liquid/tag'
require 'liquid/tag/disabler'
require 'liquid/tag/disableable'
require 'liquid/block'
require 'liquid/block_body'
require 'liquid/document'
require 'liquid/variable'
require 'liquid/variable_lookup'
require 'liquid/range_lookup'
require 'liquid/file_system'
require 'liquid/resource_limits'
require 'liquid/template'
require 'liquid/standardfilters'
require 'liquid/condition'
require 'liquid/utils'
require 'liquid/tokenizer'
@@ -86,7 +91,3 @@ require 'liquid/partial_cache'
require 'liquid/usage'
require 'liquid/registers'
require 'liquid/template_factory'
# Load all the tags of the standard library
#
Dir["#{__dir__}/liquid/tags/*.rb"].each { |f| require f }
+11 -8
View File
@@ -52,7 +52,7 @@ module Liquid
next parse_liquid_tag(markup, parse_context)
end
unless (tag = registered_tags[tag_name])
unless (tag = parse_context.environment.tag_for_name(tag_name))
# end parsing if we reach an unknown tag and let the caller decide
# determine how to proceed
return yield tag_name, markup
@@ -147,7 +147,7 @@ module Liquid
next
end
unless (tag = registered_tags[tag_name])
unless (tag = parse_context.environment.tag_for_name(tag_name))
# end parsing if we reach an unknown tag and let the caller decide
# determine how to proceed
return yield tag_name, markup
@@ -246,10 +246,17 @@ module Liquid
end
def create_variable(token, parse_context)
if token =~ ContentOfVariable
markup = Regexp.last_match(1)
if token.end_with?("}}")
i = 2
i = 3 if token[i] == "-"
parse_end = token.length - 3
parse_end -= 1 if token[parse_end] == "-"
markup_end = parse_end - i + 1
markup = markup_end <= 0 ? "" : token.byteslice(i, markup_end)
return Variable.new(markup, parse_context)
end
BlockBody.raise_missing_variable_terminator(token, parse_context)
end
@@ -262,9 +269,5 @@ module Liquid
def raise_missing_variable_terminator(token, parse_context)
BlockBody.raise_missing_variable_terminator(token, parse_context)
end
def registered_tags
Template.tags
end
end
end
+8
View File
@@ -0,0 +1,8 @@
# frozen_string_literal: true
module Liquid
module Const
EMPTY_HASH = {}.freeze
EMPTY_ARRAY = [].freeze
end
end
+9 -8
View File
@@ -15,24 +15,25 @@ module Liquid
# context['bob'] #=> nil class Context
class Context
attr_reader :scopes, :errors, :registers, :environments, :resource_limits, :static_registers, :static_environments
attr_accessor :exception_renderer, :template_name, :partial, :global_filter, :strict_variables, :strict_filters
attr_accessor :exception_renderer, :template_name, :partial, :global_filter, :strict_variables, :strict_filters, :environment
# rubocop:disable Metrics/ParameterLists
def self.build(environments: {}, outer_scope: {}, registers: {}, rethrow_errors: false, resource_limits: nil, static_environments: {}, &block)
def self.build(environment: Environment.default, environments: {}, outer_scope: {}, registers: {}, rethrow_errors: false, resource_limits: nil, static_environments: {}, &block)
new(environments, outer_scope, registers, rethrow_errors, resource_limits, static_environments, &block)
end
def initialize(environments = {}, outer_scope = {}, registers = {}, rethrow_errors = false, resource_limits = nil, static_environments = {})
def initialize(environments = {}, outer_scope = {}, registers = {}, rethrow_errors = false, resource_limits = nil, static_environments = {}, environment = Environment.default)
@environment = environment
@environments = [environments]
@environments.flatten!
@static_environments = [static_environments].flatten(1).freeze
@scopes = [(outer_scope || {})]
@scopes = [outer_scope || {}]
@registers = registers.is_a?(Registers) ? registers : Registers.new(registers)
@errors = []
@partial = false
@strict_variables = false
@resource_limits = resource_limits || ResourceLimits.new(Template.default_resource_limits)
@resource_limits = resource_limits || ResourceLimits.new(environment.default_resource_limits)
@base_scope_depth = 0
@interrupts = []
@filters = []
@@ -40,10 +41,10 @@ module Liquid
@disabled_tags = {}
@registers.static[:cached_partials] ||= {}
@registers.static[:file_system] ||= Liquid::Template.file_system
@registers.static[:file_system] ||= environment.file_system
@registers.static[:template_factory] ||= Liquid::TemplateFactory.new
self.exception_renderer = Template.default_exception_renderer
self.exception_renderer = environment.exception_renderer
if rethrow_errors
self.exception_renderer = Liquid::RAISE_EXCEPTION_LAMBDA
end
@@ -60,7 +61,7 @@ module Liquid
end
def strainer
@strainer ||= StrainerFactory.create(self, @filters)
@strainer ||= @environment.create_strainer(self, @filters)
end
# Adds filters to this context.
+22
View File
@@ -0,0 +1,22 @@
# frozen_string_literal: true
require "set"
module Liquid
class Deprecations
class << self
attr_accessor :warned
Deprecations.warned = Set.new
def warn(name, alternative)
return if warned.include?(name)
warned << name
caller_location = caller_locations(2, 1).first
Warning.warn("[DEPRECATION] #{name} is deprecated. Use #{alternative} instead. Called from #{caller_location}\n")
end
end
end
end
+159
View File
@@ -0,0 +1,159 @@
# frozen_string_literal: true
module Liquid
# The Environment is the container for all configuration options of Liquid, such as
# the registered tags, filters, and the default error mode.
class Environment
# The default error mode for all templates. This can be overridden on a
# per-template basis.
attr_accessor :error_mode
# The tags that are available to use in the template.
attr_accessor :tags
# The strainer template which is used to store filters that are available to
# use in templates.
attr_accessor :strainer_template
# The exception renderer that is used to render exceptions that are raised
# when rendering a template
attr_accessor :exception_renderer
# The default file system that is used to load templates from.
attr_accessor :file_system
# The default resource limits that are used to limit the resources that a
# template can consume.
attr_accessor :default_resource_limits
class << self
# Creates a new environment instance.
#
# @param tags [Hash] The tags that are available to use in
# the template.
# @param file_system The default file system that is used
# to load templates from.
# @param error_mode [Symbol] The default error mode for all templates
# (either :strict, :warn, or :lax).
# @param exception_renderer [Proc] The exception renderer that is used to
# render exceptions.
# @yieldparam environment [Environment] The environment instance that is being built.
# @return [Environment] The new environment instance.
def build(tags: nil, file_system: nil, error_mode: nil, exception_renderer: nil)
ret = new
ret.tags = Template::TagRegistry.new(tags) if tags
ret.file_system = file_system if file_system
ret.error_mode = error_mode if error_mode
ret.exception_renderer = exception_renderer if exception_renderer
yield ret if block_given?
ret.freeze
end
# Returns the default environment instance.
#
# @return [Environment] The default environment instance.
def default
@default ||= new
end
# Sets the default environment instance for the duration of the block
#
# @param environment [Environment] The environment instance to use as the default for the
# duration of the block.
# @yield
# @return [Object] The return value of the block.
def dangerously_override(environment)
original_default = @default
@default = environment
yield
ensure
@default = original_default
end
end
# Initializes a new environment instance.
# @api private
def initialize
@tags = Template::TagRegistry.new(Tags::STANDARD_TAGS)
@error_mode = :lax
@strainer_template = Class.new(StrainerTemplate).tap do |klass|
klass.add_filter(StandardFilters)
end
@exception_renderer = ->(exception) { exception }
@file_system = BlankFileSystem.new
@default_resource_limits = Const::EMPTY_HASH
@strainer_template_class_cache = {}
end
# Registers a new tag with the environment.
#
# @param name [String] The name of the tag.
# @param klass [Liquid::Tag] The class that implements the tag.
# @return [void]
def register_tag(name, klass)
@tags[name] = klass
end
# Registers a new filter with the environment.
#
# @param filter [Module] The module that contains the filter methods.
# @return [void]
def register_filter(filter)
@strainer_template_class_cache.clear
@strainer_template.add_filter(filter)
end
# Registers multiple filters with this environment.
#
# @param filters [Array<Module>] The modules that contain the filter methods.
# @return [self]
def register_filters(filters)
@strainer_template_class_cache.clear
filters.each { |f| @strainer_template.add_filter(f) }
self
end
# Creates a new strainer instance with the given filters, caching the result
# for faster lookup.
#
# @param context [Liquid::Context] The context that the strainer will be
# used in.
# @param filters [Array<Module>] The filters that the strainer will have
# access to.
# @return [Liquid::Strainer] The new strainer instance.
def create_strainer(context, filters = Const::EMPTY_ARRAY)
return @strainer_template.new(context) if filters.empty?
strainer_template = @strainer_template_class_cache[filters] ||= begin
klass = Class.new(@strainer_template)
filters.each { |f| klass.add_filter(f) }
klass
end
strainer_template.new(context)
end
# Returns the names of all the filter methods that are available to use in
# the strainer template.
#
# @return [Array<String>] The names of all the filter methods.
def filter_method_names
@strainer_template.filter_method_names
end
# Returns the tag class for the given tag name.
#
# @param name [String] The name of the tag.
# @return [Liquid::Tag] The tag class.
def tag_for_name(name)
@tags[name]
end
def freeze
@tags.freeze
# TODO: freeze the tags, currently this is not possible because of liquid-c
# @strainer_template.freeze
super
end
end
end
+155 -1
View File
@@ -1,8 +1,9 @@
# frozen_string_literal: true
require "strscan"
module Liquid
class Lexer
class Lexer1
SPECIALS = {
'|' => :pipe,
'.' => :dot,
@@ -58,4 +59,157 @@ module Liquid
@output << [:end_of_string]
end
end
class Lexer2
CLOSE_ROUND = [:close_round, ")"].freeze
CLOSE_SQUARE = [:close_square, "]"].freeze
COLON = [:colon, ":"].freeze
COMMA = [:comma, ","].freeze
COMPARISION_NOT_EQUAL = [:comparison, "!="].freeze
COMPARISON_CONTAINS = [:comparison, "contains"].freeze
COMPARISON_EQUAL = [:comparison, "=="].freeze
COMPARISON_GREATER_THAN = [:comparison, ">"].freeze
COMPARISON_GREATER_THAN_OR_EQUAL = [:comparison, ">="].freeze
COMPARISON_LESS_THAN = [:comparison, "<"].freeze
COMPARISON_LESS_THAN_OR_EQUAL = [:comparison, "<="].freeze
COMPARISON_NOT_EQUAL_ALT = [:comparison, "<>"].freeze
CONTAINS = /contains(?=\s)/
DASH = [:dash, "-"].freeze
DOT = [:dot, "."].freeze
DOTDOT = [:dotdot, ".."].freeze
DOT_ORD = ".".ord
DOUBLE_STRING_LITERAL = /"[^\"]*"/
EOS = [:end_of_string].freeze
IDENTIFIER = /[a-zA-Z_][\w-]*\??/
NUMBER_LITERAL = /-?\d+(\.\d+)?/
OPEN_ROUND = [:open_round, "("].freeze
OPEN_SQUARE = [:open_square, "["].freeze
PIPE = [:pipe, "|"].freeze
QUESTION = [:question, "?"].freeze
RUBY_WHITESPACE = [" ", "\t", "\r", "\n", "\f"].freeze
SINGLE_STRING_LITERAL = /'[^\']*'/
WHITESPACE_OR_NOTHING = /\s*/
COMPARISON_JUMP_TABLE = [].tap do |table|
table["=".ord] = [].tap do |sub_table|
sub_table["=".ord] = COMPARISON_EQUAL
sub_table.freeze
end
table["!".ord] = [].tap do |sub_table|
sub_table["=".ord] = COMPARISION_NOT_EQUAL
sub_table.freeze
end
table["<".ord] = [].tap do |sub_table|
sub_table["=".ord] = COMPARISON_LESS_THAN_OR_EQUAL
sub_table[">".ord] = COMPARISON_NOT_EQUAL_ALT
RUBY_WHITESPACE.each { |c| sub_table[c.ord] = COMPARISON_LESS_THAN }
sub_table.freeze
end
table[">".ord] = [].tap do |sub_table|
sub_table["=".ord] = COMPARISON_GREATER_THAN_OR_EQUAL
RUBY_WHITESPACE.each { |c| sub_table[c.ord] = COMPARISON_GREATER_THAN }
sub_table.freeze
end
table.freeze
end
NEXT_MATCHER_JUMP_TABLE = [].tap do |table|
"a".upto("z") do |c|
table[c.ord] = [:id, IDENTIFIER].freeze
table[c.upcase.ord] = [:id, IDENTIFIER].freeze
end
table["_".ord] = [:id, IDENTIFIER].freeze
"0".upto("9") do |c|
table[c.ord] = [:number, NUMBER_LITERAL].freeze
end
table["-".ord] = [:number, NUMBER_LITERAL].freeze
table["'".ord] = [:string, SINGLE_STRING_LITERAL].freeze
table["\"".ord] = [:string, DOUBLE_STRING_LITERAL].freeze
table.freeze
end
SPECIAL_TABLE = [].tap do |table|
table["|".ord] = PIPE
table[".".ord] = DOT
table[":".ord] = COLON
table[",".ord] = COMMA
table["[".ord] = OPEN_SQUARE
table["]".ord] = CLOSE_SQUARE
table["(".ord] = OPEN_ROUND
table[")".ord] = CLOSE_ROUND
table["?".ord] = QUESTION
table["-".ord] = DASH
end
NUMBER_TABLE = [].tap do |table|
"0".upto("9") do |c|
table[c.ord] = true
end
table.freeze
end
def initialize(input)
@ss = StringScanner.new(input)
end
# rubocop:disable Metrics/BlockNesting
def tokenize
@output = []
until @ss.eos?
@ss.skip(WHITESPACE_OR_NOTHING)
break if @ss.eos?
peeked = @ss.peek_byte
if (special = SPECIAL_TABLE[peeked])
@ss.scan_byte
# Special case for ".."
if special == DOT && @ss.peek_byte == DOT_ORD
@ss.scan_byte
@output << DOTDOT
elsif special == DASH
# Special case for negative numbers
if NUMBER_TABLE[@ss.peek_byte]
@ss.pos -= 1
@output << [:number, @ss.scan(NUMBER_LITERAL)]
else
@output << special
end
else
@output << special
end
elsif (sub_table = COMPARISON_JUMP_TABLE[peeked])
@ss.scan_byte
if (found = sub_table[@ss.peek_byte])
@output << found
@ss.scan_byte
else
raise SyntaxError, "Unexpected character #{peeked.chr}"
end
else
type, pattern = NEXT_MATCHER_JUMP_TABLE[peeked]
if type && (t = @ss.scan(pattern))
# Special case for "contains"
@output << if type == :id && t == "contains"
COMPARISON_CONTAINS
else
[type, t]
end
else
raise SyntaxError, "Unexpected character #{peeked.chr}"
end
end
end
# rubocop:enable Metrics/BlockNesting
@output << EOS
end
end
Lexer = StringScanner.instance_methods.include?(:scan_byte) ? Lexer2 : Lexer1
end
+4 -3
View File
@@ -3,9 +3,10 @@
module Liquid
class ParseContext
attr_accessor :locale, :line_number, :trim_whitespace, :depth
attr_reader :partial, :warnings, :error_mode
attr_reader :partial, :warnings, :error_mode, :environment
def initialize(options = {})
def initialize(options = Const::EMPTY_HASH)
@environment = options.fetch(:environment, Environment.default)
@template_options = options ? options.dup : {}
@locale = @template_options[:locale] ||= I18n.new
@@ -35,7 +36,7 @@ module Liquid
@partial = value
@options = value ? partial_options : @template_options
@error_mode = @options[:error_mode] || Template.error_mode
@error_mode = @options[:error_mode] || @environment.error_mode
end
def partial_options
+1 -1
View File
@@ -53,7 +53,7 @@ module Liquid
str = consume
str << variable_lookups
when :open_square
str = consume
str = consume.dup
str << expression
str << consume(:close_square)
str << variable_lookups
-2
View File
@@ -1001,6 +1001,4 @@ module Liquid
end
end
end
Template.register_filter(StandardFilters)
end
+9 -27
View File
@@ -5,37 +5,19 @@ module Liquid
module StrainerFactory
extend self
def add_global_filter(filter)
strainer_class_cache.clear
GlobalCache.add_filter(filter)
def add_global_filter(filter, environment = Environment.default)
Deprecations.warn("StrainerFactory.add_global_filter", "Environment#register_filter")
environment.register_filter(filter)
end
def create(context, filters = [])
strainer_from_cache(filters).new(context)
def create(context, filters = Const::EMPTY_ARRAY, environment = Environment.default)
Deprecations.warn("StrainerFactory.create", "StrainerFactory.create_strainer")
environment.create_strainer(context, filters)
end
def global_filter_names
GlobalCache.filter_method_names
end
GlobalCache = Class.new(StrainerTemplate)
private
def strainer_from_cache(filters)
if filters.empty?
GlobalCache
else
strainer_class_cache[filters] ||= begin
klass = Class.new(GlobalCache)
filters.each { |f| klass.add_filter(f) }
klass
end
end
end
def strainer_class_cache
@strainer_class_cache ||= {}
def global_filter_names(environment = Environment.default)
Deprecations.warn("StrainerFactory.global_filter_names", "Environment#filter_method_names")
Environment.strainer_template.filter_method_names
end
end
end
+3
View File
@@ -1,5 +1,8 @@
# frozen_string_literal: true
require 'liquid/tag/disabler'
require 'liquid/tag/disableable'
module Liquid
class Tag
attr_reader :nodelist, :tag_name, :line_number, :parse_context
+47
View File
@@ -0,0 +1,47 @@
# frozen_string_literal: true
require_relative "tags/table_row"
require_relative "tags/echo"
require_relative "tags/if"
require_relative "tags/break"
require_relative "tags/inline_comment"
require_relative "tags/for"
require_relative "tags/assign"
require_relative "tags/ifchanged"
require_relative "tags/case"
require_relative "tags/include"
require_relative "tags/continue"
require_relative "tags/capture"
require_relative "tags/decrement"
require_relative "tags/unless"
require_relative "tags/increment"
require_relative "tags/comment"
require_relative "tags/raw"
require_relative "tags/render"
require_relative "tags/cycle"
module Liquid
module Tags
STANDARD_TAGS = {
'cycle' => Cycle,
'render' => Render,
'raw' => Raw,
'comment' => Comment,
'increment' => Increment,
'unless' => Unless,
'decrement' => Decrement,
'capture' => Capture,
'continue' => Continue,
'include' => Include,
'case' => Case,
'ifchanged' => Ifchanged,
'assign' => Assign,
'for' => For,
'#' => InlineComment,
'break' => Break,
'if' => If,
'echo' => Echo,
'tablerow' => TableRow,
}.freeze
end
end
-2
View File
@@ -72,6 +72,4 @@ module Liquid
end
end
end
Template.register_tag('assign', Assign)
end
-2
View File
@@ -26,6 +26,4 @@ module Liquid
output
end
end
Template.register_tag('break', Break)
end
-2
View File
@@ -39,6 +39,4 @@ module Liquid
true
end
end
Template.register_tag('capture', Capture)
end
-2
View File
@@ -123,6 +123,4 @@ module Liquid
end
end
end
Template.register_tag('case', Case)
end
-2
View File
@@ -85,6 +85,4 @@ module Liquid
raise_tag_never_closed("raw")
end
end
Template.register_tag('comment', Comment)
end
-2
View File
@@ -17,6 +17,4 @@ module Liquid
output
end
end
Template.register_tag('continue', Continue)
end
+6 -2
View File
@@ -26,14 +26,20 @@ module Liquid
when NamedSyntax
@variables = variables_from_string(Regexp.last_match(2))
@name = parse_expression(Regexp.last_match(1))
@is_named = true
when SimpleSyntax
@variables = variables_from_string(markup)
@name = @variables.to_s
@is_named = !@name.match?(/\w+:0x\h{8}/)
else
raise SyntaxError, options[:locale].t("errors.syntax.cycle")
end
end
def named?
@is_named
end
def render_to_output_buffer(context, output)
context.registers[:cycle] ||= {}
@@ -72,6 +78,4 @@ module Liquid
end
end
end
Template.register_tag('cycle', Cycle)
end
-2
View File
@@ -35,6 +35,4 @@ module Liquid
output
end
end
Template.register_tag('decrement', Decrement)
end
-2
View File
@@ -36,6 +36,4 @@ module Liquid
end
end
end
Template.register_tag('echo', Echo)
end
-2
View File
@@ -201,6 +201,4 @@ module Liquid
end
end
end
Template.register_tag('for', For)
end
+1 -3
View File
@@ -111,7 +111,7 @@ module Liquid
def parse_binary_comparisons(p)
condition = parse_comparison(p)
first_condition = condition
while (op = (p.id?('and') || p.id?('or')))
while (op = p.id?('and') || p.id?('or'))
child_condition = parse_comparison(p)
condition.send(op, child_condition)
condition = child_condition
@@ -135,6 +135,4 @@ module Liquid
end
end
end
Template.register_tag('if', If)
end
-2
View File
@@ -14,6 +14,4 @@ module Liquid
output
end
end
Template.register_tag('ifchanged', Ifchanged)
end
-2
View File
@@ -110,6 +110,4 @@ module Liquid
end
end
end
Template.register_tag('include', Include)
end
-2
View File
@@ -35,6 +35,4 @@ module Liquid
output
end
end
Template.register_tag('increment', Increment)
end
-2
View File
@@ -25,6 +25,4 @@ module Liquid
true
end
end
Template.register_tag('#', InlineComment)
end
-2
View File
@@ -56,6 +56,4 @@ module Liquid
end
end
end
Template.register_tag('raw', Raw)
end
-2
View File
@@ -108,6 +108,4 @@ module Liquid
end
end
end
Template.register_tag('render', Render)
end
+6 -2
View File
@@ -65,6 +65,12 @@ module Liquid
super
output << '</td>'
# Handle any interrupts if they exist.
if context.interrupt?
interrupt = context.pop_interrupt
break if interrupt.is_a?(BreakInterrupt)
end
if tablerowloop.col_last && !tablerowloop.last
output << "</tr>\n<tr class=\"row#{tablerowloop.row + 1}\">"
end
@@ -91,6 +97,4 @@ module Liquid
raise Liquid::ArgumentError, "invalid integer"
end
end
Template.register_tag('tablerow', TableRow)
end
-2
View File
@@ -44,6 +44,4 @@ module Liquid
output
end
end
Template.register_tag('unless', Unless)
end
+55 -60
View File
@@ -18,42 +18,6 @@ module Liquid
attr_accessor :root, :name
attr_reader :resource_limits, :warnings
class TagRegistry
include Enumerable
def initialize
@tags = {}
@cache = {}
end
def [](tag_name)
return nil unless @tags.key?(tag_name)
return @cache[tag_name] if Liquid.cache_classes
lookup_class(@tags[tag_name]).tap { |o| @cache[tag_name] = o }
end
def []=(tag_name, klass)
@tags[tag_name] = klass.name
@cache[tag_name] = klass
end
def delete(tag_name)
@tags.delete(tag_name)
@cache.delete(tag_name)
end
def each(&block)
@tags.each(&block)
end
private
def lookup_class(name)
Object.const_get(name)
end
end
attr_reader :profiler
class << self
@@ -61,46 +25,71 @@ module Liquid
# :lax acts like liquid 2.5 and silently ignores malformed tags in most cases.
# :warn is the default and will give deprecation warnings when invalid syntax is used.
# :strict will enforce correct syntax.
attr_accessor :error_mode
Template.error_mode = :lax
attr_accessor :default_exception_renderer
Template.default_exception_renderer = lambda do |exception|
exception
def error_mode=(mode)
Deprecations.warn("Template.error_mode=", "Environment#error_mode=")
Environment.default.error_mode = mode
end
attr_accessor :file_system
Template.file_system = BlankFileSystem.new
def error_mode
Environment.default.error_mode
end
attr_accessor :tags
Template.tags = TagRegistry.new
private :tags=
def default_exception_renderer=(renderer)
Deprecations.warn("Template.default_exception_renderer=", "Environment#exception_renderer=")
Environment.default.exception_renderer = renderer
end
def default_exception_renderer
Environment.default.exception_renderer
end
def file_system=(file_system)
Deprecations.warn("Template.file_system=", "Environment#file_system=")
Environment.default.file_system = file_system
end
def file_system
Environment.default.file_system
end
def tags
Environment.default.tags
end
def register_tag(name, klass)
tags[name.to_s] = klass
Deprecations.warn("Template.register_tag", "Environment#register_tag")
Environment.default.register_tag(name, klass)
end
# Pass a module with filter methods which should be available
# to all liquid views. Good for registering the standard library
def register_filter(mod)
StrainerFactory.add_global_filter(mod)
Deprecations.warn("Template.register_filter", "Environment#register_filter")
Environment.default.register_filter(mod)
end
attr_accessor :default_resource_limits
Template.default_resource_limits = {}
private :default_resource_limits=
private def default_resource_limits=(limits)
Deprecations.warn("Template.default_resource_limits=", "Environment#default_resource_limits=")
Environment.default.default_resource_limits = limits
end
def default_resource_limits
Environment.default.default_resource_limits
end
# creates a new <tt>Template</tt> object from liquid source code
# To enable profiling, pass in <tt>profile: true</tt> as an option.
# See Liquid::Profiler for more information
def parse(source, options = {})
new.parse(source, options)
environment = options[:environment] || Environment.default
new(environment: environment).parse(source, options)
end
end
def initialize
def initialize(environment: Environment.default)
@environment = environment
@rethrow_errors = false
@resource_limits = ResourceLimits.new(Template.default_resource_limits)
@resource_limits = ResourceLimits.new(environment.default_resource_limits)
end
# Parse source code.
@@ -162,11 +151,11 @@ module Liquid
c
when Liquid::Drop
drop = args.shift
drop.context = Context.new([drop, assigns], instance_assigns, registers, @rethrow_errors, @resource_limits)
drop.context = Context.new([drop, assigns], instance_assigns, registers, @rethrow_errors, @resource_limits, {}, @environment)
when Hash
Context.new([args.shift, assigns], instance_assigns, registers, @rethrow_errors, @resource_limits)
Context.new([args.shift, assigns], instance_assigns, registers, @rethrow_errors, @resource_limits, {}, @environment)
when nil
Context.new(assigns, instance_assigns, registers, @rethrow_errors, @resource_limits)
Context.new(assigns, instance_assigns, registers, @rethrow_errors, @resource_limits, {}, @environment)
else
raise ArgumentError, "Expected Hash or Liquid::Context as parameter"
end
@@ -226,8 +215,14 @@ module Liquid
@options = options
@profiling = profiling
@line_numbers = options[:line_numbers] || @profiling
parse_context = options.is_a?(ParseContext) ? options : ParseContext.new(options)
@warnings = parse_context.warnings
parse_context = if options.is_a?(ParseContext)
options
else
opts = options.key?(:environment) ? options : options.merge(environment: @environment)
ParseContext.new(opts)
end
@warnings = parse_context.warnings
parse_context
end
+44
View File
@@ -0,0 +1,44 @@
# frozen_string_literal: true
module Liquid
class Template
class TagRegistry
include Enumerable
def initialize(tags = nil)
@tags = {}
@cache = {}
tags.each { |tag_name, klass| self[tag_name] = klass }
Deprecations.warn("Template::TagRegistry", "Use a Environment instance with zeitwerk")
end
def [](tag_name)
return nil unless @tags.key?(tag_name)
return @cache[tag_name] if Liquid.cache_classes
lookup_class(@tags[tag_name]).tap { |o| @cache[tag_name] = o }
end
def delete(tag_name)
Deprecations.warn("Template::TagRegistry#delete", "Use a Environment instance with immutable tags")
@tags.delete(tag_name)
@cache.delete(tag_name)
end
def []=(tag_name, klass)
@tags[tag_name] = klass.name
@cache[tag_name] = klass
end
def each(&block)
@tags.each(&block)
end
private
def lookup_class(name)
Object.const_get(name)
end
end
end
end
+1 -1
View File
@@ -2,5 +2,5 @@
# frozen_string_literal: true
module Liquid
VERSION = "5.5.0"
VERSION = "5.6.0.alpha"
end
+4 -1
View File
@@ -17,7 +17,7 @@ Gem::Specification.new do |s|
s.license = "MIT"
# s.description = "A secure, non-evaling end user template engine with aesthetic markup."
s.required_ruby_version = ">= 2.7.0"
s.required_ruby_version = ">= 3.0.0"
s.required_rubygems_version = ">= 1.3.7"
s.metadata['allowed_push_host'] = 'https://rubygems.org'
@@ -28,6 +28,9 @@ Gem::Specification.new do |s|
s.require_path = "lib"
s.add_dependency("strscan")
s.add_dependency("bigdecimal")
s.add_development_dependency('rake', '~> 13.0')
s.add_development_dependency('minitest')
end
+1
View File
@@ -3,6 +3,7 @@
require 'benchmark/ips'
require_relative 'theme_runner'
RubyVM::YJIT.enable if defined?(RubyVM::YJIT)
Liquid::Template.error_mode = ARGV.first.to_sym if ARGV.first
profiler = ThemeRunner.new
+62
View File
@@ -0,0 +1,62 @@
# frozen_string_literal: true
require "benchmark/ips"
# benchmark liquid lexing
require 'liquid'
RubyVM::YJIT.enable
EXPRESSIONS = [
"foo[1..2].baz",
"12.0",
"foo.bar.based",
"21 - 62",
"foo.bar.baz",
"foo > 12",
"foo < 12",
"foo <= 12",
"foo >= 12",
"foo <> 12",
"foo == 12",
"foo != 12",
"foo contains 12",
"foo contains 'bar'",
"foo != 'bar'",
"'foo' contains 'bar'",
'234089',
"foo | default: -1",
]
EXPRESSIONS.each do |expr|
lexer_1_result = Liquid::Lexer1.new(expr).tokenize
lexer_2_result = Liquid::Lexer2.new(expr).tokenize
next if lexer_1_result == lexer_2_result
warn "Lexer1 and Lexer2 results are different for expression: #{expr}"
warn "expected: #{lexer_1_result}"
warn "got: #{lexer_2_result}"
abort
end
Benchmark.ips do |x|
x.config(time: 10, warmup: 5)
x.report("Liquid::Lexer1#tokenize") do
EXPRESSIONS.each do |expr|
l = Liquid::Lexer1.new(expr)
l.tokenize
end
end
x.report("Liquid::Lexer2#tokenize") do
EXPRESSIONS.each do |expr|
l = Liquid::Lexer2.new(expr)
l.tokenize
end
end
x.compare!
end
+18 -9
View File
@@ -219,6 +219,21 @@ class ErrorHandlingTest < Minitest::Test
Liquid::Template.default_exception_renderer = old_exception_renderer if old_exception_renderer
end
def test_setting_exception_renderer_on_environment
exceptions = []
exception_renderer = ->(e) do
exceptions << e
''
end
environment = Liquid::Environment.build(exception_renderer: exception_renderer)
template = Liquid::Template.parse('This is a runtime error: {{ errors.argument_error }}', environment: environment)
output = template.render('errors' => ErrorDrop.new)
assert_equal('This is a runtime error: ', output)
assert_equal([Liquid::ArgumentError], template.errors.map(&:class))
end
def test_exception_renderer_exposing_non_liquid_error
template = Liquid::Template.parse('This is a runtime error: {{ errors.runtime_error }}', line_numbers: true)
exceptions = []
@@ -242,16 +257,10 @@ class ErrorHandlingTest < Minitest::Test
end
def test_included_template_name_with_line_numbers
old_file_system = Liquid::Template.file_system
environment = Liquid::Environment.build(file_system: TestFileSystem.new)
template = Liquid::Template.parse("Argument error:\n{% include 'product' %}", line_numbers: true, environment: environment)
page = template.render('errors' => ErrorDrop.new)
begin
Liquid::Template.file_system = TestFileSystem.new
template = Liquid::Template.parse("Argument error:\n{% include 'product' %}", line_numbers: true)
page = template.render('errors' => ErrorDrop.new)
ensure
Liquid::Template.file_system = old_file_system
end
assert_equal("Argument error:\nLiquid error (product line 1): argument error", page)
assert_equal("product", template.errors.first.template_name)
end
+1 -1
View File
@@ -32,7 +32,7 @@ class TestDrop < Liquid::Drop
attr_reader :value
def registers
{ @value => @context.registers[@value] }
"{#{@value.inspect}=>#{@context.registers[@value].inspect}}"
end
end
+2 -9
View File
@@ -49,14 +49,6 @@ end
class IncludeTagTest < Minitest::Test
include Liquid
def setup
@default_file_system = Liquid::Template.file_system
end
def teardown
Liquid::Template.file_system = @default_file_system
end
def test_include_tag_looks_for_file_system_in_registers_first
assert_equal(
'from OtherFileSystem',
@@ -214,9 +206,10 @@ class IncludeTagTest < Minitest::Test
def test_include_tag_caches_second_read_of_same_partial
file_system = CountingFileSystem.new
environment = Liquid::Environment.build(file_system: file_system)
assert_equal(
'from CountingFileSystemfrom CountingFileSystem',
Template.parse("{% include 'pick_a_source' %}{% include 'pick_a_source' %}").render!({}, registers: { file_system: file_system }),
Template.parse("{% include 'pick_a_source' %}{% include 'pick_a_source' %}", environment: environment).render!({}, registers: { file_system: file_system }),
)
assert_equal(1, file_system.count)
end
+48
View File
@@ -207,4 +207,52 @@ class TableRowTest < Minitest::Test
render_errors: true,
)
end
def test_table_row_handles_interrupts
assert_template_result(
"<tr class=\"row1\">\n<td class=\"col1\"> 1 </td></tr>\n",
'{% tablerow n in (1...3) cols:2 %} {{n}} {% break %} {{n}} {% endtablerow %}',
)
assert_template_result(
"<tr class=\"row1\">\n<td class=\"col1\"> 1 </td><td class=\"col2\"> 2 </td></tr>\n<tr class=\"row2\"><td class=\"col1\"> 3 </td></tr>\n",
'{% tablerow n in (1...3) cols:2 %} {{n}} {% continue %} {{n}} {% endtablerow %}',
)
end
def test_table_row_does_not_leak_interrupts
template = <<~LIQUID
{% for i in (1..2) -%}
{% for j in (1..2) -%}
{% tablerow k in (1..3) %}{% break %}{% endtablerow -%}
loop j={{ j }}
{% endfor -%}
loop i={{ i }}
{% endfor -%}
after loop
LIQUID
expected = <<~STR
<tr class="row1">
<td class="col1"></td></tr>
loop j=1
<tr class="row1">
<td class="col1"></td></tr>
loop j=2
loop i=1
<tr class="row1">
<td class="col1"></td></tr>
loop j=1
<tr class="row1">
<td class="col1"></td></tr>
loop j=2
loop i=2
after loop
STR
assert_template_result(
expected,
template,
)
end
end
+13 -29
View File
@@ -42,10 +42,11 @@ module Minitest
message: nil, partials: nil, error_mode: nil, render_errors: false,
template_factory: nil
)
template = Liquid::Template.parse(template, line_numbers: true, error_mode: error_mode&.to_sym)
file_system = StubFileSystem.new(partials || {})
environment = Liquid::Environment.build(file_system: file_system)
template = Liquid::Template.parse(template, line_numbers: true, error_mode: error_mode&.to_sym, environment: environment)
registers = Liquid::Registers.new(file_system: file_system, template_factory: template_factory)
context = Liquid::Context.build(static_environments: assigns, rethrow_errors: !render_errors, registers: registers)
context = Liquid::Context.build(static_environments: assigns, rethrow_errors: !render_errors, registers: registers, environment: environment)
output = template.render(context)
assert_equal(expected, output, message)
end
@@ -78,22 +79,12 @@ module Minitest
assert_equal(times, calls, "Number of calls to Usage.increment with #{name.inspect}")
end
def with_global_filter(*globals)
original_global_cache = Liquid::StrainerFactory::GlobalCache
Liquid::StrainerFactory.send(:remove_const, :GlobalCache)
Liquid::StrainerFactory.const_set(:GlobalCache, Class.new(Liquid::StrainerTemplate))
def with_global_filter(*globals, &blk)
environment = Liquid::Environment.build do |w|
w.register_filters(globals)
end
globals.each do |global|
Liquid::Template.register_filter(global)
end
Liquid::StrainerFactory.send(:strainer_class_cache).clear
begin
yield
ensure
Liquid::StrainerFactory.send(:remove_const, :GlobalCache)
Liquid::StrainerFactory.const_set(:GlobalCache, original_global_cache)
Liquid::StrainerFactory.send(:strainer_class_cache).clear
end
Environment.dangerously_override(environment, &blk)
end
def with_error_mode(mode)
@@ -104,18 +95,11 @@ module Minitest
Liquid::Template.error_mode = old_mode
end
def with_custom_tag(tag_name, tag_class)
old_tag = Liquid::Template.tags[tag_name]
begin
Liquid::Template.register_tag(tag_name, tag_class)
yield
ensure
if old_tag
Liquid::Template.tags[tag_name] = old_tag
else
Liquid::Template.tags.delete(tag_name)
end
end
def with_custom_tag(tag_name, tag_class, &block)
environment = Liquid::Environment.default.dup
environment.register_tag(tag_name, tag_class)
Environment.dangerously_override(environment, &block)
end
end
end
+10
View File
@@ -50,4 +50,14 @@ class LexerUnitTest < Minitest::Test
Lexer.new("%").tokenize
end
end
def test_negative_numbers
tokens = Lexer.new("foo | default: -1").tokenize
assert_equal([[:id, 'foo'], [:pipe, '|'], [:id, 'default'], [:colon, ":"], [:number, '-1'], [:end_of_string]], tokens)
end
def test_greater_than_two_digits
tokens = Lexer.new("foo > 12").tokenize
assert_equal([[:id, 'foo'], [:comparison, '>'], [:number, '12'], [:end_of_string]], tokens)
end
end
+3 -2
View File
@@ -58,8 +58,9 @@ class StrainerTemplateUnitTest < Minitest::Test
def test_add_filter_does_not_raise_when_module_overrides_previously_registered_method
with_global_filter do
strainer = Context.new.strainer
strainer.class.add_filter(PublicMethodOverrideFilter)
context = Context.new
context.add_filters([PublicMethodOverrideFilter])
strainer = context.strainer
assert(strainer.class.send(:filter_methods).include?('public_filter'))
end
end
+10 -21
View File
@@ -36,7 +36,6 @@ class TemplateUnitTest < Minitest::Test
assert(Template.tags['custom'].equal?(original_klass))
ensure
Object.send(:remove_const, :CustomTag)
Template.tags.delete('custom')
Liquid.cache_classes = original_cache_setting
end
@@ -46,36 +45,26 @@ class TemplateUnitTest < Minitest::Test
original_klass = Class.new
Object.send(:const_set, :CustomTag, original_klass)
Template.register_tag('custom', CustomTag)
with_custom_tag('custom', CustomTag) do
Object.send(:remove_const, :CustomTag)
Object.send(:remove_const, :CustomTag)
new_klass = Class.new
Object.send(:const_set, :CustomTag, new_klass)
new_klass = Class.new
Object.send(:const_set, :CustomTag, new_klass)
assert(Template.tags['custom'].equal?(new_klass))
assert(Template.tags['custom'].equal?(new_klass))
end
ensure
Object.send(:remove_const, :CustomTag)
Template.tags.delete('custom')
Liquid.cache_classes = original_cache_setting
end
class FakeTag; end
def test_tags_delete
Template.register_tag('fake', FakeTag)
assert_equal(FakeTag, Template.tags['fake'])
Template.tags.delete('fake')
assert_nil(Template.tags['fake'])
end
def test_tags_can_be_looped_over
Template.register_tag('fake', FakeTag)
result = Template.tags.map { |name, klass| [name, klass] }
assert(result.include?(["fake", "TemplateUnitTest::FakeTag"]))
ensure
Template.tags.delete('fake')
with_custom_tag('fake', FakeTag) do
result = Template.tags.map { |name, klass| [name, klass] }
assert(result.include?(["fake", "TemplateUnitTest::FakeTag"]))
end
end
class TemplateSubclass < Liquid::Template