Commit Graph
1878 Commits
Author SHA1 Message Date
Tobi Lutke 154343e64f Integrate Box and runtime into compiled template execution
- compiled_template.rb: Use Liquid::Box for secure execution on Ruby 4.0+
  - Creates box, loads runtime, locks, then evals template code
  - Provides render() method and secure? check
  - Falls back to insecure eval with warning on Ruby < 4.0

- ruby_compiler.rb: Remove inline helper generation
  - Helpers now provided by pre-loaded LR module
  - Generated code is much smaller (just control flow + LR calls)

- compile.rb: Update documentation for new security model

- template.rb: Update compile_to_ruby docs
2025-12-31 12:20:19 -04:00
Tobi Lutke 6e680a35b3 Add LR runtime module with pre-loaded helpers
The LR module provides all helper methods for compiled templates.
It is loaded into the sandbox BEFORE lock!, so helpers are defined
once and shared by all templates.

Helpers include:
- Type conversion: to_s, to_number, to_integer
- Output: output (handles nil, arrays, BigDecimal formatting)
- Lookup: lookup (hash/array access, Drop context support)
- Encoding: escape_html, url_encode/decode, base64_encode/decode
- Filters: truncate, truncatewords, slice, date, default, etc.

Method references to CGI, Base64, BigDecimal are captured at load
time, enabling safe use of these libraries within the sandbox.

Design principle: Maximize work in pre-loaded runtime, minimize
generated template code.
2025-12-31 12:20:10 -04:00
Tobi Lutke 9367b8b32e Add Liquid::Box for secure sandboxed template execution
Introduces Liquid::Box which wraps Ruby 4.0's Ruby::Box for secure
template execution. On Ruby < 4.0, provides a polyfill with security
warnings.

Key features:
- Detects Ruby::Box availability at load time
- Loads safe libraries (CGI, Base64, BigDecimal) into sandbox
- Neuters dangerous methods (file IO, process control, eval, etc.)
- Preserves user constants defined before lock!
- Provides setup_gem_load_paths! to enable gem requires in box

Security model: It is safe to expose side-effect-free, non-IO methods
that don't leak objects with dangerous methods. The sandbox blocks
capabilities, not data.
2025-12-31 12:19:53 -04:00
Claude 5cdbce7d6e Improve benchmark output clarity
Add explicit summary showing pre-compiled is X times FASTER,
and explain that "X slower" in comparison means relative to
the fastest option (pre-compiled Ruby). Higher i/s = faster.
2025-12-31 15:25:54 +00:00
Claude 520e86b8c8 Add Liquid Drop support for compiled templates
- Create CompiledContext class that duck-types to Liquid::Context
- CompiledContext provides: variable lookup, registers, strict_* flags
- Update __lookup__ helper to:
  - Set context on Drops BEFORE accessing their methods
  - Call to_liquid on objects before lookup
  - Set context on nested Drop results
- Change __lookup__ from def to lambda to capture __context__ closure
- Update expression compiler to use __lookup__.call() syntax
- Add CompiledTemplate.call options: registers, strict_variables, strict_filters

Tests added:
- test_compile_with_drop: Basic Drop property access
- test_compile_with_drop_context_access: Drop using context to access other vars
- test_compile_with_nested_drops: Chained Drop lookups
- test_compile_with_forloop_drop: Built-in forloop compatibility
- test_compile_with_registers: Drop accessing registers via context

All 51 tests pass, 30/30 benchmark templates produce matching output.
2025-12-31 15:21:30 +00:00
Claude 54e3d2328a Fix code injection vulnerability in template name handling
Use .inspect for all template/partial names inserted into generated
code strings to prevent code injection via maliciously crafted
template names like: {% render "foo'); system('rm -rf /'); #" %}
2025-12-31 15:05:20 +00:00
Claude 7be2922f91 Fix compiler output equivalence and add comprehensive tests
- Fix forloop.first/last/size lookups to try hash key before method call
- Fix tablerow cols parameter to use attributes['cols'] not @cols
- Fix tablerow output format to match interpreter (newlines, row boundaries)
- Fix capture compiler to access @body directly instead of iterating nodelist
- Add CompiledTemplate class to encapsulate code and external_tags
- Add external filter support via filter_handler
- Add debug mode warnings for external tag/filter calls
- Add Ruby 3.3 compatibility shim for peek_byte/scan_byte
- Add comprehensive unit tests for output equivalence
- Add benchmark comparing compiled vs interpreted rendering

All 30 test templates now produce identical output between compiled
Ruby and interpreted Liquid. Pre-compiled Ruby is 1.68x faster.
2025-12-31 14:58:06 +00:00
Claude e0f856fd11 Add Liquid to Ruby compiler for optimized template rendering
This adds a new `compile_to_ruby` method to Liquid::Template that compiles
Liquid templates to pure Ruby code. The compiled code can be eval'd to create
a proc that renders templates without needing the Liquid library at runtime.

## Features

- Compiles all standard Liquid tags: if/unless/case, for, assign, capture,
  cycle, increment/decrement, raw, echo, break/continue, comment, tablerow
- Compiles variable expressions with filter chains to direct Ruby method calls
- Supports static partial inlining ({% render %} and {% include %} with string
  literals are loaded and compiled at compile time)
- Dynamic partial support via runtime callbacks (__render_dynamic__, __include_dynamic__)
- Debug mode with source comments for error tracing (lightweight source map)
- SourceMapper utility to trace runtime errors back to Liquid source

## Optimization Opportunities

The compiled Ruby code has significant performance advantages:

1. No Context object - variables accessed directly from assigns hash
2. No filter invocation overhead - direct Ruby method calls
3. No resource limits tracking - no per-node render score updates
4. No stack-based scoping - uses Ruby's native block scoping
5. Direct string concatenation - no render_to_output_buffer abstraction
6. Native control flow - break/continue use Ruby's throw/catch
7. No to_liquid calls - values used directly
8. No profiling hooks - no profiler overhead
9. No exception rendering - errors propagate naturally

## Usage

```ruby
template = Liquid::Template.parse("Hello, {{ name }}!")
ruby_code = template.compile_to_ruby
render_proc = eval(ruby_code)
result = render_proc.call({ "name" => "World" })
# => "Hello, World!"

# With debug mode for error tracing:
ruby_code = template.compile_to_ruby(debug: true)
```

## Limitations

- Dynamic {% render %} and {% include %} require runtime callback methods
- Custom tags need explicit compiler implementations
- Custom filters must be available at runtime
2025-12-31 14:21:58 +00:00
James MengandGitHub c6f05eaf83 Merge pull request #1955 from Shopify/jm/trim_doc_tag_desc
Remove {{ foo }} and {{ bar }} from `doc` tag description
2025-05-06 15:29:29 -07:00
James Meng eabbd5cb6f Remove {{ foo }} and {{ bar }} from doc tag description 2025-05-06 15:20:35 -07:00
Marco Concetto RudilossoandGitHub ea864f1177 Merge pull request #1951 from Shopify/actual-revert
Fully revert calling to_s on filter array
v5.8.6
2025-04-14 17:49:47 +02:00
Marco Concetto Rudilosso c34dd812c5 Fully revert calling to_s on filter array 2025-04-14 17:36:45 +02:00
Marco Concetto RudilossoandGitHub cc04892e54 Merge pull request #1943 from Shopify/revert-to-s
Revert `Utils.to_s` on all array filters
v5.8.5
2025-04-14 17:22:10 +02:00
Marco Concetto Rudilosso f676e699a4 rollback changes to tests 2025-04-14 11:24:01 +02:00
Marco Concetto Rudilosso fc0f7f44c1 use Utils.to_s on property error 2025-04-14 11:17:59 +02:00
Marco Concetto Rudilosso b459eb656f remove test 2025-04-14 11:17:46 +02:00
Marco Concetto Rudilosso c6dcf3e714 add test 2025-04-14 11:17:46 +02:00
Marco Concetto Rudilosso 4dae678c63 Revert "Stringify properties before filtering (#1929)"
This reverts commit f5d6a36574.
2025-04-14 11:17:46 +02:00
Marco Concetto Rudilosso 4c07ff920b Revert "Always stringify properties in all array filters (#1936)"
This reverts commit aa1640035f.
2025-04-14 11:17:46 +02:00
Ian Ker-SeymerandGitHub dbe709c3bf Use to_liquid_value in uniq filter (#1948)
* Use to_liquid_value in uniq filter

* Bump version to 5.8.4
v5.8.4
2025-04-09 15:00:01 -04:00
2b75bfaff4 Fix regression when using empty/nil properties with array filters (#1944)
* Make all array filters that use `filter_array` util process empty string and nil correctly

* up version

* fix ordering of checks

* also do it for map

* Do not raise property error

* Gracefully empty property in map filter

---------

Co-authored-by: Marco Concetto Rudilosso <[email protected]>
v5.8.3
2025-04-04 11:27:29 -04:00
James MengandGitHub 87bc6e7cfa Merge pull request #1940 from Shopify/jm/update_snippet_url_links
Update snippet URLs in documentation links to point to newly created snippet page
2025-04-01 10:58:54 -07:00
James Meng 7f122aeed2 Fix snippet URLs in documentation links 2025-03-31 17:08:31 -07:00
aa1640035f Always stringify properties in all array filters (#1936)
* Always stringify sum property.

* Add test

* always stringify properties in all array filters

* fix syntax error

* up version

---------

Co-authored-by: Dominic Petrick <[email protected]>
v5.8.2
2025-03-19 13:50:01 -04:00
Ian Ker-SeymerandGitHub f5d6a36574 Stringify properties before filtering (#1929) 2025-03-17 17:43:57 -04:00
Ian Ker-SeymerandGitHub c5711c095f Improve docs of date filter (#1920) 2025-03-13 17:36:29 -04:00
James MengandGitHub 284f5fb647 Merge pull request #1928 from Shopify/jm/add_ld_link_doc_tag
Add a link to the LiquidDoc tooling reference in `doc` tag documentation
2025-03-13 09:48:40 -07:00
James MengandGitHub 21432928d0 Use full relative path for hyperlink 2025-03-13 09:47:57 -07:00
James Meng 1783c0c084 Add a link to the LiquidDoc reference in doc tag documentation comment 2025-03-12 15:31:28 -07:00
Guilherme CarreiroandGuilherme Carreiro e38f730c00 Update LiquidDoc documentation 2025-03-07 18:59:59 +01:00
2d0442798b chore: Add RUBYOPT configuration to the tests (#1859)
Co-authored-by: Ian Ker-Seymer <[email protected]>
2025-02-26 14:28:17 -05:00
Guilherme CarreiroandGuilherme Carreiro 6453a0ea48 Implement nodelist in the Doc tag so it may be visited v5.8.1 2025-02-26 13:14:39 +01:00
Guilherme CarreiroandGuilherme Carreiro a398b4cc74 Fix History.md v5.8.0 2025-02-25 08:50:46 +01:00
Guilherme CarreiroandGuilherme Carreiro cca9fe99cf Bump version to 5.8.0 2025-02-25 08:50:46 +01:00
Guilherme CarreiroandGuilherme Carreiro 17d327988d Rename {% doc %} constant strictly validates the abscense of args 2025-02-20 12:37:09 +01:00
Guilherme CarreiroandGuilherme Carreiro f643af4bac Update the implementation to make {% doc %} as strict as {% raw %} 2025-02-20 12:37:09 +01:00
Guilherme CarreiroandGuilherme Carreiro ae8a0a86ac Remove misleading unit test (thank you, @EvilGenius13) 2025-02-20 12:37:09 +01:00
Guilherme CarreiroandGuilherme Carreiro b439d0da53 Update {% doc %} to no longer support nested tags (as {% comment %} does) 2025-02-20 12:37:09 +01:00
Guilherme CarreiroandGuilherme Carreiro 16592cfb8f Add support to LiquidDoc with the new {% doc %} tag 2025-02-20 12:37:09 +01:00
Chris AtLeeandGitHub da4afd4156 Merge pull request #1905 from Shopify/catlee/invalid_utf8
Raise SyntaxError on invalid UTF8 strings in lexer/tokenizer
v5.7.3
2025-02-13 09:24:11 -05:00
Chris AtLeeandGitHub 1bb3091208 Merge pull request #1909 from Shopify/catlee/5.7.3
Bump version to 5.7.3
2025-02-13 09:22:47 -05:00
Max StoiberandGitHub 040801b32c Fix array has filters referring to some (#1910) 2025-02-12 17:37:49 +01:00
Chris AtLee 550135c0b9 Raise SyntaxError on invalid UTF8 strings in lexer/tokenizer 2025-02-11 14:23:15 -05:00
Chris AtLee aec966eed7 Bump version to 5.7.3 2025-02-11 14:21:14 -05:00
Michael GoandGitHub bfe29e11be Merge pull request #1907 from Shopify/nested-properties
Fix array filters to not support nested properties
v5.7.2
2025-01-31 12:35:03 -04:00
Guilherme Carreiro f9454d8cf3 Fix array filters to not support nested properties 2025-01-31 13:53:17 +01:00
Guilherme CarreiroandGuilherme Carreiro 8dd9279265 Fix release date on History.md 2025-01-24 15:34:48 +01:00
Guilherme CarreiroandGuilherme Carreiro bf1419b8ac Apply the same fix for find_index and has v5.7.1 2025-01-24 08:39:01 +01:00
Guilherme CarreiroandGuilherme Carreiro 5718c4cee2 Fix the find filter to return nil when filtering empty arrays 2025-01-24 08:39:01 +01:00
Guilherme CarreiroandGitHub b0dbc62696 Fix bundle exec rake example (#1900)
* Fix `bundle exec rake example`

* Move 'webrick' from 'benchmark/test' to 'development'
2025-01-23 08:21:38 +01:00