feat: remove memoryLimit; add templateLimit, outputLengthLimit, maxDepth (#937)

* feat: remove memoryLimit option (#910)

Co-authored-by: Cursor <[email protected]>

* feat: add templateLimit, outputLengthLimit, and maxDepth DoS limits

Enforce v11 resource guards in render and tags, fix for offset/else behavior, and update tutorials for Tag-class registration.

Co-authored-by: Cursor <[email protected]>

* docs: revert unnecessary tutorial churn from memoryLimit PR

Restore the two-example register-filters-tags structure (Value + Hash)
and undo unrelated constructor/emitter doc edits not required for DoS limits.

Co-authored-by: Cursor <[email protected]>

* docs: trim security-model prose and update render-tag-content

Remove diary-style engine comparisons from security-model.md.
Update render-tag-content tutorial to Tag class examples with tpls class field.

Co-authored-by: Cursor <[email protected]>

* docs: note maxDepth stack overflow applies to renderSync only

Explain why async render does not need maxDepth for stack protection based on generator/toPromise driving.

Co-authored-by: Cursor <[email protected]>

* refactor: track maxDepth via depthLimit Limiter on Context

Replace increaseDepth/decreaseDepth with a shared Limiter that supports
paired use/release, matching templateLimit and outputLengthLimit patterns.

Co-authored-by: Cursor <[email protected]>

* fix: remove spurious diff noise in filter files

Restore misc.ts from origin/next with LF line endings and re-apply only
memoryLimit removal, avoiding CRLF and blank-line churn in the export block.

Co-authored-by: Cursor <[email protected]>

* refactor: minimize PR diff noise

Co-authored-by: Cursor <[email protected]>

* feat: cap strftime pad width at 1M

docs: restructure security model with production guidance
Co-authored-by: Cursor <[email protected]>

* refactor: simplify depthLimit in partial tags and tighten security docs

Drop try/finally around depthLimit in include, layout, and render; release at generator end. Consolidate production guidance in security-model.md. Fix padded-blocks lint in dos.spec.ts.

Co-authored-by: Cursor <[email protected]>

---------

Co-authored-by: Cursor <[email protected]>
This commit is contained in:
Yang Jun
2026-07-15 22:58:13 +08:00
committed by GitHub
co-authored by Cursor
parent f0b6cd375c
commit 61ed163821
29 changed files with 305 additions and 312 deletions
+2
View File
@@ -140,6 +140,8 @@ It defaults to `false`. For example, when set to `true`, a blank string would ev
**ownPropertyOnly** hides scope variables from prototypes, useful when you're passing a not sanitized object into LiquidJS or need to hide prototypes from templates. Defaults to `true`. **ownPropertyOnly** hides scope variables from prototypes, useful when you're passing a not sanitized object into LiquidJS or need to hide prototypes from templates. Defaults to `true`.
Built-in DoS limits and host isolation guidance are documented in [Security Model](./security-model.html).
{% note info Nonexistent Tags %} {% note info Nonexistent Tags %}
Nonexistent tags always throw errors during parsing and this behavior cannot be customized. Nonexistent tags always throw errors during parsing and this behavior cannot be customized.
{% endnote %} {% endnote %}
+24 -17
View File
@@ -22,7 +22,7 @@ Expected output:
</div> </div>
``` ```
Firstly, [register][register-tags] a tag named `wrap` and parse the content into `this.tpls`. Here in `parse(tagToken, remainTokens)`: Firstly, [register][register-tags] a tag named `wrap` and parse the content into `this.tpls`. In the tag `constructor(tagToken, remainTokens, liquid)`:
- `tagToken` is current token `{%raw%}{% wrap %}{%endraw%}`, and - `tagToken` is current token `{%raw%}{% wrap %}{%endraw%}`, and
- `remainTokens` is an array of all tokens following `{%raw%}{% wrap %}{%endraw%}` until the end of this template file. - `remainTokens` is an array of all tokens following `{%raw%}{% wrap %}{%endraw%}` until the end of this template file.
@@ -30,9 +30,12 @@ Firstly, [register][register-tags] a tag named `wrap` and parse the content into
Basically, what we need to do is take/`.shift()` enough tags from `remainTokens` until we get an `endwrap` token (the name can be arbitrary, but by convention it should be `endwrap`). And if there's no `endwrap` until the end of the template file, we need to throw a tag-not-closed `Error`. Basically, what we need to do is take/`.shift()` enough tags from `remainTokens` until we get an `endwrap` token (the name can be arbitrary, but by convention it should be `endwrap`). And if there's no `endwrap` until the end of the template file, we need to throw a tag-not-closed `Error`.
```javascript ```javascript
engine.registerTag('wrap', { const { Tag } = require('liquidjs')
parse(tagToken, remainTokens) {
this.tpls = [] engine.registerTag('wrap', class WrapTag extends Tag {
tpls = []
constructor(tagToken, remainTokens, liquid) {
super(tagToken, remainTokens, liquid)
let closed = false let closed = false
while (remainTokens.length) { while (remainTokens.length) {
let token = remainTokens.shift() let token = remainTokens.shift()
@@ -44,11 +47,11 @@ engine.registerTag('wrap', {
// parse token into template // parse token into template
// parseToken() may consume more than 1 tokens // parseToken() may consume more than 1 tokens
// e.g. {% if %}...{% endif %} // e.g. {% if %}...{% endif %}
let tpl = this.liquid.parser.parseToken(token, remainTokens) let tpl = liquid.parser.parseToken(token, remainTokens)
this.tpls.push(tpl) this.tpls.push(tpl)
} }
if (!closed) throw new Error(`tag ${tagToken.getText()} not closed`) if (!closed) throw new Error(`tag ${tagToken.getText()} not closed`)
}, }
* render(context, emitter) { * render(context, emitter) {
emitter.write("<div class='wrapper'>") emitter.write("<div class='wrapper'>")
yield this.liquid.renderer.renderTemplates(this.tpls, context, emitter) yield this.liquid.renderer.renderTemplates(this.tpls, context, emitter)
@@ -57,16 +60,17 @@ engine.registerTag('wrap', {
}) })
``` ```
`.renderTemplates()` can be async; we need `yield` to wait for it to complete. For more details on async in LiquidJS, see [Sync and Async][async]. Other parts of the `render()` method are quite straightforward. Here's a JSFiddle version: <https://jsfiddle.net/por0zcn1/3/> `.renderTemplates()` can be async; we need `yield` to wait for it to complete. For more details on async in LiquidJS, see [Sync and Async][async]. Here's a JSFiddle version: <https://jsfiddle.net/por0zcn1/3/>
## Using ParseStream ## Using ParseStream
When it comes to complex tags like [for][for] and [if][if], the `parse()` can be very complicated. There's a [ParseStream][ParseStream] utility to organize the `parse()` in event-based style. Following is a re-written `parse()` using `ParseStream` that does exactly the same as the example above. For more complex tags such as [for][for] and [if][if], constructor parsing can get unwieldy. [ParseStream][ParseStream] offers an event-based API for this. The constructor below is equivalent to the example above:
```javascript ```javascript
parse(tagToken, remainTokens) { tpls = []
this.tpls = [] constructor(tagToken, remainTokens, liquid) {
this.liquid.parser.parseStream(remainTokens) super(tagToken, remainTokens, liquid)
liquid.parser.parseStream(remainTokens)
.on('template', tpl => this.tpls.push(tpl)) .on('template', tpl => this.tpls.push(tpl))
// note that we cannot use arrow function because we need `this` // note that we cannot use arrow function because we need `this`
.on('tag:endwrap', function () { this.stop() }) .on('tag:endwrap', function () { this.stop() })
@@ -103,15 +107,18 @@ As you've noticed, there's an additional `repeat.i` in the context of `repeat`.
Each time we enter a new *Context*, we need to push a new *Scope*. And when we finish rendering and exit the *Context*, we pop the *Scope* from the *Context*. As you can see in the following implementation: Each time we enter a new *Context*, we need to push a new *Scope*. And when we finish rendering and exit the *Context*, we pop the *Scope* from the *Context*. As you can see in the following implementation:
```javascript ```javascript
engine.registerTag('repeat', { const { Tag } = require('liquidjs')
parse(tagToken, remainTokens) {
this.tpls = [] engine.registerTag('repeat', class RepeatTag extends Tag {
this.liquid.parser.parseStream(remainTokens) tpls = []
constructor(tagToken, remainTokens, liquid) {
super(tagToken, remainTokens, liquid)
liquid.parser.parseStream(remainTokens)
.on('template', tpl => this.tpls.push(tpl)) .on('template', tpl => this.tpls.push(tpl))
.on('tag:endrepeat', function () { this.stop() }) .on('tag:endrepeat', function () { this.stop() })
.on('end', () => { throw new Error(`tag ${tagToken.getText()} not closed`) }) .on('end', () => { throw new Error(`tag ${tagToken.getText()} not closed`) })
.start() .start()
}, }
* render(context, emitter) { * render(context, emitter) {
const repeat = { i: 1 } const repeat = { i: 1 }
context.push({ repeat }) context.push({ repeat })
@@ -123,7 +130,7 @@ engine.registerTag('repeat', {
}) })
``` ```
The `parse()` is exactly the same as `wrap` tag, we repeat the content simply by calling `.renderTemplates(this.tpls)` twice during `render()`. Here's the JSFiddle: <https://jsfiddle.net/por0zcn1/2/> The constructor is the same as in the `wrap` tag; we repeat the content by calling `.renderTemplates(this.tpls)` twice during `render()`. Here's the JSFiddle: <https://jsfiddle.net/por0zcn1/2/>
{% note warn Use Push & Pop in Pairs %} {% note warn Use Push & Pop in Pairs %}
`context.push()` and `context.pop()` have to be used in pairs. Failing to `pop()` the *Scope* you pushed will leak the *Scope* to latter templates and may corrupt the *Context* stack. `context.push()` and `context.pop()` have to be used in pairs. Failing to `pop()` the *Scope* you pushed will leak the *Scope* to latter templates and may corrupt the *Context* stack.
+28 -39
View File
@@ -2,21 +2,19 @@
title: Security Model title: Security Model
--- ---
LiquidJS provides DoS-oriented limits (`parseLimit`, `renderLimit`, `memoryLimit`) to reduce risk. This page summarizes those limits, [`ownPropertyOnly`][ownPropertyOnly], custom [`Drop`][drop] usage, and the security boundary to assume in production. LiquidJS provides DoS-oriented limits (`parseLimit`, `templateLimit`, `outputLengthLimit`, `maxDepth`) to reduce risk. This page summarizes those limits, [`ownPropertyOnly`][ownPropertyOnly], custom [`Drop`][drop] usage, and the security boundary to assume in production.
## Security boundary ## At a glance
The built-in limits are cooperative safeguards, not strict runtime isolation. LiquidJS ships a thin cooperative DoS layer:
- They do **not** equal process RSS/heap usage.
- They do **not** sandbox JavaScript execution.
- They should be combined with process/container limits and request timeouts for defense in depth.
## Limits at a glance
- [parseLimit][parseLimit]: limit total template size per `parse()` call. - [parseLimit][parseLimit]: limit total template size per `parse()` call.
- [renderLimit][renderLimit]: limit total render time per `render()` call. - [templateLimit][templateLimit]: limit total tag/HTML/output nodes rendered per `render()` call.
- [memoryLimit][memoryLimit]: cooperatively limit memory-sensitive allocations counted by LiquidJS. - [outputLengthLimit][outputLengthLimit]: limit total output length per `render()` call.
- [maxDepth][maxDepth]: limit nesting depth of `{% render %}`, `{% include %}`, and `{% layout %}`.
- Strftime numeric pad widths in the `date` filter are capped at `1_000_000` (1M) per conversion.
These are cooperative safeguards, not runtime isolation—see [Production guidance](#production-guidance) below for host-level limits and online-service hardening.
## Limit details ## Limit details
@@ -26,9 +24,9 @@ The built-in limits are cooperative safeguards, not strict runtime isolation.
A typical PC handles `1e8` (100M) characters without issues. A typical PC handles `1e8` (100M) characters without issues.
### renderLimit ### templateLimit
Restricting template size alone is insufficient because dynamic loops with large counts can occur during rendering. [renderLimit][renderLimit] mitigates this by limiting the time consumed by each `render()` call. Restricting template size alone is insufficient because dynamic loops with large counts can occur during rendering. [templateLimit][templateLimit] mitigates this by limiting the number of tag, HTML literal, and output nodes rendered in each `render()` call.
```liquid ```liquid
{%- for i in (1..10000000) -%} {%- for i in (1..10000000) -%}
@@ -36,29 +34,19 @@ Restricting template size alone is insufficient because dynamic loops with large
{%- endfor -%} {%- endfor -%}
``` ```
Render time is checked on a per-template basis (before rendering each template). In the above example, there are 2 templates in the loop: `order: ` and `{{i}}`, render time will be checked 10000000x2 times. Each template node (the `for` tag, literal `order: `, output `{{i}}`, and so on) counts toward the limit. In the above example, a limit of `30000000` would be exceeded before the loop finishes.
`renderLimit` is not a hard CPU limiter. It is checked between template renders, so compute-intensive filters/tags/user-defined functions or deeply nested template execution between checks can still cause DoS. `templateLimit` is checked before each node render, so compute-intensive filters/tags/user-defined functions between checks can still cause DoS.
### memoryLimit ### outputLengthLimit
`memoryLimit` only limits operations that LiquidJS explicitly counts. [outputLengthLimit][outputLengthLimit] caps the cumulative length of output written during a `render()` call, including output from partials rendered via `{% render %}`.
- Counted: memory-sensitive LiquidJS operations that call internal memory accounting. ### maxDepth
- Not guaranteed counted: arbitrary user object behavior such as custom `toValue()`/`toString()` chains, or other host-side code that allocates outside LiquidJS accounting points.
In other words, `memoryLimit` limits what LiquidJS counts, not every byte your process may allocate. [maxDepth][maxDepth] limits how deeply `{% render %}`, `{% include %}`, and `{% layout %}` can nest. Defaults to `128`. In sync rendering (`renderSync`), nested tags are driven by `toValueSync`, which recursively resumes each yielded generator on the call stack—deep nesting can overflow it, and `maxDepth` caps that depth. Async `render()` resumes the same tag generators via `toPromise`/`yield` without a deep synchronous call chain, so stack overflow is not a concern there (the limit still applies as a DoS guard).
Even with a small number of templates and iterations, memory usage can grow exponentially. In the following example, memory doubles with each iteration: The `memoryLimit` option was removed in v11; enforce memory limits at the host or process level instead.
```liquid
{% assign array = "1,2,3" | split: "," %}
{% for i in (1..32) %}
{% assign array = array | concat: array %}
{% endfor %}
```
As [JavaScript uses GC to manage memory](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Memory_management), `memoryLimit` may not reflect the actual memory footprint.
## `ownPropertyOnly` and scope data ## `ownPropertyOnly` and scope data
@@ -68,20 +56,21 @@ With [`ownPropertyOnly`][ownPropertyOnly] `true`, plain scope objects only expos
[`Drop`][drop] values are not restricted the same way: LiquidJS still reads the prototype chain and may call [`liquidMethodMissing`][liquidMethodMissing]. **You** control what a drop exposes; narrow APIs and never feed unsafe data into drops unless the class is built for template access. `ownPropertyOnly` alone does not harden custom drops—audit them like any privileged code. [`Drop`][drop] values are not restricted the same way: LiquidJS still reads the prototype chain and may call [`liquidMethodMissing`][liquidMethodMissing]. **You** control what a drop exposes; narrow APIs and never feed unsafe data into drops unless the class is built for template access. `ownPropertyOnly` alone does not harden custom drops—audit them like any privileged code.
## Online service guidance ## Production guidance
If you run an online service, avoid rendering fully user-defined templates whenever possible. LiquidJS does not sandbox template code—custom filters, tags, and scope helpers run as ordinary JavaScript with your process privileges. Built-in DoS limits are one layer; production deployments, especially online services that accept template input, need additional hardening:
- Prefer curated templates or a restricted template subset. - **Prefer curated templates** over fully user-defined Liquid when possible; if users need customization, offer a restricted subset rather than open template editing.
- If user-defined templates are required, isolate rendering (worker/process/container), enforce OS/container memory and CPU limits, and apply request rate limits. - Run each render in a **worker thread or child process** with a wall-clock timeout; **kill** the worker on expiry. Libraries such as [paralleljs][paralleljs] can help for heavy single-template work.
- Treat `parseLimit`/`renderLimit`/`memoryLimit` as one layer in a broader DoS defense strategy. - Enforce **container/Kubernetes cgroup limits**, `ulimit`, or equivalent on the renderer process for memory and CPU.
- Apply **request rate limits** at the API or gateway layer.
For heavy single-template operations, process-level isolation is still recommended (for example with [paralleljs][paralleljs]). - **`node:vm`, `isolated-vm`, and Jinja/Twig-style sandbox modes are not a security boundary**—template logic runs in the same JS runtime as your app, with your privileges.
[paralleljs]: https://www.npmjs.com/package/paralleljs [paralleljs]: https://www.npmjs.com/package/paralleljs
[parseLimit]: /api/interfaces/LiquidOptions.html#parseLimit [parseLimit]: /api/interfaces/LiquidOptions.html#parseLimit
[renderLimit]: /api/interfaces/LiquidOptions.html#renderLimit [templateLimit]: /api/interfaces/LiquidOptions.html#templateLimit
[memoryLimit]: /api/interfaces/LiquidOptions.html#memoryLimit [outputLengthLimit]: /api/interfaces/LiquidOptions.html#outputLengthLimit
[maxDepth]: /api/interfaces/LiquidOptions.html#maxDepth
[ownPropertyOnly]: /api/interfaces/LiquidOptions.html#ownPropertyOnly [ownPropertyOnly]: /api/interfaces/LiquidOptions.html#ownPropertyOnly
[renderOwnPropertyOnly]: /api/interfaces/RenderOptions.html#ownPropertyOnly [renderOwnPropertyOnly]: /api/interfaces/RenderOptions.html#ownPropertyOnly
[strictVariables]: /api/interfaces/LiquidOptions.html#strictVariables [strictVariables]: /api/interfaces/LiquidOptions.html#strictVariables
+1 -2
View File
@@ -42,8 +42,7 @@
if (!/\/playground(?:\.html)?$/.test(location.pathname)) return; if (!/\/playground(?:\.html)?$/.test(location.pathname)) return;
updateVersion(liquidjs.version); updateVersion(liquidjs.version);
const engine = new liquidjs.Liquid({ const engine = new liquidjs.Liquid({
memoryLimit: 1e5, templateLimit: 1e5
renderLimit: 1e5
}); });
const colorScheme = window.matchMedia('(prefers-color-scheme: dark)'); const colorScheme = window.matchMedia('(prefers-color-scheme: dark)');
const editor = createEditor('editorEl', 'liquid'); const editor = createEditor('editorEl', 'liquid');
+10 -8
View File
@@ -1,4 +1,3 @@
import { getPerformance } from '../util/performance'
import { Drop } from '../drop/drop' import { Drop } from '../drop/drop'
import { __assign } from 'tslib' import { __assign } from 'tslib'
import { NormalizedFullOptions, defaultOptions, RenderOptions } from '../liquid-options' import { NormalizedFullOptions, defaultOptions, RenderOptions } from '../liquid-options'
@@ -36,17 +35,19 @@ export class Context {
*/ */
public strictVariables: boolean; public strictVariables: boolean;
public ownPropertyOnly: boolean; public ownPropertyOnly: boolean;
public memoryLimit: Limiter; public templateLimit: Limiter;
public renderLimit: Limiter; public outputLengthLimit: Limiter;
public constructor (env: object = {}, opts: NormalizedFullOptions = defaultOptions, renderOptions: RenderOptions = {}, { memoryLimit, renderLimit }: { [key: string]: Limiter } = {}) { public depthLimit: Limiter;
public constructor (env: object = {}, opts: NormalizedFullOptions = defaultOptions, renderOptions: RenderOptions = {}, { templateLimit, outputLengthLimit, depthLimit }: { templateLimit?: Limiter, outputLengthLimit?: Limiter, depthLimit?: Limiter } = {}) {
this.sync = !!renderOptions.sync this.sync = !!renderOptions.sync
this.opts = opts this.opts = opts
this.globals = renderOptions.globals ?? opts.globals this.globals = renderOptions.globals ?? opts.globals
this.environments = isObject(env) ? env : Object(env) this.environments = isObject(env) ? env : Object(env)
this.strictVariables = renderOptions.strictVariables ?? this.opts.strictVariables this.strictVariables = renderOptions.strictVariables ?? this.opts.strictVariables
this.ownPropertyOnly = renderOptions.ownPropertyOnly ?? opts.ownPropertyOnly this.ownPropertyOnly = renderOptions.ownPropertyOnly ?? opts.ownPropertyOnly
this.memoryLimit = memoryLimit ?? new Limiter('memory alloc', renderOptions.memoryLimit ?? opts.memoryLimit) this.templateLimit = templateLimit ?? new Limiter('template', renderOptions.templateLimit ?? opts.templateLimit)
this.renderLimit = renderLimit ?? new Limiter('template render', getPerformance().now() + (renderOptions.renderLimit ?? opts.renderLimit)) this.outputLengthLimit = outputLengthLimit ?? new Limiter('output length', renderOptions.outputLengthLimit ?? opts.outputLengthLimit)
this.depthLimit = depthLimit ?? new Limiter('template depth', opts.maxDepth)
} }
public getRegister<T> (key: string, defaultValue: T = undefined as T): T { public getRegister<T> (key: string, defaultValue: T = undefined as T): T {
return (this.registers[key] = this.registers[key] || defaultValue) return (this.registers[key] = this.registers[key] || defaultValue)
@@ -109,8 +110,9 @@ export class Context {
strictVariables: this.strictVariables, strictVariables: this.strictVariables,
ownPropertyOnly: this.ownPropertyOnly ownPropertyOnly: this.ownPropertyOnly
}, { }, {
renderLimit: this.renderLimit, templateLimit: this.templateLimit,
memoryLimit: this.memoryLimit outputLengthLimit: this.outputLengthLimit,
depthLimit: this.depthLimit
}) })
} }
private findScope (key: string | number) { private findScope (key: string | number) {
+9 -2
View File
@@ -1,10 +1,17 @@
import { stringify } from '../util' import { stringify, Limiter } from '../util'
import { Emitter } from './emitter' import { Emitter } from './emitter'
export class SimpleEmitter implements Emitter { export class SimpleEmitter implements Emitter {
public buffer = ''; public buffer = '';
private outputLengthLimit?: Limiter
constructor (outputLengthLimit?: Limiter) {
this.outputLengthLimit = outputLengthLimit
}
public write (html: any) { public write (html: any) {
this.buffer += stringify(html) const str = stringify(html)
this.outputLengthLimit?.use(str.length)
this.buffer += str
} }
} }
+10 -2
View File
@@ -1,12 +1,20 @@
import { stringify } from '../util' import { stringify, Limiter } from '../util'
import { Emitter } from './emitter' import { Emitter } from './emitter'
import { PassThrough } from 'stream' import { PassThrough } from 'stream'
export class StreamedEmitter implements Emitter { export class StreamedEmitter implements Emitter {
public buffer = ''; public buffer = '';
public stream: NodeJS.ReadWriteStream = new PassThrough() public stream: NodeJS.ReadWriteStream = new PassThrough()
private outputLengthLimit?: Limiter
constructor (outputLengthLimit?: Limiter) {
this.outputLengthLimit = outputLengthLimit
}
public write (html: any) { public write (html: any) {
this.stream.write(stringify(html)) const str = stringify(html)
this.outputLengthLimit?.use(str.length)
this.stream.write(str)
} }
public error (err: Error) { public error (err: Error) {
this.stream.emit('error', err) this.stream.emit('error', err)
-18
View File
@@ -8,9 +8,6 @@ import { EmptyDrop } from '../drop'
export const join = argumentsToValue(function (this: FilterImpl, v: any[], arg: string) { export const join = argumentsToValue(function (this: FilterImpl, v: any[], arg: string) {
const array = toArray(v) const array = toArray(v)
const sep = isNil(arg) ? ' ' : stringify(arg) const sep = isNil(arg) ? ' ' : stringify(arg)
let outputSize = sep.length * Math.max(array.length - 1, 0)
for (let i = 0; i < array.length; i++) outputSize += String(array[i]).length
this.context.memoryLimit.use(outputSize)
return Array.prototype.join.call(array, sep) return Array.prototype.join.call(array, sep)
}) })
export const last = argumentsToValue(function (this: FilterImpl, v: any) { export const last = argumentsToValue(function (this: FilterImpl, v: any) {
@@ -21,14 +18,12 @@ export const first = argumentsToValue(function (this: FilterImpl, v: any) {
}) })
export const reverse = argumentsToValue(function (this: FilterImpl, v: any[]) { export const reverse = argumentsToValue(function (this: FilterImpl, v: any[]) {
const array = toArray(v) const array = toArray(v)
this.context.memoryLimit.use(array.length)
return [...array].reverse() return [...array].reverse()
}) })
function * sortBy<T> (this: FilterImpl, arr: T[], property: string | undefined, comparator: (a: unknown, b: unknown) => number): IterableIterator<unknown> { function * sortBy<T> (this: FilterImpl, arr: T[], property: string | undefined, comparator: (a: unknown, b: unknown) => number): IterableIterator<unknown> {
const values: [T, unknown][] = [] const values: [T, unknown][] = []
const array = toArray(arr) const array = toArray(arr)
this.context.memoryLimit.use(array.length)
for (const item of array) { for (const item of array) {
values.push([ values.push([
item, item,
@@ -51,7 +46,6 @@ export const size = (v: string | any[]) => v?.length || 0
export function * map (this: FilterImpl, arr: Scope[], property: string): IterableIterator<unknown> { export function * map (this: FilterImpl, arr: Scope[], property: string): IterableIterator<unknown> {
const results = [] const results = []
const array = toArray(arr) const array = toArray(arr)
this.context.memoryLimit.use(array.length)
for (const item of array) { for (const item of array) {
results.push(yield this.context._getFromScope(item, stringify(property), false)) results.push(yield this.context._getFromScope(item, stringify(property), false))
} }
@@ -70,14 +64,12 @@ export function * sum (this: FilterImpl, arr: Scope[], property?: string): Itera
export function compact<T> (this: FilterImpl, arr: T[]) { export function compact<T> (this: FilterImpl, arr: T[]) {
const array = toArray(arr) const array = toArray(arr)
this.context.memoryLimit.use(array.length)
return Array.prototype.filter.call(array, x => !isNil(toValue(x))) return Array.prototype.filter.call(array, x => !isNil(toValue(x)))
} }
export function concat<T1, T2> (this: FilterImpl, v: T1[], arg: T2[] = []): (T1 | T2)[] { export function concat<T1, T2> (this: FilterImpl, v: T1[], arg: T2[] = []): (T1 | T2)[] {
const lhs = toArray(v) const lhs = toArray(v)
const rhs = toArray(arg) const rhs = toArray(arg)
this.context.memoryLimit.use(lhs.length + rhs.length)
return Array.prototype.concat.call(lhs, rhs) return Array.prototype.concat.call(lhs, rhs)
} }
@@ -87,7 +79,6 @@ export function push<T> (this: FilterImpl, v: T[], arg: T): T[] {
export function unshift<T> (this: FilterImpl, v: T[], arg: T): T[] { export function unshift<T> (this: FilterImpl, v: T[], arg: T): T[] {
const array = toArray(v) const array = toArray(v)
this.context.memoryLimit.use(array.length)
const clone = [...array] const clone = [...array]
clone.unshift(arg) clone.unshift(arg)
return clone return clone
@@ -95,7 +86,6 @@ export function unshift<T> (this: FilterImpl, v: T[], arg: T): T[] {
export function pop<T> (this: FilterImpl, v: T[]): T[] { export function pop<T> (this: FilterImpl, v: T[]): T[] {
const array = toArray(v) const array = toArray(v)
this.context.memoryLimit.use(array.length)
const clone = [...array] const clone = [...array]
clone.pop() clone.pop()
return clone return clone
@@ -103,7 +93,6 @@ export function pop<T> (this: FilterImpl, v: T[]): T[] {
export function shift<T> (this: FilterImpl, v: T[]): T[] { export function shift<T> (this: FilterImpl, v: T[]): T[] {
const array = toArray(v) const array = toArray(v)
this.context.memoryLimit.use(array.length)
const clone = [...array] const clone = [...array]
clone.shift() clone.shift()
return clone return clone
@@ -114,7 +103,6 @@ export function slice<T> (this: FilterImpl, v: T[] | string, begin: number, leng
if (isNil(v)) return [] if (isNil(v)) return []
if (!isArray(v)) v = stringify(v) if (!isArray(v)) v = stringify(v)
begin = begin < 0 ? v.length + begin : begin begin = begin < 0 ? v.length + begin : begin
this.context.memoryLimit.use(length)
return isArray(v) return isArray(v)
? Array.prototype.slice.call(v, begin, begin + length) ? Array.prototype.slice.call(v, begin, begin + length)
: String.prototype.slice.call(v, begin, begin + length) : String.prototype.slice.call(v, begin, begin + length)
@@ -133,7 +121,6 @@ function expectedMatcher (this: FilterImpl, expected: any): (v: any) => boolean
function * filter<T extends object> (this: FilterImpl, include: boolean, arr: T[], property: string, expected: any): IterableIterator<unknown> { function * filter<T extends object> (this: FilterImpl, include: boolean, arr: T[], property: string, expected: any): IterableIterator<unknown> {
const values: unknown[] = [] const values: unknown[] = []
arr = toArray(arr) arr = toArray(arr)
this.context.memoryLimit.use(arr.length)
const token = new Tokenizer(stringify(property)).readScopeValue() const token = new Tokenizer(stringify(property)).readScopeValue()
for (const item of arr) { for (const item of arr) {
values.push(yield evalToken(token, this.context.spawn(item))) values.push(yield evalToken(token, this.context.spawn(item)))
@@ -146,7 +133,6 @@ function * filter_exp<T extends object> (this: FilterImpl, include: boolean, arr
const filtered: unknown[] = [] const filtered: unknown[] = []
const keyTemplate = new Value(stringify(exp), this.liquid) const keyTemplate = new Value(stringify(exp), this.liquid)
const array = toArray(arr) const array = toArray(arr)
this.context.memoryLimit.use(array.length)
for (const item of array) { for (const item of array) {
this.context.push({ [itemName]: item }) this.context.push({ [itemName]: item })
const value = yield keyTemplate.value(this.context) const value = yield keyTemplate.value(this.context)
@@ -176,7 +162,6 @@ export function * group_by<T extends object> (this: FilterImpl, arr: T[], proper
const map = new Map() const map = new Map()
arr = toEnumerable(arr) arr = toEnumerable(arr)
const token = new Tokenizer(stringify(property)).readScopeValue() const token = new Tokenizer(stringify(property)).readScopeValue()
this.context.memoryLimit.use(arr.length)
for (const item of arr) { for (const item of arr) {
const key = yield evalToken(token, this.context.spawn(item)) const key = yield evalToken(token, this.context.spawn(item))
if (!map.has(key)) map.set(key, []) if (!map.has(key)) map.set(key, [])
@@ -189,7 +174,6 @@ export function * group_by_exp<T extends object> (this: FilterImpl, arr: T[], it
const map = new Map() const map = new Map()
const keyTemplate = new Value(stringify(exp), this.liquid) const keyTemplate = new Value(stringify(exp), this.liquid)
arr = toEnumerable(arr) arr = toEnumerable(arr)
this.context.memoryLimit.use(arr.length)
for (const item of arr) { for (const item of arr) {
this.context.push({ [itemName]: item }) this.context.push({ [itemName]: item })
const key = yield keyTemplate.value(this.context) const key = yield keyTemplate.value(this.context)
@@ -253,7 +237,6 @@ export function * find_exp<T extends object> (this: FilterImpl, arr: T[], itemNa
export function uniq<T> (this: FilterImpl, arr: T[]): T[] { export function uniq<T> (this: FilterImpl, arr: T[]): T[] {
arr = toArray(arr) arr = toArray(arr)
this.context.memoryLimit.use(arr.length)
return [...new Set(arr)] return [...new Set(arr)]
} }
@@ -261,7 +244,6 @@ export function sample<T> (this: FilterImpl, v: T[] | string, count = 1): T | st
v = toValue(v) v = toValue(v)
if (isNil(v)) return [] if (isNil(v)) return []
if (!isArray(v)) v = stringify(v) if (!isArray(v)) v = stringify(v)
this.context.memoryLimit.use(v.length)
const shuffled = [...v].sort(() => Math.random() - 0.5) const shuffled = [...v].sort(() => Math.random() - 0.5)
if (count === 1) return shuffled[0] if (count === 1) return shuffled[0]
return shuffled.slice(0, count) return shuffled.slice(0, count)
-3
View File
@@ -10,16 +10,13 @@ import { base64Encode, base64Decode } from './base64-impl'
export function base64_encode (this: FilterImpl, value: string | Buffer): string { export function base64_encode (this: FilterImpl, value: string | Buffer): string {
if (typeof Buffer !== 'undefined' && Buffer.isBuffer(value)) { if (typeof Buffer !== 'undefined' && Buffer.isBuffer(value)) {
this.context.memoryLimit.use(value.byteLength)
return value.toString('base64') return value.toString('base64')
} }
const str = stringify(value) const str = stringify(value)
this.context.memoryLimit.use(str.length)
return base64Encode(str) return base64Encode(str)
} }
export function base64_decode (this: FilterImpl, value: string): string { export function base64_decode (this: FilterImpl, value: string): string {
const str = stringify(value) const str = stringify(value)
this.context.memoryLimit.use(str.length)
return base64Decode(str) return base64Decode(str)
} }
-2
View File
@@ -10,13 +10,11 @@ import { sha256 as sha256Impl, hmacSha256 as hmacSha256Impl } from './crypto-imp
export function sha256 (this: FilterImpl, value: unknown): string | Promise<string> { export function sha256 (this: FilterImpl, value: unknown): string | Promise<string> {
const str = stringify(value) const str = stringify(value)
this.context.memoryLimit.use(str.length)
return sha256Impl(str) return sha256Impl(str)
} }
export function hmac_sha256 (this: FilterImpl, value: unknown, key: unknown): string | Promise<string> { export function hmac_sha256 (this: FilterImpl, value: unknown, key: unknown): string | Promise<string> {
const str = stringify(value) const str = stringify(value)
const keyStr = stringify(key) const keyStr = stringify(key)
this.context.memoryLimit.use(str.length + keyStr.length)
return hmacSha256Impl(str, keyStr) return hmacSha256Impl(str, keyStr)
} }
+4 -8
View File
@@ -3,14 +3,11 @@ import { FilterImpl } from '../template'
import { NormalizedFullOptions } from '../liquid-options' import { NormalizedFullOptions } from '../liquid-options'
export function date (this: FilterImpl, v: string | Date, format?: string, timezoneOffset?: number | string) { export function date (this: FilterImpl, v: string | Date, format?: string, timezoneOffset?: number | string) {
const size = ((v as string)?.length ?? 0) + ((timezoneOffset as string)?.length ?? 0)
this.context.memoryLimit.use(size)
const date = parseDate(v, this.context.opts, timezoneOffset) const date = parseDate(v, this.context.opts, timezoneOffset)
if (!date) return v if (!date) return v
format = toValue(format) format = toValue(format)
format = isNil(format) ? this.context.opts.dateFormat : stringify(format) format = isNil(format) ? this.context.opts.dateFormat : stringify(format)
this.context.memoryLimit.use(format.length) return strftime(date, format)
return strftime(date, format, this.context.memoryLimit)
} }
export function date_to_xmlschema (this: FilterImpl, v: string | Date) { export function date_to_xmlschema (this: FilterImpl, v: string | Date) {
@@ -32,14 +29,13 @@ export function date_to_long_string (this: FilterImpl, v: string | Date, type?:
function stringify_date (this: FilterImpl, v: string | Date, month_type: string, type?: string, style?: string) { function stringify_date (this: FilterImpl, v: string | Date, month_type: string, type?: string, style?: string) {
const date = parseDate(v, this.context.opts) const date = parseDate(v, this.context.opts)
if (!date) return v if (!date) return v
const ml = this.context.memoryLimit
if (type === 'ordinal') { if (type === 'ordinal') {
const d = date.getDate() const d = date.getDate()
return style === 'US' return style === 'US'
? strftime(date, `${month_type} ${d}%q, %Y`, ml) ? strftime(date, `${month_type} ${d}%q, %Y`)
: strftime(date, `${d}%q ${month_type} %Y`, ml) : strftime(date, `${d}%q ${month_type} %Y`)
} }
return strftime(date, `%d ${month_type} %Y`, ml) return strftime(date, `%d ${month_type} %Y`)
} }
function parseDate (v: string | Date, opts: NormalizedFullOptions, timezoneOffset?: number | string): LiquidDate | undefined { function parseDate (v: string | Date, opts: NormalizedFullOptions, timezoneOffset?: number | string): LiquidDate | undefined {
-4
View File
@@ -18,7 +18,6 @@ const unescapeMap: Record<string, string> = {
export function escape (this: FilterImpl, str: string) { export function escape (this: FilterImpl, str: string) {
str = stringify(str) str = stringify(str)
this.context.memoryLimit.use(str.length)
return str.replace(/&|<|>|"|'/g, m => escapeMap[m]) return str.replace(/&|<|>|"|'/g, m => escapeMap[m])
} }
@@ -28,7 +27,6 @@ export function xml_escape (this: FilterImpl, str: string) {
function unescape (this: FilterImpl, str: string) { function unescape (this: FilterImpl, str: string) {
str = stringify(str) str = stringify(str)
this.context.memoryLimit.use(str.length)
return str.replace(/&(amp|lt|gt|#34|#39);/g, m => unescapeMap[m]) return str.replace(/&(amp|lt|gt|#34|#39);/g, m => unescapeMap[m])
} }
@@ -38,7 +36,6 @@ export function escape_once (this: FilterImpl, str: string) {
export function newline_to_br (this: FilterImpl, v: string) { export function newline_to_br (this: FilterImpl, v: string) {
const str = stringify(v) const str = stringify(v)
this.context.memoryLimit.use(str.length)
return str.replace(/\r?\n/gm, '<br />\n') return str.replace(/\r?\n/gm, '<br />\n')
} }
@@ -46,7 +43,6 @@ export function newline_to_br (this: FilterImpl, v: string) {
// equivalent is O(n^2) in V8 on unclosed openers. // equivalent is O(n^2) in V8 on unclosed openers.
export function strip_html (this: FilterImpl, v: string) { export function strip_html (this: FilterImpl, v: string) {
const str = stringify(v) const str = stringify(v)
this.context.memoryLimit.use(str.length)
const blocks = new Map([['<script', '</script>'], ['<style', '</style>'], ['<!--', '-->'], ['<', '>']]) const blocks = new Map([['<script', '</script>'], ['<style', '</style>'], ['<!--', '-->'], ['<', '>']])
let out = '' let out = ''
let i = 0 let i = 0
+1 -21
View File
@@ -2,18 +2,6 @@ import { isFalsy } from '../render/boolean'
import { identify, isArray, isString, toValue } from '../util/underscore' import { identify, isArray, isString, toValue } from '../util/underscore'
import { FilterImpl } from '../template' import { FilterImpl } from '../template'
function chargeJsonReplacerValue (memoryLimit: { use(count: number): void }, val: unknown) {
if (typeof val === 'string') {
memoryLimit.use(val.length)
} else if (val === null || typeof val === 'number' || typeof val === 'boolean') {
memoryLimit.use(JSON.stringify(val).length)
} else if (Array.isArray(val)) {
memoryLimit.use(val.length + 1)
} else if (typeof val === 'object') {
memoryLimit.use(2)
}
}
function defaultFilter<T1 extends boolean, T2> (this: FilterImpl, value: T1, defaultValue: T2, ...args: Array<[string, any]>): T1 | T2 { function defaultFilter<T1 extends boolean, T2> (this: FilterImpl, value: T1, defaultValue: T2, ...args: Array<[string, any]>): T1 | T2 {
value = toValue(value) value = toValue(value)
if (isArray(value) || isString(value)) return value.length ? value : defaultValue if (isArray(value) || isString(value)) return value.length ? value : defaultValue
@@ -22,29 +10,21 @@ function defaultFilter<T1 extends boolean, T2> (this: FilterImpl, value: T1, def
} }
function json (this: FilterImpl, value: any, space = 0) { function json (this: FilterImpl, value: any, space = 0) {
const memoryLimit = this.context.memoryLimit return JSON.stringify(value, undefined, space)
return JSON.stringify(value, (_key, val) => {
chargeJsonReplacerValue(memoryLimit, val)
return val
}, space)
} }
function inspect (this: FilterImpl, value: any, space = 0) { function inspect (this: FilterImpl, value: any, space = 0) {
const memoryLimit = this.context.memoryLimit
const ancestors: object[] = [] const ancestors: object[] = []
return JSON.stringify(value, function (this: unknown, _key: unknown, value: any) { return JSON.stringify(value, function (this: unknown, _key: unknown, value: any) {
if (typeof value !== 'object' || value === null) { if (typeof value !== 'object' || value === null) {
chargeJsonReplacerValue(memoryLimit, value)
return value return value
} }
// `this` is the object that value is contained in, i.e., its direct parent. // `this` is the object that value is contained in, i.e., its direct parent.
while (ancestors.length > 0 && ancestors[ancestors.length - 1] !== this) ancestors.pop() while (ancestors.length > 0 && ancestors[ancestors.length - 1] !== this) ancestors.pop()
if (ancestors.includes(value)) { if (ancestors.includes(value)) {
memoryLimit.use('[Circular]'.length)
return '[Circular]' return '[Circular]'
} }
ancestors.push(value) ancestors.push(value)
chargeJsonReplacerValue(memoryLimit, value)
return value return value
}, space) }, space)
} }
-27
View File
@@ -22,7 +22,6 @@ export function append (this: FilterImpl, v: string, arg: string) {
assert(arguments.length === 2, 'append expect 2 arguments') assert(arguments.length === 2, 'append expect 2 arguments')
const lhs = stringify(v) const lhs = stringify(v)
const rhs = stringify(arg) const rhs = stringify(arg)
this.context.memoryLimit.use(lhs.length + rhs.length)
return lhs + rhs return lhs + rhs
} }
@@ -30,16 +29,13 @@ export function prepend (this: FilterImpl, v: string, arg: string) {
assert(arguments.length === 2, 'prepend expect 2 arguments') assert(arguments.length === 2, 'prepend expect 2 arguments')
const lhs = stringify(v) const lhs = stringify(v)
const rhs = stringify(arg) const rhs = stringify(arg)
this.context.memoryLimit.use(lhs.length + rhs.length)
return rhs + lhs return rhs + lhs
} }
export function lstrip (this: FilterImpl, v: string, chars?: string) { export function lstrip (this: FilterImpl, v: string, chars?: string) {
const str = stringify(v) const str = stringify(v)
this.context.memoryLimit.use(str.length)
if (chars) { if (chars) {
chars = stringify(chars) chars = stringify(chars)
this.context.memoryLimit.use(chars.length)
for (let i = 0, set = new Set(chars); i < str.length; i++) { for (let i = 0, set = new Set(chars); i < str.length; i++) {
if (!set.has(str[i])) return str.slice(i) if (!set.has(str[i])) return str.slice(i)
} }
@@ -50,34 +46,29 @@ export function lstrip (this: FilterImpl, v: string, chars?: string) {
export function downcase (this: FilterImpl, v: string) { export function downcase (this: FilterImpl, v: string) {
const str = stringify(v) const str = stringify(v)
this.context.memoryLimit.use(str.length)
return str.toLowerCase() return str.toLowerCase()
} }
export function upcase (this: FilterImpl, v: string) { export function upcase (this: FilterImpl, v: string) {
const str = stringify(v) const str = stringify(v)
this.context.memoryLimit.use(str.length)
return stringify(str).toUpperCase() return stringify(str).toUpperCase()
} }
export function remove (this: FilterImpl, v: string, arg: string) { export function remove (this: FilterImpl, v: string, arg: string) {
const str = stringify(v) const str = stringify(v)
arg = stringify(arg) arg = stringify(arg)
this.context.memoryLimit.use(str.length + arg.length)
return str.split(arg).join('') return str.split(arg).join('')
} }
export function remove_first (this: FilterImpl, v: string, l: string) { export function remove_first (this: FilterImpl, v: string, l: string) {
const str = stringify(v) const str = stringify(v)
l = stringify(l) l = stringify(l)
this.context.memoryLimit.use(str.length + l.length)
return str.replace(l, '') return str.replace(l, '')
} }
export function remove_last (this: FilterImpl, v: string, l: string) { export function remove_last (this: FilterImpl, v: string, l: string) {
const str = stringify(v) const str = stringify(v)
const pattern = stringify(l) const pattern = stringify(l)
this.context.memoryLimit.use(str.length + pattern.length)
const index = str.lastIndexOf(pattern) const index = str.lastIndexOf(pattern)
if (index === -1) return str if (index === -1) return str
return str.substring(0, index) + str.substring(index + pattern.length) return str.substring(0, index) + str.substring(index + pattern.length)
@@ -85,10 +76,8 @@ export function remove_last (this: FilterImpl, v: string, l: string) {
export function rstrip (this: FilterImpl, str: string, chars?: string) { export function rstrip (this: FilterImpl, str: string, chars?: string) {
str = stringify(str) str = stringify(str)
this.context.memoryLimit.use(str.length)
if (chars) { if (chars) {
chars = stringify(chars) chars = stringify(chars)
this.context.memoryLimit.use(chars.length)
for (let i = str.length - 1, set = new Set(chars); i >= 0; i--) { for (let i = str.length - 1, set = new Set(chars); i >= 0; i--) {
if (!set.has(str[i])) return str.slice(0, i + 1) if (!set.has(str[i])) return str.slice(0, i + 1)
} }
@@ -99,7 +88,6 @@ export function rstrip (this: FilterImpl, str: string, chars?: string) {
export function split (this: FilterImpl, v: string, arg: string) { export function split (this: FilterImpl, v: string, arg: string) {
const str = stringify(v) const str = stringify(v)
this.context.memoryLimit.use(str.length)
const arr = str.split(stringify(arg)) const arr = str.split(stringify(arg))
// align to ruby split, which is the behavior of shopify/liquid // align to ruby split, which is the behavior of shopify/liquid
// see: https://ruby-doc.org/core-2.4.0/String.html#method-i-split // see: https://ruby-doc.org/core-2.4.0/String.html#method-i-split
@@ -109,10 +97,8 @@ export function split (this: FilterImpl, v: string, arg: string) {
export function strip (this: FilterImpl, v: string, chars?: string) { export function strip (this: FilterImpl, v: string, chars?: string) {
const str = stringify(v) const str = stringify(v)
this.context.memoryLimit.use(str.length)
if (chars) { if (chars) {
const set = new Set(stringify(chars)) const set = new Set(stringify(chars))
this.context.memoryLimit.use(set.size)
let i = 0 let i = 0
let j = str.length - 1 let j = str.length - 1
while (set.has(str[i])) i++ while (set.has(str[i])) i++
@@ -124,13 +110,11 @@ export function strip (this: FilterImpl, v: string, chars?: string) {
export function strip_newlines (this: FilterImpl, v: string) { export function strip_newlines (this: FilterImpl, v: string) {
const str = stringify(v) const str = stringify(v)
this.context.memoryLimit.use(str.length)
return str.replace(/\r?\n/gm, '') return str.replace(/\r?\n/gm, '')
} }
export function capitalize (this: FilterImpl, str: string) { export function capitalize (this: FilterImpl, str: string) {
str = stringify(str) str = stringify(str)
this.context.memoryLimit.use(str.length)
return str.charAt(0).toUpperCase() + str.slice(1).toLowerCase() return str.charAt(0).toUpperCase() + str.slice(1).toLowerCase()
} }
@@ -139,8 +123,6 @@ export function replace (this: FilterImpl, v: string, pattern: string, replaceme
pattern = stringify(pattern) pattern = stringify(pattern)
replacement = stringify(replacement) replacement = stringify(replacement)
const parts = str.split(pattern) const parts = str.split(pattern)
const outputSize = str.length + (parts.length - 1) * (replacement.length - pattern.length)
this.context.memoryLimit.use(outputSize)
return parts.join(replacement) return parts.join(replacement)
} }
@@ -148,7 +130,6 @@ export function replace_first (this: FilterImpl, v: string, arg1: string, arg2:
const str = stringify(v) const str = stringify(v)
arg1 = stringify(arg1) arg1 = stringify(arg1)
arg2 = stringify(arg2) arg2 = stringify(arg2)
this.context.memoryLimit.use(str.length + arg1.length + arg2.length)
return str.replace(arg1, () => arg2) return str.replace(arg1, () => arg2)
} }
@@ -156,7 +137,6 @@ export function replace_last (this: FilterImpl, v: string, arg1: string, arg2: s
const str = stringify(v) const str = stringify(v)
const pattern = stringify(arg1) const pattern = stringify(arg1)
const replacement = stringify(arg2) const replacement = stringify(arg2)
this.context.memoryLimit.use(str.length + pattern.length + replacement.length)
const index = str.lastIndexOf(pattern) const index = str.lastIndexOf(pattern)
if (index === -1) return str if (index === -1) return str
return str.substring(0, index) + replacement + str.substring(index + pattern.length) return str.substring(0, index) + replacement + str.substring(index + pattern.length)
@@ -165,7 +145,6 @@ export function replace_last (this: FilterImpl, v: string, arg1: string, arg2: s
export function truncate (this: FilterImpl, v: string, l = 50, o = '...') { export function truncate (this: FilterImpl, v: string, l = 50, o = '...') {
const str = stringify(v) const str = stringify(v)
o = stringify(o) o = stringify(o)
this.context.memoryLimit.use(str.length + o.length)
if (str.length <= l) return v if (str.length <= l) return v
return str.substring(0, l - o.length) + o return str.substring(0, l - o.length) + o
} }
@@ -173,7 +152,6 @@ export function truncate (this: FilterImpl, v: string, l = 50, o = '...') {
export function truncatewords (this: FilterImpl, v: string, words = 15, o = '...') { export function truncatewords (this: FilterImpl, v: string, words = 15, o = '...') {
const str = stringify(v) const str = stringify(v)
o = stringify(o) o = stringify(o)
this.context.memoryLimit.use(str.length + o.length)
const arr = str.split(/\s+/) const arr = str.split(/\s+/)
if (words <= 0) words = 1 if (words <= 0) words = 1
let ret = arr.slice(0, words).join(' ') let ret = arr.slice(0, words).join(' ')
@@ -183,13 +161,11 @@ export function truncatewords (this: FilterImpl, v: string, words = 15, o = '...
export function normalize_whitespace (this: FilterImpl, v: string) { export function normalize_whitespace (this: FilterImpl, v: string) {
const str = stringify(v) const str = stringify(v)
this.context.memoryLimit.use(str.length)
return str.replace(/\s+/g, ' ') return str.replace(/\s+/g, ' ')
} }
export function number_of_words (this: FilterImpl, input: string, mode?: 'cjk' | 'auto') { export function number_of_words (this: FilterImpl, input: string, mode?: 'cjk' | 'auto') {
const str = stringify(input) const str = stringify(input)
this.context.memoryLimit.use(str.length)
input = str.trim() input = str.trim()
if (!input) return 0 if (!input) return 0
switch (mode) { switch (mode) {
@@ -209,9 +185,6 @@ export function number_of_words (this: FilterImpl, input: string, mode?: 'cjk' |
export function array_to_sentence_string (this: FilterImpl, array: unknown[], connector = 'and') { export function array_to_sentence_string (this: FilterImpl, array: unknown[], connector = 'and') {
connector = stringify(connector) connector = stringify(connector)
let outputSize = connector.length + array.length * 2
for (let i = 0; i < array.length; i++) outputSize += stringify(array[i]).length
this.context.memoryLimit.use(outputSize)
switch (array.length) { switch (array.length) {
case 0: case 0:
return '' return ''
+15 -13
View File
@@ -87,10 +87,12 @@ export interface LiquidOptions {
orderedFilterParameters?: boolean; orderedFilterParameters?: boolean;
/** For DoS handling, limit total length of templates parsed in one `parse()` call. A typical PC can handle 1e8 (100M) characters without issues. */ /** For DoS handling, limit total length of templates parsed in one `parse()` call. A typical PC can handle 1e8 (100M) characters without issues. */
parseLimit?: number; parseLimit?: number;
/** For DoS handling, limit total time (in ms) for each `render()` call. */ /** For DoS handling, limit total renders of tag/HTML/output in one `render()` call. */
renderLimit?: number; templateLimit?: number;
/** For DoS handling, limit new objects creation, including array concat/join/strftime, etc. A typical PC can handle 1e9 (1G) memory without issue. */ /** For DoS handling, limit total output length in one `render()` call. */
memoryLimit?: number; outputLengthLimit?: number;
/** For DoS handling, limit nesting depth of `{% render %}`, `{% include %}`, and `{% layout %}` tags. Defaults to `128`. */
maxDepth?: number;
} }
export interface RenderOptions { export interface RenderOptions {
@@ -110,12 +112,10 @@ export interface RenderOptions {
* Same as `ownPropertyOnly` on LiquidOptions, but only for current render() call * Same as `ownPropertyOnly` on LiquidOptions, but only for current render() call
*/ */
ownPropertyOnly?: boolean; ownPropertyOnly?: boolean;
/** For DoS handling, limit total renders of tag/HTML/output in one `render()` call. A typical PC can handle 1e5 renders of typical templates per second. */ /** For DoS handling, limit total renders of tag/HTML/output in one `render()` call. */
templateLimit?: number; templateLimit?: number;
/** For DoS handling, limit total time (in ms) for each `render()` call. */ /** For DoS handling, limit total output length in one `render()` call. */
renderLimit?: number; outputLengthLimit?: number;
/** For DoS handling, limit new objects creation, including array concat/join/strftime, etc. A typical PC can handle 1e9 (1G) memory without issue.. */
memoryLimit?: number;
} }
export interface RenderFileOptions extends RenderOptions { export interface RenderFileOptions extends RenderOptions {
@@ -160,8 +160,9 @@ export interface NormalizedFullOptions extends NormalizedOptions {
globals: object; globals: object;
operators: Operators; operators: Operators;
parseLimit: number; parseLimit: number;
renderLimit: number; templateLimit: number;
memoryLimit: number; outputLengthLimit: number;
maxDepth: number;
} }
export const defaultOptions: NormalizedFullOptions = { export const defaultOptions: NormalizedFullOptions = {
@@ -194,9 +195,10 @@ export const defaultOptions: NormalizedFullOptions = {
lenientIf: false, lenientIf: false,
globals: {}, globals: {},
operators: defaultOperators, operators: defaultOperators,
memoryLimit: Infinity,
parseLimit: Infinity, parseLimit: Infinity,
renderLimit: Infinity templateLimit: Infinity,
outputLengthLimit: Infinity,
maxDepth: 128
} }
export function normalize (options: LiquidOptions): NormalizedFullOptions { export function normalize (options: LiquidOptions): NormalizedFullOptions {
-1
View File
@@ -67,7 +67,6 @@ export function evalQuotedToken (token: QuotedToken) {
function * evalRangeToken (token: RangeToken, ctx: Context) { function * evalRangeToken (token: RangeToken, ctx: Context) {
const low: number = yield evalToken(token.lhs, ctx) const low: number = yield evalToken(token.lhs, ctx)
const high: number = yield evalToken(token.rhs, ctx) const high: number = yield evalToken(token.rhs, ctx)
ctx.memoryLimit.use(high - low + 1)
return range(+low, +high + 1) return range(+low, +high + 1)
} }
+3 -10
View File
@@ -1,4 +1,3 @@
import { getPerformance } from '../util/performance'
import { toPromise, RenderError, LiquidErrors, LiquidError } from '../util' import { toPromise, RenderError, LiquidErrors, LiquidError } from '../util'
import { Context } from '../context' import { Context } from '../context'
import { Template } from '../template' import { Template } from '../template'
@@ -6,23 +5,17 @@ import { Emitter, StreamedEmitter, SimpleEmitter } from '../emitters'
export class Render { export class Render {
public renderTemplatesToNodeStream (templates: Template[], ctx: Context): NodeJS.ReadableStream { public renderTemplatesToNodeStream (templates: Template[], ctx: Context): NodeJS.ReadableStream {
const emitter = new StreamedEmitter() const emitter = new StreamedEmitter(ctx.outputLengthLimit)
Promise.resolve().then(() => toPromise(this.renderTemplates(templates, ctx, emitter))) Promise.resolve().then(() => toPromise(this.renderTemplates(templates, ctx, emitter)))
.then(() => emitter.end(), err => emitter.error(err)) .then(() => emitter.end(), err => emitter.error(err))
return emitter.stream return emitter.stream
} }
public * renderTemplates (templates: Template[], ctx: Context, emitter?: Emitter): IterableIterator<any> { public * renderTemplates (templates: Template[], ctx: Context, emitter: Emitter = new SimpleEmitter(ctx.outputLengthLimit)): IterableIterator<any> {
if (!emitter) {
emitter = new SimpleEmitter()
}
ctx.renderLimit.check(getPerformance().now())
const errors = [] const errors = []
for (const tpl of templates) { for (const tpl of templates) {
ctx.renderLimit.check(getPerformance().now()) ctx.templateLimit.use(1)
try { try {
// if tpl.render supports emitter, it'll return empty `html`
const html = yield tpl.render(ctx, emitter) const html = yield tpl.render(ctx, emitter)
// if not, it'll return an `html`, write to the emitter for it
html && emitter.write(html) html && emitter.write(html)
if (ctx.breakCalled || ctx.continueCalled) break if (ctx.breakCalled || ctx.continueCalled) break
} catch (e) { } catch (e) {
+9 -7
View File
@@ -43,13 +43,6 @@ export default class extends Tag {
} }
* render (ctx: Context, emitter: Emitter): Generator<unknown, void | string, Template[]> { * render (ctx: Context, emitter: Emitter): Generator<unknown, void | string, Template[]> {
const r = this.liquid.renderer const r = this.liquid.renderer
let collection = toEnumerable(yield evalToken(this.collection, ctx))
if (!collection.length) {
yield r.renderTemplates(this.elseTemplates, ctx, emitter)
return
}
const continueKey = 'continue-' + this.variable + '-' + this.collection.getText() const continueKey = 'continue-' + this.variable + '-' + this.collection.getText()
ctx.push(createScope({ continue: ctx.getRegister(continueKey, {}) })) ctx.push(createScope({ continue: ctx.getRegister(continueKey, {}) }))
const hash = (yield this.hash.render(ctx)) as Record<string, any> const hash = (yield this.hash.render(ctx)) as Record<string, any>
@@ -59,6 +52,7 @@ export default class extends Tag {
? Object.keys(hash).filter(x => MODIFIERS.includes(x)) ? Object.keys(hash).filter(x => MODIFIERS.includes(x))
: MODIFIERS.filter(x => hash[x] !== undefined) : MODIFIERS.filter(x => hash[x] !== undefined)
let collection = toEnumerable(yield evalToken(this.collection, ctx))
collection = modifiers.reduce((collection, modifier: valueOf<typeof MODIFIERS>) => { collection = modifiers.reduce((collection, modifier: valueOf<typeof MODIFIERS>) => {
if (modifier === 'offset') return offset(collection, hash['offset']) if (modifier === 'offset') return offset(collection, hash['offset'])
if (modifier === 'limit') return limit(collection, hash['limit']) if (modifier === 'limit') return limit(collection, hash['limit'])
@@ -66,6 +60,14 @@ export default class extends Tag {
}, collection) }, collection)
ctx.setRegister(continueKey, (hash['offset'] || 0) + collection.length) ctx.setRegister(continueKey, (hash['offset'] || 0) + collection.length)
if (!collection.length) {
yield r.renderTemplates(this.elseTemplates, ctx, emitter)
return
}
if (!this.templates.length) return
const scope = createScope({ forloop: new ForloopDrop(collection.length, this.collection.getText(), this.variable) }) const scope = createScope({ forloop: new ForloopDrop(collection.length, this.collection.getText(), this.variable) })
ctx.push(scope) ctx.push(scope)
for (const item of collection) { for (const item of collection) {
+2
View File
@@ -28,6 +28,7 @@ export default class extends Tag {
this.hash = new Hash(tokenizer, liquid.options.jekyllInclude || liquid.options.keyValueSeparator) this.hash = new Hash(tokenizer, liquid.options.jekyllInclude || liquid.options.keyValueSeparator)
} }
* render (ctx: Context, emitter: Emitter): Generator<unknown, void, unknown> { * render (ctx: Context, emitter: Emitter): Generator<unknown, void, unknown> {
ctx.depthLimit.use(1)
const { liquid, hash, withVar } = this const { liquid, hash, withVar } = this
const { renderer } = liquid const { renderer } = liquid
const filepath = (yield renderFilePath(this.file, ctx, liquid)) as string const filepath = (yield renderFilePath(this.file, ctx, liquid)) as string
@@ -43,6 +44,7 @@ export default class extends Tag {
yield renderer.renderTemplates(templates, ctx, emitter) yield renderer.renderTemplates(templates, ctx, emitter)
ctx.pop() ctx.pop()
ctx.restoreRegister(saved) ctx.restoreRegister(saved)
ctx.depthLimit.release(1)
} }
public * children (partials: boolean, sync: boolean): Generator<unknown, Template[]> { public * children (partials: boolean, sync: boolean): Generator<unknown, Template[]> {
+2
View File
@@ -26,6 +26,7 @@ export default class extends Tag {
yield renderer.renderTemplates(this.templates, ctx, emitter) yield renderer.renderTemplates(this.templates, ctx, emitter)
return return
} }
ctx.depthLimit.use(1)
const filepath = (yield renderFilePath(this.file, ctx, liquid)) as string const filepath = (yield renderFilePath(this.file, ctx, liquid)) as string
assert(filepath, () => `illegal file path "${filepath}"`) assert(filepath, () => `illegal file path "${filepath}"`)
const templates = (yield liquid._parseLayoutFile(filepath, ctx.sync, this.currentFile)) as Template[] const templates = (yield liquid._parseLayoutFile(filepath, ctx.sync, this.currentFile)) as Template[]
@@ -43,6 +44,7 @@ export default class extends Tag {
ctx.push(createScope((yield args.render(ctx)) as Scope)) ctx.push(createScope((yield args.render(ctx)) as Scope))
yield renderer.renderTemplates(templates, ctx, emitter) yield renderer.renderTemplates(templates, ctx, emitter)
ctx.pop() ctx.pop()
ctx.depthLimit.release(1)
} }
public * children (partials: boolean): Generator<unknown, Template[]> { public * children (partials: boolean): Generator<unknown, Template[]> {
+2
View File
@@ -55,6 +55,7 @@ export default class extends Tag {
this.hash = new Hash(tokenizer, liquid.options.keyValueSeparator) this.hash = new Hash(tokenizer, liquid.options.keyValueSeparator)
} }
* render (ctx: Context, emitter: Emitter): Generator<unknown, void, unknown> { * render (ctx: Context, emitter: Emitter): Generator<unknown, void, unknown> {
ctx.depthLimit.use(1)
const { liquid, hash } = this const { liquid, hash } = this
const filepath = (yield renderFilePath(this.file, ctx, liquid)) as string const filepath = (yield renderFilePath(this.file, ctx, liquid)) as string
assert(filepath, () => `illegal file path "${filepath}"`) assert(filepath, () => `illegal file path "${filepath}"`)
@@ -81,6 +82,7 @@ export default class extends Tag {
const templates = (yield liquid._parsePartialFile(filepath, childCtx.sync, this.currentFile)) as Template[] const templates = (yield liquid._parsePartialFile(filepath, childCtx.sync, this.currentFile)) as Template[]
yield liquid.renderer.renderTemplates(templates, childCtx, emitter) yield liquid.renderer.renderTemplates(templates, childCtx, emitter)
} }
ctx.depthLimit.release(1)
} }
public * children (partials: boolean, sync: boolean): Generator<unknown, Template[]> { public * children (partials: boolean, sync: boolean): Generator<unknown, Template[]> {
+4
View File
@@ -45,6 +45,10 @@ export default class extends Tag {
const limit = (args.limit === undefined) ? collection.length : args.limit const limit = (args.limit === undefined) ? collection.length : args.limit
collection = collection.slice(offset, offset + limit) collection = collection.slice(offset, offset + limit)
if (!collection.length) return
if (!this.templates.length) return
const cols = args.cols || collection.length const cols = args.cols || collection.length
const r = this.liquid.renderer const r = this.liquid.renderer
+5
View File
@@ -14,6 +14,11 @@ export class Limiter {
this.base += +count this.base += +count
} }
} }
release (count: number) {
if (+count > 0) {
this.base -= +count
}
}
check (count: number) { check (count: number) {
if (+count > 0) { if (+count > 0) {
assert(+count <= this.limit, this.message) assert(+count <= this.limit, this.message)
+8
View File
@@ -188,6 +188,14 @@ describe('util/strftime', function () {
it('should have higher priority than H', () => { it('should have higher priority than H', () => {
expect(t(then, '%0H')).toBe('03') expect(t(then, '%0H')).toBe('03')
}) })
it('should allow pad width up to MAX_STRFTIME_PAD', () => {
expect(t(now, '%100000d').length).toBe(100000)
expect(t(now, `%${1_000_000}d`).length).toBe(1_000_000)
})
it('should throw when pad width exceeds MAX_STRFTIME_PAD', () => {
expect(() => t(now, `%${1024 * 1024 + 1}d`)).toThrow('strftime pad width limit exceeded')
expect(() => t(now, '%5000000d')).toThrow('strftime pad width limit exceeded')
})
}) })
describe('modifier field', () => { describe('modifier field', () => {
it('should ignore E modifier', () => { it('should ignore E modifier', () => {
+14 -8
View File
@@ -1,13 +1,15 @@
import { changeCase, padStart, padEnd } from './underscore' import { changeCase, padStart, padEnd } from './underscore'
import { LiquidDate } from './liquid-date' import { LiquidDate } from './liquid-date'
import type { Limiter } from './limiter' import { assert } from './assert'
/** Per-conversion numeric width cap for strftime (%N, %15d, …). */
export const MAX_STRFTIME_PAD = 1024 * 1024
const rFormat = /%([-_0^#:]+)?(\d+)?([EO])?(.)/ const rFormat = /%([-_0^#:]+)?(\d+)?([EO])?(.)/
interface FormatOptions { interface FormatOptions {
flags: Record<string, boolean>; flags: Record<string, boolean>;
width?: string; width?: string;
modifier?: string; modifier?: string;
memoryLimit?: Pick<Limiter, 'use'>;
} }
// prototype extensions // prototype extensions
@@ -98,8 +100,8 @@ const formatCodes: Record<string, FormatCodeHandler> = {
M: (d: LiquidDate) => d.getMinutes(), M: (d: LiquidDate) => d.getMinutes(),
N: (d: LiquidDate, opts: FormatOptions) => { N: (d: LiquidDate, opts: FormatOptions) => {
const width = Number(opts.width) || 9 const width = Number(opts.width) || 9
assertPadWidth(width)
const str = String(d.getMilliseconds()).slice(0, width) const str = String(d.getMilliseconds()).slice(0, width)
opts.memoryLimit?.use(width - str.length)
return padEnd(str, width, '0') return padEnd(str, width, '0')
}, },
p: (d: LiquidDate) => (d.getHours() < 12 ? 'AM' : 'PM'), p: (d: LiquidDate) => (d.getHours() < 12 ? 'AM' : 'PM'),
@@ -123,25 +125,29 @@ const formatCodes: Record<string, FormatCodeHandler> = {
} }
formatCodes.h = formatCodes.b formatCodes.h = formatCodes.b
export function strftime (d: LiquidDate, formatStr: string, memoryLimit?: Pick<Limiter, 'use'>) { export function strftime (d: LiquidDate, formatStr: string) {
let output = '' let output = ''
let remaining = formatStr let remaining = formatStr
let match let match
while ((match = rFormat.exec(remaining))) { while ((match = rFormat.exec(remaining))) {
output += remaining.slice(0, match.index) output += remaining.slice(0, match.index)
remaining = remaining.slice(match.index + match[0].length) remaining = remaining.slice(match.index + match[0].length)
output += format(d, match, memoryLimit) output += format(d, match)
} }
return output + remaining return output + remaining
} }
function format (d: LiquidDate, match: RegExpExecArray, memoryLimit?: Pick<Limiter, 'use'>) { function assertPadWidth (width: number) {
assert(width <= MAX_STRFTIME_PAD, 'strftime pad width limit exceeded')
}
function format (d: LiquidDate, match: RegExpExecArray) {
const [input, flagStr = '', width, modifier, conversion] = match const [input, flagStr = '', width, modifier, conversion] = match
const convert = formatCodes[conversion] const convert = formatCodes[conversion]
if (!convert) return input if (!convert) return input
const flags: Record<string, boolean> = {} const flags: Record<string, boolean> = {}
for (const flag of flagStr) flags[flag] = true for (const flag of flagStr) flags[flag] = true
let ret = String(convert(d, { flags, width, modifier, memoryLimit })) let ret = String(convert(d, { flags, width, modifier }))
let padChar = padSpaceChars.has(conversion) ? ' ' : '0' let padChar = padSpaceChars.has(conversion) ? ' ' : '0'
let padWidth = Number(width) || padWidths[conversion] || 0 let padWidth = Number(width) || padWidths[conversion] || 0
if (flags['^']) ret = ret.toUpperCase() if (flags['^']) ret = ret.toUpperCase()
@@ -149,6 +155,6 @@ function format (d: LiquidDate, match: RegExpExecArray, memoryLimit?: Pick<Limit
if (flags['_']) padChar = ' ' if (flags['_']) padChar = ' '
else if (flags['0']) padChar = '0' else if (flags['0']) padChar = '0'
if (flags['-']) padWidth = 0 if (flags['-']) padWidth = 0
memoryLimit?.use(Number(padWidth) - ret.length) else assertPadWidth(padWidth)
return padStart(ret, padWidth, padChar) return padStart(ret, padWidth, padChar)
} }
-7
View File
@@ -519,13 +519,6 @@ describe('Issues', function () {
const result = engine.parseAndRenderSync(`\n{{ "foo" | pos }}`) const result = engine.parseAndRenderSync(`\n{{ "foo" | pos }}`)
expect(result).toEqual('\n[2,12] foo') expect(result).toEqual('\n[2,12] foo')
}) })
it("memoryLimit doesn't work in for tag #776", () => {
const engine = new Liquid({
memoryLimit: 1e5
})
const tpl = `{% for i in (1..1000000000) %} {{'a'}} {% endfor %}`
expect(() => engine.parseAndRenderSync(tpl)).toThrow('memory alloc limit exceeded, line:1, col:1')
})
it('group_by_exp fails with object as input #785', () => { it('group_by_exp fails with object as input #785', () => {
const site = { const site = {
tags: { tags: {
+13 -23
View File
@@ -204,31 +204,21 @@ describe('filters/date', function () {
return test('{{ "1990-12-31T23:00:00Z" | date: "%Y-%m-%dT%H:%M:%S" }}', '1991-01-01T04:30:00', undefined, optsWithDateFormat) return test('{{ "1990-12-31T23:00:00Z" | date: "%Y-%m-%dT%H:%M:%S" }}', '1991-01-01T04:30:00', undefined, optsWithDateFormat)
}) })
}) })
describe('strftime width / memoryLimit', () => { describe('strftime width', () => {
it('should charge memoryLimit for huge numeric strftime widths', () => { it('should honor numeric strftime pad width', () => {
const liquid = new Liquid({ memoryLimit: 500 }) const liquid = new Liquid()
expect(() => liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: '%5000000d' }))
.toThrow('memory alloc limit exceeded')
})
it('should charge memoryLimit for array format PoC', () => {
const liquid = new Liquid({ memoryLimit: 50, renderLimit: 1e9 })
expect(() => liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: ['a'.repeat(2000000)] }))
.toThrow('memory alloc limit exceeded')
})
it('should charge memoryLimit for object toString format PoC', () => {
const liquid = new Liquid({ memoryLimit: 50, renderLimit: 1e9 })
const huge = 'a'.repeat(2000000)
const f = { toString: () => huge }
expect(() => liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f }))
.toThrow('memory alloc limit exceeded')
})
it('should honor numeric strftime pad width when memoryLimit allows', () => {
const liquid = new Liquid({ memoryLimit: 1e7 })
const out = liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: '%5000d' }) const out = liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: '%5000d' })
expect(out.length).toBe(5000) expect(out.length).toBe(5000)
const tight = new Liquid({ memoryLimit: 100 }) })
expect(() => tight.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: '%5000d' })) it('should honor large numeric strftime pad width up to the cap', () => {
.toThrow('memory alloc limit exceeded') const liquid = new Liquid()
const out = liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: '%100000d' })
expect(out.length).toBe(100000)
})
it('should throw when numeric strftime pad width is too large', () => {
const liquid = new Liquid()
expect(() => liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: '%5000000d' }))
.toThrow('strftime pad width limit exceeded')
}) })
}) })
}) })
+125 -88
View File
@@ -4,15 +4,18 @@ import { mock, restore } from '../../stub/mockfs'
describe('DoS related', function () { describe('DoS related', function () {
describe('#parseLimit', function () { describe('#parseLimit', function () {
afterEach(restore) afterEach(restore)
it('should throw when parse limit exceeded', async () => { it('should throw when parse limit exceeded', async () => {
const noLimit = new Liquid() const noLimit = new Liquid()
const limit10 = new Liquid({ parseLimit: 10 }) const limit10 = new Liquid({ parseLimit: 10 })
const limit90 = new Liquid({ parseLimit: 90 }) const limit90 = new Liquid({ parseLimit: 90 })
const template = '{% capture bar %}{{ foo | bar: 3, a[3] }}{% endcapture %}' const template = '{% capture bar %}{{ foo | bar: 3, a[3] }}{% endcapture %}'
await expect(noLimit.parseAndRender(template)).resolves.toBe('') await expect(noLimit.parseAndRender(template)).resolves.toBe('')
await expect(limit10.parseAndRender(template)).rejects.toThrow('parse length limit exceeded') await expect(limit10.parseAndRender(template)).rejects.toThrow('parse length limit exceeded')
await expect(limit90.parseAndRender(template)).resolves.toBe('') await expect(limit90.parseAndRender(template)).resolves.toBe('')
}) })
it('should take included template into account', async () => { it('should take included template into account', async () => {
mock({ mock({
'/small': 'Lorem ipsum', '/small': 'Lorem ipsum',
@@ -23,119 +26,151 @@ describe('DoS related', function () {
await expect(liquid.parseAndRender('{% include "large" %}')).rejects.toThrow('parse length limit exceeded') await expect(liquid.parseAndRender('{% include "large" %}')).rejects.toThrow('parse length limit exceeded')
}) })
}) })
describe('#renderLimit', () => {
describe('#templateLimit', () => {
it('should throw when rendering too many templates', async () => { it('should throw when rendering too many templates', async () => {
const src = '{% for i in (1..1000) %}{{i}},{% endfor %}' const src = '{% for i in (1..1000) %}{{i}},{% endfor %}'
const noLimit = new Liquid() const noLimit = new Liquid()
const limitSmall = new Liquid({ renderLimit: 0.01 }) const limitSmall = new Liquid({ templateLimit: 100 })
const limitLarge = new Liquid({ renderLimit: 2e4 }) const limitLarge = new Liquid({ templateLimit: 2001 })
await expect(noLimit.parseAndRender(src)).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/) await expect(noLimit.parseAndRender(src)).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
await expect(limitSmall.parseAndRender(src)).rejects.toThrow('template render limit exceeded') await expect(limitSmall.parseAndRender(src)).rejects.toThrow('template limit exceeded')
await expect(limitLarge.parseAndRender(src)).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/) await expect(limitLarge.parseAndRender(src)).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
}) })
it('should support reset when calling render', async () => { it('should support reset when calling render', async () => {
const src = '{% for i in (1..1000) %}{{i}},{% endfor %}' const src = '{% for i in (1..1000) %}{{i}},{% endfor %}'
const liquid = new Liquid({ renderLimit: 0.01 }) const liquid = new Liquid({ templateLimit: 100 })
await expect(liquid.parseAndRender(src)).rejects.toThrow('template render limit exceeded') await expect(liquid.parseAndRender(src)).rejects.toThrow('template limit exceeded')
await expect(liquid.parseAndRender(src, {}, { renderLimit: 1e6 })).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/) await expect(liquid.parseAndRender(src, {}, { templateLimit: 2001 })).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
}) })
it('should take partials into account', async () => { it('should take partials into account', async () => {
mock({ mock({
'/small': '{% for i in (1..5) %}{{i}}{% endfor %}', '/small': '{% for i in (1..5) %}{{i}}{% endfor %}',
'/large': '{% for i in (1..50000000) %}{{i}}{% endfor %}' '/large': '{% for i in (1..50000000) %}{{i}}{% endfor %}'
}) })
const liquid = new Liquid({ root: '/', renderLimit: 1000 }) const liquid = new Liquid({ root: '/', templateLimit: 1000 })
await expect(liquid.parseAndRender('{% render "large" %}')).rejects.toThrow('template render limit exceeded') await expect(liquid.parseAndRender('{% render "large" %}')).rejects.toThrow('template limit exceeded')
await expect(liquid.parseAndRender('{% render "small" %}')).resolves.toBe('12345') await expect(liquid.parseAndRender('{% render "small" %}')).resolves.toBe('12345')
}) })
it('should enforce renderLimit when for body has no template nodes', () => {
const liquid = new Liquid({ memoryLimit: 1e9, renderLimit: 1 })
expect(() => liquid.parseAndRenderSync('{%- for i in (1..5000000) -%}{%- endfor -%}', {}))
.toThrow('template render limit exceeded')
}) })
it('should enforce renderLimit when tablerow body has no template nodes', () => {
const liquid = new Liquid({ memoryLimit: 1e9, renderLimit: 1 }) describe('#outputLengthLimit', () => {
expect(() => liquid.parseAndRenderSync('{%- tablerow i in (1..1000000) cols:1 -%}{%- endtablerow -%}', {})) it('should throw when output length exceeded', async () => {
.toThrow('template render limit exceeded') const src = '{% for i in (1..1000) %}{{i}},{% endfor %}'
}) const noLimit = new Liquid()
}) const limitSmall = new Liquid({ outputLengthLimit: 10 })
describe('#memoryLimit', () => { const limitLarge = new Liquid({ outputLengthLimit: 5000 })
it('should throw for too many array creation in filters', async () => { await expect(noLimit.parseAndRender(src)).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
const array = Array(1e3).fill(0) await expect(limitSmall.parseAndRender(src)).rejects.toThrow('output length limit exceeded')
const liquid = new Liquid({ memoryLimit: 100 }) await expect(limitLarge.parseAndRender(src)).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
await expect(liquid.parseAndRender('{{ array | slice: 0, 3 | join }}', { array })).resolves.toBe('0 0 0')
await expect(liquid.parseAndRender('{{ array | slice: 0, 300 | join }}', { array })).rejects.toThrow('memory alloc limit exceeded, line:1, col:1')
}) })
it('should support reset when calling render', async () => { it('should support reset when calling render', async () => {
const array = Array(1e3).fill(0) const src = '{% for i in (1..1000) %}{{i}},{% endfor %}'
const liquid = new Liquid({ memoryLimit: 100 }) const liquid = new Liquid({ outputLengthLimit: 10 })
await expect(liquid.parseAndRender('{{ array | slice: 0, 300 | join }}', { array })).rejects.toThrow('memory alloc limit exceeded, line:1, col:1') await expect(liquid.parseAndRender(src)).rejects.toThrow('output length limit exceeded')
await expect(liquid.parseAndRender('{{ array | slice: 0, 300 | join }}', { array }, { memoryLimit: 1e3 })).resolves.toBe(Array(300).fill(0).join(' ')) await expect(liquid.parseAndRender(src, {}, { outputLengthLimit: 5000 })).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
}) })
it('should throw for too many array iteration in tags', async () => {
const array = ['a'] it('should take partials into account', async () => {
const liquid = new Liquid({ memoryLimit: 100 }) mock({
const src = '{% for i in (1..count) %}{% assign array = array | concat: array %}{% endfor %}{{ array | join }}' '/small': 'abc',
await expect(liquid.parseAndRender(src, { array, count: 3 })).resolves.toBe('a a a a a a a a') '/large': '{% for i in (1..1000) %}{{i}}{% endfor %}'
await expect(liquid.parseAndRender(src, { array, count: 100 })).rejects.toThrow('memory alloc limit exceeded, line:1, col:26')
}) })
it('should charge pop allocation to memoryLimit', async () => { const liquid = new Liquid({ root: '/', outputLengthLimit: 10 })
const array = Array(1e3).fill(0) await expect(liquid.parseAndRender('{% render "small" %}')).resolves.toBe('abc')
const liquid = new Liquid({ memoryLimit: 100 }) await expect(liquid.parseAndRender('{% render "large" %}')).rejects.toThrow('output length limit exceeded')
await expect(liquid.parseAndRender('{{ array | pop | size }}', { array })).rejects.toThrow('memory alloc limit exceeded')
}) })
it('should charge sample allocation to memoryLimit', async () => {
const array = Array(1e3).fill(0) it('should enforce outputLengthLimit in sync render', () => {
const liquid = new Liquid({ memoryLimit: 100 }) const liquid = new Liquid({ outputLengthLimit: 5 })
await expect(liquid.parseAndRender('{{ array | sample: 1 | size }}', { array })).rejects.toThrow('memory alloc limit exceeded') expect(() => liquid.parseAndRenderSync('{% for i in (1..100) %}{{i}}{% endfor %}'))
.toThrow('output length limit exceeded')
}) })
it('should charge join by produced output size, not element count', () => {
const array = ['a'.repeat(100), 'b'.repeat(100)] it('should enforce outputLengthLimit in stream render', async () => {
const liquid = new Liquid({ memoryLimit: 100 }) const liquid = new Liquid({ outputLengthLimit: 5 })
expect(() => liquid.parseAndRenderSync('{{ array | join: "" }}', { array })) const tpl = liquid.parse('{% for i in (1..100) %}{{i}}{% endfor %}')
.toThrow('memory alloc limit exceeded') const stream = liquid.renderToNodeStream(tpl)
}) await expect(new Promise((resolve, reject) => {
it('should allow join within memoryLimit', () => { stream.on('error', reject)
const array = ['a'.repeat(20), 'b'.repeat(20)] stream.on('end', resolve)
const liquid = new Liquid({ memoryLimit: 100 }) })).rejects.toThrow('output length limit exceeded')
expect(liquid.parseAndRenderSync('{{ array | join: "" }}', { array })).toBe('a'.repeat(20) + 'b'.repeat(20))
})
it('should prevent concat doubling from bypassing join memoryLimit', () => {
const liquid = new Liquid({ memoryLimit: 1e4 })
const src = '{%- assign a = s | split: "NOSEP" -%}' +
'{%- assign a = a | concat: a -%}{%- assign a = a | concat: a -%}{%- assign a = a | concat: a -%}' +
'{{ a | join: "" | size }}'
expect(() => liquid.parseAndRenderSync(src, { s: 'a'.repeat(5000) }))
.toThrow('memory alloc limit exceeded')
})
it('should charge array_to_sentence_string by produced output size', () => {
const array = ['a'.repeat(100), 'b'.repeat(100), 'c'.repeat(100)]
const liquid = new Liquid({ memoryLimit: 100 })
expect(() => liquid.parseAndRenderSync('{{ array | array_to_sentence_string }}', { array }))
.toThrow('memory alloc limit exceeded')
})
it('should charge json serialization of concat-doubled arrays', () => {
const liquid = new Liquid({ memoryLimit: 1e4 })
const src = '{%- assign a = s | split: "NOSEP" -%}' +
'{%- assign a = a | concat: a -%}{%- assign a = a | concat: a -%}{%- assign a = a | concat: a -%}' +
'{{ a | json | size }}'
expect(() => liquid.parseAndRenderSync(src, { s: 'a'.repeat(5000) }))
.toThrow('memory alloc limit exceeded')
})
it('should charge inspect serialization of concat-doubled arrays', () => {
const liquid = new Liquid({ memoryLimit: 1e4 })
const src = '{%- assign a = s | split: "NOSEP" -%}' +
'{%- assign a = a | concat: a -%}{%- assign a = a | concat: a -%}{%- assign a = a | concat: a -%}' +
'{{ a | inspect | size }}'
expect(() => liquid.parseAndRenderSync(src, { s: 'a'.repeat(5000) }))
.toThrow('memory alloc limit exceeded')
})
it('should charge strip_html input length to memoryLimit', () => {
const liquid = new Liquid({ memoryLimit: 100 })
expect(() => liquid.parseAndRenderSync('{{ s | strip_html }}', { s: 'a'.repeat(200) }))
.toThrow('memory alloc limit exceeded')
}) })
}) })
describe('#maxDepth', () => {
function chain (depth: number, tag: string) {
const templates: Record<string, string> = {}
for (let i = 0; i < depth; i++) {
templates[`t${i}`] = i === depth - 1 ? 'done' : `{% ${tag} "t${i + 1}" %}`
}
return templates
}
it('should throw when include depth exceeded', async () => {
const liquid = new Liquid({ templates: chain(3, 'include'), maxDepth: 2 })
await expect(liquid.parseAndRender('{% include "t0" %}')).rejects.toThrow('template depth limit exceeded')
})
it('should allow include within maxDepth', async () => {
const liquid = new Liquid({ templates: chain(2, 'include'), maxDepth: 2 })
await expect(liquid.parseAndRender('{% include "t0" %}')).resolves.toBe('done')
})
it('should throw when render depth exceeded', async () => {
const liquid = new Liquid({ templates: chain(3, 'render'), maxDepth: 2 })
await expect(liquid.parseAndRender('{% render "t0" %}')).rejects.toThrow('template depth limit exceeded')
})
it('should allow render within maxDepth', async () => {
const liquid = new Liquid({ templates: chain(2, 'render'), maxDepth: 2 })
await expect(liquid.parseAndRender('{% render "t0" %}')).resolves.toBe('done')
})
it('should throw when layout depth exceeded', async () => {
const liquid = new Liquid({
templates: {
a: '{% layout "b" %}body-a',
b: '{% layout "c" %}body-b',
c: 'body-c'
},
maxDepth: 2
})
await expect(liquid.parseAndRender('{% layout "a" %}root')).rejects.toThrow('template depth limit exceeded')
})
it('should allow layout within maxDepth', async () => {
const liquid = new Liquid({
templates: {
a: '{% layout "b" %}body-a',
b: 'body-b'
},
maxDepth: 2
})
await expect(liquid.parseAndRender('{% layout "a" %}root')).resolves.toBe('body-b')
})
it('should not count layout none toward depth', async () => {
const liquid = new Liquid({ maxDepth: 0 })
await expect(liquid.parseAndRender('{% layout none %}ok')).resolves.toBe('ok')
})
it('should default maxDepth to 128', async () => {
const liquid = new Liquid({ templates: chain(128, 'include') })
await expect(liquid.parseAndRender('{% include "t0" %}')).resolves.toBe('done')
const overflow = new Liquid({ templates: chain(129, 'include') })
await expect(overflow.parseAndRender('{% include "t0" %}')).rejects.toThrow('template depth limit exceeded')
})
it('should enforce maxDepth in sync render', () => {
const liquid = new Liquid({ templates: chain(3, 'include'), maxDepth: 2 })
expect(() => liquid.parseAndRenderSync('{% include "t0" %}')).toThrow('template depth limit exceeded')
})
})
describe('strip_html ReDoS', () => { describe('strip_html ReDoS', () => {
// Regression for O(n^2) backtracking on unclosed `<script` / `<style` openers. // Regression for O(n^2) backtracking on unclosed `<script` / `<style` openers.
// The previous regex stalled the event loop for ~10s on 350KB of `'<script'.repeat`. // The previous regex stalled the event loop for ~10s on 350KB of `'<script'.repeat`.
@@ -145,11 +180,13 @@ describe('DoS related', function () {
const payload = '<script'.repeat(50000) const payload = '<script'.repeat(50000)
expect(liquid.parseAndRenderSync('{{ x | strip_html }}', { x: payload })).toBe(payload) expect(liquid.parseAndRenderSync('{{ x | strip_html }}', { x: payload })).toBe(payload)
}, 1000) }, 1000)
it('should handle many unclosed <style openers in linear time', () => { it('should handle many unclosed <style openers in linear time', () => {
const liquid = new Liquid() const liquid = new Liquid()
const payload = '<style'.repeat(50000) const payload = '<style'.repeat(50000)
expect(liquid.parseAndRenderSync('{{ x | strip_html }}', { x: payload })).toBe(payload) expect(liquid.parseAndRenderSync('{{ x | strip_html }}', { x: payload })).toBe(payload)
}, 1000) }, 1000)
it('should handle <script openers that have > but no </script> in linear time', () => { it('should handle <script openers that have > but no </script> in linear time', () => {
const liquid = new Liquid() const liquid = new Liquid()
const payload = '<script>foo'.repeat(50000) const payload = '<script>foo'.repeat(50000)
+12
View File
@@ -120,6 +120,18 @@ describe('tags/for', function () {
const html = await liquid.parseAndRender(src, scope) const html = await liquid.parseAndRender(src, scope)
return expect(html).toBe('b') return expect(html).toBe('b')
}) })
it('should goto else when limit empties collection', async function () {
const src = '{%for c in alpha limit:0%}a{%else%}b{%endfor%}'
const html = await liquid.parseAndRender(src, scope)
return expect(html).toBe('b')
})
it('should goto else when offset past end', async function () {
const src = '{%for c in alpha offset:10%}a{%else%}b{%endfor%}'
const html = await liquid.parseAndRender(src, scope)
return expect(html).toBe('b')
})
}) })
it('should support for with forloop', async function () { it('should support for with forloop', async function () {