feat: remove memoryLimit; add templateLimit, outputLengthLimit, maxDepth (#937)

* feat: remove memoryLimit option (#910)

Co-authored-by: Cursor <[email protected]>

* feat: add templateLimit, outputLengthLimit, and maxDepth DoS limits

Enforce v11 resource guards in render and tags, fix for offset/else behavior, and update tutorials for Tag-class registration.

Co-authored-by: Cursor <[email protected]>

* docs: revert unnecessary tutorial churn from memoryLimit PR

Restore the two-example register-filters-tags structure (Value + Hash)
and undo unrelated constructor/emitter doc edits not required for DoS limits.

Co-authored-by: Cursor <[email protected]>

* docs: trim security-model prose and update render-tag-content

Remove diary-style engine comparisons from security-model.md.
Update render-tag-content tutorial to Tag class examples with tpls class field.

Co-authored-by: Cursor <[email protected]>

* docs: note maxDepth stack overflow applies to renderSync only

Explain why async render does not need maxDepth for stack protection based on generator/toPromise driving.

Co-authored-by: Cursor <[email protected]>

* refactor: track maxDepth via depthLimit Limiter on Context

Replace increaseDepth/decreaseDepth with a shared Limiter that supports
paired use/release, matching templateLimit and outputLengthLimit patterns.

Co-authored-by: Cursor <[email protected]>

* fix: remove spurious diff noise in filter files

Restore misc.ts from origin/next with LF line endings and re-apply only
memoryLimit removal, avoiding CRLF and blank-line churn in the export block.

Co-authored-by: Cursor <[email protected]>

* refactor: minimize PR diff noise

Co-authored-by: Cursor <[email protected]>

* feat: cap strftime pad width at 1M

docs: restructure security model with production guidance
Co-authored-by: Cursor <[email protected]>

* refactor: simplify depthLimit in partial tags and tighten security docs

Drop try/finally around depthLimit in include, layout, and render; release at generator end. Consolidate production guidance in security-model.md. Fix padded-blocks lint in dos.spec.ts.

Co-authored-by: Cursor <[email protected]>

---------

Co-authored-by: Cursor <[email protected]>
This commit is contained in:
Yang Jun
2026-07-15 22:58:13 +08:00
committed by GitHub
co-authored by Cursor
parent f0b6cd375c
commit 61ed163821
29 changed files with 305 additions and 312 deletions
+2
View File
@@ -140,6 +140,8 @@ It defaults to `false`. For example, when set to `true`, a blank string would ev
**ownPropertyOnly** hides scope variables from prototypes, useful when you're passing a not sanitized object into LiquidJS or need to hide prototypes from templates. Defaults to `true`.
Built-in DoS limits and host isolation guidance are documented in [Security Model](./security-model.html).
{% note info Nonexistent Tags %}
Nonexistent tags always throw errors during parsing and this behavior cannot be customized.
{% endnote %}
+25 -18
View File
@@ -22,7 +22,7 @@ Expected output:
</div>
```
Firstly, [register][register-tags] a tag named `wrap` and parse the content into `this.tpls`. Here in `parse(tagToken, remainTokens)`:
Firstly, [register][register-tags] a tag named `wrap` and parse the content into `this.tpls`. In the tag `constructor(tagToken, remainTokens, liquid)`:
- `tagToken` is current token `{%raw%}{% wrap %}{%endraw%}`, and
- `remainTokens` is an array of all tokens following `{%raw%}{% wrap %}{%endraw%}` until the end of this template file.
@@ -30,11 +30,14 @@ Firstly, [register][register-tags] a tag named `wrap` and parse the content into
Basically, what we need to do is take/`.shift()` enough tags from `remainTokens` until we get an `endwrap` token (the name can be arbitrary, but by convention it should be `endwrap`). And if there's no `endwrap` until the end of the template file, we need to throw a tag-not-closed `Error`.
```javascript
engine.registerTag('wrap', {
parse(tagToken, remainTokens) {
this.tpls = []
const { Tag } = require('liquidjs')
engine.registerTag('wrap', class WrapTag extends Tag {
tpls = []
constructor(tagToken, remainTokens, liquid) {
super(tagToken, remainTokens, liquid)
let closed = false
while(remainTokens.length) {
while (remainTokens.length) {
let token = remainTokens.shift()
// we got the end tag! stop taking tokens
if (token.name === 'endwrap') {
@@ -44,11 +47,11 @@ engine.registerTag('wrap', {
// parse token into template
// parseToken() may consume more than 1 tokens
// e.g. {% if %}...{% endif %}
let tpl = this.liquid.parser.parseToken(token, remainTokens)
let tpl = liquid.parser.parseToken(token, remainTokens)
this.tpls.push(tpl)
}
if (!closed) throw new Error(`tag ${tagToken.getText()} not closed`)
},
}
* render(context, emitter) {
emitter.write("<div class='wrapper'>")
yield this.liquid.renderer.renderTemplates(this.tpls, context, emitter)
@@ -57,16 +60,17 @@ engine.registerTag('wrap', {
})
```
`.renderTemplates()` can be async; we need `yield` to wait for it to complete. For more details on async in LiquidJS, see [Sync and Async][async]. Other parts of the `render()` method are quite straightforward. Here's a JSFiddle version: <https://jsfiddle.net/por0zcn1/3/>
`.renderTemplates()` can be async; we need `yield` to wait for it to complete. For more details on async in LiquidJS, see [Sync and Async][async]. Here's a JSFiddle version: <https://jsfiddle.net/por0zcn1/3/>
## Using ParseStream
When it comes to complex tags like [for][for] and [if][if], the `parse()` can be very complicated. There's a [ParseStream][ParseStream] utility to organize the `parse()` in event-based style. Following is a re-written `parse()` using `ParseStream` that does exactly the same as the example above.
For more complex tags such as [for][for] and [if][if], constructor parsing can get unwieldy. [ParseStream][ParseStream] offers an event-based API for this. The constructor below is equivalent to the example above:
```javascript
parse(tagToken, remainTokens) {
this.tpls = []
this.liquid.parser.parseStream(remainTokens)
tpls = []
constructor(tagToken, remainTokens, liquid) {
super(tagToken, remainTokens, liquid)
liquid.parser.parseStream(remainTokens)
.on('template', tpl => this.tpls.push(tpl))
// note that we cannot use arrow function because we need `this`
.on('tag:endwrap', function () { this.stop() })
@@ -103,15 +107,18 @@ As you've noticed, there's an additional `repeat.i` in the context of `repeat`.
Each time we enter a new *Context*, we need to push a new *Scope*. And when we finish rendering and exit the *Context*, we pop the *Scope* from the *Context*. As you can see in the following implementation:
```javascript
engine.registerTag('repeat', {
parse(tagToken, remainTokens) {
this.tpls = []
this.liquid.parser.parseStream(remainTokens)
const { Tag } = require('liquidjs')
engine.registerTag('repeat', class RepeatTag extends Tag {
tpls = []
constructor(tagToken, remainTokens, liquid) {
super(tagToken, remainTokens, liquid)
liquid.parser.parseStream(remainTokens)
.on('template', tpl => this.tpls.push(tpl))
.on('tag:endrepeat', function () { this.stop() })
.on('end', () => { throw new Error(`tag ${tagToken.getText()} not closed`) })
.start()
},
}
* render(context, emitter) {
const repeat = { i: 1 }
context.push({ repeat })
@@ -123,7 +130,7 @@ engine.registerTag('repeat', {
})
```
The `parse()` is exactly the same as `wrap` tag, we repeat the content simply by calling `.renderTemplates(this.tpls)` twice during `render()`. Here's the JSFiddle: <https://jsfiddle.net/por0zcn1/2/>
The constructor is the same as in the `wrap` tag; we repeat the content by calling `.renderTemplates(this.tpls)` twice during `render()`. Here's the JSFiddle: <https://jsfiddle.net/por0zcn1/2/>
{% note warn Use Push & Pop in Pairs %}
`context.push()` and `context.pop()` have to be used in pairs. Failing to `pop()` the *Scope* you pushed will leak the *Scope* to latter templates and may corrupt the *Context* stack.
+28 -39
View File
@@ -2,21 +2,19 @@
title: Security Model
---
LiquidJS provides DoS-oriented limits (`parseLimit`, `renderLimit`, `memoryLimit`) to reduce risk. This page summarizes those limits, [`ownPropertyOnly`][ownPropertyOnly], custom [`Drop`][drop] usage, and the security boundary to assume in production.
LiquidJS provides DoS-oriented limits (`parseLimit`, `templateLimit`, `outputLengthLimit`, `maxDepth`) to reduce risk. This page summarizes those limits, [`ownPropertyOnly`][ownPropertyOnly], custom [`Drop`][drop] usage, and the security boundary to assume in production.
## Security boundary
## At a glance
The built-in limits are cooperative safeguards, not strict runtime isolation.
- They do **not** equal process RSS/heap usage.
- They do **not** sandbox JavaScript execution.
- They should be combined with process/container limits and request timeouts for defense in depth.
## Limits at a glance
LiquidJS ships a thin cooperative DoS layer:
- [parseLimit][parseLimit]: limit total template size per `parse()` call.
- [renderLimit][renderLimit]: limit total render time per `render()` call.
- [memoryLimit][memoryLimit]: cooperatively limit memory-sensitive allocations counted by LiquidJS.
- [templateLimit][templateLimit]: limit total tag/HTML/output nodes rendered per `render()` call.
- [outputLengthLimit][outputLengthLimit]: limit total output length per `render()` call.
- [maxDepth][maxDepth]: limit nesting depth of `{% render %}`, `{% include %}`, and `{% layout %}`.
- Strftime numeric pad widths in the `date` filter are capped at `1_000_000` (1M) per conversion.
These are cooperative safeguards, not runtime isolation—see [Production guidance](#production-guidance) below for host-level limits and online-service hardening.
## Limit details
@@ -26,9 +24,9 @@ The built-in limits are cooperative safeguards, not strict runtime isolation.
A typical PC handles `1e8` (100M) characters without issues.
### renderLimit
### templateLimit
Restricting template size alone is insufficient because dynamic loops with large counts can occur during rendering. [renderLimit][renderLimit] mitigates this by limiting the time consumed by each `render()` call.
Restricting template size alone is insufficient because dynamic loops with large counts can occur during rendering. [templateLimit][templateLimit] mitigates this by limiting the number of tag, HTML literal, and output nodes rendered in each `render()` call.
```liquid
{%- for i in (1..10000000) -%}
@@ -36,29 +34,19 @@ Restricting template size alone is insufficient because dynamic loops with large
{%- endfor -%}
```
Render time is checked on a per-template basis (before rendering each template). In the above example, there are 2 templates in the loop: `order: ` and `{{i}}`, render time will be checked 10000000x2 times.
Each template node (the `for` tag, literal `order: `, output `{{i}}`, and so on) counts toward the limit. In the above example, a limit of `30000000` would be exceeded before the loop finishes.
`renderLimit` is not a hard CPU limiter. It is checked between template renders, so compute-intensive filters/tags/user-defined functions or deeply nested template execution between checks can still cause DoS.
`templateLimit` is checked before each node render, so compute-intensive filters/tags/user-defined functions between checks can still cause DoS.
### memoryLimit
### outputLengthLimit
`memoryLimit` only limits operations that LiquidJS explicitly counts.
[outputLengthLimit][outputLengthLimit] caps the cumulative length of output written during a `render()` call, including output from partials rendered via `{% render %}`.
- Counted: memory-sensitive LiquidJS operations that call internal memory accounting.
- Not guaranteed counted: arbitrary user object behavior such as custom `toValue()`/`toString()` chains, or other host-side code that allocates outside LiquidJS accounting points.
### maxDepth
In other words, `memoryLimit` limits what LiquidJS counts, not every byte your process may allocate.
[maxDepth][maxDepth] limits how deeply `{% render %}`, `{% include %}`, and `{% layout %}` can nest. Defaults to `128`. In sync rendering (`renderSync`), nested tags are driven by `toValueSync`, which recursively resumes each yielded generator on the call stack—deep nesting can overflow it, and `maxDepth` caps that depth. Async `render()` resumes the same tag generators via `toPromise`/`yield` without a deep synchronous call chain, so stack overflow is not a concern there (the limit still applies as a DoS guard).
Even with a small number of templates and iterations, memory usage can grow exponentially. In the following example, memory doubles with each iteration:
```liquid
{% assign array = "1,2,3" | split: "," %}
{% for i in (1..32) %}
{% assign array = array | concat: array %}
{% endfor %}
```
As [JavaScript uses GC to manage memory](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Memory_management), `memoryLimit` may not reflect the actual memory footprint.
The `memoryLimit` option was removed in v11; enforce memory limits at the host or process level instead.
## `ownPropertyOnly` and scope data
@@ -68,20 +56,21 @@ With [`ownPropertyOnly`][ownPropertyOnly] `true`, plain scope objects only expos
[`Drop`][drop] values are not restricted the same way: LiquidJS still reads the prototype chain and may call [`liquidMethodMissing`][liquidMethodMissing]. **You** control what a drop exposes; narrow APIs and never feed unsafe data into drops unless the class is built for template access. `ownPropertyOnly` alone does not harden custom drops—audit them like any privileged code.
## Online service guidance
## Production guidance
If you run an online service, avoid rendering fully user-defined templates whenever possible.
LiquidJS does not sandbox template code—custom filters, tags, and scope helpers run as ordinary JavaScript with your process privileges. Built-in DoS limits are one layer; production deployments, especially online services that accept template input, need additional hardening:
- Prefer curated templates or a restricted template subset.
- If user-defined templates are required, isolate rendering (worker/process/container), enforce OS/container memory and CPU limits, and apply request rate limits.
- Treat `parseLimit`/`renderLimit`/`memoryLimit` as one layer in a broader DoS defense strategy.
For heavy single-template operations, process-level isolation is still recommended (for example with [paralleljs][paralleljs]).
- **Prefer curated templates** over fully user-defined Liquid when possible; if users need customization, offer a restricted subset rather than open template editing.
- Run each render in a **worker thread or child process** with a wall-clock timeout; **kill** the worker on expiry. Libraries such as [paralleljs][paralleljs] can help for heavy single-template work.
- Enforce **container/Kubernetes cgroup limits**, `ulimit`, or equivalent on the renderer process for memory and CPU.
- Apply **request rate limits** at the API or gateway layer.
- **`node:vm`, `isolated-vm`, and Jinja/Twig-style sandbox modes are not a security boundary**—template logic runs in the same JS runtime as your app, with your privileges.
[paralleljs]: https://www.npmjs.com/package/paralleljs
[parseLimit]: /api/interfaces/LiquidOptions.html#parseLimit
[renderLimit]: /api/interfaces/LiquidOptions.html#renderLimit
[memoryLimit]: /api/interfaces/LiquidOptions.html#memoryLimit
[templateLimit]: /api/interfaces/LiquidOptions.html#templateLimit
[outputLengthLimit]: /api/interfaces/LiquidOptions.html#outputLengthLimit
[maxDepth]: /api/interfaces/LiquidOptions.html#maxDepth
[ownPropertyOnly]: /api/interfaces/LiquidOptions.html#ownPropertyOnly
[renderOwnPropertyOnly]: /api/interfaces/RenderOptions.html#ownPropertyOnly
[strictVariables]: /api/interfaces/LiquidOptions.html#strictVariables
+1 -2
View File
@@ -42,8 +42,7 @@
if (!/\/playground(?:\.html)?$/.test(location.pathname)) return;
updateVersion(liquidjs.version);
const engine = new liquidjs.Liquid({
memoryLimit: 1e5,
renderLimit: 1e5
templateLimit: 1e5
});
const colorScheme = window.matchMedia('(prefers-color-scheme: dark)');
const editor = createEditor('editorEl', 'liquid');
+10 -8
View File
@@ -1,4 +1,3 @@
import { getPerformance } from '../util/performance'
import { Drop } from '../drop/drop'
import { __assign } from 'tslib'
import { NormalizedFullOptions, defaultOptions, RenderOptions } from '../liquid-options'
@@ -36,17 +35,19 @@ export class Context {
*/
public strictVariables: boolean;
public ownPropertyOnly: boolean;
public memoryLimit: Limiter;
public renderLimit: Limiter;
public constructor (env: object = {}, opts: NormalizedFullOptions = defaultOptions, renderOptions: RenderOptions = {}, { memoryLimit, renderLimit }: { [key: string]: Limiter } = {}) {
public templateLimit: Limiter;
public outputLengthLimit: Limiter;
public depthLimit: Limiter;
public constructor (env: object = {}, opts: NormalizedFullOptions = defaultOptions, renderOptions: RenderOptions = {}, { templateLimit, outputLengthLimit, depthLimit }: { templateLimit?: Limiter, outputLengthLimit?: Limiter, depthLimit?: Limiter } = {}) {
this.sync = !!renderOptions.sync
this.opts = opts
this.globals = renderOptions.globals ?? opts.globals
this.environments = isObject(env) ? env : Object(env)
this.strictVariables = renderOptions.strictVariables ?? this.opts.strictVariables
this.ownPropertyOnly = renderOptions.ownPropertyOnly ?? opts.ownPropertyOnly
this.memoryLimit = memoryLimit ?? new Limiter('memory alloc', renderOptions.memoryLimit ?? opts.memoryLimit)
this.renderLimit = renderLimit ?? new Limiter('template render', getPerformance().now() + (renderOptions.renderLimit ?? opts.renderLimit))
this.templateLimit = templateLimit ?? new Limiter('template', renderOptions.templateLimit ?? opts.templateLimit)
this.outputLengthLimit = outputLengthLimit ?? new Limiter('output length', renderOptions.outputLengthLimit ?? opts.outputLengthLimit)
this.depthLimit = depthLimit ?? new Limiter('template depth', opts.maxDepth)
}
public getRegister<T> (key: string, defaultValue: T = undefined as T): T {
return (this.registers[key] = this.registers[key] || defaultValue)
@@ -109,8 +110,9 @@ export class Context {
strictVariables: this.strictVariables,
ownPropertyOnly: this.ownPropertyOnly
}, {
renderLimit: this.renderLimit,
memoryLimit: this.memoryLimit
templateLimit: this.templateLimit,
outputLengthLimit: this.outputLengthLimit,
depthLimit: this.depthLimit
})
}
private findScope (key: string | number) {
+9 -2
View File
@@ -1,10 +1,17 @@
import { stringify } from '../util'
import { stringify, Limiter } from '../util'
import { Emitter } from './emitter'
export class SimpleEmitter implements Emitter {
public buffer = '';
private outputLengthLimit?: Limiter
constructor (outputLengthLimit?: Limiter) {
this.outputLengthLimit = outputLengthLimit
}
public write (html: any) {
this.buffer += stringify(html)
const str = stringify(html)
this.outputLengthLimit?.use(str.length)
this.buffer += str
}
}
+10 -2
View File
@@ -1,12 +1,20 @@
import { stringify } from '../util'
import { stringify, Limiter } from '../util'
import { Emitter } from './emitter'
import { PassThrough } from 'stream'
export class StreamedEmitter implements Emitter {
public buffer = '';
public stream: NodeJS.ReadWriteStream = new PassThrough()
private outputLengthLimit?: Limiter
constructor (outputLengthLimit?: Limiter) {
this.outputLengthLimit = outputLengthLimit
}
public write (html: any) {
this.stream.write(stringify(html))
const str = stringify(html)
this.outputLengthLimit?.use(str.length)
this.stream.write(str)
}
public error (err: Error) {
this.stream.emit('error', err)
-18
View File
@@ -8,9 +8,6 @@ import { EmptyDrop } from '../drop'
export const join = argumentsToValue(function (this: FilterImpl, v: any[], arg: string) {
const array = toArray(v)
const sep = isNil(arg) ? ' ' : stringify(arg)
let outputSize = sep.length * Math.max(array.length - 1, 0)
for (let i = 0; i < array.length; i++) outputSize += String(array[i]).length
this.context.memoryLimit.use(outputSize)
return Array.prototype.join.call(array, sep)
})
export const last = argumentsToValue(function (this: FilterImpl, v: any) {
@@ -21,14 +18,12 @@ export const first = argumentsToValue(function (this: FilterImpl, v: any) {
})
export const reverse = argumentsToValue(function (this: FilterImpl, v: any[]) {
const array = toArray(v)
this.context.memoryLimit.use(array.length)
return [...array].reverse()
})
function * sortBy<T> (this: FilterImpl, arr: T[], property: string | undefined, comparator: (a: unknown, b: unknown) => number): IterableIterator<unknown> {
const values: [T, unknown][] = []
const array = toArray(arr)
this.context.memoryLimit.use(array.length)
for (const item of array) {
values.push([
item,
@@ -51,7 +46,6 @@ export const size = (v: string | any[]) => v?.length || 0
export function * map (this: FilterImpl, arr: Scope[], property: string): IterableIterator<unknown> {
const results = []
const array = toArray(arr)
this.context.memoryLimit.use(array.length)
for (const item of array) {
results.push(yield this.context._getFromScope(item, stringify(property), false))
}
@@ -70,14 +64,12 @@ export function * sum (this: FilterImpl, arr: Scope[], property?: string): Itera
export function compact<T> (this: FilterImpl, arr: T[]) {
const array = toArray(arr)
this.context.memoryLimit.use(array.length)
return Array.prototype.filter.call(array, x => !isNil(toValue(x)))
}
export function concat<T1, T2> (this: FilterImpl, v: T1[], arg: T2[] = []): (T1 | T2)[] {
const lhs = toArray(v)
const rhs = toArray(arg)
this.context.memoryLimit.use(lhs.length + rhs.length)
return Array.prototype.concat.call(lhs, rhs)
}
@@ -87,7 +79,6 @@ export function push<T> (this: FilterImpl, v: T[], arg: T): T[] {
export function unshift<T> (this: FilterImpl, v: T[], arg: T): T[] {
const array = toArray(v)
this.context.memoryLimit.use(array.length)
const clone = [...array]
clone.unshift(arg)
return clone
@@ -95,7 +86,6 @@ export function unshift<T> (this: FilterImpl, v: T[], arg: T): T[] {
export function pop<T> (this: FilterImpl, v: T[]): T[] {
const array = toArray(v)
this.context.memoryLimit.use(array.length)
const clone = [...array]
clone.pop()
return clone
@@ -103,7 +93,6 @@ export function pop<T> (this: FilterImpl, v: T[]): T[] {
export function shift<T> (this: FilterImpl, v: T[]): T[] {
const array = toArray(v)
this.context.memoryLimit.use(array.length)
const clone = [...array]
clone.shift()
return clone
@@ -114,7 +103,6 @@ export function slice<T> (this: FilterImpl, v: T[] | string, begin: number, leng
if (isNil(v)) return []
if (!isArray(v)) v = stringify(v)
begin = begin < 0 ? v.length + begin : begin
this.context.memoryLimit.use(length)
return isArray(v)
? Array.prototype.slice.call(v, begin, begin + length)
: String.prototype.slice.call(v, begin, begin + length)
@@ -133,7 +121,6 @@ function expectedMatcher (this: FilterImpl, expected: any): (v: any) => boolean
function * filter<T extends object> (this: FilterImpl, include: boolean, arr: T[], property: string, expected: any): IterableIterator<unknown> {
const values: unknown[] = []
arr = toArray(arr)
this.context.memoryLimit.use(arr.length)
const token = new Tokenizer(stringify(property)).readScopeValue()
for (const item of arr) {
values.push(yield evalToken(token, this.context.spawn(item)))
@@ -146,7 +133,6 @@ function * filter_exp<T extends object> (this: FilterImpl, include: boolean, arr
const filtered: unknown[] = []
const keyTemplate = new Value(stringify(exp), this.liquid)
const array = toArray(arr)
this.context.memoryLimit.use(array.length)
for (const item of array) {
this.context.push({ [itemName]: item })
const value = yield keyTemplate.value(this.context)
@@ -176,7 +162,6 @@ export function * group_by<T extends object> (this: FilterImpl, arr: T[], proper
const map = new Map()
arr = toEnumerable(arr)
const token = new Tokenizer(stringify(property)).readScopeValue()
this.context.memoryLimit.use(arr.length)
for (const item of arr) {
const key = yield evalToken(token, this.context.spawn(item))
if (!map.has(key)) map.set(key, [])
@@ -189,7 +174,6 @@ export function * group_by_exp<T extends object> (this: FilterImpl, arr: T[], it
const map = new Map()
const keyTemplate = new Value(stringify(exp), this.liquid)
arr = toEnumerable(arr)
this.context.memoryLimit.use(arr.length)
for (const item of arr) {
this.context.push({ [itemName]: item })
const key = yield keyTemplate.value(this.context)
@@ -253,7 +237,6 @@ export function * find_exp<T extends object> (this: FilterImpl, arr: T[], itemNa
export function uniq<T> (this: FilterImpl, arr: T[]): T[] {
arr = toArray(arr)
this.context.memoryLimit.use(arr.length)
return [...new Set(arr)]
}
@@ -261,7 +244,6 @@ export function sample<T> (this: FilterImpl, v: T[] | string, count = 1): T | st
v = toValue(v)
if (isNil(v)) return []
if (!isArray(v)) v = stringify(v)
this.context.memoryLimit.use(v.length)
const shuffled = [...v].sort(() => Math.random() - 0.5)
if (count === 1) return shuffled[0]
return shuffled.slice(0, count)
-3
View File
@@ -10,16 +10,13 @@ import { base64Encode, base64Decode } from './base64-impl'
export function base64_encode (this: FilterImpl, value: string | Buffer): string {
if (typeof Buffer !== 'undefined' && Buffer.isBuffer(value)) {
this.context.memoryLimit.use(value.byteLength)
return value.toString('base64')
}
const str = stringify(value)
this.context.memoryLimit.use(str.length)
return base64Encode(str)
}
export function base64_decode (this: FilterImpl, value: string): string {
const str = stringify(value)
this.context.memoryLimit.use(str.length)
return base64Decode(str)
}
-2
View File
@@ -10,13 +10,11 @@ import { sha256 as sha256Impl, hmacSha256 as hmacSha256Impl } from './crypto-imp
export function sha256 (this: FilterImpl, value: unknown): string | Promise<string> {
const str = stringify(value)
this.context.memoryLimit.use(str.length)
return sha256Impl(str)
}
export function hmac_sha256 (this: FilterImpl, value: unknown, key: unknown): string | Promise<string> {
const str = stringify(value)
const keyStr = stringify(key)
this.context.memoryLimit.use(str.length + keyStr.length)
return hmacSha256Impl(str, keyStr)
}
+4 -8
View File
@@ -3,14 +3,11 @@ import { FilterImpl } from '../template'
import { NormalizedFullOptions } from '../liquid-options'
export function date (this: FilterImpl, v: string | Date, format?: string, timezoneOffset?: number | string) {
const size = ((v as string)?.length ?? 0) + ((timezoneOffset as string)?.length ?? 0)
this.context.memoryLimit.use(size)
const date = parseDate(v, this.context.opts, timezoneOffset)
if (!date) return v
format = toValue(format)
format = isNil(format) ? this.context.opts.dateFormat : stringify(format)
this.context.memoryLimit.use(format.length)
return strftime(date, format, this.context.memoryLimit)
return strftime(date, format)
}
export function date_to_xmlschema (this: FilterImpl, v: string | Date) {
@@ -32,14 +29,13 @@ export function date_to_long_string (this: FilterImpl, v: string | Date, type?:
function stringify_date (this: FilterImpl, v: string | Date, month_type: string, type?: string, style?: string) {
const date = parseDate(v, this.context.opts)
if (!date) return v
const ml = this.context.memoryLimit
if (type === 'ordinal') {
const d = date.getDate()
return style === 'US'
? strftime(date, `${month_type} ${d}%q, %Y`, ml)
: strftime(date, `${d}%q ${month_type} %Y`, ml)
? strftime(date, `${month_type} ${d}%q, %Y`)
: strftime(date, `${d}%q ${month_type} %Y`)
}
return strftime(date, `%d ${month_type} %Y`, ml)
return strftime(date, `%d ${month_type} %Y`)
}
function parseDate (v: string | Date, opts: NormalizedFullOptions, timezoneOffset?: number | string): LiquidDate | undefined {
-4
View File
@@ -18,7 +18,6 @@ const unescapeMap: Record<string, string> = {
export function escape (this: FilterImpl, str: string) {
str = stringify(str)
this.context.memoryLimit.use(str.length)
return str.replace(/&|<|>|"|'/g, m => escapeMap[m])
}
@@ -28,7 +27,6 @@ export function xml_escape (this: FilterImpl, str: string) {
function unescape (this: FilterImpl, str: string) {
str = stringify(str)
this.context.memoryLimit.use(str.length)
return str.replace(/&(amp|lt|gt|#34|#39);/g, m => unescapeMap[m])
}
@@ -38,7 +36,6 @@ export function escape_once (this: FilterImpl, str: string) {
export function newline_to_br (this: FilterImpl, v: string) {
const str = stringify(v)
this.context.memoryLimit.use(str.length)
return str.replace(/\r?\n/gm, '<br />\n')
}
@@ -46,7 +43,6 @@ export function newline_to_br (this: FilterImpl, v: string) {
// equivalent is O(n^2) in V8 on unclosed openers.
export function strip_html (this: FilterImpl, v: string) {
const str = stringify(v)
this.context.memoryLimit.use(str.length)
const blocks = new Map([['<script', '</script>'], ['<style', '</style>'], ['<!--', '-->'], ['<', '>']])
let out = ''
let i = 0
+1 -21
View File
@@ -2,18 +2,6 @@ import { isFalsy } from '../render/boolean'
import { identify, isArray, isString, toValue } from '../util/underscore'
import { FilterImpl } from '../template'
function chargeJsonReplacerValue (memoryLimit: { use(count: number): void }, val: unknown) {
if (typeof val === 'string') {
memoryLimit.use(val.length)
} else if (val === null || typeof val === 'number' || typeof val === 'boolean') {
memoryLimit.use(JSON.stringify(val).length)
} else if (Array.isArray(val)) {
memoryLimit.use(val.length + 1)
} else if (typeof val === 'object') {
memoryLimit.use(2)
}
}
function defaultFilter<T1 extends boolean, T2> (this: FilterImpl, value: T1, defaultValue: T2, ...args: Array<[string, any]>): T1 | T2 {
value = toValue(value)
if (isArray(value) || isString(value)) return value.length ? value : defaultValue
@@ -22,29 +10,21 @@ function defaultFilter<T1 extends boolean, T2> (this: FilterImpl, value: T1, def
}
function json (this: FilterImpl, value: any, space = 0) {
const memoryLimit = this.context.memoryLimit
return JSON.stringify(value, (_key, val) => {
chargeJsonReplacerValue(memoryLimit, val)
return val
}, space)
return JSON.stringify(value, undefined, space)
}
function inspect (this: FilterImpl, value: any, space = 0) {
const memoryLimit = this.context.memoryLimit
const ancestors: object[] = []
return JSON.stringify(value, function (this: unknown, _key: unknown, value: any) {
if (typeof value !== 'object' || value === null) {
chargeJsonReplacerValue(memoryLimit, value)
return value
}
// `this` is the object that value is contained in, i.e., its direct parent.
while (ancestors.length > 0 && ancestors[ancestors.length - 1] !== this) ancestors.pop()
if (ancestors.includes(value)) {
memoryLimit.use('[Circular]'.length)
return '[Circular]'
}
ancestors.push(value)
chargeJsonReplacerValue(memoryLimit, value)
return value
}, space)
}
-27
View File
@@ -22,7 +22,6 @@ export function append (this: FilterImpl, v: string, arg: string) {
assert(arguments.length === 2, 'append expect 2 arguments')
const lhs = stringify(v)
const rhs = stringify(arg)
this.context.memoryLimit.use(lhs.length + rhs.length)
return lhs + rhs
}
@@ -30,16 +29,13 @@ export function prepend (this: FilterImpl, v: string, arg: string) {
assert(arguments.length === 2, 'prepend expect 2 arguments')
const lhs = stringify(v)
const rhs = stringify(arg)
this.context.memoryLimit.use(lhs.length + rhs.length)
return rhs + lhs
}
export function lstrip (this: FilterImpl, v: string, chars?: string) {
const str = stringify(v)
this.context.memoryLimit.use(str.length)
if (chars) {
chars = stringify(chars)
this.context.memoryLimit.use(chars.length)
for (let i = 0, set = new Set(chars); i < str.length; i++) {
if (!set.has(str[i])) return str.slice(i)
}
@@ -50,34 +46,29 @@ export function lstrip (this: FilterImpl, v: string, chars?: string) {
export function downcase (this: FilterImpl, v: string) {
const str = stringify(v)
this.context.memoryLimit.use(str.length)
return str.toLowerCase()
}
export function upcase (this: FilterImpl, v: string) {
const str = stringify(v)
this.context.memoryLimit.use(str.length)
return stringify(str).toUpperCase()
}
export function remove (this: FilterImpl, v: string, arg: string) {
const str = stringify(v)
arg = stringify(arg)
this.context.memoryLimit.use(str.length + arg.length)
return str.split(arg).join('')
}
export function remove_first (this: FilterImpl, v: string, l: string) {
const str = stringify(v)
l = stringify(l)
this.context.memoryLimit.use(str.length + l.length)
return str.replace(l, '')
}
export function remove_last (this: FilterImpl, v: string, l: string) {
const str = stringify(v)
const pattern = stringify(l)
this.context.memoryLimit.use(str.length + pattern.length)
const index = str.lastIndexOf(pattern)
if (index === -1) return str
return str.substring(0, index) + str.substring(index + pattern.length)
@@ -85,10 +76,8 @@ export function remove_last (this: FilterImpl, v: string, l: string) {
export function rstrip (this: FilterImpl, str: string, chars?: string) {
str = stringify(str)
this.context.memoryLimit.use(str.length)
if (chars) {
chars = stringify(chars)
this.context.memoryLimit.use(chars.length)
for (let i = str.length - 1, set = new Set(chars); i >= 0; i--) {
if (!set.has(str[i])) return str.slice(0, i + 1)
}
@@ -99,7 +88,6 @@ export function rstrip (this: FilterImpl, str: string, chars?: string) {
export function split (this: FilterImpl, v: string, arg: string) {
const str = stringify(v)
this.context.memoryLimit.use(str.length)
const arr = str.split(stringify(arg))
// align to ruby split, which is the behavior of shopify/liquid
// see: https://ruby-doc.org/core-2.4.0/String.html#method-i-split
@@ -109,10 +97,8 @@ export function split (this: FilterImpl, v: string, arg: string) {
export function strip (this: FilterImpl, v: string, chars?: string) {
const str = stringify(v)
this.context.memoryLimit.use(str.length)
if (chars) {
const set = new Set(stringify(chars))
this.context.memoryLimit.use(set.size)
let i = 0
let j = str.length - 1
while (set.has(str[i])) i++
@@ -124,13 +110,11 @@ export function strip (this: FilterImpl, v: string, chars?: string) {
export function strip_newlines (this: FilterImpl, v: string) {
const str = stringify(v)
this.context.memoryLimit.use(str.length)
return str.replace(/\r?\n/gm, '')
}
export function capitalize (this: FilterImpl, str: string) {
str = stringify(str)
this.context.memoryLimit.use(str.length)
return str.charAt(0).toUpperCase() + str.slice(1).toLowerCase()
}
@@ -139,8 +123,6 @@ export function replace (this: FilterImpl, v: string, pattern: string, replaceme
pattern = stringify(pattern)
replacement = stringify(replacement)
const parts = str.split(pattern)
const outputSize = str.length + (parts.length - 1) * (replacement.length - pattern.length)
this.context.memoryLimit.use(outputSize)
return parts.join(replacement)
}
@@ -148,7 +130,6 @@ export function replace_first (this: FilterImpl, v: string, arg1: string, arg2:
const str = stringify(v)
arg1 = stringify(arg1)
arg2 = stringify(arg2)
this.context.memoryLimit.use(str.length + arg1.length + arg2.length)
return str.replace(arg1, () => arg2)
}
@@ -156,7 +137,6 @@ export function replace_last (this: FilterImpl, v: string, arg1: string, arg2: s
const str = stringify(v)
const pattern = stringify(arg1)
const replacement = stringify(arg2)
this.context.memoryLimit.use(str.length + pattern.length + replacement.length)
const index = str.lastIndexOf(pattern)
if (index === -1) return str
return str.substring(0, index) + replacement + str.substring(index + pattern.length)
@@ -165,7 +145,6 @@ export function replace_last (this: FilterImpl, v: string, arg1: string, arg2: s
export function truncate (this: FilterImpl, v: string, l = 50, o = '...') {
const str = stringify(v)
o = stringify(o)
this.context.memoryLimit.use(str.length + o.length)
if (str.length <= l) return v
return str.substring(0, l - o.length) + o
}
@@ -173,7 +152,6 @@ export function truncate (this: FilterImpl, v: string, l = 50, o = '...') {
export function truncatewords (this: FilterImpl, v: string, words = 15, o = '...') {
const str = stringify(v)
o = stringify(o)
this.context.memoryLimit.use(str.length + o.length)
const arr = str.split(/\s+/)
if (words <= 0) words = 1
let ret = arr.slice(0, words).join(' ')
@@ -183,13 +161,11 @@ export function truncatewords (this: FilterImpl, v: string, words = 15, o = '...
export function normalize_whitespace (this: FilterImpl, v: string) {
const str = stringify(v)
this.context.memoryLimit.use(str.length)
return str.replace(/\s+/g, ' ')
}
export function number_of_words (this: FilterImpl, input: string, mode?: 'cjk' | 'auto') {
const str = stringify(input)
this.context.memoryLimit.use(str.length)
input = str.trim()
if (!input) return 0
switch (mode) {
@@ -209,9 +185,6 @@ export function number_of_words (this: FilterImpl, input: string, mode?: 'cjk' |
export function array_to_sentence_string (this: FilterImpl, array: unknown[], connector = 'and') {
connector = stringify(connector)
let outputSize = connector.length + array.length * 2
for (let i = 0; i < array.length; i++) outputSize += stringify(array[i]).length
this.context.memoryLimit.use(outputSize)
switch (array.length) {
case 0:
return ''
+15 -13
View File
@@ -87,10 +87,12 @@ export interface LiquidOptions {
orderedFilterParameters?: boolean;
/** For DoS handling, limit total length of templates parsed in one `parse()` call. A typical PC can handle 1e8 (100M) characters without issues. */
parseLimit?: number;
/** For DoS handling, limit total time (in ms) for each `render()` call. */
renderLimit?: number;
/** For DoS handling, limit new objects creation, including array concat/join/strftime, etc. A typical PC can handle 1e9 (1G) memory without issue. */
memoryLimit?: number;
/** For DoS handling, limit total renders of tag/HTML/output in one `render()` call. */
templateLimit?: number;
/** For DoS handling, limit total output length in one `render()` call. */
outputLengthLimit?: number;
/** For DoS handling, limit nesting depth of `{% render %}`, `{% include %}`, and `{% layout %}` tags. Defaults to `128`. */
maxDepth?: number;
}
export interface RenderOptions {
@@ -110,12 +112,10 @@ export interface RenderOptions {
* Same as `ownPropertyOnly` on LiquidOptions, but only for current render() call
*/
ownPropertyOnly?: boolean;
/** For DoS handling, limit total renders of tag/HTML/output in one `render()` call. A typical PC can handle 1e5 renders of typical templates per second. */
/** For DoS handling, limit total renders of tag/HTML/output in one `render()` call. */
templateLimit?: number;
/** For DoS handling, limit total time (in ms) for each `render()` call. */
renderLimit?: number;
/** For DoS handling, limit new objects creation, including array concat/join/strftime, etc. A typical PC can handle 1e9 (1G) memory without issue.. */
memoryLimit?: number;
/** For DoS handling, limit total output length in one `render()` call. */
outputLengthLimit?: number;
}
export interface RenderFileOptions extends RenderOptions {
@@ -160,8 +160,9 @@ export interface NormalizedFullOptions extends NormalizedOptions {
globals: object;
operators: Operators;
parseLimit: number;
renderLimit: number;
memoryLimit: number;
templateLimit: number;
outputLengthLimit: number;
maxDepth: number;
}
export const defaultOptions: NormalizedFullOptions = {
@@ -194,9 +195,10 @@ export const defaultOptions: NormalizedFullOptions = {
lenientIf: false,
globals: {},
operators: defaultOperators,
memoryLimit: Infinity,
parseLimit: Infinity,
renderLimit: Infinity
templateLimit: Infinity,
outputLengthLimit: Infinity,
maxDepth: 128
}
export function normalize (options: LiquidOptions): NormalizedFullOptions {
-1
View File
@@ -67,7 +67,6 @@ export function evalQuotedToken (token: QuotedToken) {
function * evalRangeToken (token: RangeToken, ctx: Context) {
const low: number = yield evalToken(token.lhs, ctx)
const high: number = yield evalToken(token.rhs, ctx)
ctx.memoryLimit.use(high - low + 1)
return range(+low, +high + 1)
}
+3 -10
View File
@@ -1,4 +1,3 @@
import { getPerformance } from '../util/performance'
import { toPromise, RenderError, LiquidErrors, LiquidError } from '../util'
import { Context } from '../context'
import { Template } from '../template'
@@ -6,23 +5,17 @@ import { Emitter, StreamedEmitter, SimpleEmitter } from '../emitters'
export class Render {
public renderTemplatesToNodeStream (templates: Template[], ctx: Context): NodeJS.ReadableStream {
const emitter = new StreamedEmitter()
const emitter = new StreamedEmitter(ctx.outputLengthLimit)
Promise.resolve().then(() => toPromise(this.renderTemplates(templates, ctx, emitter)))
.then(() => emitter.end(), err => emitter.error(err))
return emitter.stream
}
public * renderTemplates (templates: Template[], ctx: Context, emitter?: Emitter): IterableIterator<any> {
if (!emitter) {
emitter = new SimpleEmitter()
}
ctx.renderLimit.check(getPerformance().now())
public * renderTemplates (templates: Template[], ctx: Context, emitter: Emitter = new SimpleEmitter(ctx.outputLengthLimit)): IterableIterator<any> {
const errors = []
for (const tpl of templates) {
ctx.renderLimit.check(getPerformance().now())
ctx.templateLimit.use(1)
try {
// if tpl.render supports emitter, it'll return empty `html`
const html = yield tpl.render(ctx, emitter)
// if not, it'll return an `html`, write to the emitter for it
html && emitter.write(html)
if (ctx.breakCalled || ctx.continueCalled) break
} catch (e) {
+9 -7
View File
@@ -43,13 +43,6 @@ export default class extends Tag {
}
* render (ctx: Context, emitter: Emitter): Generator<unknown, void | string, Template[]> {
const r = this.liquid.renderer
let collection = toEnumerable(yield evalToken(this.collection, ctx))
if (!collection.length) {
yield r.renderTemplates(this.elseTemplates, ctx, emitter)
return
}
const continueKey = 'continue-' + this.variable + '-' + this.collection.getText()
ctx.push(createScope({ continue: ctx.getRegister(continueKey, {}) }))
const hash = (yield this.hash.render(ctx)) as Record<string, any>
@@ -59,6 +52,7 @@ export default class extends Tag {
? Object.keys(hash).filter(x => MODIFIERS.includes(x))
: MODIFIERS.filter(x => hash[x] !== undefined)
let collection = toEnumerable(yield evalToken(this.collection, ctx))
collection = modifiers.reduce((collection, modifier: valueOf<typeof MODIFIERS>) => {
if (modifier === 'offset') return offset(collection, hash['offset'])
if (modifier === 'limit') return limit(collection, hash['limit'])
@@ -66,6 +60,14 @@ export default class extends Tag {
}, collection)
ctx.setRegister(continueKey, (hash['offset'] || 0) + collection.length)
if (!collection.length) {
yield r.renderTemplates(this.elseTemplates, ctx, emitter)
return
}
if (!this.templates.length) return
const scope = createScope({ forloop: new ForloopDrop(collection.length, this.collection.getText(), this.variable) })
ctx.push(scope)
for (const item of collection) {
+2
View File
@@ -28,6 +28,7 @@ export default class extends Tag {
this.hash = new Hash(tokenizer, liquid.options.jekyllInclude || liquid.options.keyValueSeparator)
}
* render (ctx: Context, emitter: Emitter): Generator<unknown, void, unknown> {
ctx.depthLimit.use(1)
const { liquid, hash, withVar } = this
const { renderer } = liquid
const filepath = (yield renderFilePath(this.file, ctx, liquid)) as string
@@ -43,6 +44,7 @@ export default class extends Tag {
yield renderer.renderTemplates(templates, ctx, emitter)
ctx.pop()
ctx.restoreRegister(saved)
ctx.depthLimit.release(1)
}
public * children (partials: boolean, sync: boolean): Generator<unknown, Template[]> {
+2
View File
@@ -26,6 +26,7 @@ export default class extends Tag {
yield renderer.renderTemplates(this.templates, ctx, emitter)
return
}
ctx.depthLimit.use(1)
const filepath = (yield renderFilePath(this.file, ctx, liquid)) as string
assert(filepath, () => `illegal file path "${filepath}"`)
const templates = (yield liquid._parseLayoutFile(filepath, ctx.sync, this.currentFile)) as Template[]
@@ -43,6 +44,7 @@ export default class extends Tag {
ctx.push(createScope((yield args.render(ctx)) as Scope))
yield renderer.renderTemplates(templates, ctx, emitter)
ctx.pop()
ctx.depthLimit.release(1)
}
public * children (partials: boolean): Generator<unknown, Template[]> {
+2
View File
@@ -55,6 +55,7 @@ export default class extends Tag {
this.hash = new Hash(tokenizer, liquid.options.keyValueSeparator)
}
* render (ctx: Context, emitter: Emitter): Generator<unknown, void, unknown> {
ctx.depthLimit.use(1)
const { liquid, hash } = this
const filepath = (yield renderFilePath(this.file, ctx, liquid)) as string
assert(filepath, () => `illegal file path "${filepath}"`)
@@ -81,6 +82,7 @@ export default class extends Tag {
const templates = (yield liquid._parsePartialFile(filepath, childCtx.sync, this.currentFile)) as Template[]
yield liquid.renderer.renderTemplates(templates, childCtx, emitter)
}
ctx.depthLimit.release(1)
}
public * children (partials: boolean, sync: boolean): Generator<unknown, Template[]> {
+4
View File
@@ -45,6 +45,10 @@ export default class extends Tag {
const limit = (args.limit === undefined) ? collection.length : args.limit
collection = collection.slice(offset, offset + limit)
if (!collection.length) return
if (!this.templates.length) return
const cols = args.cols || collection.length
const r = this.liquid.renderer
+5
View File
@@ -14,6 +14,11 @@ export class Limiter {
this.base += +count
}
}
release (count: number) {
if (+count > 0) {
this.base -= +count
}
}
check (count: number) {
if (+count > 0) {
assert(+count <= this.limit, this.message)
+8
View File
@@ -188,6 +188,14 @@ describe('util/strftime', function () {
it('should have higher priority than H', () => {
expect(t(then, '%0H')).toBe('03')
})
it('should allow pad width up to MAX_STRFTIME_PAD', () => {
expect(t(now, '%100000d').length).toBe(100000)
expect(t(now, `%${1_000_000}d`).length).toBe(1_000_000)
})
it('should throw when pad width exceeds MAX_STRFTIME_PAD', () => {
expect(() => t(now, `%${1024 * 1024 + 1}d`)).toThrow('strftime pad width limit exceeded')
expect(() => t(now, '%5000000d')).toThrow('strftime pad width limit exceeded')
})
})
describe('modifier field', () => {
it('should ignore E modifier', () => {
+14 -8
View File
@@ -1,13 +1,15 @@
import { changeCase, padStart, padEnd } from './underscore'
import { LiquidDate } from './liquid-date'
import type { Limiter } from './limiter'
import { assert } from './assert'
/** Per-conversion numeric width cap for strftime (%N, %15d, …). */
export const MAX_STRFTIME_PAD = 1024 * 1024
const rFormat = /%([-_0^#:]+)?(\d+)?([EO])?(.)/
interface FormatOptions {
flags: Record<string, boolean>;
width?: string;
modifier?: string;
memoryLimit?: Pick<Limiter, 'use'>;
}
// prototype extensions
@@ -98,8 +100,8 @@ const formatCodes: Record<string, FormatCodeHandler> = {
M: (d: LiquidDate) => d.getMinutes(),
N: (d: LiquidDate, opts: FormatOptions) => {
const width = Number(opts.width) || 9
assertPadWidth(width)
const str = String(d.getMilliseconds()).slice(0, width)
opts.memoryLimit?.use(width - str.length)
return padEnd(str, width, '0')
},
p: (d: LiquidDate) => (d.getHours() < 12 ? 'AM' : 'PM'),
@@ -123,25 +125,29 @@ const formatCodes: Record<string, FormatCodeHandler> = {
}
formatCodes.h = formatCodes.b
export function strftime (d: LiquidDate, formatStr: string, memoryLimit?: Pick<Limiter, 'use'>) {
export function strftime (d: LiquidDate, formatStr: string) {
let output = ''
let remaining = formatStr
let match
while ((match = rFormat.exec(remaining))) {
output += remaining.slice(0, match.index)
remaining = remaining.slice(match.index + match[0].length)
output += format(d, match, memoryLimit)
output += format(d, match)
}
return output + remaining
}
function format (d: LiquidDate, match: RegExpExecArray, memoryLimit?: Pick<Limiter, 'use'>) {
function assertPadWidth (width: number) {
assert(width <= MAX_STRFTIME_PAD, 'strftime pad width limit exceeded')
}
function format (d: LiquidDate, match: RegExpExecArray) {
const [input, flagStr = '', width, modifier, conversion] = match
const convert = formatCodes[conversion]
if (!convert) return input
const flags: Record<string, boolean> = {}
for (const flag of flagStr) flags[flag] = true
let ret = String(convert(d, { flags, width, modifier, memoryLimit }))
let ret = String(convert(d, { flags, width, modifier }))
let padChar = padSpaceChars.has(conversion) ? ' ' : '0'
let padWidth = Number(width) || padWidths[conversion] || 0
if (flags['^']) ret = ret.toUpperCase()
@@ -149,6 +155,6 @@ function format (d: LiquidDate, match: RegExpExecArray, memoryLimit?: Pick<Limit
if (flags['_']) padChar = ' '
else if (flags['0']) padChar = '0'
if (flags['-']) padWidth = 0
memoryLimit?.use(Number(padWidth) - ret.length)
else assertPadWidth(padWidth)
return padStart(ret, padWidth, padChar)
}
-7
View File
@@ -519,13 +519,6 @@ describe('Issues', function () {
const result = engine.parseAndRenderSync(`\n{{ "foo" | pos }}`)
expect(result).toEqual('\n[2,12] foo')
})
it("memoryLimit doesn't work in for tag #776", () => {
const engine = new Liquid({
memoryLimit: 1e5
})
const tpl = `{% for i in (1..1000000000) %} {{'a'}} {% endfor %}`
expect(() => engine.parseAndRenderSync(tpl)).toThrow('memory alloc limit exceeded, line:1, col:1')
})
it('group_by_exp fails with object as input #785', () => {
const site = {
tags: {
+13 -23
View File
@@ -204,31 +204,21 @@ describe('filters/date', function () {
return test('{{ "1990-12-31T23:00:00Z" | date: "%Y-%m-%dT%H:%M:%S" }}', '1991-01-01T04:30:00', undefined, optsWithDateFormat)
})
})
describe('strftime width / memoryLimit', () => {
it('should charge memoryLimit for huge numeric strftime widths', () => {
const liquid = new Liquid({ memoryLimit: 500 })
expect(() => liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: '%5000000d' }))
.toThrow('memory alloc limit exceeded')
})
it('should charge memoryLimit for array format PoC', () => {
const liquid = new Liquid({ memoryLimit: 50, renderLimit: 1e9 })
expect(() => liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: ['a'.repeat(2000000)] }))
.toThrow('memory alloc limit exceeded')
})
it('should charge memoryLimit for object toString format PoC', () => {
const liquid = new Liquid({ memoryLimit: 50, renderLimit: 1e9 })
const huge = 'a'.repeat(2000000)
const f = { toString: () => huge }
expect(() => liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f }))
.toThrow('memory alloc limit exceeded')
})
it('should honor numeric strftime pad width when memoryLimit allows', () => {
const liquid = new Liquid({ memoryLimit: 1e7 })
describe('strftime width', () => {
it('should honor numeric strftime pad width', () => {
const liquid = new Liquid()
const out = liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: '%5000d' })
expect(out.length).toBe(5000)
const tight = new Liquid({ memoryLimit: 100 })
expect(() => tight.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: '%5000d' }))
.toThrow('memory alloc limit exceeded')
})
it('should honor large numeric strftime pad width up to the cap', () => {
const liquid = new Liquid()
const out = liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: '%100000d' })
expect(out.length).toBe(100000)
})
it('should throw when numeric strftime pad width is too large', () => {
const liquid = new Liquid()
expect(() => liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: '%5000000d' }))
.toThrow('strftime pad width limit exceeded')
})
})
})
+126 -89
View File
@@ -4,15 +4,18 @@ import { mock, restore } from '../../stub/mockfs'
describe('DoS related', function () {
describe('#parseLimit', function () {
afterEach(restore)
it('should throw when parse limit exceeded', async () => {
const noLimit = new Liquid()
const limit10 = new Liquid({ parseLimit: 10 })
const limit90 = new Liquid({ parseLimit: 90 })
const template = '{% capture bar %}{{ foo | bar: 3, a[3] }}{% endcapture %}'
await expect(noLimit.parseAndRender(template)).resolves.toBe('')
await expect(limit10.parseAndRender(template)).rejects.toThrow('parse length limit exceeded')
await expect(limit90.parseAndRender(template)).resolves.toBe('')
})
it('should take included template into account', async () => {
mock({
'/small': 'Lorem ipsum',
@@ -23,119 +26,151 @@ describe('DoS related', function () {
await expect(liquid.parseAndRender('{% include "large" %}')).rejects.toThrow('parse length limit exceeded')
})
})
describe('#renderLimit', () => {
describe('#templateLimit', () => {
it('should throw when rendering too many templates', async () => {
const src = '{% for i in (1..1000) %}{{i}},{% endfor %}'
const noLimit = new Liquid()
const limitSmall = new Liquid({ renderLimit: 0.01 })
const limitLarge = new Liquid({ renderLimit: 2e4 })
const limitSmall = new Liquid({ templateLimit: 100 })
const limitLarge = new Liquid({ templateLimit: 2001 })
await expect(noLimit.parseAndRender(src)).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
await expect(limitSmall.parseAndRender(src)).rejects.toThrow('template render limit exceeded')
await expect(limitSmall.parseAndRender(src)).rejects.toThrow('template limit exceeded')
await expect(limitLarge.parseAndRender(src)).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
})
it('should support reset when calling render', async () => {
const src = '{% for i in (1..1000) %}{{i}},{% endfor %}'
const liquid = new Liquid({ renderLimit: 0.01 })
await expect(liquid.parseAndRender(src)).rejects.toThrow('template render limit exceeded')
await expect(liquid.parseAndRender(src, {}, { renderLimit: 1e6 })).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
const liquid = new Liquid({ templateLimit: 100 })
await expect(liquid.parseAndRender(src)).rejects.toThrow('template limit exceeded')
await expect(liquid.parseAndRender(src, {}, { templateLimit: 2001 })).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
})
it('should take partials into account', async () => {
mock({
'/small': '{% for i in (1..5) %}{{i}}{% endfor %}',
'/large': '{% for i in (1..50000000) %}{{i}}{% endfor %}'
})
const liquid = new Liquid({ root: '/', renderLimit: 1000 })
await expect(liquid.parseAndRender('{% render "large" %}')).rejects.toThrow('template render limit exceeded')
const liquid = new Liquid({ root: '/', templateLimit: 1000 })
await expect(liquid.parseAndRender('{% render "large" %}')).rejects.toThrow('template limit exceeded')
await expect(liquid.parseAndRender('{% render "small" %}')).resolves.toBe('12345')
})
it('should enforce renderLimit when for body has no template nodes', () => {
const liquid = new Liquid({ memoryLimit: 1e9, renderLimit: 1 })
expect(() => liquid.parseAndRenderSync('{%- for i in (1..5000000) -%}{%- endfor -%}', {}))
.toThrow('template render limit exceeded')
})
it('should enforce renderLimit when tablerow body has no template nodes', () => {
const liquid = new Liquid({ memoryLimit: 1e9, renderLimit: 1 })
expect(() => liquid.parseAndRenderSync('{%- tablerow i in (1..1000000) cols:1 -%}{%- endtablerow -%}', {}))
.toThrow('template render limit exceeded')
})
})
describe('#memoryLimit', () => {
it('should throw for too many array creation in filters', async () => {
const array = Array(1e3).fill(0)
const liquid = new Liquid({ memoryLimit: 100 })
await expect(liquid.parseAndRender('{{ array | slice: 0, 3 | join }}', { array })).resolves.toBe('0 0 0')
await expect(liquid.parseAndRender('{{ array | slice: 0, 300 | join }}', { array })).rejects.toThrow('memory alloc limit exceeded, line:1, col:1')
describe('#outputLengthLimit', () => {
it('should throw when output length exceeded', async () => {
const src = '{% for i in (1..1000) %}{{i}},{% endfor %}'
const noLimit = new Liquid()
const limitSmall = new Liquid({ outputLengthLimit: 10 })
const limitLarge = new Liquid({ outputLengthLimit: 5000 })
await expect(noLimit.parseAndRender(src)).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
await expect(limitSmall.parseAndRender(src)).rejects.toThrow('output length limit exceeded')
await expect(limitLarge.parseAndRender(src)).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
})
it('should support reset when calling render', async () => {
const array = Array(1e3).fill(0)
const liquid = new Liquid({ memoryLimit: 100 })
await expect(liquid.parseAndRender('{{ array | slice: 0, 300 | join }}', { array })).rejects.toThrow('memory alloc limit exceeded, line:1, col:1')
await expect(liquid.parseAndRender('{{ array | slice: 0, 300 | join }}', { array }, { memoryLimit: 1e3 })).resolves.toBe(Array(300).fill(0).join(' '))
const src = '{% for i in (1..1000) %}{{i}},{% endfor %}'
const liquid = new Liquid({ outputLengthLimit: 10 })
await expect(liquid.parseAndRender(src)).rejects.toThrow('output length limit exceeded')
await expect(liquid.parseAndRender(src, {}, { outputLengthLimit: 5000 })).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
})
it('should throw for too many array iteration in tags', async () => {
const array = ['a']
const liquid = new Liquid({ memoryLimit: 100 })
const src = '{% for i in (1..count) %}{% assign array = array | concat: array %}{% endfor %}{{ array | join }}'
await expect(liquid.parseAndRender(src, { array, count: 3 })).resolves.toBe('a a a a a a a a')
await expect(liquid.parseAndRender(src, { array, count: 100 })).rejects.toThrow('memory alloc limit exceeded, line:1, col:26')
it('should take partials into account', async () => {
mock({
'/small': 'abc',
'/large': '{% for i in (1..1000) %}{{i}}{% endfor %}'
})
const liquid = new Liquid({ root: '/', outputLengthLimit: 10 })
await expect(liquid.parseAndRender('{% render "small" %}')).resolves.toBe('abc')
await expect(liquid.parseAndRender('{% render "large" %}')).rejects.toThrow('output length limit exceeded')
})
it('should charge pop allocation to memoryLimit', async () => {
const array = Array(1e3).fill(0)
const liquid = new Liquid({ memoryLimit: 100 })
await expect(liquid.parseAndRender('{{ array | pop | size }}', { array })).rejects.toThrow('memory alloc limit exceeded')
it('should enforce outputLengthLimit in sync render', () => {
const liquid = new Liquid({ outputLengthLimit: 5 })
expect(() => liquid.parseAndRenderSync('{% for i in (1..100) %}{{i}}{% endfor %}'))
.toThrow('output length limit exceeded')
})
it('should charge sample allocation to memoryLimit', async () => {
const array = Array(1e3).fill(0)
const liquid = new Liquid({ memoryLimit: 100 })
await expect(liquid.parseAndRender('{{ array | sample: 1 | size }}', { array })).rejects.toThrow('memory alloc limit exceeded')
})
it('should charge join by produced output size, not element count', () => {
const array = ['a'.repeat(100), 'b'.repeat(100)]
const liquid = new Liquid({ memoryLimit: 100 })
expect(() => liquid.parseAndRenderSync('{{ array | join: "" }}', { array }))
.toThrow('memory alloc limit exceeded')
})
it('should allow join within memoryLimit', () => {
const array = ['a'.repeat(20), 'b'.repeat(20)]
const liquid = new Liquid({ memoryLimit: 100 })
expect(liquid.parseAndRenderSync('{{ array | join: "" }}', { array })).toBe('a'.repeat(20) + 'b'.repeat(20))
})
it('should prevent concat doubling from bypassing join memoryLimit', () => {
const liquid = new Liquid({ memoryLimit: 1e4 })
const src = '{%- assign a = s | split: "NOSEP" -%}' +
'{%- assign a = a | concat: a -%}{%- assign a = a | concat: a -%}{%- assign a = a | concat: a -%}' +
'{{ a | join: "" | size }}'
expect(() => liquid.parseAndRenderSync(src, { s: 'a'.repeat(5000) }))
.toThrow('memory alloc limit exceeded')
})
it('should charge array_to_sentence_string by produced output size', () => {
const array = ['a'.repeat(100), 'b'.repeat(100), 'c'.repeat(100)]
const liquid = new Liquid({ memoryLimit: 100 })
expect(() => liquid.parseAndRenderSync('{{ array | array_to_sentence_string }}', { array }))
.toThrow('memory alloc limit exceeded')
})
it('should charge json serialization of concat-doubled arrays', () => {
const liquid = new Liquid({ memoryLimit: 1e4 })
const src = '{%- assign a = s | split: "NOSEP" -%}' +
'{%- assign a = a | concat: a -%}{%- assign a = a | concat: a -%}{%- assign a = a | concat: a -%}' +
'{{ a | json | size }}'
expect(() => liquid.parseAndRenderSync(src, { s: 'a'.repeat(5000) }))
.toThrow('memory alloc limit exceeded')
})
it('should charge inspect serialization of concat-doubled arrays', () => {
const liquid = new Liquid({ memoryLimit: 1e4 })
const src = '{%- assign a = s | split: "NOSEP" -%}' +
'{%- assign a = a | concat: a -%}{%- assign a = a | concat: a -%}{%- assign a = a | concat: a -%}' +
'{{ a | inspect | size }}'
expect(() => liquid.parseAndRenderSync(src, { s: 'a'.repeat(5000) }))
.toThrow('memory alloc limit exceeded')
})
it('should charge strip_html input length to memoryLimit', () => {
const liquid = new Liquid({ memoryLimit: 100 })
expect(() => liquid.parseAndRenderSync('{{ s | strip_html }}', { s: 'a'.repeat(200) }))
.toThrow('memory alloc limit exceeded')
it('should enforce outputLengthLimit in stream render', async () => {
const liquid = new Liquid({ outputLengthLimit: 5 })
const tpl = liquid.parse('{% for i in (1..100) %}{{i}}{% endfor %}')
const stream = liquid.renderToNodeStream(tpl)
await expect(new Promise((resolve, reject) => {
stream.on('error', reject)
stream.on('end', resolve)
})).rejects.toThrow('output length limit exceeded')
})
})
describe('#maxDepth', () => {
function chain (depth: number, tag: string) {
const templates: Record<string, string> = {}
for (let i = 0; i < depth; i++) {
templates[`t${i}`] = i === depth - 1 ? 'done' : `{% ${tag} "t${i + 1}" %}`
}
return templates
}
it('should throw when include depth exceeded', async () => {
const liquid = new Liquid({ templates: chain(3, 'include'), maxDepth: 2 })
await expect(liquid.parseAndRender('{% include "t0" %}')).rejects.toThrow('template depth limit exceeded')
})
it('should allow include within maxDepth', async () => {
const liquid = new Liquid({ templates: chain(2, 'include'), maxDepth: 2 })
await expect(liquid.parseAndRender('{% include "t0" %}')).resolves.toBe('done')
})
it('should throw when render depth exceeded', async () => {
const liquid = new Liquid({ templates: chain(3, 'render'), maxDepth: 2 })
await expect(liquid.parseAndRender('{% render "t0" %}')).rejects.toThrow('template depth limit exceeded')
})
it('should allow render within maxDepth', async () => {
const liquid = new Liquid({ templates: chain(2, 'render'), maxDepth: 2 })
await expect(liquid.parseAndRender('{% render "t0" %}')).resolves.toBe('done')
})
it('should throw when layout depth exceeded', async () => {
const liquid = new Liquid({
templates: {
a: '{% layout "b" %}body-a',
b: '{% layout "c" %}body-b',
c: 'body-c'
},
maxDepth: 2
})
await expect(liquid.parseAndRender('{% layout "a" %}root')).rejects.toThrow('template depth limit exceeded')
})
it('should allow layout within maxDepth', async () => {
const liquid = new Liquid({
templates: {
a: '{% layout "b" %}body-a',
b: 'body-b'
},
maxDepth: 2
})
await expect(liquid.parseAndRender('{% layout "a" %}root')).resolves.toBe('body-b')
})
it('should not count layout none toward depth', async () => {
const liquid = new Liquid({ maxDepth: 0 })
await expect(liquid.parseAndRender('{% layout none %}ok')).resolves.toBe('ok')
})
it('should default maxDepth to 128', async () => {
const liquid = new Liquid({ templates: chain(128, 'include') })
await expect(liquid.parseAndRender('{% include "t0" %}')).resolves.toBe('done')
const overflow = new Liquid({ templates: chain(129, 'include') })
await expect(overflow.parseAndRender('{% include "t0" %}')).rejects.toThrow('template depth limit exceeded')
})
it('should enforce maxDepth in sync render', () => {
const liquid = new Liquid({ templates: chain(3, 'include'), maxDepth: 2 })
expect(() => liquid.parseAndRenderSync('{% include "t0" %}')).toThrow('template depth limit exceeded')
})
})
describe('strip_html ReDoS', () => {
// Regression for O(n^2) backtracking on unclosed `<script` / `<style` openers.
// The previous regex stalled the event loop for ~10s on 350KB of `'<script'.repeat`.
@@ -145,11 +180,13 @@ describe('DoS related', function () {
const payload = '<script'.repeat(50000)
expect(liquid.parseAndRenderSync('{{ x | strip_html }}', { x: payload })).toBe(payload)
}, 1000)
it('should handle many unclosed <style openers in linear time', () => {
const liquid = new Liquid()
const payload = '<style'.repeat(50000)
expect(liquid.parseAndRenderSync('{{ x | strip_html }}', { x: payload })).toBe(payload)
}, 1000)
it('should handle <script openers that have > but no </script> in linear time', () => {
const liquid = new Liquid()
const payload = '<script>foo'.repeat(50000)
+12
View File
@@ -120,6 +120,18 @@ describe('tags/for', function () {
const html = await liquid.parseAndRender(src, scope)
return expect(html).toBe('b')
})
it('should goto else when limit empties collection', async function () {
const src = '{%for c in alpha limit:0%}a{%else%}b{%endfor%}'
const html = await liquid.parseAndRender(src, scope)
return expect(html).toBe('b')
})
it('should goto else when offset past end', async function () {
const src = '{%for c in alpha offset:10%}a{%else%}b{%endfor%}'
const html = await liquid.parseAndRender(src, scope)
return expect(html).toBe('b')
})
})
it('should support for with forloop', async function () {