mirror of
https://github.com/harttle/liquidjs.git
synced 2026-09-17 21:40:39 -07:00
feat: remove memoryLimit; add templateLimit, outputLengthLimit, maxDepth (#937)
* feat: remove memoryLimit option (#910) Co-authored-by: Cursor <[email protected]> * feat: add templateLimit, outputLengthLimit, and maxDepth DoS limits Enforce v11 resource guards in render and tags, fix for offset/else behavior, and update tutorials for Tag-class registration. Co-authored-by: Cursor <[email protected]> * docs: revert unnecessary tutorial churn from memoryLimit PR Restore the two-example register-filters-tags structure (Value + Hash) and undo unrelated constructor/emitter doc edits not required for DoS limits. Co-authored-by: Cursor <[email protected]> * docs: trim security-model prose and update render-tag-content Remove diary-style engine comparisons from security-model.md. Update render-tag-content tutorial to Tag class examples with tpls class field. Co-authored-by: Cursor <[email protected]> * docs: note maxDepth stack overflow applies to renderSync only Explain why async render does not need maxDepth for stack protection based on generator/toPromise driving. Co-authored-by: Cursor <[email protected]> * refactor: track maxDepth via depthLimit Limiter on Context Replace increaseDepth/decreaseDepth with a shared Limiter that supports paired use/release, matching templateLimit and outputLengthLimit patterns. Co-authored-by: Cursor <[email protected]> * fix: remove spurious diff noise in filter files Restore misc.ts from origin/next with LF line endings and re-apply only memoryLimit removal, avoiding CRLF and blank-line churn in the export block. Co-authored-by: Cursor <[email protected]> * refactor: minimize PR diff noise Co-authored-by: Cursor <[email protected]> * feat: cap strftime pad width at 1M docs: restructure security model with production guidance Co-authored-by: Cursor <[email protected]> * refactor: simplify depthLimit in partial tags and tighten security docs Drop try/finally around depthLimit in include, layout, and render; release at generator end. Consolidate production guidance in security-model.md. Fix padded-blocks lint in dos.spec.ts. Co-authored-by: Cursor <[email protected]> --------- Co-authored-by: Cursor <[email protected]>
This commit is contained in:
@@ -140,6 +140,8 @@ It defaults to `false`. For example, when set to `true`, a blank string would ev
|
||||
|
||||
**ownPropertyOnly** hides scope variables from prototypes, useful when you're passing a not sanitized object into LiquidJS or need to hide prototypes from templates. Defaults to `true`.
|
||||
|
||||
Built-in DoS limits and host isolation guidance are documented in [Security Model](./security-model.html).
|
||||
|
||||
{% note info Nonexistent Tags %}
|
||||
Nonexistent tags always throw errors during parsing and this behavior cannot be customized.
|
||||
{% endnote %}
|
||||
|
||||
@@ -22,7 +22,7 @@ Expected output:
|
||||
</div>
|
||||
```
|
||||
|
||||
Firstly, [register][register-tags] a tag named `wrap` and parse the content into `this.tpls`. Here in `parse(tagToken, remainTokens)`:
|
||||
Firstly, [register][register-tags] a tag named `wrap` and parse the content into `this.tpls`. In the tag `constructor(tagToken, remainTokens, liquid)`:
|
||||
|
||||
- `tagToken` is current token `{%raw%}{% wrap %}{%endraw%}`, and
|
||||
- `remainTokens` is an array of all tokens following `{%raw%}{% wrap %}{%endraw%}` until the end of this template file.
|
||||
@@ -30,11 +30,14 @@ Firstly, [register][register-tags] a tag named `wrap` and parse the content into
|
||||
Basically, what we need to do is take/`.shift()` enough tags from `remainTokens` until we get an `endwrap` token (the name can be arbitrary, but by convention it should be `endwrap`). And if there's no `endwrap` until the end of the template file, we need to throw a tag-not-closed `Error`.
|
||||
|
||||
```javascript
|
||||
engine.registerTag('wrap', {
|
||||
parse(tagToken, remainTokens) {
|
||||
this.tpls = []
|
||||
const { Tag } = require('liquidjs')
|
||||
|
||||
engine.registerTag('wrap', class WrapTag extends Tag {
|
||||
tpls = []
|
||||
constructor(tagToken, remainTokens, liquid) {
|
||||
super(tagToken, remainTokens, liquid)
|
||||
let closed = false
|
||||
while(remainTokens.length) {
|
||||
while (remainTokens.length) {
|
||||
let token = remainTokens.shift()
|
||||
// we got the end tag! stop taking tokens
|
||||
if (token.name === 'endwrap') {
|
||||
@@ -44,11 +47,11 @@ engine.registerTag('wrap', {
|
||||
// parse token into template
|
||||
// parseToken() may consume more than 1 tokens
|
||||
// e.g. {% if %}...{% endif %}
|
||||
let tpl = this.liquid.parser.parseToken(token, remainTokens)
|
||||
let tpl = liquid.parser.parseToken(token, remainTokens)
|
||||
this.tpls.push(tpl)
|
||||
}
|
||||
if (!closed) throw new Error(`tag ${tagToken.getText()} not closed`)
|
||||
},
|
||||
}
|
||||
* render(context, emitter) {
|
||||
emitter.write("<div class='wrapper'>")
|
||||
yield this.liquid.renderer.renderTemplates(this.tpls, context, emitter)
|
||||
@@ -57,16 +60,17 @@ engine.registerTag('wrap', {
|
||||
})
|
||||
```
|
||||
|
||||
`.renderTemplates()` can be async; we need `yield` to wait for it to complete. For more details on async in LiquidJS, see [Sync and Async][async]. Other parts of the `render()` method are quite straightforward. Here's a JSFiddle version: <https://jsfiddle.net/por0zcn1/3/>
|
||||
`.renderTemplates()` can be async; we need `yield` to wait for it to complete. For more details on async in LiquidJS, see [Sync and Async][async]. Here's a JSFiddle version: <https://jsfiddle.net/por0zcn1/3/>
|
||||
|
||||
## Using ParseStream
|
||||
|
||||
When it comes to complex tags like [for][for] and [if][if], the `parse()` can be very complicated. There's a [ParseStream][ParseStream] utility to organize the `parse()` in event-based style. Following is a re-written `parse()` using `ParseStream` that does exactly the same as the example above.
|
||||
For more complex tags such as [for][for] and [if][if], constructor parsing can get unwieldy. [ParseStream][ParseStream] offers an event-based API for this. The constructor below is equivalent to the example above:
|
||||
|
||||
```javascript
|
||||
parse(tagToken, remainTokens) {
|
||||
this.tpls = []
|
||||
this.liquid.parser.parseStream(remainTokens)
|
||||
tpls = []
|
||||
constructor(tagToken, remainTokens, liquid) {
|
||||
super(tagToken, remainTokens, liquid)
|
||||
liquid.parser.parseStream(remainTokens)
|
||||
.on('template', tpl => this.tpls.push(tpl))
|
||||
// note that we cannot use arrow function because we need `this`
|
||||
.on('tag:endwrap', function () { this.stop() })
|
||||
@@ -103,15 +107,18 @@ As you've noticed, there's an additional `repeat.i` in the context of `repeat`.
|
||||
Each time we enter a new *Context*, we need to push a new *Scope*. And when we finish rendering and exit the *Context*, we pop the *Scope* from the *Context*. As you can see in the following implementation:
|
||||
|
||||
```javascript
|
||||
engine.registerTag('repeat', {
|
||||
parse(tagToken, remainTokens) {
|
||||
this.tpls = []
|
||||
this.liquid.parser.parseStream(remainTokens)
|
||||
const { Tag } = require('liquidjs')
|
||||
|
||||
engine.registerTag('repeat', class RepeatTag extends Tag {
|
||||
tpls = []
|
||||
constructor(tagToken, remainTokens, liquid) {
|
||||
super(tagToken, remainTokens, liquid)
|
||||
liquid.parser.parseStream(remainTokens)
|
||||
.on('template', tpl => this.tpls.push(tpl))
|
||||
.on('tag:endrepeat', function () { this.stop() })
|
||||
.on('end', () => { throw new Error(`tag ${tagToken.getText()} not closed`) })
|
||||
.start()
|
||||
},
|
||||
}
|
||||
* render(context, emitter) {
|
||||
const repeat = { i: 1 }
|
||||
context.push({ repeat })
|
||||
@@ -123,7 +130,7 @@ engine.registerTag('repeat', {
|
||||
})
|
||||
```
|
||||
|
||||
The `parse()` is exactly the same as `wrap` tag, we repeat the content simply by calling `.renderTemplates(this.tpls)` twice during `render()`. Here's the JSFiddle: <https://jsfiddle.net/por0zcn1/2/>
|
||||
The constructor is the same as in the `wrap` tag; we repeat the content by calling `.renderTemplates(this.tpls)` twice during `render()`. Here's the JSFiddle: <https://jsfiddle.net/por0zcn1/2/>
|
||||
|
||||
{% note warn Use Push & Pop in Pairs %}
|
||||
`context.push()` and `context.pop()` have to be used in pairs. Failing to `pop()` the *Scope* you pushed will leak the *Scope* to latter templates and may corrupt the *Context* stack.
|
||||
|
||||
@@ -2,21 +2,19 @@
|
||||
title: Security Model
|
||||
---
|
||||
|
||||
LiquidJS provides DoS-oriented limits (`parseLimit`, `renderLimit`, `memoryLimit`) to reduce risk. This page summarizes those limits, [`ownPropertyOnly`][ownPropertyOnly], custom [`Drop`][drop] usage, and the security boundary to assume in production.
|
||||
LiquidJS provides DoS-oriented limits (`parseLimit`, `templateLimit`, `outputLengthLimit`, `maxDepth`) to reduce risk. This page summarizes those limits, [`ownPropertyOnly`][ownPropertyOnly], custom [`Drop`][drop] usage, and the security boundary to assume in production.
|
||||
|
||||
## Security boundary
|
||||
## At a glance
|
||||
|
||||
The built-in limits are cooperative safeguards, not strict runtime isolation.
|
||||
|
||||
- They do **not** equal process RSS/heap usage.
|
||||
- They do **not** sandbox JavaScript execution.
|
||||
- They should be combined with process/container limits and request timeouts for defense in depth.
|
||||
|
||||
## Limits at a glance
|
||||
LiquidJS ships a thin cooperative DoS layer:
|
||||
|
||||
- [parseLimit][parseLimit]: limit total template size per `parse()` call.
|
||||
- [renderLimit][renderLimit]: limit total render time per `render()` call.
|
||||
- [memoryLimit][memoryLimit]: cooperatively limit memory-sensitive allocations counted by LiquidJS.
|
||||
- [templateLimit][templateLimit]: limit total tag/HTML/output nodes rendered per `render()` call.
|
||||
- [outputLengthLimit][outputLengthLimit]: limit total output length per `render()` call.
|
||||
- [maxDepth][maxDepth]: limit nesting depth of `{% render %}`, `{% include %}`, and `{% layout %}`.
|
||||
- Strftime numeric pad widths in the `date` filter are capped at `1_000_000` (1M) per conversion.
|
||||
|
||||
These are cooperative safeguards, not runtime isolation—see [Production guidance](#production-guidance) below for host-level limits and online-service hardening.
|
||||
|
||||
## Limit details
|
||||
|
||||
@@ -26,9 +24,9 @@ The built-in limits are cooperative safeguards, not strict runtime isolation.
|
||||
|
||||
A typical PC handles `1e8` (100M) characters without issues.
|
||||
|
||||
### renderLimit
|
||||
### templateLimit
|
||||
|
||||
Restricting template size alone is insufficient because dynamic loops with large counts can occur during rendering. [renderLimit][renderLimit] mitigates this by limiting the time consumed by each `render()` call.
|
||||
Restricting template size alone is insufficient because dynamic loops with large counts can occur during rendering. [templateLimit][templateLimit] mitigates this by limiting the number of tag, HTML literal, and output nodes rendered in each `render()` call.
|
||||
|
||||
```liquid
|
||||
{%- for i in (1..10000000) -%}
|
||||
@@ -36,29 +34,19 @@ Restricting template size alone is insufficient because dynamic loops with large
|
||||
{%- endfor -%}
|
||||
```
|
||||
|
||||
Render time is checked on a per-template basis (before rendering each template). In the above example, there are 2 templates in the loop: `order: ` and `{{i}}`, render time will be checked 10000000x2 times.
|
||||
Each template node (the `for` tag, literal `order: `, output `{{i}}`, and so on) counts toward the limit. In the above example, a limit of `30000000` would be exceeded before the loop finishes.
|
||||
|
||||
`renderLimit` is not a hard CPU limiter. It is checked between template renders, so compute-intensive filters/tags/user-defined functions or deeply nested template execution between checks can still cause DoS.
|
||||
`templateLimit` is checked before each node render, so compute-intensive filters/tags/user-defined functions between checks can still cause DoS.
|
||||
|
||||
### memoryLimit
|
||||
### outputLengthLimit
|
||||
|
||||
`memoryLimit` only limits operations that LiquidJS explicitly counts.
|
||||
[outputLengthLimit][outputLengthLimit] caps the cumulative length of output written during a `render()` call, including output from partials rendered via `{% render %}`.
|
||||
|
||||
- Counted: memory-sensitive LiquidJS operations that call internal memory accounting.
|
||||
- Not guaranteed counted: arbitrary user object behavior such as custom `toValue()`/`toString()` chains, or other host-side code that allocates outside LiquidJS accounting points.
|
||||
### maxDepth
|
||||
|
||||
In other words, `memoryLimit` limits what LiquidJS counts, not every byte your process may allocate.
|
||||
[maxDepth][maxDepth] limits how deeply `{% render %}`, `{% include %}`, and `{% layout %}` can nest. Defaults to `128`. In sync rendering (`renderSync`), nested tags are driven by `toValueSync`, which recursively resumes each yielded generator on the call stack—deep nesting can overflow it, and `maxDepth` caps that depth. Async `render()` resumes the same tag generators via `toPromise`/`yield` without a deep synchronous call chain, so stack overflow is not a concern there (the limit still applies as a DoS guard).
|
||||
|
||||
Even with a small number of templates and iterations, memory usage can grow exponentially. In the following example, memory doubles with each iteration:
|
||||
|
||||
```liquid
|
||||
{% assign array = "1,2,3" | split: "," %}
|
||||
{% for i in (1..32) %}
|
||||
{% assign array = array | concat: array %}
|
||||
{% endfor %}
|
||||
```
|
||||
|
||||
As [JavaScript uses GC to manage memory](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Memory_management), `memoryLimit` may not reflect the actual memory footprint.
|
||||
The `memoryLimit` option was removed in v11; enforce memory limits at the host or process level instead.
|
||||
|
||||
## `ownPropertyOnly` and scope data
|
||||
|
||||
@@ -68,20 +56,21 @@ With [`ownPropertyOnly`][ownPropertyOnly] `true`, plain scope objects only expos
|
||||
|
||||
[`Drop`][drop] values are not restricted the same way: LiquidJS still reads the prototype chain and may call [`liquidMethodMissing`][liquidMethodMissing]. **You** control what a drop exposes; narrow APIs and never feed unsafe data into drops unless the class is built for template access. `ownPropertyOnly` alone does not harden custom drops—audit them like any privileged code.
|
||||
|
||||
## Online service guidance
|
||||
## Production guidance
|
||||
|
||||
If you run an online service, avoid rendering fully user-defined templates whenever possible.
|
||||
LiquidJS does not sandbox template code—custom filters, tags, and scope helpers run as ordinary JavaScript with your process privileges. Built-in DoS limits are one layer; production deployments, especially online services that accept template input, need additional hardening:
|
||||
|
||||
- Prefer curated templates or a restricted template subset.
|
||||
- If user-defined templates are required, isolate rendering (worker/process/container), enforce OS/container memory and CPU limits, and apply request rate limits.
|
||||
- Treat `parseLimit`/`renderLimit`/`memoryLimit` as one layer in a broader DoS defense strategy.
|
||||
|
||||
For heavy single-template operations, process-level isolation is still recommended (for example with [paralleljs][paralleljs]).
|
||||
- **Prefer curated templates** over fully user-defined Liquid when possible; if users need customization, offer a restricted subset rather than open template editing.
|
||||
- Run each render in a **worker thread or child process** with a wall-clock timeout; **kill** the worker on expiry. Libraries such as [paralleljs][paralleljs] can help for heavy single-template work.
|
||||
- Enforce **container/Kubernetes cgroup limits**, `ulimit`, or equivalent on the renderer process for memory and CPU.
|
||||
- Apply **request rate limits** at the API or gateway layer.
|
||||
- **`node:vm`, `isolated-vm`, and Jinja/Twig-style sandbox modes are not a security boundary**—template logic runs in the same JS runtime as your app, with your privileges.
|
||||
|
||||
[paralleljs]: https://www.npmjs.com/package/paralleljs
|
||||
[parseLimit]: /api/interfaces/LiquidOptions.html#parseLimit
|
||||
[renderLimit]: /api/interfaces/LiquidOptions.html#renderLimit
|
||||
[memoryLimit]: /api/interfaces/LiquidOptions.html#memoryLimit
|
||||
[templateLimit]: /api/interfaces/LiquidOptions.html#templateLimit
|
||||
[outputLengthLimit]: /api/interfaces/LiquidOptions.html#outputLengthLimit
|
||||
[maxDepth]: /api/interfaces/LiquidOptions.html#maxDepth
|
||||
[ownPropertyOnly]: /api/interfaces/LiquidOptions.html#ownPropertyOnly
|
||||
[renderOwnPropertyOnly]: /api/interfaces/RenderOptions.html#ownPropertyOnly
|
||||
[strictVariables]: /api/interfaces/LiquidOptions.html#strictVariables
|
||||
|
||||
Vendored
+1
-2
@@ -42,8 +42,7 @@
|
||||
if (!/\/playground(?:\.html)?$/.test(location.pathname)) return;
|
||||
updateVersion(liquidjs.version);
|
||||
const engine = new liquidjs.Liquid({
|
||||
memoryLimit: 1e5,
|
||||
renderLimit: 1e5
|
||||
templateLimit: 1e5
|
||||
});
|
||||
const colorScheme = window.matchMedia('(prefers-color-scheme: dark)');
|
||||
const editor = createEditor('editorEl', 'liquid');
|
||||
|
||||
+10
-8
@@ -1,4 +1,3 @@
|
||||
import { getPerformance } from '../util/performance'
|
||||
import { Drop } from '../drop/drop'
|
||||
import { __assign } from 'tslib'
|
||||
import { NormalizedFullOptions, defaultOptions, RenderOptions } from '../liquid-options'
|
||||
@@ -36,17 +35,19 @@ export class Context {
|
||||
*/
|
||||
public strictVariables: boolean;
|
||||
public ownPropertyOnly: boolean;
|
||||
public memoryLimit: Limiter;
|
||||
public renderLimit: Limiter;
|
||||
public constructor (env: object = {}, opts: NormalizedFullOptions = defaultOptions, renderOptions: RenderOptions = {}, { memoryLimit, renderLimit }: { [key: string]: Limiter } = {}) {
|
||||
public templateLimit: Limiter;
|
||||
public outputLengthLimit: Limiter;
|
||||
public depthLimit: Limiter;
|
||||
public constructor (env: object = {}, opts: NormalizedFullOptions = defaultOptions, renderOptions: RenderOptions = {}, { templateLimit, outputLengthLimit, depthLimit }: { templateLimit?: Limiter, outputLengthLimit?: Limiter, depthLimit?: Limiter } = {}) {
|
||||
this.sync = !!renderOptions.sync
|
||||
this.opts = opts
|
||||
this.globals = renderOptions.globals ?? opts.globals
|
||||
this.environments = isObject(env) ? env : Object(env)
|
||||
this.strictVariables = renderOptions.strictVariables ?? this.opts.strictVariables
|
||||
this.ownPropertyOnly = renderOptions.ownPropertyOnly ?? opts.ownPropertyOnly
|
||||
this.memoryLimit = memoryLimit ?? new Limiter('memory alloc', renderOptions.memoryLimit ?? opts.memoryLimit)
|
||||
this.renderLimit = renderLimit ?? new Limiter('template render', getPerformance().now() + (renderOptions.renderLimit ?? opts.renderLimit))
|
||||
this.templateLimit = templateLimit ?? new Limiter('template', renderOptions.templateLimit ?? opts.templateLimit)
|
||||
this.outputLengthLimit = outputLengthLimit ?? new Limiter('output length', renderOptions.outputLengthLimit ?? opts.outputLengthLimit)
|
||||
this.depthLimit = depthLimit ?? new Limiter('template depth', opts.maxDepth)
|
||||
}
|
||||
public getRegister<T> (key: string, defaultValue: T = undefined as T): T {
|
||||
return (this.registers[key] = this.registers[key] || defaultValue)
|
||||
@@ -109,8 +110,9 @@ export class Context {
|
||||
strictVariables: this.strictVariables,
|
||||
ownPropertyOnly: this.ownPropertyOnly
|
||||
}, {
|
||||
renderLimit: this.renderLimit,
|
||||
memoryLimit: this.memoryLimit
|
||||
templateLimit: this.templateLimit,
|
||||
outputLengthLimit: this.outputLengthLimit,
|
||||
depthLimit: this.depthLimit
|
||||
})
|
||||
}
|
||||
private findScope (key: string | number) {
|
||||
|
||||
@@ -1,10 +1,17 @@
|
||||
import { stringify } from '../util'
|
||||
import { stringify, Limiter } from '../util'
|
||||
import { Emitter } from './emitter'
|
||||
|
||||
export class SimpleEmitter implements Emitter {
|
||||
public buffer = '';
|
||||
private outputLengthLimit?: Limiter
|
||||
|
||||
constructor (outputLengthLimit?: Limiter) {
|
||||
this.outputLengthLimit = outputLengthLimit
|
||||
}
|
||||
|
||||
public write (html: any) {
|
||||
this.buffer += stringify(html)
|
||||
const str = stringify(html)
|
||||
this.outputLengthLimit?.use(str.length)
|
||||
this.buffer += str
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,12 +1,20 @@
|
||||
import { stringify } from '../util'
|
||||
import { stringify, Limiter } from '../util'
|
||||
import { Emitter } from './emitter'
|
||||
import { PassThrough } from 'stream'
|
||||
|
||||
export class StreamedEmitter implements Emitter {
|
||||
public buffer = '';
|
||||
public stream: NodeJS.ReadWriteStream = new PassThrough()
|
||||
private outputLengthLimit?: Limiter
|
||||
|
||||
constructor (outputLengthLimit?: Limiter) {
|
||||
this.outputLengthLimit = outputLengthLimit
|
||||
}
|
||||
|
||||
public write (html: any) {
|
||||
this.stream.write(stringify(html))
|
||||
const str = stringify(html)
|
||||
this.outputLengthLimit?.use(str.length)
|
||||
this.stream.write(str)
|
||||
}
|
||||
public error (err: Error) {
|
||||
this.stream.emit('error', err)
|
||||
|
||||
@@ -8,9 +8,6 @@ import { EmptyDrop } from '../drop'
|
||||
export const join = argumentsToValue(function (this: FilterImpl, v: any[], arg: string) {
|
||||
const array = toArray(v)
|
||||
const sep = isNil(arg) ? ' ' : stringify(arg)
|
||||
let outputSize = sep.length * Math.max(array.length - 1, 0)
|
||||
for (let i = 0; i < array.length; i++) outputSize += String(array[i]).length
|
||||
this.context.memoryLimit.use(outputSize)
|
||||
return Array.prototype.join.call(array, sep)
|
||||
})
|
||||
export const last = argumentsToValue(function (this: FilterImpl, v: any) {
|
||||
@@ -21,14 +18,12 @@ export const first = argumentsToValue(function (this: FilterImpl, v: any) {
|
||||
})
|
||||
export const reverse = argumentsToValue(function (this: FilterImpl, v: any[]) {
|
||||
const array = toArray(v)
|
||||
this.context.memoryLimit.use(array.length)
|
||||
return [...array].reverse()
|
||||
})
|
||||
|
||||
function * sortBy<T> (this: FilterImpl, arr: T[], property: string | undefined, comparator: (a: unknown, b: unknown) => number): IterableIterator<unknown> {
|
||||
const values: [T, unknown][] = []
|
||||
const array = toArray(arr)
|
||||
this.context.memoryLimit.use(array.length)
|
||||
for (const item of array) {
|
||||
values.push([
|
||||
item,
|
||||
@@ -51,7 +46,6 @@ export const size = (v: string | any[]) => v?.length || 0
|
||||
export function * map (this: FilterImpl, arr: Scope[], property: string): IterableIterator<unknown> {
|
||||
const results = []
|
||||
const array = toArray(arr)
|
||||
this.context.memoryLimit.use(array.length)
|
||||
for (const item of array) {
|
||||
results.push(yield this.context._getFromScope(item, stringify(property), false))
|
||||
}
|
||||
@@ -70,14 +64,12 @@ export function * sum (this: FilterImpl, arr: Scope[], property?: string): Itera
|
||||
|
||||
export function compact<T> (this: FilterImpl, arr: T[]) {
|
||||
const array = toArray(arr)
|
||||
this.context.memoryLimit.use(array.length)
|
||||
return Array.prototype.filter.call(array, x => !isNil(toValue(x)))
|
||||
}
|
||||
|
||||
export function concat<T1, T2> (this: FilterImpl, v: T1[], arg: T2[] = []): (T1 | T2)[] {
|
||||
const lhs = toArray(v)
|
||||
const rhs = toArray(arg)
|
||||
this.context.memoryLimit.use(lhs.length + rhs.length)
|
||||
return Array.prototype.concat.call(lhs, rhs)
|
||||
}
|
||||
|
||||
@@ -87,7 +79,6 @@ export function push<T> (this: FilterImpl, v: T[], arg: T): T[] {
|
||||
|
||||
export function unshift<T> (this: FilterImpl, v: T[], arg: T): T[] {
|
||||
const array = toArray(v)
|
||||
this.context.memoryLimit.use(array.length)
|
||||
const clone = [...array]
|
||||
clone.unshift(arg)
|
||||
return clone
|
||||
@@ -95,7 +86,6 @@ export function unshift<T> (this: FilterImpl, v: T[], arg: T): T[] {
|
||||
|
||||
export function pop<T> (this: FilterImpl, v: T[]): T[] {
|
||||
const array = toArray(v)
|
||||
this.context.memoryLimit.use(array.length)
|
||||
const clone = [...array]
|
||||
clone.pop()
|
||||
return clone
|
||||
@@ -103,7 +93,6 @@ export function pop<T> (this: FilterImpl, v: T[]): T[] {
|
||||
|
||||
export function shift<T> (this: FilterImpl, v: T[]): T[] {
|
||||
const array = toArray(v)
|
||||
this.context.memoryLimit.use(array.length)
|
||||
const clone = [...array]
|
||||
clone.shift()
|
||||
return clone
|
||||
@@ -114,7 +103,6 @@ export function slice<T> (this: FilterImpl, v: T[] | string, begin: number, leng
|
||||
if (isNil(v)) return []
|
||||
if (!isArray(v)) v = stringify(v)
|
||||
begin = begin < 0 ? v.length + begin : begin
|
||||
this.context.memoryLimit.use(length)
|
||||
return isArray(v)
|
||||
? Array.prototype.slice.call(v, begin, begin + length)
|
||||
: String.prototype.slice.call(v, begin, begin + length)
|
||||
@@ -133,7 +121,6 @@ function expectedMatcher (this: FilterImpl, expected: any): (v: any) => boolean
|
||||
function * filter<T extends object> (this: FilterImpl, include: boolean, arr: T[], property: string, expected: any): IterableIterator<unknown> {
|
||||
const values: unknown[] = []
|
||||
arr = toArray(arr)
|
||||
this.context.memoryLimit.use(arr.length)
|
||||
const token = new Tokenizer(stringify(property)).readScopeValue()
|
||||
for (const item of arr) {
|
||||
values.push(yield evalToken(token, this.context.spawn(item)))
|
||||
@@ -146,7 +133,6 @@ function * filter_exp<T extends object> (this: FilterImpl, include: boolean, arr
|
||||
const filtered: unknown[] = []
|
||||
const keyTemplate = new Value(stringify(exp), this.liquid)
|
||||
const array = toArray(arr)
|
||||
this.context.memoryLimit.use(array.length)
|
||||
for (const item of array) {
|
||||
this.context.push({ [itemName]: item })
|
||||
const value = yield keyTemplate.value(this.context)
|
||||
@@ -176,7 +162,6 @@ export function * group_by<T extends object> (this: FilterImpl, arr: T[], proper
|
||||
const map = new Map()
|
||||
arr = toEnumerable(arr)
|
||||
const token = new Tokenizer(stringify(property)).readScopeValue()
|
||||
this.context.memoryLimit.use(arr.length)
|
||||
for (const item of arr) {
|
||||
const key = yield evalToken(token, this.context.spawn(item))
|
||||
if (!map.has(key)) map.set(key, [])
|
||||
@@ -189,7 +174,6 @@ export function * group_by_exp<T extends object> (this: FilterImpl, arr: T[], it
|
||||
const map = new Map()
|
||||
const keyTemplate = new Value(stringify(exp), this.liquid)
|
||||
arr = toEnumerable(arr)
|
||||
this.context.memoryLimit.use(arr.length)
|
||||
for (const item of arr) {
|
||||
this.context.push({ [itemName]: item })
|
||||
const key = yield keyTemplate.value(this.context)
|
||||
@@ -253,7 +237,6 @@ export function * find_exp<T extends object> (this: FilterImpl, arr: T[], itemNa
|
||||
|
||||
export function uniq<T> (this: FilterImpl, arr: T[]): T[] {
|
||||
arr = toArray(arr)
|
||||
this.context.memoryLimit.use(arr.length)
|
||||
return [...new Set(arr)]
|
||||
}
|
||||
|
||||
@@ -261,7 +244,6 @@ export function sample<T> (this: FilterImpl, v: T[] | string, count = 1): T | st
|
||||
v = toValue(v)
|
||||
if (isNil(v)) return []
|
||||
if (!isArray(v)) v = stringify(v)
|
||||
this.context.memoryLimit.use(v.length)
|
||||
const shuffled = [...v].sort(() => Math.random() - 0.5)
|
||||
if (count === 1) return shuffled[0]
|
||||
return shuffled.slice(0, count)
|
||||
|
||||
@@ -10,16 +10,13 @@ import { base64Encode, base64Decode } from './base64-impl'
|
||||
|
||||
export function base64_encode (this: FilterImpl, value: string | Buffer): string {
|
||||
if (typeof Buffer !== 'undefined' && Buffer.isBuffer(value)) {
|
||||
this.context.memoryLimit.use(value.byteLength)
|
||||
return value.toString('base64')
|
||||
}
|
||||
const str = stringify(value)
|
||||
this.context.memoryLimit.use(str.length)
|
||||
return base64Encode(str)
|
||||
}
|
||||
|
||||
export function base64_decode (this: FilterImpl, value: string): string {
|
||||
const str = stringify(value)
|
||||
this.context.memoryLimit.use(str.length)
|
||||
return base64Decode(str)
|
||||
}
|
||||
|
||||
@@ -10,13 +10,11 @@ import { sha256 as sha256Impl, hmacSha256 as hmacSha256Impl } from './crypto-imp
|
||||
|
||||
export function sha256 (this: FilterImpl, value: unknown): string | Promise<string> {
|
||||
const str = stringify(value)
|
||||
this.context.memoryLimit.use(str.length)
|
||||
return sha256Impl(str)
|
||||
}
|
||||
|
||||
export function hmac_sha256 (this: FilterImpl, value: unknown, key: unknown): string | Promise<string> {
|
||||
const str = stringify(value)
|
||||
const keyStr = stringify(key)
|
||||
this.context.memoryLimit.use(str.length + keyStr.length)
|
||||
return hmacSha256Impl(str, keyStr)
|
||||
}
|
||||
|
||||
+4
-8
@@ -3,14 +3,11 @@ import { FilterImpl } from '../template'
|
||||
import { NormalizedFullOptions } from '../liquid-options'
|
||||
|
||||
export function date (this: FilterImpl, v: string | Date, format?: string, timezoneOffset?: number | string) {
|
||||
const size = ((v as string)?.length ?? 0) + ((timezoneOffset as string)?.length ?? 0)
|
||||
this.context.memoryLimit.use(size)
|
||||
const date = parseDate(v, this.context.opts, timezoneOffset)
|
||||
if (!date) return v
|
||||
format = toValue(format)
|
||||
format = isNil(format) ? this.context.opts.dateFormat : stringify(format)
|
||||
this.context.memoryLimit.use(format.length)
|
||||
return strftime(date, format, this.context.memoryLimit)
|
||||
return strftime(date, format)
|
||||
}
|
||||
|
||||
export function date_to_xmlschema (this: FilterImpl, v: string | Date) {
|
||||
@@ -32,14 +29,13 @@ export function date_to_long_string (this: FilterImpl, v: string | Date, type?:
|
||||
function stringify_date (this: FilterImpl, v: string | Date, month_type: string, type?: string, style?: string) {
|
||||
const date = parseDate(v, this.context.opts)
|
||||
if (!date) return v
|
||||
const ml = this.context.memoryLimit
|
||||
if (type === 'ordinal') {
|
||||
const d = date.getDate()
|
||||
return style === 'US'
|
||||
? strftime(date, `${month_type} ${d}%q, %Y`, ml)
|
||||
: strftime(date, `${d}%q ${month_type} %Y`, ml)
|
||||
? strftime(date, `${month_type} ${d}%q, %Y`)
|
||||
: strftime(date, `${d}%q ${month_type} %Y`)
|
||||
}
|
||||
return strftime(date, `%d ${month_type} %Y`, ml)
|
||||
return strftime(date, `%d ${month_type} %Y`)
|
||||
}
|
||||
|
||||
function parseDate (v: string | Date, opts: NormalizedFullOptions, timezoneOffset?: number | string): LiquidDate | undefined {
|
||||
|
||||
@@ -18,7 +18,6 @@ const unescapeMap: Record<string, string> = {
|
||||
|
||||
export function escape (this: FilterImpl, str: string) {
|
||||
str = stringify(str)
|
||||
this.context.memoryLimit.use(str.length)
|
||||
return str.replace(/&|<|>|"|'/g, m => escapeMap[m])
|
||||
}
|
||||
|
||||
@@ -28,7 +27,6 @@ export function xml_escape (this: FilterImpl, str: string) {
|
||||
|
||||
function unescape (this: FilterImpl, str: string) {
|
||||
str = stringify(str)
|
||||
this.context.memoryLimit.use(str.length)
|
||||
return str.replace(/&(amp|lt|gt|#34|#39);/g, m => unescapeMap[m])
|
||||
}
|
||||
|
||||
@@ -38,7 +36,6 @@ export function escape_once (this: FilterImpl, str: string) {
|
||||
|
||||
export function newline_to_br (this: FilterImpl, v: string) {
|
||||
const str = stringify(v)
|
||||
this.context.memoryLimit.use(str.length)
|
||||
return str.replace(/\r?\n/gm, '<br />\n')
|
||||
}
|
||||
|
||||
@@ -46,7 +43,6 @@ export function newline_to_br (this: FilterImpl, v: string) {
|
||||
// equivalent is O(n^2) in V8 on unclosed openers.
|
||||
export function strip_html (this: FilterImpl, v: string) {
|
||||
const str = stringify(v)
|
||||
this.context.memoryLimit.use(str.length)
|
||||
const blocks = new Map([['<script', '</script>'], ['<style', '</style>'], ['<!--', '-->'], ['<', '>']])
|
||||
let out = ''
|
||||
let i = 0
|
||||
|
||||
+1
-21
@@ -2,18 +2,6 @@ import { isFalsy } from '../render/boolean'
|
||||
import { identify, isArray, isString, toValue } from '../util/underscore'
|
||||
import { FilterImpl } from '../template'
|
||||
|
||||
function chargeJsonReplacerValue (memoryLimit: { use(count: number): void }, val: unknown) {
|
||||
if (typeof val === 'string') {
|
||||
memoryLimit.use(val.length)
|
||||
} else if (val === null || typeof val === 'number' || typeof val === 'boolean') {
|
||||
memoryLimit.use(JSON.stringify(val).length)
|
||||
} else if (Array.isArray(val)) {
|
||||
memoryLimit.use(val.length + 1)
|
||||
} else if (typeof val === 'object') {
|
||||
memoryLimit.use(2)
|
||||
}
|
||||
}
|
||||
|
||||
function defaultFilter<T1 extends boolean, T2> (this: FilterImpl, value: T1, defaultValue: T2, ...args: Array<[string, any]>): T1 | T2 {
|
||||
value = toValue(value)
|
||||
if (isArray(value) || isString(value)) return value.length ? value : defaultValue
|
||||
@@ -22,29 +10,21 @@ function defaultFilter<T1 extends boolean, T2> (this: FilterImpl, value: T1, def
|
||||
}
|
||||
|
||||
function json (this: FilterImpl, value: any, space = 0) {
|
||||
const memoryLimit = this.context.memoryLimit
|
||||
return JSON.stringify(value, (_key, val) => {
|
||||
chargeJsonReplacerValue(memoryLimit, val)
|
||||
return val
|
||||
}, space)
|
||||
return JSON.stringify(value, undefined, space)
|
||||
}
|
||||
|
||||
function inspect (this: FilterImpl, value: any, space = 0) {
|
||||
const memoryLimit = this.context.memoryLimit
|
||||
const ancestors: object[] = []
|
||||
return JSON.stringify(value, function (this: unknown, _key: unknown, value: any) {
|
||||
if (typeof value !== 'object' || value === null) {
|
||||
chargeJsonReplacerValue(memoryLimit, value)
|
||||
return value
|
||||
}
|
||||
// `this` is the object that value is contained in, i.e., its direct parent.
|
||||
while (ancestors.length > 0 && ancestors[ancestors.length - 1] !== this) ancestors.pop()
|
||||
if (ancestors.includes(value)) {
|
||||
memoryLimit.use('[Circular]'.length)
|
||||
return '[Circular]'
|
||||
}
|
||||
ancestors.push(value)
|
||||
chargeJsonReplacerValue(memoryLimit, value)
|
||||
return value
|
||||
}, space)
|
||||
}
|
||||
|
||||
@@ -22,7 +22,6 @@ export function append (this: FilterImpl, v: string, arg: string) {
|
||||
assert(arguments.length === 2, 'append expect 2 arguments')
|
||||
const lhs = stringify(v)
|
||||
const rhs = stringify(arg)
|
||||
this.context.memoryLimit.use(lhs.length + rhs.length)
|
||||
return lhs + rhs
|
||||
}
|
||||
|
||||
@@ -30,16 +29,13 @@ export function prepend (this: FilterImpl, v: string, arg: string) {
|
||||
assert(arguments.length === 2, 'prepend expect 2 arguments')
|
||||
const lhs = stringify(v)
|
||||
const rhs = stringify(arg)
|
||||
this.context.memoryLimit.use(lhs.length + rhs.length)
|
||||
return rhs + lhs
|
||||
}
|
||||
|
||||
export function lstrip (this: FilterImpl, v: string, chars?: string) {
|
||||
const str = stringify(v)
|
||||
this.context.memoryLimit.use(str.length)
|
||||
if (chars) {
|
||||
chars = stringify(chars)
|
||||
this.context.memoryLimit.use(chars.length)
|
||||
for (let i = 0, set = new Set(chars); i < str.length; i++) {
|
||||
if (!set.has(str[i])) return str.slice(i)
|
||||
}
|
||||
@@ -50,34 +46,29 @@ export function lstrip (this: FilterImpl, v: string, chars?: string) {
|
||||
|
||||
export function downcase (this: FilterImpl, v: string) {
|
||||
const str = stringify(v)
|
||||
this.context.memoryLimit.use(str.length)
|
||||
return str.toLowerCase()
|
||||
}
|
||||
|
||||
export function upcase (this: FilterImpl, v: string) {
|
||||
const str = stringify(v)
|
||||
this.context.memoryLimit.use(str.length)
|
||||
return stringify(str).toUpperCase()
|
||||
}
|
||||
|
||||
export function remove (this: FilterImpl, v: string, arg: string) {
|
||||
const str = stringify(v)
|
||||
arg = stringify(arg)
|
||||
this.context.memoryLimit.use(str.length + arg.length)
|
||||
return str.split(arg).join('')
|
||||
}
|
||||
|
||||
export function remove_first (this: FilterImpl, v: string, l: string) {
|
||||
const str = stringify(v)
|
||||
l = stringify(l)
|
||||
this.context.memoryLimit.use(str.length + l.length)
|
||||
return str.replace(l, '')
|
||||
}
|
||||
|
||||
export function remove_last (this: FilterImpl, v: string, l: string) {
|
||||
const str = stringify(v)
|
||||
const pattern = stringify(l)
|
||||
this.context.memoryLimit.use(str.length + pattern.length)
|
||||
const index = str.lastIndexOf(pattern)
|
||||
if (index === -1) return str
|
||||
return str.substring(0, index) + str.substring(index + pattern.length)
|
||||
@@ -85,10 +76,8 @@ export function remove_last (this: FilterImpl, v: string, l: string) {
|
||||
|
||||
export function rstrip (this: FilterImpl, str: string, chars?: string) {
|
||||
str = stringify(str)
|
||||
this.context.memoryLimit.use(str.length)
|
||||
if (chars) {
|
||||
chars = stringify(chars)
|
||||
this.context.memoryLimit.use(chars.length)
|
||||
for (let i = str.length - 1, set = new Set(chars); i >= 0; i--) {
|
||||
if (!set.has(str[i])) return str.slice(0, i + 1)
|
||||
}
|
||||
@@ -99,7 +88,6 @@ export function rstrip (this: FilterImpl, str: string, chars?: string) {
|
||||
|
||||
export function split (this: FilterImpl, v: string, arg: string) {
|
||||
const str = stringify(v)
|
||||
this.context.memoryLimit.use(str.length)
|
||||
const arr = str.split(stringify(arg))
|
||||
// align to ruby split, which is the behavior of shopify/liquid
|
||||
// see: https://ruby-doc.org/core-2.4.0/String.html#method-i-split
|
||||
@@ -109,10 +97,8 @@ export function split (this: FilterImpl, v: string, arg: string) {
|
||||
|
||||
export function strip (this: FilterImpl, v: string, chars?: string) {
|
||||
const str = stringify(v)
|
||||
this.context.memoryLimit.use(str.length)
|
||||
if (chars) {
|
||||
const set = new Set(stringify(chars))
|
||||
this.context.memoryLimit.use(set.size)
|
||||
let i = 0
|
||||
let j = str.length - 1
|
||||
while (set.has(str[i])) i++
|
||||
@@ -124,13 +110,11 @@ export function strip (this: FilterImpl, v: string, chars?: string) {
|
||||
|
||||
export function strip_newlines (this: FilterImpl, v: string) {
|
||||
const str = stringify(v)
|
||||
this.context.memoryLimit.use(str.length)
|
||||
return str.replace(/\r?\n/gm, '')
|
||||
}
|
||||
|
||||
export function capitalize (this: FilterImpl, str: string) {
|
||||
str = stringify(str)
|
||||
this.context.memoryLimit.use(str.length)
|
||||
return str.charAt(0).toUpperCase() + str.slice(1).toLowerCase()
|
||||
}
|
||||
|
||||
@@ -139,8 +123,6 @@ export function replace (this: FilterImpl, v: string, pattern: string, replaceme
|
||||
pattern = stringify(pattern)
|
||||
replacement = stringify(replacement)
|
||||
const parts = str.split(pattern)
|
||||
const outputSize = str.length + (parts.length - 1) * (replacement.length - pattern.length)
|
||||
this.context.memoryLimit.use(outputSize)
|
||||
return parts.join(replacement)
|
||||
}
|
||||
|
||||
@@ -148,7 +130,6 @@ export function replace_first (this: FilterImpl, v: string, arg1: string, arg2:
|
||||
const str = stringify(v)
|
||||
arg1 = stringify(arg1)
|
||||
arg2 = stringify(arg2)
|
||||
this.context.memoryLimit.use(str.length + arg1.length + arg2.length)
|
||||
return str.replace(arg1, () => arg2)
|
||||
}
|
||||
|
||||
@@ -156,7 +137,6 @@ export function replace_last (this: FilterImpl, v: string, arg1: string, arg2: s
|
||||
const str = stringify(v)
|
||||
const pattern = stringify(arg1)
|
||||
const replacement = stringify(arg2)
|
||||
this.context.memoryLimit.use(str.length + pattern.length + replacement.length)
|
||||
const index = str.lastIndexOf(pattern)
|
||||
if (index === -1) return str
|
||||
return str.substring(0, index) + replacement + str.substring(index + pattern.length)
|
||||
@@ -165,7 +145,6 @@ export function replace_last (this: FilterImpl, v: string, arg1: string, arg2: s
|
||||
export function truncate (this: FilterImpl, v: string, l = 50, o = '...') {
|
||||
const str = stringify(v)
|
||||
o = stringify(o)
|
||||
this.context.memoryLimit.use(str.length + o.length)
|
||||
if (str.length <= l) return v
|
||||
return str.substring(0, l - o.length) + o
|
||||
}
|
||||
@@ -173,7 +152,6 @@ export function truncate (this: FilterImpl, v: string, l = 50, o = '...') {
|
||||
export function truncatewords (this: FilterImpl, v: string, words = 15, o = '...') {
|
||||
const str = stringify(v)
|
||||
o = stringify(o)
|
||||
this.context.memoryLimit.use(str.length + o.length)
|
||||
const arr = str.split(/\s+/)
|
||||
if (words <= 0) words = 1
|
||||
let ret = arr.slice(0, words).join(' ')
|
||||
@@ -183,13 +161,11 @@ export function truncatewords (this: FilterImpl, v: string, words = 15, o = '...
|
||||
|
||||
export function normalize_whitespace (this: FilterImpl, v: string) {
|
||||
const str = stringify(v)
|
||||
this.context.memoryLimit.use(str.length)
|
||||
return str.replace(/\s+/g, ' ')
|
||||
}
|
||||
|
||||
export function number_of_words (this: FilterImpl, input: string, mode?: 'cjk' | 'auto') {
|
||||
const str = stringify(input)
|
||||
this.context.memoryLimit.use(str.length)
|
||||
input = str.trim()
|
||||
if (!input) return 0
|
||||
switch (mode) {
|
||||
@@ -209,9 +185,6 @@ export function number_of_words (this: FilterImpl, input: string, mode?: 'cjk' |
|
||||
|
||||
export function array_to_sentence_string (this: FilterImpl, array: unknown[], connector = 'and') {
|
||||
connector = stringify(connector)
|
||||
let outputSize = connector.length + array.length * 2
|
||||
for (let i = 0; i < array.length; i++) outputSize += stringify(array[i]).length
|
||||
this.context.memoryLimit.use(outputSize)
|
||||
switch (array.length) {
|
||||
case 0:
|
||||
return ''
|
||||
|
||||
+15
-13
@@ -87,10 +87,12 @@ export interface LiquidOptions {
|
||||
orderedFilterParameters?: boolean;
|
||||
/** For DoS handling, limit total length of templates parsed in one `parse()` call. A typical PC can handle 1e8 (100M) characters without issues. */
|
||||
parseLimit?: number;
|
||||
/** For DoS handling, limit total time (in ms) for each `render()` call. */
|
||||
renderLimit?: number;
|
||||
/** For DoS handling, limit new objects creation, including array concat/join/strftime, etc. A typical PC can handle 1e9 (1G) memory without issue. */
|
||||
memoryLimit?: number;
|
||||
/** For DoS handling, limit total renders of tag/HTML/output in one `render()` call. */
|
||||
templateLimit?: number;
|
||||
/** For DoS handling, limit total output length in one `render()` call. */
|
||||
outputLengthLimit?: number;
|
||||
/** For DoS handling, limit nesting depth of `{% render %}`, `{% include %}`, and `{% layout %}` tags. Defaults to `128`. */
|
||||
maxDepth?: number;
|
||||
}
|
||||
|
||||
export interface RenderOptions {
|
||||
@@ -110,12 +112,10 @@ export interface RenderOptions {
|
||||
* Same as `ownPropertyOnly` on LiquidOptions, but only for current render() call
|
||||
*/
|
||||
ownPropertyOnly?: boolean;
|
||||
/** For DoS handling, limit total renders of tag/HTML/output in one `render()` call. A typical PC can handle 1e5 renders of typical templates per second. */
|
||||
/** For DoS handling, limit total renders of tag/HTML/output in one `render()` call. */
|
||||
templateLimit?: number;
|
||||
/** For DoS handling, limit total time (in ms) for each `render()` call. */
|
||||
renderLimit?: number;
|
||||
/** For DoS handling, limit new objects creation, including array concat/join/strftime, etc. A typical PC can handle 1e9 (1G) memory without issue.. */
|
||||
memoryLimit?: number;
|
||||
/** For DoS handling, limit total output length in one `render()` call. */
|
||||
outputLengthLimit?: number;
|
||||
}
|
||||
|
||||
export interface RenderFileOptions extends RenderOptions {
|
||||
@@ -160,8 +160,9 @@ export interface NormalizedFullOptions extends NormalizedOptions {
|
||||
globals: object;
|
||||
operators: Operators;
|
||||
parseLimit: number;
|
||||
renderLimit: number;
|
||||
memoryLimit: number;
|
||||
templateLimit: number;
|
||||
outputLengthLimit: number;
|
||||
maxDepth: number;
|
||||
}
|
||||
|
||||
export const defaultOptions: NormalizedFullOptions = {
|
||||
@@ -194,9 +195,10 @@ export const defaultOptions: NormalizedFullOptions = {
|
||||
lenientIf: false,
|
||||
globals: {},
|
||||
operators: defaultOperators,
|
||||
memoryLimit: Infinity,
|
||||
parseLimit: Infinity,
|
||||
renderLimit: Infinity
|
||||
templateLimit: Infinity,
|
||||
outputLengthLimit: Infinity,
|
||||
maxDepth: 128
|
||||
}
|
||||
|
||||
export function normalize (options: LiquidOptions): NormalizedFullOptions {
|
||||
|
||||
@@ -67,7 +67,6 @@ export function evalQuotedToken (token: QuotedToken) {
|
||||
function * evalRangeToken (token: RangeToken, ctx: Context) {
|
||||
const low: number = yield evalToken(token.lhs, ctx)
|
||||
const high: number = yield evalToken(token.rhs, ctx)
|
||||
ctx.memoryLimit.use(high - low + 1)
|
||||
return range(+low, +high + 1)
|
||||
}
|
||||
|
||||
|
||||
+3
-10
@@ -1,4 +1,3 @@
|
||||
import { getPerformance } from '../util/performance'
|
||||
import { toPromise, RenderError, LiquidErrors, LiquidError } from '../util'
|
||||
import { Context } from '../context'
|
||||
import { Template } from '../template'
|
||||
@@ -6,23 +5,17 @@ import { Emitter, StreamedEmitter, SimpleEmitter } from '../emitters'
|
||||
|
||||
export class Render {
|
||||
public renderTemplatesToNodeStream (templates: Template[], ctx: Context): NodeJS.ReadableStream {
|
||||
const emitter = new StreamedEmitter()
|
||||
const emitter = new StreamedEmitter(ctx.outputLengthLimit)
|
||||
Promise.resolve().then(() => toPromise(this.renderTemplates(templates, ctx, emitter)))
|
||||
.then(() => emitter.end(), err => emitter.error(err))
|
||||
return emitter.stream
|
||||
}
|
||||
public * renderTemplates (templates: Template[], ctx: Context, emitter?: Emitter): IterableIterator<any> {
|
||||
if (!emitter) {
|
||||
emitter = new SimpleEmitter()
|
||||
}
|
||||
ctx.renderLimit.check(getPerformance().now())
|
||||
public * renderTemplates (templates: Template[], ctx: Context, emitter: Emitter = new SimpleEmitter(ctx.outputLengthLimit)): IterableIterator<any> {
|
||||
const errors = []
|
||||
for (const tpl of templates) {
|
||||
ctx.renderLimit.check(getPerformance().now())
|
||||
ctx.templateLimit.use(1)
|
||||
try {
|
||||
// if tpl.render supports emitter, it'll return empty `html`
|
||||
const html = yield tpl.render(ctx, emitter)
|
||||
// if not, it'll return an `html`, write to the emitter for it
|
||||
html && emitter.write(html)
|
||||
if (ctx.breakCalled || ctx.continueCalled) break
|
||||
} catch (e) {
|
||||
|
||||
+9
-7
@@ -43,13 +43,6 @@ export default class extends Tag {
|
||||
}
|
||||
* render (ctx: Context, emitter: Emitter): Generator<unknown, void | string, Template[]> {
|
||||
const r = this.liquid.renderer
|
||||
let collection = toEnumerable(yield evalToken(this.collection, ctx))
|
||||
|
||||
if (!collection.length) {
|
||||
yield r.renderTemplates(this.elseTemplates, ctx, emitter)
|
||||
return
|
||||
}
|
||||
|
||||
const continueKey = 'continue-' + this.variable + '-' + this.collection.getText()
|
||||
ctx.push(createScope({ continue: ctx.getRegister(continueKey, {}) }))
|
||||
const hash = (yield this.hash.render(ctx)) as Record<string, any>
|
||||
@@ -59,6 +52,7 @@ export default class extends Tag {
|
||||
? Object.keys(hash).filter(x => MODIFIERS.includes(x))
|
||||
: MODIFIERS.filter(x => hash[x] !== undefined)
|
||||
|
||||
let collection = toEnumerable(yield evalToken(this.collection, ctx))
|
||||
collection = modifiers.reduce((collection, modifier: valueOf<typeof MODIFIERS>) => {
|
||||
if (modifier === 'offset') return offset(collection, hash['offset'])
|
||||
if (modifier === 'limit') return limit(collection, hash['limit'])
|
||||
@@ -66,6 +60,14 @@ export default class extends Tag {
|
||||
}, collection)
|
||||
|
||||
ctx.setRegister(continueKey, (hash['offset'] || 0) + collection.length)
|
||||
|
||||
if (!collection.length) {
|
||||
yield r.renderTemplates(this.elseTemplates, ctx, emitter)
|
||||
return
|
||||
}
|
||||
|
||||
if (!this.templates.length) return
|
||||
|
||||
const scope = createScope({ forloop: new ForloopDrop(collection.length, this.collection.getText(), this.variable) })
|
||||
ctx.push(scope)
|
||||
for (const item of collection) {
|
||||
|
||||
@@ -28,6 +28,7 @@ export default class extends Tag {
|
||||
this.hash = new Hash(tokenizer, liquid.options.jekyllInclude || liquid.options.keyValueSeparator)
|
||||
}
|
||||
* render (ctx: Context, emitter: Emitter): Generator<unknown, void, unknown> {
|
||||
ctx.depthLimit.use(1)
|
||||
const { liquid, hash, withVar } = this
|
||||
const { renderer } = liquid
|
||||
const filepath = (yield renderFilePath(this.file, ctx, liquid)) as string
|
||||
@@ -43,6 +44,7 @@ export default class extends Tag {
|
||||
yield renderer.renderTemplates(templates, ctx, emitter)
|
||||
ctx.pop()
|
||||
ctx.restoreRegister(saved)
|
||||
ctx.depthLimit.release(1)
|
||||
}
|
||||
|
||||
public * children (partials: boolean, sync: boolean): Generator<unknown, Template[]> {
|
||||
|
||||
@@ -26,6 +26,7 @@ export default class extends Tag {
|
||||
yield renderer.renderTemplates(this.templates, ctx, emitter)
|
||||
return
|
||||
}
|
||||
ctx.depthLimit.use(1)
|
||||
const filepath = (yield renderFilePath(this.file, ctx, liquid)) as string
|
||||
assert(filepath, () => `illegal file path "${filepath}"`)
|
||||
const templates = (yield liquid._parseLayoutFile(filepath, ctx.sync, this.currentFile)) as Template[]
|
||||
@@ -43,6 +44,7 @@ export default class extends Tag {
|
||||
ctx.push(createScope((yield args.render(ctx)) as Scope))
|
||||
yield renderer.renderTemplates(templates, ctx, emitter)
|
||||
ctx.pop()
|
||||
ctx.depthLimit.release(1)
|
||||
}
|
||||
|
||||
public * children (partials: boolean): Generator<unknown, Template[]> {
|
||||
|
||||
@@ -55,6 +55,7 @@ export default class extends Tag {
|
||||
this.hash = new Hash(tokenizer, liquid.options.keyValueSeparator)
|
||||
}
|
||||
* render (ctx: Context, emitter: Emitter): Generator<unknown, void, unknown> {
|
||||
ctx.depthLimit.use(1)
|
||||
const { liquid, hash } = this
|
||||
const filepath = (yield renderFilePath(this.file, ctx, liquid)) as string
|
||||
assert(filepath, () => `illegal file path "${filepath}"`)
|
||||
@@ -81,6 +82,7 @@ export default class extends Tag {
|
||||
const templates = (yield liquid._parsePartialFile(filepath, childCtx.sync, this.currentFile)) as Template[]
|
||||
yield liquid.renderer.renderTemplates(templates, childCtx, emitter)
|
||||
}
|
||||
ctx.depthLimit.release(1)
|
||||
}
|
||||
|
||||
public * children (partials: boolean, sync: boolean): Generator<unknown, Template[]> {
|
||||
|
||||
@@ -45,6 +45,10 @@ export default class extends Tag {
|
||||
const limit = (args.limit === undefined) ? collection.length : args.limit
|
||||
|
||||
collection = collection.slice(offset, offset + limit)
|
||||
if (!collection.length) return
|
||||
|
||||
if (!this.templates.length) return
|
||||
|
||||
const cols = args.cols || collection.length
|
||||
|
||||
const r = this.liquid.renderer
|
||||
|
||||
@@ -14,6 +14,11 @@ export class Limiter {
|
||||
this.base += +count
|
||||
}
|
||||
}
|
||||
release (count: number) {
|
||||
if (+count > 0) {
|
||||
this.base -= +count
|
||||
}
|
||||
}
|
||||
check (count: number) {
|
||||
if (+count > 0) {
|
||||
assert(+count <= this.limit, this.message)
|
||||
|
||||
@@ -188,6 +188,14 @@ describe('util/strftime', function () {
|
||||
it('should have higher priority than H', () => {
|
||||
expect(t(then, '%0H')).toBe('03')
|
||||
})
|
||||
it('should allow pad width up to MAX_STRFTIME_PAD', () => {
|
||||
expect(t(now, '%100000d').length).toBe(100000)
|
||||
expect(t(now, `%${1_000_000}d`).length).toBe(1_000_000)
|
||||
})
|
||||
it('should throw when pad width exceeds MAX_STRFTIME_PAD', () => {
|
||||
expect(() => t(now, `%${1024 * 1024 + 1}d`)).toThrow('strftime pad width limit exceeded')
|
||||
expect(() => t(now, '%5000000d')).toThrow('strftime pad width limit exceeded')
|
||||
})
|
||||
})
|
||||
describe('modifier field', () => {
|
||||
it('should ignore E modifier', () => {
|
||||
|
||||
+14
-8
@@ -1,13 +1,15 @@
|
||||
import { changeCase, padStart, padEnd } from './underscore'
|
||||
import { LiquidDate } from './liquid-date'
|
||||
import type { Limiter } from './limiter'
|
||||
import { assert } from './assert'
|
||||
|
||||
/** Per-conversion numeric width cap for strftime (%N, %15d, …). */
|
||||
export const MAX_STRFTIME_PAD = 1024 * 1024
|
||||
|
||||
const rFormat = /%([-_0^#:]+)?(\d+)?([EO])?(.)/
|
||||
interface FormatOptions {
|
||||
flags: Record<string, boolean>;
|
||||
width?: string;
|
||||
modifier?: string;
|
||||
memoryLimit?: Pick<Limiter, 'use'>;
|
||||
}
|
||||
|
||||
// prototype extensions
|
||||
@@ -98,8 +100,8 @@ const formatCodes: Record<string, FormatCodeHandler> = {
|
||||
M: (d: LiquidDate) => d.getMinutes(),
|
||||
N: (d: LiquidDate, opts: FormatOptions) => {
|
||||
const width = Number(opts.width) || 9
|
||||
assertPadWidth(width)
|
||||
const str = String(d.getMilliseconds()).slice(0, width)
|
||||
opts.memoryLimit?.use(width - str.length)
|
||||
return padEnd(str, width, '0')
|
||||
},
|
||||
p: (d: LiquidDate) => (d.getHours() < 12 ? 'AM' : 'PM'),
|
||||
@@ -123,25 +125,29 @@ const formatCodes: Record<string, FormatCodeHandler> = {
|
||||
}
|
||||
formatCodes.h = formatCodes.b
|
||||
|
||||
export function strftime (d: LiquidDate, formatStr: string, memoryLimit?: Pick<Limiter, 'use'>) {
|
||||
export function strftime (d: LiquidDate, formatStr: string) {
|
||||
let output = ''
|
||||
let remaining = formatStr
|
||||
let match
|
||||
while ((match = rFormat.exec(remaining))) {
|
||||
output += remaining.slice(0, match.index)
|
||||
remaining = remaining.slice(match.index + match[0].length)
|
||||
output += format(d, match, memoryLimit)
|
||||
output += format(d, match)
|
||||
}
|
||||
return output + remaining
|
||||
}
|
||||
|
||||
function format (d: LiquidDate, match: RegExpExecArray, memoryLimit?: Pick<Limiter, 'use'>) {
|
||||
function assertPadWidth (width: number) {
|
||||
assert(width <= MAX_STRFTIME_PAD, 'strftime pad width limit exceeded')
|
||||
}
|
||||
|
||||
function format (d: LiquidDate, match: RegExpExecArray) {
|
||||
const [input, flagStr = '', width, modifier, conversion] = match
|
||||
const convert = formatCodes[conversion]
|
||||
if (!convert) return input
|
||||
const flags: Record<string, boolean> = {}
|
||||
for (const flag of flagStr) flags[flag] = true
|
||||
let ret = String(convert(d, { flags, width, modifier, memoryLimit }))
|
||||
let ret = String(convert(d, { flags, width, modifier }))
|
||||
let padChar = padSpaceChars.has(conversion) ? ' ' : '0'
|
||||
let padWidth = Number(width) || padWidths[conversion] || 0
|
||||
if (flags['^']) ret = ret.toUpperCase()
|
||||
@@ -149,6 +155,6 @@ function format (d: LiquidDate, match: RegExpExecArray, memoryLimit?: Pick<Limit
|
||||
if (flags['_']) padChar = ' '
|
||||
else if (flags['0']) padChar = '0'
|
||||
if (flags['-']) padWidth = 0
|
||||
memoryLimit?.use(Number(padWidth) - ret.length)
|
||||
else assertPadWidth(padWidth)
|
||||
return padStart(ret, padWidth, padChar)
|
||||
}
|
||||
|
||||
@@ -519,13 +519,6 @@ describe('Issues', function () {
|
||||
const result = engine.parseAndRenderSync(`\n{{ "foo" | pos }}`)
|
||||
expect(result).toEqual('\n[2,12] foo')
|
||||
})
|
||||
it("memoryLimit doesn't work in for tag #776", () => {
|
||||
const engine = new Liquid({
|
||||
memoryLimit: 1e5
|
||||
})
|
||||
const tpl = `{% for i in (1..1000000000) %} {{'a'}} {% endfor %}`
|
||||
expect(() => engine.parseAndRenderSync(tpl)).toThrow('memory alloc limit exceeded, line:1, col:1')
|
||||
})
|
||||
it('group_by_exp fails with object as input #785', () => {
|
||||
const site = {
|
||||
tags: {
|
||||
|
||||
@@ -204,31 +204,21 @@ describe('filters/date', function () {
|
||||
return test('{{ "1990-12-31T23:00:00Z" | date: "%Y-%m-%dT%H:%M:%S" }}', '1991-01-01T04:30:00', undefined, optsWithDateFormat)
|
||||
})
|
||||
})
|
||||
describe('strftime width / memoryLimit', () => {
|
||||
it('should charge memoryLimit for huge numeric strftime widths', () => {
|
||||
const liquid = new Liquid({ memoryLimit: 500 })
|
||||
expect(() => liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: '%5000000d' }))
|
||||
.toThrow('memory alloc limit exceeded')
|
||||
})
|
||||
it('should charge memoryLimit for array format PoC', () => {
|
||||
const liquid = new Liquid({ memoryLimit: 50, renderLimit: 1e9 })
|
||||
expect(() => liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: ['a'.repeat(2000000)] }))
|
||||
.toThrow('memory alloc limit exceeded')
|
||||
})
|
||||
it('should charge memoryLimit for object toString format PoC', () => {
|
||||
const liquid = new Liquid({ memoryLimit: 50, renderLimit: 1e9 })
|
||||
const huge = 'a'.repeat(2000000)
|
||||
const f = { toString: () => huge }
|
||||
expect(() => liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f }))
|
||||
.toThrow('memory alloc limit exceeded')
|
||||
})
|
||||
it('should honor numeric strftime pad width when memoryLimit allows', () => {
|
||||
const liquid = new Liquid({ memoryLimit: 1e7 })
|
||||
describe('strftime width', () => {
|
||||
it('should honor numeric strftime pad width', () => {
|
||||
const liquid = new Liquid()
|
||||
const out = liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: '%5000d' })
|
||||
expect(out.length).toBe(5000)
|
||||
const tight = new Liquid({ memoryLimit: 100 })
|
||||
expect(() => tight.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: '%5000d' }))
|
||||
.toThrow('memory alloc limit exceeded')
|
||||
})
|
||||
it('should honor large numeric strftime pad width up to the cap', () => {
|
||||
const liquid = new Liquid()
|
||||
const out = liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: '%100000d' })
|
||||
expect(out.length).toBe(100000)
|
||||
})
|
||||
it('should throw when numeric strftime pad width is too large', () => {
|
||||
const liquid = new Liquid()
|
||||
expect(() => liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: '%5000000d' }))
|
||||
.toThrow('strftime pad width limit exceeded')
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
@@ -4,15 +4,18 @@ import { mock, restore } from '../../stub/mockfs'
|
||||
describe('DoS related', function () {
|
||||
describe('#parseLimit', function () {
|
||||
afterEach(restore)
|
||||
|
||||
it('should throw when parse limit exceeded', async () => {
|
||||
const noLimit = new Liquid()
|
||||
const limit10 = new Liquid({ parseLimit: 10 })
|
||||
const limit90 = new Liquid({ parseLimit: 90 })
|
||||
const template = '{% capture bar %}{{ foo | bar: 3, a[3] }}{% endcapture %}'
|
||||
|
||||
await expect(noLimit.parseAndRender(template)).resolves.toBe('')
|
||||
await expect(limit10.parseAndRender(template)).rejects.toThrow('parse length limit exceeded')
|
||||
await expect(limit90.parseAndRender(template)).resolves.toBe('')
|
||||
})
|
||||
|
||||
it('should take included template into account', async () => {
|
||||
mock({
|
||||
'/small': 'Lorem ipsum',
|
||||
@@ -23,119 +26,151 @@ describe('DoS related', function () {
|
||||
await expect(liquid.parseAndRender('{% include "large" %}')).rejects.toThrow('parse length limit exceeded')
|
||||
})
|
||||
})
|
||||
describe('#renderLimit', () => {
|
||||
|
||||
describe('#templateLimit', () => {
|
||||
it('should throw when rendering too many templates', async () => {
|
||||
const src = '{% for i in (1..1000) %}{{i}},{% endfor %}'
|
||||
const noLimit = new Liquid()
|
||||
const limitSmall = new Liquid({ renderLimit: 0.01 })
|
||||
const limitLarge = new Liquid({ renderLimit: 2e4 })
|
||||
const limitSmall = new Liquid({ templateLimit: 100 })
|
||||
const limitLarge = new Liquid({ templateLimit: 2001 })
|
||||
await expect(noLimit.parseAndRender(src)).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
|
||||
await expect(limitSmall.parseAndRender(src)).rejects.toThrow('template render limit exceeded')
|
||||
await expect(limitSmall.parseAndRender(src)).rejects.toThrow('template limit exceeded')
|
||||
await expect(limitLarge.parseAndRender(src)).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
|
||||
})
|
||||
|
||||
it('should support reset when calling render', async () => {
|
||||
const src = '{% for i in (1..1000) %}{{i}},{% endfor %}'
|
||||
const liquid = new Liquid({ renderLimit: 0.01 })
|
||||
await expect(liquid.parseAndRender(src)).rejects.toThrow('template render limit exceeded')
|
||||
await expect(liquid.parseAndRender(src, {}, { renderLimit: 1e6 })).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
|
||||
const liquid = new Liquid({ templateLimit: 100 })
|
||||
await expect(liquid.parseAndRender(src)).rejects.toThrow('template limit exceeded')
|
||||
await expect(liquid.parseAndRender(src, {}, { templateLimit: 2001 })).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
|
||||
})
|
||||
|
||||
it('should take partials into account', async () => {
|
||||
mock({
|
||||
'/small': '{% for i in (1..5) %}{{i}}{% endfor %}',
|
||||
'/large': '{% for i in (1..50000000) %}{{i}}{% endfor %}'
|
||||
})
|
||||
const liquid = new Liquid({ root: '/', renderLimit: 1000 })
|
||||
await expect(liquid.parseAndRender('{% render "large" %}')).rejects.toThrow('template render limit exceeded')
|
||||
const liquid = new Liquid({ root: '/', templateLimit: 1000 })
|
||||
await expect(liquid.parseAndRender('{% render "large" %}')).rejects.toThrow('template limit exceeded')
|
||||
await expect(liquid.parseAndRender('{% render "small" %}')).resolves.toBe('12345')
|
||||
})
|
||||
it('should enforce renderLimit when for body has no template nodes', () => {
|
||||
const liquid = new Liquid({ memoryLimit: 1e9, renderLimit: 1 })
|
||||
expect(() => liquid.parseAndRenderSync('{%- for i in (1..5000000) -%}{%- endfor -%}', {}))
|
||||
.toThrow('template render limit exceeded')
|
||||
})
|
||||
it('should enforce renderLimit when tablerow body has no template nodes', () => {
|
||||
const liquid = new Liquid({ memoryLimit: 1e9, renderLimit: 1 })
|
||||
expect(() => liquid.parseAndRenderSync('{%- tablerow i in (1..1000000) cols:1 -%}{%- endtablerow -%}', {}))
|
||||
.toThrow('template render limit exceeded')
|
||||
})
|
||||
})
|
||||
describe('#memoryLimit', () => {
|
||||
it('should throw for too many array creation in filters', async () => {
|
||||
const array = Array(1e3).fill(0)
|
||||
const liquid = new Liquid({ memoryLimit: 100 })
|
||||
await expect(liquid.parseAndRender('{{ array | slice: 0, 3 | join }}', { array })).resolves.toBe('0 0 0')
|
||||
await expect(liquid.parseAndRender('{{ array | slice: 0, 300 | join }}', { array })).rejects.toThrow('memory alloc limit exceeded, line:1, col:1')
|
||||
|
||||
describe('#outputLengthLimit', () => {
|
||||
it('should throw when output length exceeded', async () => {
|
||||
const src = '{% for i in (1..1000) %}{{i}},{% endfor %}'
|
||||
const noLimit = new Liquid()
|
||||
const limitSmall = new Liquid({ outputLengthLimit: 10 })
|
||||
const limitLarge = new Liquid({ outputLengthLimit: 5000 })
|
||||
await expect(noLimit.parseAndRender(src)).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
|
||||
await expect(limitSmall.parseAndRender(src)).rejects.toThrow('output length limit exceeded')
|
||||
await expect(limitLarge.parseAndRender(src)).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
|
||||
})
|
||||
|
||||
it('should support reset when calling render', async () => {
|
||||
const array = Array(1e3).fill(0)
|
||||
const liquid = new Liquid({ memoryLimit: 100 })
|
||||
await expect(liquid.parseAndRender('{{ array | slice: 0, 300 | join }}', { array })).rejects.toThrow('memory alloc limit exceeded, line:1, col:1')
|
||||
await expect(liquid.parseAndRender('{{ array | slice: 0, 300 | join }}', { array }, { memoryLimit: 1e3 })).resolves.toBe(Array(300).fill(0).join(' '))
|
||||
const src = '{% for i in (1..1000) %}{{i}},{% endfor %}'
|
||||
const liquid = new Liquid({ outputLengthLimit: 10 })
|
||||
await expect(liquid.parseAndRender(src)).rejects.toThrow('output length limit exceeded')
|
||||
await expect(liquid.parseAndRender(src, {}, { outputLengthLimit: 5000 })).resolves.toMatch(/^1,2,3,4,5,.*,999,1000,$/)
|
||||
})
|
||||
it('should throw for too many array iteration in tags', async () => {
|
||||
const array = ['a']
|
||||
const liquid = new Liquid({ memoryLimit: 100 })
|
||||
const src = '{% for i in (1..count) %}{% assign array = array | concat: array %}{% endfor %}{{ array | join }}'
|
||||
await expect(liquid.parseAndRender(src, { array, count: 3 })).resolves.toBe('a a a a a a a a')
|
||||
await expect(liquid.parseAndRender(src, { array, count: 100 })).rejects.toThrow('memory alloc limit exceeded, line:1, col:26')
|
||||
|
||||
it('should take partials into account', async () => {
|
||||
mock({
|
||||
'/small': 'abc',
|
||||
'/large': '{% for i in (1..1000) %}{{i}}{% endfor %}'
|
||||
})
|
||||
const liquid = new Liquid({ root: '/', outputLengthLimit: 10 })
|
||||
await expect(liquid.parseAndRender('{% render "small" %}')).resolves.toBe('abc')
|
||||
await expect(liquid.parseAndRender('{% render "large" %}')).rejects.toThrow('output length limit exceeded')
|
||||
})
|
||||
it('should charge pop allocation to memoryLimit', async () => {
|
||||
const array = Array(1e3).fill(0)
|
||||
const liquid = new Liquid({ memoryLimit: 100 })
|
||||
await expect(liquid.parseAndRender('{{ array | pop | size }}', { array })).rejects.toThrow('memory alloc limit exceeded')
|
||||
|
||||
it('should enforce outputLengthLimit in sync render', () => {
|
||||
const liquid = new Liquid({ outputLengthLimit: 5 })
|
||||
expect(() => liquid.parseAndRenderSync('{% for i in (1..100) %}{{i}}{% endfor %}'))
|
||||
.toThrow('output length limit exceeded')
|
||||
})
|
||||
it('should charge sample allocation to memoryLimit', async () => {
|
||||
const array = Array(1e3).fill(0)
|
||||
const liquid = new Liquid({ memoryLimit: 100 })
|
||||
await expect(liquid.parseAndRender('{{ array | sample: 1 | size }}', { array })).rejects.toThrow('memory alloc limit exceeded')
|
||||
})
|
||||
it('should charge join by produced output size, not element count', () => {
|
||||
const array = ['a'.repeat(100), 'b'.repeat(100)]
|
||||
const liquid = new Liquid({ memoryLimit: 100 })
|
||||
expect(() => liquid.parseAndRenderSync('{{ array | join: "" }}', { array }))
|
||||
.toThrow('memory alloc limit exceeded')
|
||||
})
|
||||
it('should allow join within memoryLimit', () => {
|
||||
const array = ['a'.repeat(20), 'b'.repeat(20)]
|
||||
const liquid = new Liquid({ memoryLimit: 100 })
|
||||
expect(liquid.parseAndRenderSync('{{ array | join: "" }}', { array })).toBe('a'.repeat(20) + 'b'.repeat(20))
|
||||
})
|
||||
it('should prevent concat doubling from bypassing join memoryLimit', () => {
|
||||
const liquid = new Liquid({ memoryLimit: 1e4 })
|
||||
const src = '{%- assign a = s | split: "NOSEP" -%}' +
|
||||
'{%- assign a = a | concat: a -%}{%- assign a = a | concat: a -%}{%- assign a = a | concat: a -%}' +
|
||||
'{{ a | join: "" | size }}'
|
||||
expect(() => liquid.parseAndRenderSync(src, { s: 'a'.repeat(5000) }))
|
||||
.toThrow('memory alloc limit exceeded')
|
||||
})
|
||||
it('should charge array_to_sentence_string by produced output size', () => {
|
||||
const array = ['a'.repeat(100), 'b'.repeat(100), 'c'.repeat(100)]
|
||||
const liquid = new Liquid({ memoryLimit: 100 })
|
||||
expect(() => liquid.parseAndRenderSync('{{ array | array_to_sentence_string }}', { array }))
|
||||
.toThrow('memory alloc limit exceeded')
|
||||
})
|
||||
it('should charge json serialization of concat-doubled arrays', () => {
|
||||
const liquid = new Liquid({ memoryLimit: 1e4 })
|
||||
const src = '{%- assign a = s | split: "NOSEP" -%}' +
|
||||
'{%- assign a = a | concat: a -%}{%- assign a = a | concat: a -%}{%- assign a = a | concat: a -%}' +
|
||||
'{{ a | json | size }}'
|
||||
expect(() => liquid.parseAndRenderSync(src, { s: 'a'.repeat(5000) }))
|
||||
.toThrow('memory alloc limit exceeded')
|
||||
})
|
||||
it('should charge inspect serialization of concat-doubled arrays', () => {
|
||||
const liquid = new Liquid({ memoryLimit: 1e4 })
|
||||
const src = '{%- assign a = s | split: "NOSEP" -%}' +
|
||||
'{%- assign a = a | concat: a -%}{%- assign a = a | concat: a -%}{%- assign a = a | concat: a -%}' +
|
||||
'{{ a | inspect | size }}'
|
||||
expect(() => liquid.parseAndRenderSync(src, { s: 'a'.repeat(5000) }))
|
||||
.toThrow('memory alloc limit exceeded')
|
||||
})
|
||||
it('should charge strip_html input length to memoryLimit', () => {
|
||||
const liquid = new Liquid({ memoryLimit: 100 })
|
||||
expect(() => liquid.parseAndRenderSync('{{ s | strip_html }}', { s: 'a'.repeat(200) }))
|
||||
.toThrow('memory alloc limit exceeded')
|
||||
|
||||
it('should enforce outputLengthLimit in stream render', async () => {
|
||||
const liquid = new Liquid({ outputLengthLimit: 5 })
|
||||
const tpl = liquid.parse('{% for i in (1..100) %}{{i}}{% endfor %}')
|
||||
const stream = liquid.renderToNodeStream(tpl)
|
||||
await expect(new Promise((resolve, reject) => {
|
||||
stream.on('error', reject)
|
||||
stream.on('end', resolve)
|
||||
})).rejects.toThrow('output length limit exceeded')
|
||||
})
|
||||
})
|
||||
|
||||
describe('#maxDepth', () => {
|
||||
function chain (depth: number, tag: string) {
|
||||
const templates: Record<string, string> = {}
|
||||
for (let i = 0; i < depth; i++) {
|
||||
templates[`t${i}`] = i === depth - 1 ? 'done' : `{% ${tag} "t${i + 1}" %}`
|
||||
}
|
||||
return templates
|
||||
}
|
||||
|
||||
it('should throw when include depth exceeded', async () => {
|
||||
const liquid = new Liquid({ templates: chain(3, 'include'), maxDepth: 2 })
|
||||
await expect(liquid.parseAndRender('{% include "t0" %}')).rejects.toThrow('template depth limit exceeded')
|
||||
})
|
||||
|
||||
it('should allow include within maxDepth', async () => {
|
||||
const liquid = new Liquid({ templates: chain(2, 'include'), maxDepth: 2 })
|
||||
await expect(liquid.parseAndRender('{% include "t0" %}')).resolves.toBe('done')
|
||||
})
|
||||
|
||||
it('should throw when render depth exceeded', async () => {
|
||||
const liquid = new Liquid({ templates: chain(3, 'render'), maxDepth: 2 })
|
||||
await expect(liquid.parseAndRender('{% render "t0" %}')).rejects.toThrow('template depth limit exceeded')
|
||||
})
|
||||
|
||||
it('should allow render within maxDepth', async () => {
|
||||
const liquid = new Liquid({ templates: chain(2, 'render'), maxDepth: 2 })
|
||||
await expect(liquid.parseAndRender('{% render "t0" %}')).resolves.toBe('done')
|
||||
})
|
||||
|
||||
it('should throw when layout depth exceeded', async () => {
|
||||
const liquid = new Liquid({
|
||||
templates: {
|
||||
a: '{% layout "b" %}body-a',
|
||||
b: '{% layout "c" %}body-b',
|
||||
c: 'body-c'
|
||||
},
|
||||
maxDepth: 2
|
||||
})
|
||||
await expect(liquid.parseAndRender('{% layout "a" %}root')).rejects.toThrow('template depth limit exceeded')
|
||||
})
|
||||
|
||||
it('should allow layout within maxDepth', async () => {
|
||||
const liquid = new Liquid({
|
||||
templates: {
|
||||
a: '{% layout "b" %}body-a',
|
||||
b: 'body-b'
|
||||
},
|
||||
maxDepth: 2
|
||||
})
|
||||
await expect(liquid.parseAndRender('{% layout "a" %}root')).resolves.toBe('body-b')
|
||||
})
|
||||
|
||||
it('should not count layout none toward depth', async () => {
|
||||
const liquid = new Liquid({ maxDepth: 0 })
|
||||
await expect(liquid.parseAndRender('{% layout none %}ok')).resolves.toBe('ok')
|
||||
})
|
||||
|
||||
it('should default maxDepth to 128', async () => {
|
||||
const liquid = new Liquid({ templates: chain(128, 'include') })
|
||||
await expect(liquid.parseAndRender('{% include "t0" %}')).resolves.toBe('done')
|
||||
const overflow = new Liquid({ templates: chain(129, 'include') })
|
||||
await expect(overflow.parseAndRender('{% include "t0" %}')).rejects.toThrow('template depth limit exceeded')
|
||||
})
|
||||
|
||||
it('should enforce maxDepth in sync render', () => {
|
||||
const liquid = new Liquid({ templates: chain(3, 'include'), maxDepth: 2 })
|
||||
expect(() => liquid.parseAndRenderSync('{% include "t0" %}')).toThrow('template depth limit exceeded')
|
||||
})
|
||||
})
|
||||
|
||||
describe('strip_html ReDoS', () => {
|
||||
// Regression for O(n^2) backtracking on unclosed `<script` / `<style` openers.
|
||||
// The previous regex stalled the event loop for ~10s on 350KB of `'<script'.repeat`.
|
||||
@@ -145,11 +180,13 @@ describe('DoS related', function () {
|
||||
const payload = '<script'.repeat(50000)
|
||||
expect(liquid.parseAndRenderSync('{{ x | strip_html }}', { x: payload })).toBe(payload)
|
||||
}, 1000)
|
||||
|
||||
it('should handle many unclosed <style openers in linear time', () => {
|
||||
const liquid = new Liquid()
|
||||
const payload = '<style'.repeat(50000)
|
||||
expect(liquid.parseAndRenderSync('{{ x | strip_html }}', { x: payload })).toBe(payload)
|
||||
}, 1000)
|
||||
|
||||
it('should handle <script openers that have > but no </script> in linear time', () => {
|
||||
const liquid = new Liquid()
|
||||
const payload = '<script>foo'.repeat(50000)
|
||||
|
||||
@@ -120,6 +120,18 @@ describe('tags/for', function () {
|
||||
const html = await liquid.parseAndRender(src, scope)
|
||||
return expect(html).toBe('b')
|
||||
})
|
||||
|
||||
it('should goto else when limit empties collection', async function () {
|
||||
const src = '{%for c in alpha limit:0%}a{%else%}b{%endfor%}'
|
||||
const html = await liquid.parseAndRender(src, scope)
|
||||
return expect(html).toBe('b')
|
||||
})
|
||||
|
||||
it('should goto else when offset past end', async function () {
|
||||
const src = '{%for c in alpha offset:10%}a{%else%}b{%endfor%}'
|
||||
const html = await liquid.parseAndRender(src, scope)
|
||||
return expect(html).toBe('b')
|
||||
})
|
||||
})
|
||||
|
||||
it('should support for with forloop', async function () {
|
||||
|
||||
Reference in New Issue
Block a user