fix(date): cap strftime widths and account padding in memoryLimit (#895)

* fix(date): cap strftime widths and account padding in memoryLimit

- Clamp numeric strftime pad widths to MAX_STRFTIME_PAD (1024)
- Export estimateStrftimePaddingMemory for the date filter to charge memoryLimit
- Replace unbounded pad() concatenation loop with ch.repeat + single concat
- Add regression tests for clamping and memoryLimit on huge %width directives

Co-authored-by: Cursor <[email protected]>

* fix(date): harden strftime memory accounting and document security model

Move strftime memory charging into the same formatting path used for padding, enforce pre-allocation checks, and add regression tests for non-string date format PoCs. Add dedicated docs clarifying that memoryLimit is cooperative DoS mitigation and not strict heap isolation.

Co-authored-by: Cursor <[email protected]>

* docs(zh-cn): add security model docs for DoS limits

Add a Chinese security-model tutorial and link it from the Chinese DoS guide to clarify that memoryLimit is cooperative accounting, list uncounted custom conversion cases, and recommend avoiding fully user-defined templates in online services.

Co-authored-by: Cursor <[email protected]>

* docs: consolidate DoS docs into security-model pages

Merge DoS guidance into security-model docs in both English and Chinese, and remove the placeholder dos.md pages to avoid duplicate/redirect-only docs.

Co-authored-by: Cursor <[email protected]>

* docs: merge DoS details into security-model docs

Move the detailed parseLimit/renderLimit/memoryLimit explanations and examples into the English and Chinese security-model pages so content from the removed dos pages is preserved.

Co-authored-by: Cursor <[email protected]>

* docs: reorganize security-model structure for clarity

Restructure English and Chinese security-model docs into a consistent flow: security boundary, limits overview, per-limit details, and online service guidance.

Co-authored-by: Cursor <[email protected]>

* refactor(strftime): simplify %N width parsing logic

Use regex-backed width assumptions to simplify %N width normalization and padding memory accounting while keeping behavior equivalent.

Co-authored-by: Cursor <[email protected]>

* refactor(strftime): rely on memoryLimit for width control

Remove MAX_STRFTIME_PAD hard capping and rely on memoryLimit enforcement before padding allocation. Update strftime/date tests and security-model docs to match the new boundary and renderLimit caveats.

Co-authored-by: Cursor <[email protected]>

* fix(strftime): use add() once for padding, minimize churn

- pad(): replace per-char loop with a single add(str, ch.repeat(n)) call.
  The earlier `probe[0] === ch` heuristic was wrong when ch happened to
  equal a leading char of 'probe' (e.g. ch === 'p').
- strftime.ts: revert unrelated typing/structural refactors so the diff
  contains only the memoryLimit threading and the %N memory charge.
- docs: rewire the deleted dos.html sidebar entry to security-model.html
  (with localized labels) so the deleted page does not 404 from the
  sidebar.

Co-authored-by: Cursor <[email protected]>

---------

Co-authored-by: Cursor <[email protected]>
This commit is contained in:
Yang Jun
2026-05-10 14:35:28 +08:00
committed by GitHub
co-authored by Cursor
parent 5b9c346908
commit 3129d46dc9
11 changed files with 200 additions and 127 deletions
+1 -1
View File
@@ -19,7 +19,7 @@ tutorials:
plugins: plugins.html
operators: operators.html
truth: truthy-and-falsy.html
dos: dos.html
security_model: security-model.html
static_analysis: static-analysis.html
miscellaneous:
migration9: migrate-to-9.html
-57
View File
@@ -1,57 +0,0 @@
---
title: DoS Prevention
---
When the template or data context cannot be trusted, enabling DoS prevention options is crucial. LiquidJS provides 3 options for this purpose: `parseLimit`, `renderLimit`, and `memoryLimit`.
## TL;DR
Setting these options can largely ensure that your LiquidJS instance won't hang for extended periods or consume excessive memory. These limits are based on the available JavaScript APIs, so they are not precise hard limits but thresholds to help prevent your process from failing or hanging.
```typescript
const liquid = new Liquid({
parseLimit: 1e8, // typical size of your templates in each render
renderLimit: 1000, // limit each render to be completed in 1s
memoryLimit: 1e9, // memory available for LiquidJS (1e9 for 1GB)
})
```
When a `parse()` or `render()` cannot be completed within given resource, it throws.
## parseLimit
[parseLimit][parseLimit] restricts the size (character length) of templates parsed in each `.parse()` call, including referenced partials and layouts. Since LiquidJS parses template strings in near O(n) time, limiting total template length is usually sufficient.
A typical PC handles `1e8` (100M) characters without issues.
## renderLimit
Restricting template size alone is insufficient because dynamic loops with large counts can occur in render time. [renderLimit][renderLimit] mitigates this by limiting the time consumed by each `render()` call.
```liquid
{%- for i in (1..10000000) -%}
order: {{i}}
{%- endfor -%}
```
Render time is checked on a per-template basis (before rendering each template). In the above example, there are 2 templates in the loop: `order: ` and `{{i}}`, render time will be checked 10000000x2 times.
For time-consuming tags and filters within a single template, the process can still hang. For fully controlled rendering, consider using a process manager like [paralleljs][paralleljs].
## memoryLimit
Even with small number of templates and iterations, memory usage can grow exponentially. In the following example, memory doubles with each iteration:
```liquid
{% assign array = "1,2,3" | split: "," %}
{% for i in (1..32) %}
{% assign array = array | concat: array %}
{% endfor %}
```
[memoryLimit][memoryLimit] restricts memory-sensitive filters to prevent excessive memory allocation. As [JavaScript uses GC to manage memory](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Memory_management), `memoryLimit` limits only the total number of objects allocated by memory sensitive filters in LiquidJS thus may not reflect the actual memory footprint.
[paralleljs]: https://www.npmjs.com/package/paralleljs
[parseLimit]: /api/interfaces/LiquidOptions.html#parseLimit
[renderLimit]: /api/interfaces/LiquidOptions.html#renderLimit
[memoryLimit]: /api/interfaces/LiquidOptions.html#memoryLimit
+76
View File
@@ -0,0 +1,76 @@
---
title: Security Model
---
LiquidJS provides DoS-oriented limits (`parseLimit`, `renderLimit`, `memoryLimit`) to reduce risk. This page explains what each limit protects, and the security boundary you should assume in production.
## Security boundary
The built-in limits are cooperative safeguards, not strict runtime isolation.
- They do **not** equal process RSS/heap usage.
- They do **not** sandbox JavaScript execution.
- They should be combined with process/container limits and request timeouts for defense in depth.
## Limits at a glance
- [parseLimit][parseLimit]: limit total template size per `parse()` call.
- [renderLimit][renderLimit]: limit total render time per `render()` call.
- [memoryLimit][memoryLimit]: cooperatively limit memory-sensitive allocations counted by LiquidJS.
## Limit details
### parseLimit
[parseLimit][parseLimit] restricts the size (character length) of templates parsed in each `.parse()` call, including referenced partials and layouts. Since LiquidJS parses template strings in near O(n) time, limiting total template length is usually sufficient.
A typical PC handles `1e8` (100M) characters without issues.
### renderLimit
Restricting template size alone is insufficient because dynamic loops with large counts can occur in render time. [renderLimit][renderLimit] mitigates this by limiting the time consumed by each `render()` call.
```liquid
{%- for i in (1..10000000) -%}
order: {{i}}
{%- endfor -%}
```
Render time is checked on a per-template basis (before rendering each template). In the above example, there are 2 templates in the loop: `order: ` and `{{i}}`, render time will be checked 10000000x2 times.
`renderLimit` is not a hard CPU limiter. It is checked between template renders, so compute-intensive filters/tags/user-defined functions or deeply nested template execution between checks can still cause DoS.
### memoryLimit
`memoryLimit` only limits operations that LiquidJS explicitly counts.
- Counted: memory-sensitive LiquidJS operations that call internal memory accounting.
- Not guaranteed counted: arbitrary user object behavior such as custom `toValue()`/`toString()` chains, or other host-side code that allocates outside LiquidJS accounting points.
In other words, `memoryLimit` limits what LiquidJS counts, not every byte your process may allocate.
Even with small number of templates and iterations, memory usage can grow exponentially. In the following example, memory doubles with each iteration:
```liquid
{% assign array = "1,2,3" | split: "," %}
{% for i in (1..32) %}
{% assign array = array | concat: array %}
{% endfor %}
```
As [JavaScript uses GC to manage memory](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Memory_management), `memoryLimit` may not reflect the actual memory footprint.
## Online service guidance
If you run an online service, avoid rendering fully user-defined templates whenever possible.
- Prefer curated templates or a restricted template subset.
- If user-defined templates are required, isolate rendering (worker/process/container), enforce OS/container memory and CPU limits, and apply request rate limits.
- Treat `parseLimit`/`renderLimit`/`memoryLimit` as one layer in a broader DoS defense strategy.
For heavy single-template operations, process-level isolation is still recommended (for example with [paralleljs][paralleljs]).
[paralleljs]: https://www.npmjs.com/package/paralleljs
[parseLimit]: /api/interfaces/LiquidOptions.html#parseLimit
[renderLimit]: /api/interfaces/LiquidOptions.html#renderLimit
[memoryLimit]: /api/interfaces/LiquidOptions.html#memoryLimit
-55
View File
@@ -1,55 +0,0 @@
---
title: 防止 DoS 攻击
---
当模板或数据上下文不可信时,启用DoS预防选项至关重要。LiquidJS 提供了三个选项用于此目的:`parseLimit``renderLimit``memoryLimit`
## TL;DR
设置这些选项可以在很大程度上确保你的 LiquidJS 实例不会长时间挂起或消耗过多内存。这些限制基于可用的 JavaScript API,因此它们不是精确的硬性限制,而是确保你的进程不会失败或挂起的阈值。
```typescript
const liquid = new Liquid({
parseLimit: 1e8, // 每次渲染的模板的典型大小
renderLimit: 1000, // 每次渲染最多 1s
memoryLimit: 1e9, // LiquidJS 可用的内存(1e9 对应 1GB
})
```
## parseLimit
[parseLimit][parseLimit] 限制每次 `.parse()` 调用中解析的模板大小(字符长度),包括引用的 partials 和 layouts。由于 LiquidJS 解析模板字符串的时间复杂度接近 O(n),限制模板总长度通常就足够了。
普通电脑可以很容易处理 `1e8`100M)个字符的模板。
## renderLimit
仅限制模板大小是不够的,因为在渲染时可能会出现动态的数组和循环。[renderLimit][renderLimit] 通过限制每次 `render()` 调用的时间来缓解这些问题。
```liquid
{%- for i in (1..10000000) -%}
order: {{i}}
{%- endfor -%}
```
渲染时间是在渲染每个模板之前检查的。在上面的例子中,循环中有两个模板:`order: ``{{i}}`,因此会检查 2x10000000 次。
单个模板内的标签和过滤器仍然可能把进程挂起。要完全控制渲染过程,建议使用类似 [paralleljs][paralleljs] 的进程管理器。
## memoryLimit
即使模板和迭代次数较少,内存使用量也可能呈指数增长。在下面的示例中,内存会在每次迭代中翻倍:
```liquid
{% assign array = "1,2,3" | split: "," %}
{% for i in (1..32) %}
{% assign array = array | concat: array %}
{% endfor %}
```
[memoryLimit][memoryLimit] 限制内存敏感的过滤器,以防止过度的内存分配。由于 [JavaScript 使用 GC 来管理内存](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Memory_management)`memoryLimit` 仅限制 LiquidJS 中内存敏感过滤器分配的对象总数,因此可能无法反映实际的内存占用。
[paralleljs]: https://www.npmjs.com/package/paralleljs
[parseLimit]: /api/interfaces/LiquidOptions.html#parseLimit
[renderLimit]: /api/interfaces/LiquidOptions.html#renderLimit
[memoryLimit]: /api/interfaces/LiquidOptions.html#memoryLimit
@@ -0,0 +1,76 @@
---
title: 安全模型
---
LiquidJS 提供了面向 DoS 的限制选项(`parseLimit``renderLimit``memoryLimit`)来降低风险。本文按统一结构说明每个限制的作用范围,以及你在生产环境应采用的安全边界。
## 安全边界
内置限制是协作式防护,不是严格的运行时隔离。
- 它**不等于**进程的 RSS/heap 实际占用。
- 它**不是** JavaScript 沙箱。
- 在生产环境中应结合进程/容器资源限制和请求超时做分层防护。
## 限制速览
- [parseLimit][parseLimit]:限制每次 `parse()` 的模板总长度。
- [renderLimit][renderLimit]:限制每次 `render()` 的总渲染时间。
- [memoryLimit][memoryLimit]:协作式限制 LiquidJS 已记账的内存敏感分配。
## 限制详解
### parseLimit
[parseLimit][parseLimit] 限制每次 `.parse()` 调用中解析的模板大小(字符长度),包括引用的 partials 和 layouts。由于 LiquidJS 解析模板字符串的时间复杂度接近 O(n),限制模板总长度通常就足够了。
普通电脑可以很容易处理 `1e8`100M)个字符的模板。
### renderLimit
仅限制模板大小是不够的,因为在渲染时可能会出现动态的数组和循环。[renderLimit][renderLimit] 通过限制每次 `render()` 调用的时间来缓解这些问题。
```liquid
{%- for i in (1..10000000) -%}
order: {{i}}
{%- endfor -%}
```
渲染时间是在渲染每个模板之前检查的。在上面的例子中,循环中有两个模板:`order: ``{{i}}`,因此会检查 2x10000000 次。
`renderLimit` 不是硬性的 CPU 限制器。它是在模板渲染边界做检查,因此检查点之间的高计算开销过滤器/标签/用户自定义函数,或深层模板嵌套,仍可能导致 DoS。
### memoryLimit
`memoryLimit` 只限制 LiquidJS 显式记账到的操作。
- 会被统计:LiquidJS 内部调用了内存记账逻辑的内存敏感操作。
- 不保证被统计:任意用户对象行为(例如自定义 `toValue()` / `toString()` 链)以及其他发生在 LiquidJS 记账点之外的宿主侧分配。
换句话说,`memoryLimit` 限制的是 LiquidJS 的“已记账分配”,而不是进程里每一个字节的分配。
即使模板和迭代次数较少,内存使用量也可能呈指数增长。在下面的示例中,内存会在每次迭代中翻倍:
```liquid
{% assign array = "1,2,3" | split: "," %}
{% for i in (1..32) %}
{% assign array = array | concat: array %}
{% endfor %}
```
由于 [JavaScript 使用 GC 来管理内存](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Memory_management)`memoryLimit` 可能无法反映实际的内存占用。
## 在线服务建议
如果你运行在线服务,建议尽量避免渲染完全由用户定义的模板。
- 优先使用受控模板或受限模板子集。
- 如果必须支持用户自定义模板,请隔离渲染(worker/进程/容器),并同时配置操作系统或容器级的内存/CPU 限额与请求限流。
-`parseLimit` / `renderLimit` / `memoryLimit` 视为 DoS 防护体系中的一层,而不是唯一防线。
对于单个模板中的重型操作,仍建议使用进程级隔离(例如 [paralleljs][paralleljs])。
[paralleljs]: https://www.npmjs.com/package/paralleljs
[parseLimit]: /api/interfaces/LiquidOptions.html#parseLimit
[renderLimit]: /api/interfaces/LiquidOptions.html#renderLimit
[memoryLimit]: /api/interfaces/LiquidOptions.html#memoryLimit
+1 -1
View File
@@ -51,7 +51,7 @@ sidebar:
plugins: Plugins
operators: Operators
truth: Truthy and Falsy
dos: DoS
security_model: Security Model
static_analysis: Static Analysis
miscellaneous: Miscellaneous
+1 -1
View File
@@ -51,7 +51,7 @@ sidebar:
plugins: 插件
operators: 运算符
truth: 真和假
dos: DoS
security_model: 安全模型
static_analysis: 静态分析
miscellaneous: 其他
+7 -5
View File
@@ -3,13 +3,14 @@ import { FilterImpl } from '../template'
import { NormalizedFullOptions } from '../liquid-options'
export function date (this: FilterImpl, v: string | Date, format?: string, timezoneOffset?: number | string) {
const size = ((v as string)?.length ?? 0) + (format?.length ?? 0) + ((timezoneOffset as string)?.length ?? 0)
const size = ((v as string)?.length ?? 0) + ((timezoneOffset as string)?.length ?? 0)
this.context.memoryLimit.use(size)
const date = parseDate(v, this.context.opts, timezoneOffset)
if (!date) return v
format = toValue(format)
format = isNil(format) ? this.context.opts.dateFormat : stringify(format)
return strftime(date, format)
this.context.memoryLimit.use(format.length)
return strftime(date, format, this.context.memoryLimit)
}
export function date_to_xmlschema (this: FilterImpl, v: string | Date) {
@@ -31,13 +32,14 @@ export function date_to_long_string (this: FilterImpl, v: string | Date, type?:
function stringify_date (this: FilterImpl, v: string | Date, month_type: string, type?: string, style?: string) {
const date = parseDate(v, this.context.opts)
if (!date) return v
const ml = this.context.memoryLimit
if (type === 'ordinal') {
const d = date.getDate()
return style === 'US'
? strftime(date, `${month_type} ${d}%q, %Y`)
: strftime(date, `${d}%q ${month_type} %Y`)
? strftime(date, `${month_type} ${d}%q, %Y`, ml)
: strftime(date, `${d}%q ${month_type} %Y`, ml)
}
return strftime(date, `%d ${month_type} %Y`)
return strftime(date, `%d ${month_type} %Y`, ml)
}
function parseDate (v: string | Date, opts: NormalizedFullOptions, timezoneOffset?: number | string): LiquidDate | undefined {
+8 -4
View File
@@ -1,11 +1,13 @@
import { changeCase, padStart, padEnd } from './underscore'
import { LiquidDate } from './liquid-date'
import type { Limiter } from './limiter'
const rFormat = /%([-_0^#:]+)?(\d+)?([EO])?(.)/
interface FormatOptions {
flags: object;
width?: string;
modifier?: string;
memoryLimit?: Pick<Limiter, 'use'>;
}
// prototype extensions
@@ -95,6 +97,7 @@ const formatCodes = {
N: (d: LiquidDate, opts: FormatOptions) => {
const width = Number(opts.width) || 9
const str = String(d.getMilliseconds()).slice(0, width)
opts.memoryLimit?.use(width - str.length)
return padEnd(str, width, '0')
},
p: (d: LiquidDate) => (d.getHours() < 12 ? 'AM' : 'PM'),
@@ -118,25 +121,25 @@ const formatCodes = {
};
(formatCodes as any).h = formatCodes.b
export function strftime (d: LiquidDate, formatStr: string) {
export function strftime (d: LiquidDate, formatStr: string, memoryLimit?: Pick<Limiter, 'use'>) {
let output = ''
let remaining = formatStr
let match
while ((match = rFormat.exec(remaining))) {
output += remaining.slice(0, match.index)
remaining = remaining.slice(match.index + match[0].length)
output += format(d, match)
output += format(d, match, memoryLimit)
}
return output + remaining
}
function format (d: LiquidDate, match: RegExpExecArray) {
function format (d: LiquidDate, match: RegExpExecArray, memoryLimit?: Pick<Limiter, 'use'>) {
const [input, flagStr = '', width, modifier, conversion] = match
const convert = formatCodes[conversion]
if (!convert) return input
const flags = {}
for (const flag of flagStr) flags[flag] = true
let ret = String(convert(d, { flags, width, modifier }))
let ret = String(convert(d, { flags, width, modifier, memoryLimit }))
let padChar = padSpaceChars.has(conversion) ? ' ' : '0'
let padWidth = width || padWidths[conversion] || 0
if (flags['^']) ret = ret.toUpperCase()
@@ -144,5 +147,6 @@ function format (d: LiquidDate, match: RegExpExecArray) {
if (flags['_']) padChar = ' '
else if (flags['0']) padChar = '0'
if (flags['-']) padWidth = 0
memoryLimit?.use(Number(padWidth) - ret.length)
return padStart(ret, padWidth, padChar)
}
+3 -3
View File
@@ -152,9 +152,9 @@ export function padEnd (str: any, length: number, ch = ' ') {
export function pad (str: any, length: number, ch: string, add: (str: string, ch: string) => string) {
str = String(str)
let n = length - str.length
while (n-- > 0) str = add(str, ch)
return str
const n = length - str.length
if (n <= 0) return str
return add(str, ch.repeat(n))
}
export function identify<T> (val: T): T {
+27
View File
@@ -204,6 +204,33 @@ describe('filters/date', function () {
return test('{{ "1990-12-31T23:00:00Z" | date: "%Y-%m-%dT%H:%M:%S" }}', '1991-01-01T04:30:00', undefined, optsWithDateFormat)
})
})
describe('strftime width / memoryLimit', () => {
it('should charge memoryLimit for huge numeric strftime widths', () => {
const liquid = new Liquid({ memoryLimit: 500 })
expect(() => liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: '%5000000d' }))
.toThrow('memory alloc limit exceeded')
})
it('should charge memoryLimit for array format PoC', () => {
const liquid = new Liquid({ memoryLimit: 50, renderLimit: 1e9 })
expect(() => liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: ['a'.repeat(2000000)] }))
.toThrow('memory alloc limit exceeded')
})
it('should charge memoryLimit for object toString format PoC', () => {
const liquid = new Liquid({ memoryLimit: 50, renderLimit: 1e9 })
const huge = 'a'.repeat(2000000)
const f = { toString: () => huge }
expect(() => liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f }))
.toThrow('memory alloc limit exceeded')
})
it('should honor numeric strftime pad width when memoryLimit allows', () => {
const liquid = new Liquid({ memoryLimit: 1e7 })
const out = liquid.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: '%5000d' })
expect(out.length).toBe(5000)
const tight = new Liquid({ memoryLimit: 100 })
expect(() => tight.parseAndRenderSync('{{ d | date: f }}', { d: 'now', f: '%5000d' }))
.toThrow('memory alloc limit exceeded')
})
})
})
describe('filters/date_to_xmlschema', function () {
const liquid = new Liquid()