CWE-150: Improper Neutralization of Escape, Meta, or Control Sequences

Causes

• Consider a situation in which an AI agent uses LLM output based on training data from untrusted sources. Suppose an attacker is able to inject input into the training data that is used by the LLM. If a person is using a terminal emulator to interact with the LLM on the command line, if the output contains escape codes, then the emulator might act on those codes, e.g., to change color, flash text, or (in some cases) leak sensitive information or execute code. In one documented case, a terminal would handle an OSC (Operating System Command) for setting the working directory by processing a file:// URL in a way that could perform a DNS lookup for an adversary-controlled domain, leading to an information leak [REF-1526].

Remediation

• Developers should anticipate that escape, meta and control characters/sequences will be injected/removed/manipulated in the input vectors of their product. Use an appropriate combination of denylists and allowlists to ensure only valid, expected and appropriate input is processed by the system.
• Implementation: [object Object]
• Implementation: While it is risky to use dynamically-generated query strings, code, or commands that mix control and data together, sometimes it may be unavoidable. Properly quote arguments and escape any special characters within those arguments. The most conservative approach is to escape or filter all characters that do not pass an extremely strict allowlist (such as everything that is not alphanumeric or white space). If some special characters are still needed, such as white space, wrap each argument in quotes after the escaping/filtering step. Be careful of argument injection (CWE-88).
• Implementation: Inputs should be decoded and canonicalized to the application's current internal representation before being validated (CWE-180). Make sure that the application does not decode the same input twice (CWE-174). Such errors could be used to bypass allowlist validation schemes by introducing dangerous inputs after they have been checked.
• Implementation: When using output from an LLM, neutralize or strip escape codes before redirecting output to the terminal or other rendering engine that would process the codes. The neutralization could require that the character be printable and/or allowable whitespace, such as a carriage return or newline. Be deliberate about what to allow.
• Build and Compilation: When using an LLM: during tokenizer training, suppress escape codes from the tokenizer's vocabulary. Depending on context, this could be accomplished by removing the codes from input to the tokenizer, or removing the map from the string to its token ID. It is generally unlikely that this removal would adversely affect the quality or correctness of what is generated, e.g. advice requests for terminal settings to change colors.

Detection

• Automated Static Analysis: Automated static analysis, commonly referred to as Static Application Security Testing (SAST), can find some instances of this weakness by analyzing source code (or binary/compiled code) without having to execute it. Typically, this is done by building a model of data flow and control flow, then searching for potentially-vulnerable patterns that connect "sources" (origins of input) with "sinks" (destinations where the data interacts with external components, a lower layer such as the OS, etc.)

Related CWEs

• CWE-138: Improper Neutralization of Special Elements

Related CVEs

Related CVE mappings appear after CVE records are cross-indexed.

ATT&CK Relevance

ATT&CK relevance is shown only when reviewed or responsibly inferred.