CWE-1427: Improper Neutralization of Input Used for LLM… | Glexia
CWE-1427 (Improper Neutralization of Input Used for LLM Prompting) weakness overview with consequences, detection methods, mitigations, related CVEs and MITRE…
Glexia's Take · Automated analysis
CWE-1427: prompt injection
Improper Neutralization of Input Used for LLM Prompting represents a recurring weakness pattern that can create exploitable paths when design, validation, or implementation controls are missing.
Executive Impact
- Confidentiality,Integrity,Availability: Execute Unauthorized Code or Commands,Varies by Context:
- Confidentiality: Read Application Data:
- Integrity: Modify Application Data,Execute Unauthorized Code or Commands:
- Access Control: Read Application Data,Modify Application Data,Gain Privileges or Assume Identity:
Developer Pattern
CWE-1427 is the kind of defect developers can usually prevent with explicit validation, safer framework defaults, and tests that exercise hostile input or unsafe state transitions.
Automation confidence
high confidence from CWE-1427, 4.20.
Generated from the cited source records. This long-tail analysis has not been individually reviewed by a named human.
Official CWE Definition
CWE-1427: Improper Neutralization of Input Used for LLM Prompting
The product uses externally-provided data to build prompts provided to large language models (LLMs), but the way these prompts are constructed causes the LLM to fail to distinguish between user-supplied inputs and developer provided system directives.
Developer And Remediation Guidance
How teams prevent and detect this weakness
Causes
- Consider a "CWE Differentiator" application that uses an an LLM generative AI based "chatbot" to explain the difference between two weaknesses. As input, it accepts two CWE IDs, constructs a prompt string, sends the prompt to the chatbot, and prints the results. The prompt string effectively acts as a command to the chatbot component. Assume that invokeChatbot() calls the chatbot and returns the response as a string; the implementation details are not important here. To avoid XSS risks, the code ensures that the response from the chatbot is properly encoded for HTML output. If the user provides CWE-77 and CWE-78, then the resulting prompt would look like:,However, the attacker could provide malformed CWE IDs containing malicious prompts such as:,This would produce a prompt like:,Instead of providing well-formed CWE IDs, the adversary has performed a "prompt injection" attack by adding an additional prompt that was not intended by the developer. The result from the maliciously modified prompt might be something like this:,While the attack in this example is not serious, it shows the risk of unexpected results. Prompts can be constructed to steal private information, invoke unexpected agents, etc.,In this case, it might be easiest to fix the code by validating the input CWE IDs:
- , , ,Note: due to the non-deterministic nature of LLMs, eradication of dangerous behavior cannot be confirmed without thorough testing and continuous monitoring in addition to the provided prompt engineering. The previous code can be improved by modifying the system prompt to direct the system to avoid leaking the token. This could be done by appending instructions to the end of system_prompt, stating that requests for the token should be denied, and no information about the token should be included in responses:, ,To further address this weakness, the design could be changed so that secrets do not need to be included within system instructions, since any information provided to the LLM is at risk of being returned to the user.
Remediation
- Architecture and Design:
- Implementation:
- Installation,Operation:
- System Configuration:
Detection
- Dynamic Analysis with Manual Results Interpretation:
- Dynamic Analysis with Automated Results Interpretation:
- Architecture or Design Review:
Mappings
Related CVEs, CWEs, and ATT&CK context
Related CWEs
ATT&CK Relevance
ATT&CK relevance is shown only when reviewed or responsibly inferred.