Library hallucinations in AI generated code creates risk of loading malware in your programs

Code-generating Large Language Models (LLMs) are seen as powerful productivity tools for developers, but they also expose code to serious risks to the software supply chain. 

A study examining 16 popular LLMs across 576,000 code samples found that LLM models frequently recommend non-existent software packages—a phenomenon termed "package hallucinations." These "hallucinated" packages can open multiple attack vectors that malicious actors can exploit:

• Using hallucinated package names to create malicious package libraries - The research found that 58% of hallucinated packages persist across multiple iterations, with 43% repeating in all ten query attempts. Because these hallucinated packages are predictable patterns, attackers can identify the most consistently hallucinated package names and build malicious versions.
• Prompt poisoning of the LLM through repeated interactions to train it on their malicious package libraries - Attackers, especially Nation-State threat actors can invest their resources to strategically poison the training data of LLMs with the names and code examples that use their malicious packages. With this and a presence of source code and packages in librarieas, there is a feedback loop where these packages become part of the hallucination, increasing the reach and persistence of the attack.

In a rush to get assistance for some code and to meet deadlines, developers very often simply load the code and see it run as expected, never checking for anything malicious in the code.

The risk is especially high for "vibe coding"— where programmers rely heavily or exclusively on AI assistance to generate large portions of code with minimal verification. Such code is massively more likely to accept and implement package recommendations without any validation. 

Commercial models hallucinate significantly less often (5.2%) than open-source models (21.7%), but the sheer volume of unique hallucinated packages (205,474 across all tests) provides attackers with an abundance of opportunities.

This type of attack is very difficult to detect, even when the development lifecycle has security checks in it. Automated security tools that scan for known malicious code signatures may miss these attacks entirely since the package inclusion appears legitimate— the library exists, it's in the package repository and is regularly loaded. When the programming topics are very new, the automated scanners are even less successful since they don't have enough detected malware for new technologies or frameworks. 

Organizations working on their own product should consider maintaining approved package lists, implementing automated package validation systems or loading from local repositories. This should be paired with educating developers about the risks of unverified package installation—especially during rapid development cycles and vibe coding.

For individuals - even those that are vibe coding - check the imported libraries manually - check that the repositories are very active (a lot of updates) from many contributors and that they have been around for over 2-3 years in active development. Large number of contributors and users enables more frequent peer review and detection of malicious code by someone else. If the library has only one contributor, is very new or is not active, try to find another one. Check for the best library on stackoverflow, then ask the LLM to use that.