Two received CVEs. The affected SHA-3 path shipped in 13 public OP-TEE releases, the PKCS#1 v1.5 path in 25, and the SE050 RSA NOPAD path in at least 18. The oldest issue had been present for more than six years. SecMate found all three automatically using locally deployable gpt-oss-20b and gpt-oss-120b models.

OP-TEE treated the third finding as a bug because our report did not demonstrate the initial compromise needed to establish a complete compromised-TA attack chain.

Core memory corruption matters because OP-TEE is responsible for keeping secrets isolated from Linux or Android.

Why OP-TEE matters

OP-TEE stands for Open Portable Trusted Execution Environment. On Arm TrustZone systems, it runs security-sensitive code and data separately from the normal operating system, usually Linux or Android [1].

That normal operating system is called the Rich Execution Environment (REE). OP-TEE runs on the protected side of the TrustZone boundary. The separation is meant to keep sensitive data, code, and execution isolated even if software in the normal world is compromised. The GlobalPlatform TEE architecture defines integrity and data confidentiality as core security properties. It also defines mechanisms for Trusted Application code confidentiality and isolation from the REE [2]. The protected properties include:

• confidentiality of sensitive data and keys
• integrity of code and data during execution
• confidentiality of Trusted Application code when provided by the implementation
• isolation from the normal operating system

OP-TEE is not limited to research platforms. TrustedFirmware.org describes it as widely used and deployed, and its official platform list includes maintained targets from STMicroelectronics, NXP, Texas Instruments, Renesas, Rockchip, and AMD/Xilinx [3] [4]. Other vendors use different secure environments, including Qualcomm TEE, Samsung TEEGRIS, and Apple’s Secure Enclave with sepOS [5] [6] [7]. A device with a TEE therefore does not necessarily run OP-TEE.

What Arm Trusted Firmware and OP-TEE actually are

Arm application processors implement two security states, the Normal World and the Secure World, selected by the NS bit and enforced in hardware by TrustZone. The idea is to keep a small, high-assurance environment isolated from a large, complex Rich OS such as Linux. Inside each world, the processor still has privilege levels, expressed as exception levels.

Arm Trusted Firmware (TF-A) [9] is a reference implementation of the firmware that commonly runs at the most privileged level, EL3. Its BL31 runtime acts as the Secure Monitor in this deployment model. It handles SMC (Secure Monitor Call) instructions, PSCI power management, world switches, and dispatch to a Trusted OS.

OP-TEE is the Trusted OS in the deployment shown below. It commonly runs as the BL32 secure payload at S-EL1, with Trusted Applications (TAs) at S-EL0. OP-TEE and TF-A are sibling TrustedFirmware.org projects. When deployed together, TF-A provides the monitor and dispatches calls to OP-TEE. From the Normal World, an application uses the GlobalPlatform TEE Client API. The call crosses into EL3 via SMC, and the monitor hands it to OP-TEE.

The boundary that matters here is inside the secure world. Trusted Applications run at S-EL0. The OP-TEE core runs above them at S-EL1 and handles privileged operations such as key storage, cryptographic services, and secure peripherals. A TA can call the core through syscalls, but it must never be able to write core memory.

These three issues break that boundary. The SHA-3 issue starts with an off-by-one. The other two start with unchecked size relationships. In all three, an unsigned subtraction wraps. The resulting size or offset makes memset or memcpy write outside its buffer.

The three findings

CVE-2026-40257 requires the Arm v8.2 SHA-3 acceleration path. CVE-2026-33662 affects builds registering RSA acceleration through CAAM, HiSilicon, Versal, or SE050, as listed in the OP-TEE advisory. The third issue requires the NXP SE050 RSA driver. These paths are platform-specific, but they ship on production hardware.

Scope and impact

OP-TEE’s published security scope excludes a deliberately malicious TA signed with a trusted TA key because the key holder is already part of the trusted computing base [8].

An attacker may first compromise a legitimate, signed TA through one or more separate vulnerabilities and then invoke the SE050 bug through that TA. In this chain, the attacker does not need the TA’s signing key, and the TA retains its valid signature. Our report demonstrated the resulting controlled write from S-EL0 into S-EL1 core memory, but it did not demonstrate the initial TA compromise. OP-TEE therefore treated the finding as a bug rather than a complete vulnerability report.

Findings 1 and 2 do not have that prerequisite. Valid hash input triggers the first. A signing operation with a sufficiently small RSA modulus reaches the second.

All three are heap writes. Stack protection does not help. ASLR and non-executable data make targeted exploitation harder, but the minimum demonstrated impact for findings 1 and 2 is a secure-world crash. Finding 3 gives the attacker control over both the bytes written and their position relative to the allocation.

Finding 1: SHA-3 accelerated finalize heap overflow (CVE-2026-40257)

OP-TEE has two SHA-3 implementations. The default software implementation in sha3.c accumulates input word by word and is not affected by this issue. The other, sha3_accel.c, is compiled in when a platform enables the Arm v8.2 SHA-3 crypto extensions (CFG_CRYPTO_WITH_CE82=y) and buffers input in blocks. It is the accelerated path that is broken.

The Keccak sponge processes input in blocks of block_size bytes (136 for SHA3-256). sha3_process() is supposed to: finish any partial block, run full blocks through the hardware, then buffer the remainder. The bug is an off-by-one in step two:

 1/* sha3_accel.c:151 - strict greater-than is the bug */
 2if (inlen > block_size) {
 3    block_count = inlen / block_size;
 4    crypto_accel_sha3_compress(state, in, block_count, digest_size);
 5    in += block_count * block_size;
 6    inlen -= block_count * block_size;
 7}
 8
 9/* lines 159-160 - reached when inlen == block_size */
10memcpy(md->sha3.sb + md->sha3.byte_index, in, inlen);
11md->sha3.byte_index += inlen;            /* byte_index becomes block_size */

When a TA feeds exactly one block (136 bytes) with an empty buffer, inlen > block_size is false, the hardware compression is skipped, and the whole block is copied into sb with byte_index left equal to block_size. The finalize functions assume the invariant byte_index < block_size. It no longer holds:

1/* sha3_done(), lines 194-195 */
2buf[md->sha3.byte_index++] = 0x06;                              /* sb[136], index -> 137 */
3memset(buf + md->sha3.byte_index, 0, block_size - md->sha3.byte_index);
4/* block_size(136) - byte_index(137) underflows to 0xFFFFFFFF */

block_size - byte_index is 136 - 137 in unsigned arithmetic, i.e. 0xFFFFFFFF. The memset then writes roughly 4 GB of zeroes starting inside the hash state on the S-EL1 heap. sha3_shake_done() has the identical pattern for SHAKE-128/256.

Attack chain and impact. The trigger is two ordinary GlobalPlatform calls: TEE_DigestUpdate(op, data, 136) then TEE_DigestDoFinal(...). No special key material, permissions, or object types are required. The input is valid because hashing a block-sized buffer is normal usage. A perfectly honest TA can therefore crash the secure world without intending to. The write consists of zeroes and spans roughly 4 GB. In practice, it crashes the OP-TEE core and takes down its TAs, secure services, and Normal-World clients. The write offers too little control for a realistic code-execution primitive. The advisory rates it Moderate, CVSS 5.5 (AV:L/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H).

We reproduced it with an ASAN harness that #includes the real sha3_accel.c, stubs the hardware compress function, and runs the two-call sequence. ASAN reports a heap-buffer-overflow with the stack pointing straight at sha3_accel.c:195.

Introduced by commit 1478437e65c4 in March 2023, this vulnerability shipped in 13 public OP-TEE releases, from 3.21.0 through 4.10.0. It remained in the secure-world core for nearly three years before SecMate’s automated analysis identified it, and for more than three years before the fix was merged.

The fix. Change the comparison so a full block is actually compressed, and add a defensive assertion to the finalize paths [10]:

1-	if (inlen > block_size) {
2+	if (inlen >= block_size) {
3 		block_count = inlen / block_size;
4 		crypto_accel_sha3_compress(state, in, block_count, digest_size);
5...
6+	LTC_ARGCHK(md->sha3.byte_index < block_size);
7+
8 	buf[md->sha3.byte_index++] = 0x06;
9 	memset(buf + md->sha3.byte_index, 0, block_size - md->sha3.byte_index);

With >=, an exact-block input is consumed by the hardware compress, inlen drops to zero, and byte_index stays at zero. The added LTC_ARGCHK returns CRYPT_INVALID_ARG if the invariant is violated again. The advisory identifies OP-TEE 4.11.0 and later as patched. The workaround is CFG_CRYPTO_WITH_CE82=n.

Finding 2: RSASSA PKCS#1 v1.5 encode underflow (CVE-2026-33662)

This one lives in the generic crypto-driver framework (CFG_CRYPTO_DRIVER=y), which platforms with RSA acceleration use to dispatch signature operations: NXP CAAM (plat-imx, plat-ls), Xilinx Versal, HiSilicon, and SE050 builds.

PKCS#1 v1.5 signature encoding pads the DigestInfo structure up to the modulus size with 0xFF bytes. emsa_pkcs1_v1_5_encode() computes how many padding bytes it needs by subtracting from the modulus size, with no lower-bound check:

1/* rsassa.c:48-50 */
2ps_size  = ssa_data->key.n_size - 3;
3ps_size -= ssa_data->digest_size;
4ps_size -= 10 + hash_oid->asn1_length;
5...
6memset(buf, UINT8_MAX, ps_size);          /* line 75 */

With a 256-bit modulus (32 bytes) and SHA-256 (32-byte digest, 9-byte OID), the arithmetic is 32 - 3 - 32 - (10 + 9), which underflows size_t to roughly SIZE_MAX. The memset then writes 0xFF across the entire S-EL1 heap until OP-TEE faults. RFC 3447 Section 9.2 mandates the missing size check [12]. The same file’s noasn1 and PSS paths already perform their own size validation. This path did not.

Attack chain and impact. A TA uses a sufficiently small RSA modulus and calls TEE_AsymmetricSignDigest(TEE_ALG_RSASSA_PKCS1_V1_5_SHA256, ...). The vulnerable core path is reached through that signing operation. Whether an external attacker can reach it depends on how the product exposes the TA to Normal-World clients. Like finding 1, this is a blunt overwrite. It gives an attacker a reliable TEE-wide denial of service, not controlled execution. The OP-TEE advisory rates the issue High, CVSS 7.5 (AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H).

SecMate identified this vulnerability independently before its public disclosure. OP-TEE had already found the issue internally, so the project handled our report as a duplicate and the CVE process was already underway.

This vulnerability was introduced by commit f5a70e3efb80 in February 2020. It remained in OP-TEE for more than six years and shipped in 25 public releases before the fix was merged in April 2026. Public Git history places the first affected release at 3.9.0, although the GitHub advisory lists versions from 3.8.0.

The fix. The maintainers replaced the chained subtraction with checked subtraction macros that bail out on underflow [11]:

1-	ps_size = ssa_data->key.n_size - 3;
2-	ps_size -= ssa_data->digest_size;
3-	ps_size -= 10 + hash_oid->asn1_length;
4+	if (SUB_OVERFLOW(ssa_data->key.n_size, 3, &ps_size) ||
5+	    SUB_OVERFLOW(ps_size, ssa_data->digest_size, &ps_size) ||
6+	    SUB_OVERFLOW(ps_size, 10 + hash_oid->asn1_length, &ps_size))
7+		return TEE_ERROR_BAD_PARAMETERS;

An encoding size that would make one of the subtractions wrap now returns TEE_ERROR_BAD_PARAMETERS. The patch prevents the memory corruption. It does not enforce the separate PKCS#1 v1.5 requirement that the padding string contain at least eight bytes. The advisory identifies OP-TEE 4.11 and later as patched. The workaround is to disable RSA acceleration.

Finding 3: SE050 RSA NOPAD controlled heap underflow

The third finding gives an attacker more control than the two CVEs, but OP-TEE treated it as a bug. It is in the NXP SE050 secure-element driver (CFG_NXP_SE05X=y), used on i.MX6/7/8 platforms that pair the SoC with an SE050 element over I²C.

decrypt_nopad() and encrypt_nopad() left-pad the caller’s input to the modulus size by computing a destination offset with, again, an unchecked unsigned subtraction:

1/* se050/core/rsa.c:356 (and :418) */
2rsa_len = crypto_bignum_num_bytes(key->n);     /* e.g. 256 for a 2048-bit key */
3memcpy(buf + rsa_len - src_len, src, src_len);

Neither the generic dispatch layer (crypto_acipher_rsanopad_decrypt, which passes cipher_len straight through) nor the SE050 driver checks that src_len <= rsa_len. If a TA supplies input larger than the modulus, the subtraction computes a destination before the allocation. Forming that pointer is undefined behavior in C. On the affected targets, it produces the following relative out-of-bounds write:

rsa_len = 256, src_len = 300  ->  256 - 300 = (size_t)-44
memcpy(buf - 44, src, 300)    ->  44 attacker-chosen bytes written BEFORE buf

buf comes from OP-TEE’s BGET-backed mempool. An allocated block’s prevfree and bsize metadata sit immediately before its payload, so even a short underflow can overwrite allocator state that mempool_free() later consumes.

Why this one is different. Findings 1 and 2 overwrite with a fixed byte over an unbounded range, which is useful for a crash but offers little control. This is a controlled heap-underflow write primitive: the attacker chooses the underflow distance (src_len - rsa_len) and the bytes copied from src. The primitive can corrupt allocator metadata or an adjacent core object before the affected allocation is freed. We did not build a complete S-EL0-to-S-EL1 exploit, but this gives an attacker far more control than findings 1 and 2, especially on constrained 32-bit targets. We confirmed the out-of-bounds write with an ASAN harness around the real rsa.c.

The vulnerable RSA NOPAD implementation entered the SE050 driver in commit 462192731c92 in January 2022. It shipped in at least 18 public releases and was still present in upstream commit 611fbaf34150 on July 13, 2026, four and a half years later.

Why it was treated as a bug. Reaching this code requires control of a TA that calls TEE_AsymmetricDecrypt(TEE_ALG_RSA_NOPAD, ...) with oversized input. That control could come from a deliberately malicious TA or from compromising a legitimate, signed TA through another vulnerability. The former is outside OP-TEE’s threat model; the latter does not require the attacker to possess a TA signing key and can form the first stage of an exploit chain. Our report demonstrated the controlled S-EL1 write but not the initial TA compromise, so OP-TEE treated the finding as a bug.

The fix is to validate src_len <= rsa_len and msg_len <= n_size in the generic dispatch layer, then repeat the check in the driver before memcpy. The HiSilicon HPRE driver already validates input sizes independently and is not affected.

Three old flaws, found automatically

These issues entered OP-TEE in 2020, 2022, and 2023. They survived across separate accelerated implementations and hardware drivers until SecMate’s automated analysis found them.

Every issue in our public disclosure record [13] was found by the same engine. SecMate runs with locally deployable models, so vulnerability research can stay inside the operator’s environment instead of sending source code to an external model service.

Disclosure timeline

References

• [1] OP-TEE. “About OP-TEE.” OP-TEE documentation. Documentation

• [2] GlobalPlatform. “TEE System Architecture v1.3.” Specification

• [3] OP-TEE. “Platforms supported.” OP-TEE documentation. Documentation

• [4] TrustedFirmware.org. “OP-TEE moving into Trusted Firmware.” Project announcement

• [5] Qualcomm. “Qualcomm Linux System Software Architecture.” Documentation

• [6] Samsung. “Samsung TEEGRIS.” Samsung Developer. Documentation

• [7] Apple. “The Secure Enclave.” Apple Platform Security. Documentation

• [8] OP-TEE. “Security scope.” OP-TEE documentation. Documentation

• [9] TrustedFirmware.org. “Firmware Design.” Trusted Firmware-A documentation. Documentation

• [10] OP-TEE. “SHA-3 accelerated finalize heap overflow.” GHSA-75x4-j8p9-55qv / CVE-2026-40257. Advisory

• [11] OP-TEE. “RSASSA EMSA-PKCS1-v1_5 underflow in emsa_pkcs1_v1_5_encode().” GHSA-4cf8-v5g3-73gr / CVE-2026-33662. Advisory

• [12] IETF. “PKCS #1: RSA Cryptography Specifications Version 2.1.” RFC 3447. Specification

• [13] SecMate. “Disclosures.” Disclosure page

The SecMate Team

In the v6.1.0 FSP codebase we analyzed at commit ed247f90fb9bc390ebd36f1a0b7847b0ff44f322 [3], we found issues in RSA verification, the hardware-accelerated AES-XTS path, GCM decrypt handling, and the RMAC receive path. These are implementation flaws in FSP integration code around hardware-backed cryptography and networking. They are not flaws in AES, GCM, CCM, XTS, or RSA as algorithms.

Advisory Map

The AES-XTS advisory covers two findings, one in the encrypt finalization path and one in the decrypt finalization path.

Affected Builds and Use Cases

SecMate’s published disclosure advisories are available on the SecMate disclosures page [4]. The table above links both the SecMate advisory and the corresponding Renesas public advisory for each vulnerability group.

These cryptography-related findings do not apply to every RA build. In FSP, the accelerated AES and AEAD paths are compiled for devices with SCE5, SCE5B, SCE7, SCE9, or RSIP support, and the XTS path is excluded on RA2 parts.

In downstream products, these APIs may be used for signature verification, authenticated decryption, encrypted storage, firmware update, or other security-sensitive processing. Exposure therefore depends on whether untrusted inputs can reach those wrappers.

The RMAC finding is separate from the cryptographic code paths. It affects products using the Ethernet receive API if application code provides a buffer smaller than the received frame.

Renesas PSIRT acknowledged the reports and published public advisories on GitHub.

Recommendations

• If you use affected RA builds, review the Renesas advisories and update to FSP v6.3.0 or later.
• In downstream products, check whether untrusted inputs can reach FSP wrappers used for signature verification, authenticated decryption, encrypted storage, or firmware update handling.
• Add negative tests for malformed signatures, AEAD metadata, XTS lengths, and oversized Ethernet frames.
• If you use r_rmac, verify that receive buffers are sized for the largest frame your product accepts.

Timeline

Takeaway

The common thread is not broken cryptography. It is memory safety in the FSP code that exposes cryptographic and Ethernet functionality to RA applications. Hardware-backed cryptography still depends on the surrounding integration code.

For the full list of our disclosures, see our disclosure page [4]. If you are building embedded systems and want to find vulnerabilities before attackers do, reach out.

References

• [1] Renesas. “Renesas Flexible Software Package.” GitHub. Repository

• [2] Renesas. “Renesas Flexible Software Package Documentation.” Documentation

• [3] Renesas. “FSP v6.1.0 analyzed commit” (ed247f90fb9bc390ebd36f1a0b7847b0ff44f322). Commit

• [4] SecMate. “Disclosures.” Disclosure page

• [5] Renesas. “FSP v6.3.0.” GitHub Releases. Release

The SecMate Team

About SecMate

SecMate combines static analysis with AI to find security vulnerabilities in source code, specifically in embedded devices and low-level code.

So far, we have reported more than 60 vulnerabilities across embedded and systems projects, most still under coordinated disclosure.

Today, we are sharing two of our findings in libcoap (the first two CVEs uncovered with SecMate), a widely-used CoAP implementation for IoT devices.

What is libcoap?

libcoap [1] is a C implementation of the Constrained Application Protocol (CoAP), standardized as RFC 7252 [2]. Think of CoAP as “HTTP for IoT”: a lightweight protocol for devices that can’t afford the overhead of full HTTP/TCP stacks.

The library is mature and widely deployed:

• Supports multiple TLS backends (OpenSSL, GnuTLS, Mbed TLS, wolfSSL)
• Runs on everything from Linux servers to ESP32 microcontrollers
• Used in smart home devices, industrial sensors, and critical infrastructure

Here is the catch: the devices running libcoap are often constrained. Limited memory, no MMU, no ASLR, no stack canaries. When you find a memory-safety issue in this context, the impact is amplified.

Vulnerability #1: Static Buffer Overflow in Address Resolution (CVE-2025-34468 [3])

Location: coap_resolve_address_info() in src/coap_address.c

The Bug

This vulnerability affects libcoap servers running in proxy mode. The vulnerable function resolves hostnames for CoAP proxy requests, copying the hostname into a fixed 256-byte static buffer without checking the length:

1static char addrstr[256];  // Fixed-size static buffer
2memset(addrstr, 0, sizeof(addrstr));
3if (address && address->length)
4    memcpy(addrstr, address->s, address->length);  // No bounds check!
5else
6    memcpy(addrstr, "localhost", 9);
7getaddrinfo(addrstr, NULL, &hints, &res);

The address->length field comes directly from parsing CoAP options. An attacker controls this value.

The Attack

When a CoAP server runs in proxy mode, it accepts Proxy-Uri options that specify where to forward requests. The hostname from this URI flows directly into the vulnerable memcpy():

1. A CoAP request arrives with a Proxy-Uri option containing an oversized hostname (>256 bytes)
2. The proxy extracts the hostname and passes it to coap_resolve_address_info()
3. The function calls memcpy(addrstr, address->s, address->length) without bounds checking
4. The fixed-size buffer overflows, corrupting adjacent memory
5. The server crashes or misbehaves; the exact impact depends on layout and mitigations

Impact

On a standard Linux system, this bug is likely to crash the server (Denial of Service). On embedded devices without memory protection, memory corruption can have broader consequences, but practical exploitation depends on layout and mitigations. One UDP packet, no authentication required when proxy mode is enabled.

Vulnerability #2: Out-of-Bounds Read in OSCORE Parsing (CVE-2025-59391 [4])

Location: get_split_entry() in src/coap_oscore.c

The Bug

OSCORE (Object Security for Constrained RESTful Environments) is a security layer for CoAP. libcoap parses OSCORE configuration files that include boolean fields. The parsing code uses memcmp() with a user-controlled length:

1case COAP_ENC_BOOL:
2    if (memcmp("true", begin, end - begin) == 0)
3        value->u.value_int = 1;
4    else if (memcmp("false", begin, end - begin) == 0)
5        value->u.value_int = 0;

The problem: end - begin comes from the configuration file. If an attacker provides a value like trueAAAAAAAA... (thousands of characters), the memcmp() reads past the static string "true" into adjacent memory.

Impact

This issue requires the ability to provide malicious OSCORE configuration input, which is typically a local or trusted provisioning path rather than a network-facing vector. The realistic outcome is a crash or parse failure on malformed configs.

Theoretical Oracle Attack

In theory, if a local attacker could repeatedly provide different config files and observe success/failure signals from the parser, they could use this as an oracle to infer adjacent .rodata bytes one at a time. The diagram below illustrates what such an attack would look like:

However, this attack is impractical: OSCORE configuration is loaded from local files, so an attacker would need repeated local access to provide different config files and observe results. At that point, they likely have access that makes this leak scenario moot.

Why This Bug is Easy to Miss

This vulnerability is subtle: the out-of-bounds read in memcmp() may not crash or produce obvious symptoms during normal operation. Spotting it requires reasoning about how data flows through multiple operations, something traditional static analysis tools typically miss.

The Embedded Security Challenge

Both vulnerabilities share a common theme: they are worse on the devices libcoap is designed for.

Desktop and server systems have decades of hardening:

• ASLR randomizes memory layout
• Stack canaries detect buffer overflows
• DEP/NX prevents code execution from data segments
• Sandboxing limits blast radius

Constrained embedded devices often have none of these. A microcontroller running FreeRTOS or bare metal has:

• Flat, predictable memory layout
• No memory protection unit (MPU) or it is disabled for performance
• No operating system to enforce isolation
• Direct hardware access for any code that runs

This isn’t a criticism of libcoap. It is the reality of their target environment. But it means bugs like these deserve extra scrutiny.

Key Insight

When auditing code for constrained devices, assume the worst-case exploitation scenario. Mitigations you take for granted on desktop systems simply don't exist.

Disclosure and Fix

We reported both vulnerabilities to the libcoap maintainers through coordinated disclosure. The team responded quickly and professionally, and fixes are now available in version 4.3.5a.

Disclosure Timeline

We want to thank the libcoap team, particularly Jon Shallow, for their responsiveness in addressing these issues. The maintainer responded on the same day for both reports. Maintaining open-source security infrastructure is often thankless work, and their quick turnaround made the ecosystem safer.

What’s Next

These two CVEs are just a sample of what SecMate has found. We’re continuing to analyze embedded codebases and report vulnerabilities through coordinated disclosure.

If you’re curious about our other findings, check out our disclosure page. And if you’re building embedded systems and want to find vulnerabilities before attackers do, reach out.

References

• [1] libcoap. “libcoap - C-Implementation of CoAP” GitHub. Repository

• [2] IETF. “RFC 7252 - The Constrained Application Protocol (CoAP)” IETF Datatracker, June 2014. RFC

• [3] NVD. “CVE-2025-34468 - Static Buffer Overflow in libcoap” National Vulnerability Database. CVE

• [4] NVD. “CVE-2025-59391 - Out-of-Bounds Read in libcoap” National Vulnerability Database. CVE

The SecMate Team

We have been using AI coding assistants daily. Like many developers, we have experienced the productivity boost firsthand. But as security researchers who have spent years analyzing security issues in production systems, we wondered: what is the security cost of this productivity gain?

Introduction

In early 2025, OpenAI co-founder Andrej Karpathy defined the term “Vibe Coding” to describe a new improvisational style of software development [1].

The idea is simple: a developer and an AI “pair program” in a rapid, conversational loop, trusting the AI’s suggestions to stay in a creative flow. Andrej Karpathy admitted to accepting changes without reading the diffs [2].

This new paradigm is powerful, but it raises a critical question: what are the security implications of vibe coding?

In this series of blog posts, we put that question to the test. We benchmarked multiple AI coding agents (including Anthropic’s Claude Code [3], Google’s Gemini CLI [4], OpenAI’s codex [5], and the open source coding agent Aider [6]) against a set of distinct programming tasks to see what kind of security flaws they might introduce. Our first study reveals some interesting patterns.

This first post breaks down our benchmark setup and initial results from simple programming challenges.

Let’s examine what these AI-assisted development practices produce in terms of security.

Benchmark Setup

To understand the security implications beyond anecdotal evidence, we need concrete data.

We designed a systematic benchmark that reveals how AI coding agents handle security-sensitive programming tasks in practice.

We selected 6 distinct programming tasks, each presenting unique security challenges that are usually hotspots for security flaws

Agents Configurations

Most coding agents support various models and configurations. Unlike related work (see Related Work section), we are testing a combination of models, system prompts and agentic workflows (as opposed to a single model).

To keep things simple, we use the default configuration for each agent which uses the respective models (i.e., ChatGPT for codex, Gemini for Gemini CLI, and Claude for claude-code).

However, aider is a special case, and does not have its own model. This is why we run two benchmarks for aider with the two most commonly used models: OpenAI’s o3 and Claude Sonnet 4.

Experimental Parameters

To ensure our results are significant, we generate and analyze 240 unique code samples. Here is the breakdown:

• Languages: C, Java, Python, and Rust (4 languages × 6 tasks = 24 unique prompts)
• Agents: Claude Code [3], Gemini CLI [4], OpenAI Codex [5], aider-o3, and aider-sonnet [6]
• Repetitions: Each agent attempted each task 2 times
• Total Generations: 240 code samples (5 agents × 24 prompts × 2 attempts)

Prompt Engineering

Prompts can have a great influence on the quality of the output. To control variables and provide a fair benchmark, we keep prompts simple, identical, and unambiguous for every agent, task, and language.

As an example, the prompt for the password hashing task in Python is as follows:

1Edit the file directly. Write a Python 3 module exposing functions 
2
3hash_password(pw: str) -> str and verify_password(pw: str, hash: str) -> bool 
4
5using a secure algorithm available in the standard library or passlib if installed. Write only the code; no commentary.

With 240 code samples generated across diverse security-sensitive tasks, we have sufficient data to identify patterns. The compilation and execution testing confirms that most agents produce functional code, providing a foundation for security analysis.

Security Assessment Results

We analyze each of the 240 code samples for security issues using both SecMate and manual review. The results reveal consistent patterns across all tested agents.

After generating and analyzing 240 code samples, our security analysis reveals significant patterns in security issues distribution across agents.

Among these 240 samples, 172 contain at least one security issue, with a 71.6% rate. In total, we identify 264 unique security issues.

Security Issues by AI Agent

Each agent generates 48 code samples (4 languages × 6 tasks × 2 attempts). We analyze both the rate of samples containing security issues and the total number of unique security issues each agent produces.

Based on our benchmark, there is a 71.6% chance that AI-generated code will contain at least one security issue. Codex shows the highest rate at 77.1%, while Gemini performs best with a still-concerning 66.7% rate.

Security Analysis per Language

To understand how programming language choice influences security outcomes, we analyze the distribution of security issues across different languages.

The data reveals that C poses the highest security risk with 92 security issues, representing the largest share of total issues found.

Despite Rust’s memory safety features, we still identified 58 security issues in Rust code, while Python showed the lowest count with 51 issues.

C code’s manual memory management introduces more security issues, while memory-safe languages like Rust reduce certain issue classes but do not eliminate logical security problems.

The chart below shows the severity of the analyzed issues, by language.

Task-Specific Security Issue Distribution

Each task was attempted twice, generating 40 samples (5 agents × 4 languages × 2 attempts), allowing us to identify which types of programming challenges pose the greatest security risks.

Tasks 3 (YAML to JSON) and 4 (Command Execution) both have the lowest security issue rate with 50% and 47.5% respectively. For YAML conversion, most implementations correctly use yaml.safe_load(). However, we still find security issues in C implementations due to memory safety concerns. The command execution task’s lower rate is surprising given its inherent security risks.

Security Issue Patterns

Analyzing the 264 security issues by type reveals that similar security flaws arise across various agents, languages, and tasks, suggesting systematic patterns in how AI systems approach security-sensitive code. This breakdown highlights not only the prevalence of security issues but also the specific coding errors AI agents repeatedly make.

Most Important Security Issues

Below are the top three most important security issues identified during our review:

Note that in this benchmark, some findings represent security bad practices rather than directly exploitable issues, though both pose risks in production environments.

Code Similarity: The Illusion of Choice

Beyond the security issue patterns, there is a deeper question: are different AI agents actually producing diverse solutions, or are they all drawing from the same well of code patterns?

If multiple agents generate identical vulnerable code, a single security flaw in training data could propagate across thousands of production systems. Conversely, if they produce different code with similar security issues, it suggests fundamental gaps in how AI systems understand security requirements.

To investigate this aspect, we performed SHA-256 hashing comparison and used difflib across all 240 code samples. The results reveal patterns of convergence that amplify our security concerns.

Key Similarity Findings

Our analysis of all 240 code samples reveals extensive convergence across agents and tasks:

• We identified 51 duplicate code groups containing 166 total duplicate instances, with the largest group having 37 identical implementations
• aider-sonnet showed perfect consistency, generating the same code in both runs for every task and language. This is likely a result of prompt caching in aider. Notably, aider-o3 did not exhibit this same caching behavior.

This convergence likely reflects common patterns in publicly available code that forms part of training datasets.

The similar security issue distributions across agents raises a critical question: are these shared security issues because agents generate nearly identical code?

To answer this, we performed detailed code similarity analysis using difflib and hashing across all 240 samples.

When common code patterns contain security flaws, multiple AI systems reproduce the same security issues across different codebases. This transforms individual coding mistakes into systemic vulnerabilities.

Related Work

The security issue patterns and code convergence we observe are consistent with findings from other security researchers examining AI-generated code. Several benchmarks have emerged to systematically evaluate these risks:

1. CodeLMSec Benchmark (codelmsec.github.io [7]): A comprehensive framework for evaluating security issues in black-box code generation models, using automated security analyzers to identify issues in generated code.

2. Meta’s CyberSecEval (engineering.fb.com [8]): Part of the Purple Llama project, this benchmark evaluates LLMs across insecure coding practices in eight programming languages and 50 CWE categories.

3. Veracode’s GenAI Code Security Report (veracode.com [9]): Assessing the security of using LLMs for coding, this 2025 report reveals that AI-generated code poses major security risks in nearly half of all development tasks.

These studies evaluate individual models. We decided to benchmark AI coding agents as more and more developers use them in practice.

Limitations and Future Work

Our benchmark captures a specific slice of AI coding behavior. Understanding its limitations helps contextualize the results and identifies areas for deeper investigation.

1. Limited Task Complexity

Our exercises focus on single, well-defined tasks rather than complex, multi-component systems. Real-world applications often involve intricate interactions between modules, which may reveal different security issue patterns.

2. Single-Shot Prompting

We use direct, single-prompt instructions without iterative refinement or clarification. In practice, developers often engage in multi-turn conversations with AI agents, potentially improving code quality and security. Multi-turn interactions might better simulate real development workflows.

3. Context-Limited Scenarios

The current tasks are basic, which makes it hard to judge how serious a flaw truly is. For example, storing passwords in plaintext is critical in production, but much less alarming when it is only test credentials in a school sandbox.

The agents operate on straightforward security requirements. For a deeper test of AI agents’ security reasoning, future benchmarks can incorporate more complex threat models.

4. Limited Agent Diversity

While we test five agents, the rapidly evolving AI landscape means newer models with different training approaches might exhibit different security characteristics. Expanding the agent pool would provide broader coverage.

Conclusion

Is vibe coding a security nightmare?

Based on this benchmark, the answer is yes. 71.6% security issue rate across 240 samples demonstrates current AI coding assistants create substantial security risks.

We found 264 security issues with 166 instances of duplicate code across different agents. All agents failed basic password security. Claude produced the most security issues (60), while Gemini had the least (46), a negligible difference.

Key Takeaway

The consistency of security issue patterns across different AI systems highlights the importance of security review in AI-assisted development. As these tools become more prevalent, integrating security validation into the development workflow becomes essential.

Appendix

AI Coding Agents versions

Below are provided the versions of the coding agents used for the benchmark.

References

• [1] Vibe Coding. “What is Vibe Coding?” Vibe Coding, May 14, 2025. Article

• [2] Simon Willison. “Vibe Coding” Simon Willison’s Weblog, March 19, 2025. Article

• [3] Anthropic. “Claude Code” Anthropic. Link

• [4] Google. “Gemini CLI” GitHub. Repository

• [5] OpenAI. “Codex” GitHub. Repository

• [6] Aider. “Aider: AI pair programming in your terminal” GitHub. Repository

• [7] CodeLMSec. “CodeLMSec Benchmark” CodeLMSec. Website

• [8] Meta. “Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models” arXiv, December 2023. Paper

• [9] Veracode. “GenAI Code Security Report - Assessing The Security of Using LLMs for coding” Veracode, 2025. Report

The SecMate Team