Corrupting the Secure World: Three Memory-Safety Flaws in OP-TEE

SecMate’s automated analysis found three security issues in OP-TEE [1] , the open-source Trusted OS for Arm TrustZone that is commonly deployed with Arm Trusted Firmware. All three corrupt memory in the privileged OP-TEE core. We reported them against commit f2a7ad0638aeff5243593b33cc56ad064cae7615.

Two received CVEs. The affected SHA-3 path shipped in 13 public OP-TEE releases, the PKCS#1 v1.5 path in 25, and the SE050 RSA NOPAD path in at least 18. The oldest issue had been present for more than six years. SecMate found all three automatically using locally deployable gpt-oss-20b and gpt-oss-120b models.

OP-TEE treated the third finding as a bug because our report did not demonstrate the initial compromise needed to establish a complete compromised-TA attack chain.

Core memory corruption matters because OP-TEE is responsible for keeping secrets isolated from Linux or Android.

Why OP-TEE matters

OP-TEE stands for Open Portable Trusted Execution Environment. On Arm TrustZone systems, it runs security-sensitive code and data separately from the normal operating system, usually Linux or Android [1] .

That normal operating system is called the Rich Execution Environment (REE). OP-TEE runs on the protected side of the TrustZone boundary. The separation is meant to keep sensitive data, code, and execution isolated even if software in the normal world is compromised. The GlobalPlatform TEE architecture defines integrity and data confidentiality as core security properties. It also defines mechanisms for Trusted Application code confidentiality and isolation from the REE [2] . The protected properties include:

• confidentiality of sensitive data and keys
• integrity of code and data during execution
• confidentiality of Trusted Application code when provided by the implementation
• isolation from the normal operating system

OP-TEE is not limited to research platforms. TrustedFirmware.org describes it as widely used and deployed, and its official platform list includes maintained targets from STMicroelectronics, NXP, Texas Instruments, Renesas, Rockchip, and AMD/Xilinx [3] [4] . Other vendors use different secure environments, including Qualcomm TEE, Samsung TEEGRIS, and Apple’s Secure Enclave with sepOS [5] [6] [7] . A device with a TEE therefore does not necessarily run OP-TEE.

What Arm Trusted Firmware and OP-TEE actually are

Arm application processors implement two security states, the Normal World and the Secure World, selected by the NS bit and enforced in hardware by TrustZone. The idea is to keep a small, high-assurance environment isolated from a large, complex Rich OS such as Linux. Inside each world, the processor still has privilege levels, expressed as exception levels.

Arm Trusted Firmware (TF-A) [9] is a reference implementation of the firmware that commonly runs at the most privileged level, EL3. Its BL31 runtime acts as the Secure Monitor in this deployment model. It handles SMC (Secure Monitor Call) instructions, PSCI power management, world switches, and dispatch to a Trusted OS.

OP-TEE is the Trusted OS in the deployment shown below. It commonly runs as the BL32 secure payload at S-EL1, with Trusted Applications (TAs) at S-EL0. OP-TEE and TF-A are sibling TrustedFirmware.org projects. When deployed together, TF-A provides the monitor and dispatches calls to OP-TEE. From the Normal World, an application uses the GlobalPlatform TEE Client API. The call crosses into EL3 via SMC, and the monitor hands it to OP-TEE.

The boundary that matters here is inside the secure world. Trusted Applications run at S-EL0. The OP-TEE core runs above them at S-EL1 and handles privileged operations such as key storage, cryptographic services, and secure peripherals. A TA can call the core through syscalls, but it must never be able to write core memory.

These three issues break that boundary. The SHA-3 issue starts with an off-by-one. The other two start with unchecked size relationships. In all three, an unsigned subtraction wraps. The resulting size or offset makes memset or memcpy write outside its buffer.

The three findings

CVE-2026-40257 requires the Arm v8.2 SHA-3 acceleration path. CVE-2026-33662 affects builds registering RSA acceleration through CAAM, HiSilicon, Versal, or SE050, as listed in the OP-TEE advisory. The third issue requires the NXP SE050 RSA driver. These paths are platform-specific, but they ship on production hardware.

Scope and impact

OP-TEE’s published security scope excludes a deliberately malicious TA signed with a trusted TA key because the key holder is already part of the trusted computing base [8] .

An attacker may first compromise a legitimate, signed TA through one or more separate vulnerabilities and then invoke the SE050 bug through that TA. In this chain, the attacker does not need the TA’s signing key, and the TA retains its valid signature. Our report demonstrated the resulting controlled write from S-EL0 into S-EL1 core memory, but it did not demonstrate the initial TA compromise. OP-TEE therefore treated the finding as a bug rather than a complete vulnerability report.

Findings 1 and 2 do not have that prerequisite. Valid hash input triggers the first. A signing operation with a sufficiently small RSA modulus reaches the second.

All three are heap writes. Stack protection does not help. ASLR and non-executable data make targeted exploitation harder, but the minimum demonstrated impact for findings 1 and 2 is a secure-world crash. Finding 3 gives the attacker control over both the bytes written and their position relative to the allocation.

Finding 1: SHA-3 accelerated finalize heap overflow (CVE-2026-40257)

OP-TEE has two SHA-3 implementations. The default software implementation in sha3.c accumulates input word by word and is not affected by this issue. The other, sha3_accel.c, is compiled in when a platform enables the Arm v8.2 SHA-3 crypto extensions (CFG_CRYPTO_WITH_CE82=y) and buffers input in blocks. It is the accelerated path that is broken.

The Keccak sponge processes input in blocks of block_size bytes (136 for SHA3-256). sha3_process() is supposed to: finish any partial block, run full blocks through the hardware, then buffer the remainder. The bug is an off-by-one in step two:

 1/* sha3_accel.c:151 - strict greater-than is the bug */
 2if (inlen > block_size) {
 3    block_count = inlen / block_size;
 4    crypto_accel_sha3_compress(state, in, block_count, digest_size);
 5    in += block_count * block_size;
 6    inlen -= block_count * block_size;
 7}
 8
 9/* lines 159-160 - reached when inlen == block_size */
10memcpy(md->sha3.sb + md->sha3.byte_index, in, inlen);
11md->sha3.byte_index += inlen;            /* byte_index becomes block_size */

When a TA feeds exactly one block (136 bytes) with an empty buffer, inlen > block_size is false, the hardware compression is skipped, and the whole block is copied into sb with byte_index left equal to block_size. The finalize functions assume the invariant byte_index < block_size. It no longer holds:

1/* sha3_done(), lines 194-195 */
2buf[md->sha3.byte_index++] = 0x06;                              /* sb[136], index -> 137 */
3memset(buf + md->sha3.byte_index, 0, block_size - md->sha3.byte_index);
4/* block_size(136) - byte_index(137) underflows to 0xFFFFFFFF */

block_size - byte_index is 136 - 137 in unsigned arithmetic, i.e. 0xFFFFFFFF. The memset then writes roughly 4 GB of zeroes starting inside the hash state on the S-EL1 heap. sha3_shake_done() has the identical pattern for SHAKE-128/256.

Attack chain and impact. The trigger is two ordinary GlobalPlatform calls: TEE_DigestUpdate(op, data, 136) then TEE_DigestDoFinal(...). No special key material, permissions, or object types are required. The input is valid because hashing a block-sized buffer is normal usage. A perfectly honest TA can therefore crash the secure world without intending to. The write consists of zeroes and spans roughly 4 GB. In practice, it crashes the OP-TEE core and takes down its TAs, secure services, and Normal-World clients. The write offers too little control for a realistic code-execution primitive. The advisory rates it Moderate, CVSS 5.5 (AV:L/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H).

We reproduced it with an ASAN harness that #includes the real sha3_accel.c, stubs the hardware compress function, and runs the two-call sequence. ASAN reports a heap-buffer-overflow with the stack pointing straight at sha3_accel.c:195.

Introduced by commit 1478437e65c4 in March 2023, this vulnerability shipped in 13 public OP-TEE releases, from 3.21.0 through 4.10.0. It remained in the secure-world core for nearly three years before SecMate’s automated analysis identified it, and for more than three years before the fix was merged.

The fix. Change the comparison so a full block is actually compressed, and add a defensive assertion to the finalize paths [10] :

1-	if (inlen > block_size) {
2+	if (inlen >= block_size) {
3 		block_count = inlen / block_size;
4 		crypto_accel_sha3_compress(state, in, block_count, digest_size);
5...
6+	LTC_ARGCHK(md->sha3.byte_index < block_size);
7+
8 	buf[md->sha3.byte_index++] = 0x06;
9 	memset(buf + md->sha3.byte_index, 0, block_size - md->sha3.byte_index);

With >=, an exact-block input is consumed by the hardware compress, inlen drops to zero, and byte_index stays at zero. The added LTC_ARGCHK returns CRYPT_INVALID_ARG if the invariant is violated again. The advisory identifies OP-TEE 4.11.0 and later as patched. The workaround is CFG_CRYPTO_WITH_CE82=n.

Finding 2: RSASSA PKCS#1 v1.5 encode underflow (CVE-2026-33662)

This one lives in the generic crypto-driver framework (CFG_CRYPTO_DRIVER=y), which platforms with RSA acceleration use to dispatch signature operations: NXP CAAM (plat-imx, plat-ls), Xilinx Versal, HiSilicon, and SE050 builds.

PKCS#1 v1.5 signature encoding pads the DigestInfo structure up to the modulus size with 0xFF bytes. emsa_pkcs1_v1_5_encode() computes how many padding bytes it needs by subtracting from the modulus size, with no lower-bound check:

1/* rsassa.c:48-50 */
2ps_size  = ssa_data->key.n_size - 3;
3ps_size -= ssa_data->digest_size;
4ps_size -= 10 + hash_oid->asn1_length;
5...
6memset(buf, UINT8_MAX, ps_size);          /* line 75 */

With a 256-bit modulus (32 bytes) and SHA-256 (32-byte digest, 9-byte OID), the arithmetic is 32 - 3 - 32 - (10 + 9), which underflows size_t to roughly SIZE_MAX. The memset then writes 0xFF across the entire S-EL1 heap until OP-TEE faults. RFC 3447 Section 9.2 mandates the missing size check [12] . The same file’s noasn1 and PSS paths already perform their own size validation. This path did not.

Attack chain and impact. A TA uses a sufficiently small RSA modulus and calls TEE_AsymmetricSignDigest(TEE_ALG_RSASSA_PKCS1_V1_5_SHA256, ...). The vulnerable core path is reached through that signing operation. Whether an external attacker can reach it depends on how the product exposes the TA to Normal-World clients. Like finding 1, this is a blunt overwrite. It gives an attacker a reliable TEE-wide denial of service, not controlled execution. The OP-TEE advisory rates the issue High, CVSS 7.5 (AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H).

SecMate identified this vulnerability independently before its public disclosure. OP-TEE had already found the issue internally, so the project handled our report as a duplicate and the CVE process was already underway.

This vulnerability was introduced by commit f5a70e3efb80 in February 2020. It remained in OP-TEE for more than six years and shipped in 25 public releases before the fix was merged in April 2026. Public Git history places the first affected release at 3.9.0, although the GitHub advisory lists versions from 3.8.0.

The fix. The maintainers replaced the chained subtraction with checked subtraction macros that bail out on underflow [11] :

1-	ps_size = ssa_data->key.n_size - 3;
2-	ps_size -= ssa_data->digest_size;
3-	ps_size -= 10 + hash_oid->asn1_length;
4+	if (SUB_OVERFLOW(ssa_data->key.n_size, 3, &ps_size) ||
5+	    SUB_OVERFLOW(ps_size, ssa_data->digest_size, &ps_size) ||
6+	    SUB_OVERFLOW(ps_size, 10 + hash_oid->asn1_length, &ps_size))
7+		return TEE_ERROR_BAD_PARAMETERS;

An encoding size that would make one of the subtractions wrap now returns TEE_ERROR_BAD_PARAMETERS. The patch prevents the memory corruption. It does not enforce the separate PKCS#1 v1.5 requirement that the padding string contain at least eight bytes. The advisory identifies OP-TEE 4.11 and later as patched. The workaround is to disable RSA acceleration.

Finding 3: SE050 RSA NOPAD controlled heap underflow

The third finding gives an attacker more control than the two CVEs, but OP-TEE treated it as a bug. It is in the NXP SE050 secure-element driver (CFG_NXP_SE05X=y), used on i.MX6/7/8 platforms that pair the SoC with an SE050 element over I²C.

decrypt_nopad() and encrypt_nopad() left-pad the caller’s input to the modulus size by computing a destination offset with, again, an unchecked unsigned subtraction:

1/* se050/core/rsa.c:356 (and :418) */
2rsa_len = crypto_bignum_num_bytes(key->n);     /* e.g. 256 for a 2048-bit key */
3memcpy(buf + rsa_len - src_len, src, src_len);

Neither the generic dispatch layer (crypto_acipher_rsanopad_decrypt, which passes cipher_len straight through) nor the SE050 driver checks that src_len <= rsa_len. If a TA supplies input larger than the modulus, the subtraction computes a destination before the allocation. Forming that pointer is undefined behavior in C. On the affected targets, it produces the following relative out-of-bounds write:

rsa_len = 256, src_len = 300  ->  256 - 300 = (size_t)-44
memcpy(buf - 44, src, 300)    ->  44 attacker-chosen bytes written BEFORE buf

buf comes from OP-TEE’s BGET-backed mempool. An allocated block’s prevfree and bsize metadata sit immediately before its payload, so even a short underflow can overwrite allocator state that mempool_free() later consumes.

Why this one is different. Findings 1 and 2 overwrite with a fixed byte over an unbounded range, which is useful for a crash but offers little control. This is a controlled heap-underflow write primitive: the attacker chooses the underflow distance (src_len - rsa_len) and the bytes copied from src. The primitive can corrupt allocator metadata or an adjacent core object before the affected allocation is freed. We did not build a complete S-EL0-to-S-EL1 exploit, but this gives an attacker far more control than findings 1 and 2, especially on constrained 32-bit targets. We confirmed the out-of-bounds write with an ASAN harness around the real rsa.c.

The vulnerable RSA NOPAD implementation entered the SE050 driver in commit 462192731c92 in January 2022. It shipped in at least 18 public releases and was still present in upstream commit 611fbaf34150 on July 13, 2026, four and a half years later.

Why it was treated as a bug. Reaching this code requires control of a TA that calls TEE_AsymmetricDecrypt(TEE_ALG_RSA_NOPAD, ...) with oversized input. That control could come from a deliberately malicious TA or from compromising a legitimate, signed TA through another vulnerability. The former is outside OP-TEE’s threat model; the latter does not require the attacker to possess a TA signing key and can form the first stage of an exploit chain. Our report demonstrated the controlled S-EL1 write but not the initial TA compromise, so OP-TEE treated the finding as a bug.

The fix is to validate src_len <= rsa_len and msg_len <= n_size in the generic dispatch layer, then repeat the check in the driver before memcpy. The HiSilicon HPRE driver already validates input sizes independently and is not affected.

Three old flaws, found automatically

These issues entered OP-TEE in 2020, 2022, and 2023. They survived across separate accelerated implementations and hardware drivers until SecMate’s automated analysis found them.

Every issue in our public disclosure record [13] was found by the same engine. SecMate runs with locally deployable models, so vulnerability research can stay inside the operator’s environment instead of sending source code to an external model service.

Disclosure timeline

References

• [1] OP-TEE. “About OP-TEE.” OP-TEE documentation. Documentation

• [2] GlobalPlatform. “TEE System Architecture v1.3.” Specification

• [3] OP-TEE. “Platforms supported.” OP-TEE documentation. Documentation

• [4] TrustedFirmware.org. “OP-TEE moving into Trusted Firmware.” Project announcement

• [5] Qualcomm. “Qualcomm Linux System Software Architecture.” Documentation

• [6] Samsung. “Samsung TEEGRIS.” Samsung Developer. Documentation

• [7] Apple. “The Secure Enclave.” Apple Platform Security. Documentation

• [8] OP-TEE. “Security scope.” OP-TEE documentation. Documentation

• [9] TrustedFirmware.org. “Firmware Design.” Trusted Firmware-A documentation. Documentation

• [10] OP-TEE. “SHA-3 accelerated finalize heap overflow.” GHSA-75x4-j8p9-55qv / CVE-2026-40257. Advisory

• [11] OP-TEE. “RSASSA EMSA-PKCS1-v1_5 underflow in emsa_pkcs1_v1_5_encode().” GHSA-4cf8-v5g3-73gr / CVE-2026-33662. Advisory

• [12] IETF. “PKCS #1: RSA Cryptography Specifications Version 2.1.” RFC 3447. Specification

• [13] SecMate. “Disclosures.” Disclosure page

The SecMate Team