Skip to content

[Bug] DefaultSerializationExceptionWrapper does not implement Cleanable, so cleanup() is never invoked by codecs #16537

Open

Description

@binbin0325

Pre-check

  • I am sure that all the content I provide is in English.

Search before asking

  • I had searched in the issues and found no similar issues.

Apache Dubbo Component

Java SDK (apache/dubbo)

Dubbo Version

Dubbo 3.3.6 (also reproducible on the latest 3.3 branch), JDK 25 (Amazon Corretto 25.0.4), Linux (Kubernetes). Not JDK specific.

Steps to reproduce this issue

DefaultSerializationExceptionWrapper is registered as an SPI wrapper for Serialization
(META-INF/dubbo/internal/org.apache.dubbo.common.serialize.Serialization), so every
ObjectInput / ObjectOutput returned to the codecs is a ProxyObjectInput / ProxyObjectOutput.
Neither proxy implements Cleanable or forwards cleanup() to its target.

Serialization serialization = FrameworkModel.defaultModel()
        .getExtensionLoader(Serialization.class)
        .getExtension("hessian2");
ObjectInput in = serialization.deserialize(url, new ByteArrayInputStream(bytes));
// false: in is DefaultSerializationExceptionWrapper$ProxyObjectInput,
// although Hessian2ObjectInput itself implements Cleanable
System.out.println(in instanceof Cleanable);

As a result, every instanceof Cleanable check in the codecs is always false and cleanup() is never called:

  • ExchangeCodec#encodeRequest / #encodeResponse (3 call sites)
  • TransportCodec#encode / #decode (2 call sites)
  • DecodeableRpcResult#decode(Channel, InputStream)
  • DecodeableRpcInvocation#decode(Channel, InputStream)

What you expected to happen

The wrapper should be transparent for Cleanable: ProxyObjectInput / ProxyObjectOutput should implement Cleanable and forward cleanup() to the wrapped object, so that codecs can release per-message resources as they did before this wrapper was introduced.

Anything else

Impact in production: we use a custom serialization whose ObjectInput decompresses the body with GZIPInputStream and closes it in cleanup(). Because cleanup() was never invoked, every compressed response left its native java.util.zip.Inflater (~40 KB of off-heap memory) to the Cleaner. With generational ZGC and a large, mostly empty heap, major collections were rare, so native memory grew by several GiB per minute under load and the container was OOMKilled.

NMT showed no growth (zlib memory is not tracked by NMT). A native-memory flame graph pointed at:
DecodeableRpcResult.decode -> DefaultSerializationExceptionWrapper.deserialize -> ... -> GZIPInputStream. -> Inflater.init

For built-in Hessian2 the effect is minor (cleanup() only resets the reused Hessian2Input / Hessian2Output), which is probably why it went unnoticed.

Also noticed while reading the class: ProxyObjectOutput#writeBytes(byte[] v, int off, int len) calls target.writeBytes(v) and ignores off / len.

Do you have a (mini) reproduction demo?

  • Yes, I have a minimal reproduction demo to help resolve this issue more effectively!

Are you willing to submit a pull request to fix on your own?

  • Yes I am willing to submit a pull request on my own!

Code of Conduct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions