当前位置:网站首页>Performance comparison of serialization tools such as Jackson, fastjson, kryo, protostuff

Performance comparison of serialization tools such as Jackson, fastjson, kryo, protostuff

2020-11-09 14:32:35 Ziyuesheng

brief introduction

In actual projects , We often need to use sequential tools to store and transfer objects . At present, the more widely used sequence chemical industry has :jackson、fastjson、kryo、protostuff、fst etc. , This article will briefly compare the serialization and deserialization performance of these tools .

Project environment

This article USES the jmh As a test tool .

os:win 10

jdk:1.8.0_231

jmh:1.25

The selected serialization tool and the corresponding version are as follows :

fastjson:1.2.74

jackson:2.11.3

kryo:5.0.0

fst:2.57

protostuff:1.7.2

Test code

For the sake of fairness , I try to make the use of serialization tools in test cases closer to the actual project , for example ,kryo Of Kryo Object is not thread safe , In real projects, we don't just use it every time new A new object , But use ThreadLocal Or pooling to reduce the overhead of creating objects .

This article uses java bean as follows . A user object , One to one relationship between department object and position object , Among them, the Department objects are self related .

public class User implements Serializable {
    private static final long serialVersionUID = 1L;
    //  General attribute --29 individual 
    private String id;
    private String account;
    private String password;
    private Integer status;
    // ······
    
    /**
     *  Department 
     */
    private Department department;
    /**
     *  Position 
     */
    private Position position;
    
    //  Omitted below setter/getter Method 
}
public class Department implements Serializable {
    private static final long serialVersionUID = 1L;
    //  General attribute --7 individual 
    private String id;
    private String parentId;
    // ······
    /**
     *  subsidiary department 
     */
    private List<Department> children;
    
    //  Omitted below setter/getter Method 
}
public class Position implements Serializable {
    private static final long serialVersionUID = 1L;
    //  General attribute --6 individual 
    private String id;
    private String name;
    // ······
    //  Omitted below setter/getter Method 
}

Part of the test code is shown below , See the link at the end for the complete code .

JDK The serialization tool comes with

JDK Provides ObjectOutputStream Used to support serialization ,ObjectInputStream For deserializing . Be careful , Use JDK When the serialization tool comes with it ,java bean Must be realized Serializable, Otherwise it will throw NotSerializableException abnormal . Use keywords transient Decorated member properties are not serialized .

    //  serialize 
    @Benchmark
    public byte[] jdkSerialize(CommonState commonState) throws Exception {
        ByteArrayOutputStream byteArray = new ByteArrayOutputStream();
        ObjectOutputStream outputStream = new ObjectOutputStream(byteArray);
        outputStream.writeObject(commonState.user);
        outputStream.flush();
        outputStream.close();
        return byteArray.toByteArray();
    }
    //  Deserialization 
    @Benchmark
    public User jdkDeSerialize(JdkState jdkState) throws Exception {
        ByteArrayInputStream byteArray = new ByteArrayInputStream(jdkState.bytes);
        ObjectInputStream inputStream = new ObjectInputStream(byteArray);
        User user = (User)inputStream.readObject();
        inputStream.close();
        assert "zzs0".equals(user.getName());
        return user;
    }

fastjson

fastjson Developed by Ali team , It's the fastest Java Realized json library . fastjson Of API Very concise , And support a certain degree of customization ( for example , annotation @JSONField、 enumeration Feature Wait for custom serialization ). Criticized by others , May be fastjson Of bug More .

    //  serialize 
    @Benchmark
    public byte[] fastJsonSerialize(CommonState commonState) {
        return JSON.toJSONBytes(commonState.user);
    }
    //  Deserialization 
    @Benchmark
    public User fastJsonDeSerialize(FastJsonState fastJsonState) {
        User user = JSON.parseObject(fastJsonState.bytes, User.class);
        assert "zzs0".equals(user.getName());
        return user;
    }

jackson

jackson from fasterxml Organizational development , comparison fastjson, With more powerful functions 、 Higher stability 、 Better scalability 、 Richer customization support .Spring Default json The parsing tool is jackson.

Use jackson We need to pay attention to ,ObjectMapper Object is thread safe , Can be reused .

    //  serialize 
    @Benchmark
    public byte[] jacksonSerialize(CommonState commonState, JacksonState jacksonState) throws Exception {
        return jacksonState.objectMapper.writeValueAsBytes(commonState.user);
    }
    //  Deserialization 
    @Benchmark
    public User jacksonDeSerialize(JacksonState jacksonState) throws Exception {
        User user = jacksonState.objectMapper.readValue(jacksonState.bytes, User.class);
        assert "zzs0".equals(user.getName());
        return user;
    }

kryo

kryo from EsotericSoftware Organizational development , Are not compatible jdk Data with its own serialization tool ,kryo The serialized data should be smaller , as for API In terms of simplicity , I think it's still a little bit worse , If you're not careful, you'll dig . Use kryo The following points need to be noted :

  1. Kryo Object is not thread safe , have access to ThreadLocal Or pool to get ( This article uses pools to get );
  2. kryo Through class registration, you can write the class's class id, Instead of the fully qualified class name of the class , This reduces the size of the serialized data . however , It's hard to guarantee that the same class is registered on different machines class id, therefore , It is recommended to set kryo.setRegistrationRequired(false);, Because of the same Class It is difficult to guarantee the consistency of the registration number on different machines ;
  3. When java bean When a circular reference occurs , Use kryo Stack memory overflow may occur , At this time, you can set kryo.setReferences(true); To avoid . If circular references are not possible in a project , Can be set to false To improve performance .
	//  serialize 
    @Benchmark
    public byte[] kryoSerialize(CommonState commonState, KryoState kryoState) {
        ByteArrayOutputStream byteArray = new ByteArrayOutputStream();
        Output output = new Output(byteArray);
        Kryo kryo = kryoState.kryoPool.obtain();
        kryo.writeClassAndObject(output, commonState.user);
        kryoState.kryoPool.free(kryo);
        output.flush();
        output.close();
        return byteArray.toByteArray();
    }
	// Deserialization 
    @Benchmark
    public User kryoDeSerialize(KryoState kryoState) throws Exception {
        ByteArrayInputStream byteArray = new ByteArrayInputStream(kryoState.bytes);
        Input input = new Input(byteArray);
        Kryo kryo = kryoState.kryoPool.obtain();
        User user = (User)kryo.readClassAndObject(input);
        kryoState.kryoPool.free(kryo);
        input.close();
        assert "zzs0".equals(user.getName());
        return user;
    }

fst

fst(fast-serialization) By RuedigerMoeller Development ,API Very concise . Attention should be paid when using ,FSTConfiguration Objects can be reused .

Actually ,fst I also support json Formal serialization , But the performance of this piece is a little worse, and less people use it , I won't mention it here .

	//  serialize 
    @Benchmark
    public byte[] fstSerialize(CommonState commonState, FSTConfigurationState fSTConfigurationState) {
        return fSTConfigurationState.fSTConfiguration.asByteArray(commonState.user);
    }
	//  Deserialization 
    @Benchmark
    public User fstDeSerialize(FSTState fstState) throws Exception {
        User user = (User)fstState.fSTConfiguration.asObject(fstState.bytes);
        assert "zzs0".equals(user.getName());
        return user;
    }

protostuff

protostuff Is based on google protobuf Developed from ( And protobuf comparison ,protostuff In the case of almost no loss of performance, no need to write .proto File to achieve serialization ), Are not compatible jdk Data with its own serialization tool , The serialized data should be smaller . Use protostuff There are a few points to pay attention to :

  1. protostuff Using the order in which fields are defined tag, When adding new fields, the order of the original fields must be kept unchanged , Otherwise, the old data may fail to deserialize ;
  2. protostuff Can't serialize directly Array、List、Map, If you need to serialize , It needs to be packaged first java bean;
	//  serialize 
    @Benchmark
    public byte[] protostuffSerialize(CommonState commonStateme) {
        Schema<User> schema = (Schema<User>)RuntimeSchema.getSchema(User.class);
        return ProtostuffIOUtil.toByteArray(commonStateme.user, schema, LinkedBuffer.allocate(LinkedBuffer.DEFAULT_BUFFER_SIZE));
    }
	//  Deserialization 
    @Benchmark
    public User protostuffDeSerialize(ProtostuffState protostuffState) throws Exception {
        User user = new User();
        Schema<User> schema = (Schema<User>)RuntimeSchema.getSchema(User.class);
        ProtostuffIOUtil.mergeFrom(protostuffState.bytes, user, schema);
        assert "zzs0".equals(user.getName());
        return user;
    }

test result

The following is a measure of throughput , Under the same conditions , The higher the throughput, the better .

serialize

cmd The instructions are as follows :

mvn clean package
java -ea -jar target/benchmarks.jar cn.zzs.serialize.SerializeTest -f 1 -t 1 -wi 10 -i 10

test result :

# JMH version: 1.25
# VM version: JDK 1.8.0_231, Java HotSpot(TM) 64-Bit Server VM, 25.231-b11
# VM invoker: D:\growUp\installation\jdk1.8.0_231\jre\bin\java.exe
# VM options: -ea
# Warmup: 10 iterations, 10 s each
# Measurement: 10 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
Benchmark                           Mode  Cnt    Score    Error   Units
SerializeTest.fastJsonSerialize    thrpt   10  449.474 ±  1.851  ops/ms
SerializeTest.fstSerialize         thrpt   10  694.716 ±  7.240  ops/ms
SerializeTest.jacksonSerialize     thrpt   10  330.610 ±  6.968  ops/ms
SerializeTest.jdkSerialize         thrpt   10  132.483 ±  0.379  ops/ms
SerializeTest.kryoSerialize        thrpt   10  609.969 ±  3.288  ops/ms
SerializeTest.protostuffSerialize  thrpt   10  762.737 ± 10.918  ops/ms

You can see , In terms of serialization speed :protostuff > fst > kryo > fastjson > jackson > jdk.

Deserialization

cmd The instructions are as follows :

mvn clean package
java -ea -jar target/benchmarks.jar cn.zzs.serialize.DeSerializeTest -f 1 -t 1 -wi 10 -i 10

test result :

# JMH version: 1.25
# VM version: JDK 1.8.0_231, Java HotSpot(TM) 64-Bit Server VM, 25.231-b11
# VM invoker: D:\growUp\installation\jdk1.8.0_231\jre\bin\java.exe
# VM options: -ea
# Warmup: 10 iterations, 10 s each
# Measurement: 10 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time

fastjson serialized data size:1044
fst serialized data size:607
jackson serialized data size:1060
jdk serialized data size:1700
kryo serialized data size:597
protostuff serialized data size:543

Benchmark                               Mode  Cnt    Score   Error   Units
DeSerializeTest.fastJsonDeSerialize    thrpt   10  397.069 ± 3.507  ops/ms
DeSerializeTest.fstDeSerialize         thrpt   10  465.813 ± 4.700  ops/ms
DeSerializeTest.jacksonDeSerialize     thrpt   10  226.412 ± 1.436  ops/ms
DeSerializeTest.jdkDeSerialize         thrpt   10   27.919 ± 0.352  ops/ms
DeSerializeTest.kryoDeSerialize        thrpt   10  448.978 ± 3.504  ops/ms
DeSerializeTest.protostuffDeSerialize  thrpt   10  499.328 ± 4.485  ops/ms

You can see , In terms of deserialization speed :protostuff > fst > kryo > fastjson > jackson > jdk, The result is consistent with serialization .

The size of the serialized data :protostuff < kryo < fst < fastjson < jackson < jdk.

The above data is for reference only . Thank you for reading .

Related source code please move : serialize-tool-demo

This is an original article , Please attach a link to the source of the original :https://www.cnblogs.com/ZhangZiSheng001/p/13948414.html

版权声明
本文为[Ziyuesheng]所创,转载请带上原文链接,感谢