The JDK’s java.io.Externalizable interface is simple to implement. Just 2 methods to implement and you are ready to go. Unfortunately there are a few ways you can shoot yourself in the foot. In this post we look at 3 ways you can make mistakes when implementing Externalizable the readExternal() and writeExternal() methods. This page provides a minimal code snippet to illustrate each mistake.

The first mistake is trivial and it will result in an exception being thrown at runtime.  The second and third types of mistake are more dangerous!  There will be no problem at runtime but the state of the deserialized objects will be incorrect! Not only are you losing information! You may not even notice it.

Utility

Before we take a look at the different mistakes first a bit of blumping. We’ll use the static inOut() method shown below throughout our tests. It is just a helper method which we’ll use to serialize and deserialize objects. The serialization step will call the writeExternal() method of our classes. Deserialization in turn uses the readExternal() method.blumping

package mistakes;

import java.io.*;

public class ExternalizableMistake {
    public static  T inOut(T in) throws IOException, ClassNotFoundException {
        ByteArrayOutputStream boas = new ByteArrayOutputStream();
        ObjectOutputStream oos = new ObjectOutputStream(boas);
        oos.writeObject(in);
        oos.flush();
        ByteArrayInputStream bais = new ByteArrayInputStream(boas.toByteArray());
        return (T) new ObjectInputStream(bais).readObject();
    }
}

Mistake 1: lost ticket

concertLet’s start by selling some tickets; very simple tickets as you can see below. The ConcertTicket class has 2 simple fields, implements Externalizable as well as equals() and hashCode().

Can you see the mistake? The equals(), hashCode() and toString() methods are correct. Look at the writeExternal() and readExternal() methods.

 

package mistakes;

import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;
import java.util.Date;

public class ConcertTicket implements Externalizable {
    Date date;
    String area;

    public void writeExternal(ObjectOutput out) throws IOException {
        out.writeObject(date);
        out.writeUTF(area);
    }

    public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
        area = in.readUTF();
        date = (Date) in.readObject();
    }

    @Override
    public boolean equals(Object o) {

        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;

        ConcertTicket that = (ConcertTicket) o;

        if (date != null ? !date.equals(that.date) : that.date != null) return false;
        if (area != null ? !area.equals(that.area) : that.area != null) return false;

        return true;
    }

    @Override
    public int hashCode() {
        int result = date != null ? date.hashCode() : 0;
        result = 31 * result + (area != null ? area.hashCode() : 0);
        return result;
    }

    @Override
    public String toString() {
        return "ConcertTicket{" +
                "date=" + date +
                ", area='" + area + '\'' +
                '}';
    }

    public static void main(String[] args) throws IOException, ClassNotFoundException {
        //
        ConcertTicket concertTicket = new ConcertTicket();
        concertTicket.area = "Arena";
        concertTicket.date = new Date(); // whatever
        System.out.println("Ticket in  : " +concertTicket);
        System.out.println("Ticket out : " +ExternalizableMistake.inOut(concertTicket));
        System.out.println(concertTicket.equals(ExternalizableMistake.inOut(concertTicket))); // prints? Nothing! IOException
    }
}

readExternal: not what you ordered

We reversed the order in which we serialize and deserialize the fields! We first we serialize the date field with writeObject() and then the area field with writeUTF(). The serialization will go just fine. Everybody is happy. But when we want to deserialize we have a big problem as shown on the illustration below. The stacktrace shows that the readUTF() method fails. The fact that deserialization fails is a good thing! Yes, it is a feature not a bug. The JDK serialization checks the data to avoid data corruption.

In this example the first bytes represent the state of a Java Date instance. If readtUTF() did not perform this check it would create a new but most likely corrupt String. How does those checks work?

The methods of ObjectOutputStream class add a small amount of meta data in the serialized data. This meta data are small headers consisting of a few bytes which describe the data in the output. Methods like writeObject() and writeUTF() etc write this metadata in the output. So it is the ObjectOutputStream which adds the data  but it is the ObjectInputStream class which performs the checks at runtime. When we call readUTF() instead of readObject() in our example the JDK algorthim will check whether the first bytes, the header with meta data, to confirm that the bytes in the input correspond to an UTF encoded String. In our example this is not the case and as precaution an IOException gets thrown.

ConcertTick_IOException_during_deserialization

The good news is that our data is not lost! Sure you can not read it. But as soon as we fix readExternal() we will be able to deserialize the data again!

Lesson I

Watch out for the order in which you call the serialization and deserialization methods. The java compiler can not help us here! All the types match. Only the order of the methods calls differs. A good way to avoid this pitfall is to write unit tests for your Externalizable classes;

Mistake 2: Employee vs Manager

Meet the next Externalizable POJO: the Employee class. This our base class which has just one String field name. Now scroll down and look at the Manager class and its main() method.

package mistakes;

import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;

public class Employee implements Externalizable {
    protected String name;

    public void writeExternal(ObjectOutput out) throws IOException {
        out.writeUTF(name);
    }

    public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
        name = in.readUTF();
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;

        Employee employee = (Employee) o;

        return !(name != null ? !name.equals(employee.name) : employee.name != null);

    }

    @Override
    public int hashCode() {
        return name != null ? name.hashCode() : 0;
    }

    @Override
    public String toString() {
        return "Employee{" +
                "name='" + name + '\'' +
                '}';
    }
}

Meet the Manager class. This a class of its own. It is more than an Employee… it deserves a big bonus from time to time. So the Manager class extends Employee and it adds a boolean field hasBigBonus and override the equals() and hashCode() methods. Look at the main() now.

package mistakes;

import java.io.IOException;

public class Manager extends Employee {
    boolean hasBigBonus = false;  // I knew it!!!

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;
        if (!super.equals(o)) return false;

        Manager manager = (Manager) o;

        return hasBigBonus == manager.hasBigBonus;

    }

    @Override
    public int hashCode() {
        int result = super.hashCode();
        result = 31 * result + (hasBigBonus ? 1 : 0);
        return result;
    }

    @Override
    public String toString() {
        return "Manager{" +
                "name=" + name +
                "hasBigBonus=" + hasBigBonus +
                '}';
    }

    public static void main(String[] args) throws IOException, ClassNotFoundException {
        Employee hardWorking = new Employee();
        hardWorking.name = "Robert";
        System.out.println(hardWorking.equals(hardWorking));  // prints true

        //
        Manager boss = new Manager();
        boss.name = "Kurt";
        boss.hasBigBonus = true; // Why!?!!
        System.out.println("Manager  in: " + boss);
        System.out.println("Manager out: " + ExternalizableMistake.inOut(boss));
        // The next line prints false
        System.out.println(boss.equals(ExternalizableMistake.inOut(boss)));
    }
}

Externalizable and inheritance

Same question as above: did you see the mistake? The first three lines work just fine. The employe can be serialized and deserialized correct. The deserialized instance is equal to the original instance. Why is this not the case for Manager instances? Why would the last line print false? Why is the deserialized Manager instance we different from the the ‘boss’ instance we serialized?

Our mistake: we did not implement Externalizable for the Manager class! And the java language does not force us to. Employee is Externalizable and so is the Manager class. The JDK serialization and deserialization casts the object to a Externalizable reference and invokes the readExternal() and writeExternal() methods.

But in this example we did not override readExternal() and writeExternal() for the Manager class. That’s not a problem of course. The implementation of the parent class, the Employee class, will be used. But these methods only serialize and deserialize the ‘name’ field of the Employee class. The state of the ‘hasBigBonus’ field is forgotten.

This behaviour is very different from what we are used to when using Serializable! In that case we do not run into this problem. The algorithm implemented by the JDK’s ObjectOutputStream automatically collects all the fields, declared and inherited, of an object.

There will be no compile time warning and no exception will be thrown at runtime. This kind of error is very very hard to find.

Lesson II

Java’s inheritance model does not force us to implement the writeExternal and readExternal methods for our subclasses. If the base class implements those methods there will be no warning. Always take extra care when dealing with a hierarchy of Externalizable classes. Ensure that each class implements the necessary methods.

Mistake 3: books

Forget the ConcertTicket class and the Manager class with its big bonus. Change your mind reading a book. Why not keep track of the books in our library by writing a very simple application to do that. We’ll keep it very simple by only recording the titles and the booksnumber of pages. Instead of using a DB as a backend we simply serialize all the data to disk. Should work just fine. Look at the Book class below. Of course it implements Externalizable and its methods.

For the last time look at the main() method below.

package mistakes;

import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;

public class Book implements Externalizable {
    String title;
    short pages;

    public void writeExternal(ObjectOutput out) throws IOException {
        out.writeObject(title);
        out.writeInt(pages);
    }

    public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
        title = (String) in.readObject();
        pages = in.readShort();
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;

        Book book = (Book) o;

        if (pages != book.pages) return false;
        return !(title != null ? !title.equals(book.title) : book.title != null);

    }

    @Override
    public int hashCode() {
        int result = title != null ? title.hashCode() : 0;
        result = 31 * result + (int) pages;
        return result;
    }

    @Override
    public String toString() {
        return "Book{" +
                "title='" + title + '\'' +
                ", pages=" + pages +
                '}';
    }

    public static void main(String[] args) throws IOException, ClassNotFoundException {
        //
        Book book = new Book();
        book.title = "Great title";
        book.pages = 999;
        System.out.println("Book in:  " +book);                      // prints the original instance
        System.out.println("Book out: " +ExternalizableMistake.inOut(book));      // prints the serialized-deserialized instance
        // prints false
        System.out.println(book.equals(ExternalizableMistake.inOut(book)));
    }
}

writeExternal and readExternal: type mismatch

What’s wrong this time!?! How is it possible that the deserialized Book instance differs from the original? There are just 2 fields to serialize. A String an a single short what can go wrong.

Answer: the mistake here is the use of the wrong methods to serialize and deserialized the pages field. Instead of using the writeShort(int v) method this example uses writeInt(int v). Nothing wrong there except that the wrong method is used to deserialized the bytes. Instead of using readInt() we used readShort(). And it seems to work, no exception! But the object state is incorrect, what happens?

In the first mistake we mixed up the order in which we serialized and deserialized the fields. This resulted in a IOException at runtime because extra metadata is added in the output to detect such mistakes. Well it so happens that not all serialization methods add metadata to the output! The methods to serialize primitive types do not put extra information in the output. Types like byte, char, short, int and long get converted into bytes in a much more straightforward way compared to String and other Object types. Primitive value are converted to bytes using simple bit shift operations. No extra header, no detection of the mismatch between the deserialization method and the data in the input.

Back to our last mistake. Two things happen here. First of all notice that we use writeInt(int v) to serialize a short and get no warning. The type of conversion is safe since there is no loss of precision.  This so called widening primitive conversion (see JLS) is very handy most of the time. But here we quietly convert a 2 byte integer values into a 4 byte one. So we put 4 bytes in the output when we only needed 2 bytes. The second error stems from the fact that we use readShort(), which returns a short (so here the type system is helping us) to deserialize the data. The readShort() method just take the 2 next bytes from the input and performs some bit shifts and additions to compute the corresponding short value. Except of course the first two byte are those of a signed two’s complement representation of the integer value 999. These bytes both containing 0x00 (hex).  And the  short value consisting of all bits set to zero is 0 (zero). That’s why the pages field becomes 0 in our example.

A last important detail. In this scenario we don’t consume all the bytes in the input. There are at least 2 bytes left from the encoded integer in the input. So chances are that in a more realistic setup we’ll get an IOException when consuming the remainder of the input. But if we don’t make additional call on the ObjectInputStream, as the example shows, we will not notice the error!

Lesson III

The serialization of primitive values has some subtle pitfalls. Not in the least because the ObjectOutputStream API has several methods with an int parameter although they are meant to serialize different type. An accidental mismatch will go unnoticed at compile time. Since there are not checks for serializing primitive we can end up losing some data is during serialization.

Summary

This post has shown 3 different mistakes you can make when implementing Externalizable. In conclusion:

  • Pay close attention to the order in which you serialize (writeExternal()) and deserialize (readExternal()) fields.
  • The errors illustrated in this post can not be compile time warning only an IOException at runtime or no exception at all!
  • Take into account the fact you MUST implement Externalizable for all the subclasses of a class implements Externalizable
  • When implementing Externalizable by hand consider writing simple a simple unit test to test the seriailzation (and deserialization) of each class.

 

check-your-code-for-mistakesUse it totally FREE!

Resources