Thread: Look Ma, no files! Portable object persistence

Where can I store my application data?
I see a lot of people starting threads because they're struggling to persist (often) a little bit of data from one run of their Java application to the next. The offered solutions are often some kind of jiggery-pokery involving java.io.File. Not only is File handling a cross-platform morass (our favourite OSes can't even agree on path separators, let alone filesystem roots and GoodPlacesToPutApplicationData™), but every so often another limited-resource computing device appears, some of them without a natural filesystem at all.

So if we can't store data in files, where can we store it? Enter the Java Preferences API in the java.util.prefs package. Preferences is a platform neutral way to store a small amount of simple data. It offers support to store Strings and built-in data types and retrieve them without having to worry how they are persisted. You simply obtain a Preferences instance, store your value-to-be-persisted on a key, exit your app and sleep easy. The next time you start your app, you obtain the Preferences instance and invoke get([key you used previously], [default if nothing's there]). Ideal - for simple use.

Preferences was designed to be simple and lightweight, so it doesn't include any support for storing Java Objects. Worse, it imposes severe constraints on the size of values that can be stored. My Java 6 API docs say that a String stored by a Preferences instance can be up to 8,192 characters long (Preferences.MAX_VALUE_LENGTH). And only built-in datatypes and Strings may be stored! So if we can accept the constraints of the Preferences class, how would we use it to store arbitrary objects? Here's my first stab at it.

An Object to String bridge
We're going to store Objects as Strings. Java offers Serialization as a means of storing and retrieving objects to and from streams, so that's what we're going to do - serialize an Object to a stream. What kind of stream? A stream that creates a valid String object. Then we're going to store that String in a Preferences instance. When we later retrieve the String from the Preferences instance, we have to go back across the bridge - deserialize the String - back into our original Object-that-was-stored.

We know we can create Strings from byte arrays, and we know we can get a byte array from a String, so it seems like we should be able to use ByteArrayInputStream and ByteArrayOutputStream to bridge between byte arrays and streams. So far so good: Serialization does Object-to-Stream and the ByteArray{In|Out}putStreams will do Stream-to-byte-array. The problem with this first bridge should be evident from the API docs for the String(byte[]) constructor:

Constructs a new String by decoding the specified array of bytes using the platform's default charset. ...
The behavior of this constructor when the given bytes are not valid in the default charset is unspecified.

That's clearly not good. ObjectOutputStream creates a stream of bytes that is a valid object, not a valid String in any Charset. So to bridge this gap we need another pair of Streams between streams of valid Object bytes and streams of valid String bytes. FilterInputStream and FilterOutputStream are a pair of handy Streams for just this purpose.

What I chose to do (I can post the code if anyone is interested) was to extend Filter{In|Out}putStream to accept a stream of arbitrary bytes on one side and output a 'hex' representation of those bytes on the other side. Every byte that is written to my HexOutputStream (the class I extend from FilterOutputStream) by ObjectOutputStream is converted to a 2-character hex representation, which is in turn written to the ByteArrayOutputStream as 2 bytes. OK, so I'm obviously doubling the size of the stored object, but I can absolutely guarantee that a string of bytes which are '0' to '9' and 'a' to 'f' are valid String objects in any Charset.

So far so good - my Object to String bridge is now

mySeriliazableObject -> ObjectOutputStream -> HexOutputStream -> ByteArrayOutputStream -> new String(byte[])

and my String to Object bridge is

String.getBytes() -> ByteArrayInputStream -> HexInputStream -> ObjectInputStream -> mySerializableObject

Let's have a demo.

A suitable problem
I'm not very good at remembering birthdays, particularly bad when it's the birthday of my Significant Other. I could write a command-line application to keep a note of the birthday, but I'm a platform butterfly and I know I'm going to change my PC soon, but I can't be sure whether I'll be using an Android phone, an Ubuntu nettop, a LEGO NXT brick, an iPad or a Windo... no, let's not go too far. I want this app to remember the special birthday, so I create a serializable class so that I can create birthday instances from the keyboard and store and retrieve them from some persistence backend. I don't care what the persistence back-end is, so Serializable at least bridges my class to streams. Streams are good. I can write stuff into them, read stuff out of them, and I don't care what's at the other end.
Here's Birthday.java:

package com.javaprogrammingforums.domyhomework;
 
import java.io.*;
 
import java.util.regex.*;
 
public class Birthday implements Serializable
{
  public final static long serialVersionUID = 42l;
  public final static Pattern PAT_BIRTHDAY = Pattern.compile("^([0-9]{1,2})[a-z]{0,} ([JFMASOND][a-z]{2,8})$");
  public enum Month
   {
    January(31), February(29), March(31), April(30), May(31), June(30), July(31), August(31), September(30), October(31), November(30), December(31);
    private final int MAX_DAYS;
    Month(int maxDays)
    { MAX_DAYS = maxDays; }
    public void validate(int day) throws IllegalArgumentException
    {
      if (day < 1 || day > MAX_DAYS)
        throw new IllegalArgumentException("Not a valid day in " + this + ": " + day);
    }
   };
  private final int day;
  private final Month month;
  public Birthday(int dayOfMonth, Month m) throws IllegalArgumentException
  {
    m.validate(dayOfMonth);
    day = dayOfMonth;
    month = m;
  }
  public static Birthday parse(String s) throws IllegalArgumentException, NullPointerException
  {
    if (s == null)
      throw new NullPointerException();
    Matcher mat = PAT_BIRTHDAY.matcher(s);
    if (!mat.matches())
      throw new IllegalArgumentException("Bad format for birthday: '" + s + "'");
    try
    {
      Month m = Enum.valueOf(Month.class, mat.group(2));
      return new Birthday(Integer.parseInt(mat.group(1)), m);
    }
    catch (Exception e)
    {
      throw new IllegalArgumentException("Bad month: '" + mat.group(2) + "'", e);
    }
  }
  public String toString()
  {
    return month + " " + day;
  }
}

Marvellous, a Birthday class that stores only a day-of-month and a month as an int and an Enum. Should be plenty good enough for a demo of Object-into-Preferences.

Now for the application. The application PreferencesSOBirthday starts up, tries to retrieve a Birthday from the Preferences back-end, displays it as a reminder if there's one in there. If there isn't a Birthday object in the Preferences instance, it asks you for one and stores it. Nowhere in the code will you find any mention of a File. For that matter, nowhere in the code will you find any explicit choice of persistent storage at all - all it does is to obtain a Preferences instance that's suitable for the user/application, and stores the object. The next time you run the application - tadaa - the object is magically there!

package com.javaprogrammingforums.domyhomework;
 
import java.io.*;
 
import java.util.prefs.Preferences;
 
public class PreferencesSOBirthday
{
  private final static String PREFS_KEY_BIRTHDAY = "significant.other.birthday";
  public static void main(String[] args) throws Exception
  {
    /* may throw SecurityException - just die */
    Preferences prefs = Preferences.userNodeForPackage(PreferencesSOBirthday.class);
    if (args.length > 0 && "reset".equalsIgnoreCase(args[0]))
    {
      System.out.println("Command line argument 'reset' found - data removed.");
      prefs.remove(PREFS_KEY_BIRTHDAY);
    }
    // fetch the string-encoded object
    String sSOBD = prefs.get(PREFS_KEY_BIRTHDAY, null);
    if (sSOBD != null)
      try
      {
        /* check preferences string actually contains a Birthday */
        getBirthday(sSOBD);
      }
      catch (Exception e)
      {
        System.out.println("Ouch, my brain hurts!");
        sSOBD = null;
        prefs.remove(PREFS_KEY_BIRTHDAY);
      }
    while (sSOBD == null)
    {
      System.out.print("OMG are you in trouble, I don't know your SO's birthday - enter it now: ");
      BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
      String sBirthday = br.readLine();
      if (sBirthday == null)
      {
        System.out.println("OK, be like that then");
        System.exit(0);
      }
      try
      {
        Birthday b = Birthday.parse(sBirthday);
        sSOBD = toPreference(b);
        prefs.put(PREFS_KEY_BIRTHDAY, sSOBD);
      }
      catch (Exception e)
      {
        System.out.println(e);
        sSOBD = null;
      }
    }
    System.out.println("Be prepared - your Significant Other's birthday is " + getBirthday(sSOBD));
  }
  /* String to Birthday bridge */
  private static Birthday getBirthday(String s) throws NullPointerException, IOException, ClassNotFoundException
  {
    return (Birthday)new ObjectInputStream(new HexInputStream(new ByteArrayInputStream(s.getBytes()))).readObject();
  }
  /* Birthday to String bridge */
  private static String toPreference(Birthday b) throws IOException
  {
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    ObjectOutputStream oos = new ObjectOutputStream(new HexOutputStream(baos));
    oos.writeObject(b);
    oos.close();
    return new String(baos.toByteArray());
  }
}

I've tested this on Ubuntu and Windows XP. The code was compiled on Ubuntu and I copied the class files over to Windows XP and they worked exactly the same there. You'll have to run the application once to be told there's no data and enter a date - the application exits. Run it again, it always 'remembers' the date. Run it with a single command line argument of 'reset' and it'll remove the object (OK, the encoded string value) from the Preferences instance. Here's a command line to run the app:

java com.javaprogrammingforums.domyhomework.PreferencesSOBirthday

Remember to run the application at least twice, or you won't see persistence 'in action'. Here's a command line to reset (remove) the stored data:

java com.javaprogrammingforums.domyhomework.PreferencesSOBirthday reset

Where next?
It shouldn't be difficult to turn this trivial example into a general-purpose Object Persistance engine based on the Preferences API, but I would caution against it because of the arguments put forward by Sun in the first article linked above. Preferences really isn't for arbitrary Object storage, it's for storing simple configuration data. Still, if a solution such as the one above fits your requirements and you are disciplined enough to avoid the temptation to base a mega-project on it instead of using a proper persistence back-end, then I think it has some kilometrage. Enjoy.

edit: Some time later I realised you can't actually run this without the HexStreams classes. I'm not going to post those right now, because I think they're not necessary to understand what's going on. Attached instead is an executable jar file - execute it with 'java -jar SOBirthday.jar'

nother edit: Just seen an article at IBM by an author who can read the API docs and notice the putByteArray() method in Preferences! That would make the code slightly less complicated, but not much. The IBM article also suggests breaking your objects up into under-8KB chunks, but I think if you're going to go that far, you probably need a proper DB: http://www.ibm.com/developerworks/ja...api/index.html

Great post Sean4u. I've promoted it to a front page article.