Thread: Ways of Transferring Data

Let me start of with an introduction. My name is Zyle. I have been programming with Java since late 2006. It would be safe to say that I've programmed more than 4 hours a day since then. I am currently attending a university for Software Development and Software Application Technology. I wouldn't say that I know a lot, but I certainly have experience with Java, as well as other popular languages.

Now that you know a little bit about me, let's get to the good stuff. I am currently trying to create a simple way for transmitting data in a client-server model. I have used the standard java.net classes in the past, and have recently been working with NIO. I have used MINA and Netty, and while they both have their pros/cons, I'd prefer to write my own library. I have asked this question in another community (not specifically focused on Java), and got different answers. However I though that I'd post here to see what your opinions are. (Note that the data transferred will be using the TCP, not UDP)

Currently, I am thinking about having 5 classes.

1. Packet - Pretty simple, just a class to hold data being transmitted to the client/server.
2. Client - Just a class to represent a client connected to the server.
   
   public class Client {
       private final SocketChannel channel_;
    
       protected Client(SocketChannel channel) {
           channel_ = channel;
       }
    
       public final SocketChannel getChannel() {
           return channel_;
       }
   }

3. PacketDecoder - Again pretty simple. I'm thinking that it should look something like
   
   public interface PacketDecoder {
       public Packet decode(Client client, ByteBuffer buffer);
   }

   Basically what it will do is convert some data from a ByteBuffer into a usable Packet object.
4. PacketHandler - These classes will be what perform the logic for the packet.
   
   public interface PacketHandler {
       public void handle(Client client, Packet packet);
   }

5. PacketEncoder - These classes will turn a Packet object into a ByteBuffer which can be written directly to the client/server.
   
   public interface PacketEncoder {
       public ByteBuffer encode(Client client, Packet packet);
   }

Note that this is just a brief summary of how I see it. Now that you have all the background information, here are my questions:

1. What are any/all improvements you can suggest for this design?
2. How should I be handling Packets? Should each packet have its own class? Should I have the size of each packet predetermined, or have the length in the first 1-2 bytes before the data?
3. If each Packet were its own class, how would I handle the PacketEncoder? If I have all the data hard-coded into the PacketEncoder, then I end up with an enormous switch statement. If I were to add a
   
   public ByteBuffer getData();

   method to the Packet class, then wouldn't that defeat the purpose of the PacketEncoder?

I had more questions but neglected to write them down. I'll add them to the list when I can remember them.

Thank you for reading my thread, if you have any information that might be of assistance, I would be grateful for your response.

Hello there Zyle,

I would really consider using a library already available out there, maybe MINA and Netty aren't to your liking but it's well worth it not having to deal with all of the network stuff. I would suggest you have a look at KryoNet which is a networking library built on top of the Kryo Serialization Library. It offers really nice ways of sending POJO's across the wire.

However if you still insist on writing your own, I'd say you're on the right track, the important thing I'd say is the encoder/decoder of packets. If the first byte in the packet bytearray defines what type of packet you're dealing with the decoder will easily know how to decode it (what things to read and in what order).

In all fairness, Packet could be an interface with a read method and a write method. You would also need a way of registering your classes.

For instance your Packet.java interface could look like this:

import java.nio.ByteBuffer;
 
public interface Packet {
 
    public void encode(final ByteBuffer byteBuffer);
 
    public void decode(final ByteBuffer byteBuffer);
}

And you could have a packet implementation like this:

import java.nio.ByteBuffer;
 
public class SimplePacket implements Packet {
 
    private int someInteger;
 
    private float someFloat;
 
    @Override
    public void encode(final ByteBuffer byteBuffer) {
        byteBuffer.putInt(this.someInteger);
        byteBuffer.putFloat(someFloat);
    }
 
    @Override
    public void decode(final ByteBuffer byteBuffer) {
        this.someInteger = byteBuffer.getInt();
        this.someFloat = byteBuffer.getFloat();
    }
 
    // Setters and getters here
}

Then you might need some sort of converter or similar to translate packets and bytebuffers:

import java.nio.ByteBuffer;
import java.util.HashMap;
import java.util.Map;
 
public class PacketConverter {
 
    private static Map<Integer, Class<? extends Packet>> packetClasses = new HashMap<Integer, Class<? extends Packet>>();
 
    public synchronized static void registerClass(final Class<? extends Packet> packetClass) {
        final int id = packetClasses.size() + 1;
        packetClasses.put(id, packetClass);
    }
 
    private int getIdForPacketInstance(final Packet packet) {
        for (final Map.Entry<Integer, Class<? extends Packet>> entry : packetClasses.entrySet()) {
            if (entry.getValue().getName().equals(packet.getClass().getName())) {
                return entry.getKey();
            }
        }
 
        return -1;
    }
 
    public ByteBuffer serialize(final Packet packet) {
        final int id = getIdForPacketInstance(packet);
 
        if (id != -1) {
            final ByteBuffer byteBuffer = ByteBuffer.allocate(Integer.MAX_VALUE);
            byteBuffer.putInt(getIdForPacketInstance(packet));
            packet.encode(byteBuffer);
            byteBuffer.flip();
 
            return byteBuffer;
        }
 
        return null;
    }
 
    public Packet deserialize(final ByteBuffer byteBuffer) throws Exception {
        final int packetId = byteBuffer.getInt();
        final Class<? extends Packet> packetClass = packetClasses.get(packetId);
 
        if (packetClass != null) {
            final Packet packet = packetClass.newInstance();
            packet.decode(byteBuffer);
 
            return packet;
        }
 
        return null;
    }
}

As you can see the PacketConverter allows you to register packet classes. This model is really similar to the KryoNet model which is why I would suggest you have a look at that first since that would take care of the object serialization for you, sparing you from dealing with the bytebuffers.

Also note that I've not tested this code so there might be issues.

Hope this helps you in some way.


Daniel