Java, Bytes, the Network and parsing ( 2 Views )

1. Hello all,
   
   I am a bit on the low side with sleep, so it is likely that I am overlooking an easy solution for my problem, but I am getting rather frustrated with Java right now.
   
   The setup is this: I have a C++ server and a Java client that talk to each other using sockets (TCP). I have defined a couple of "commands" which consist of a 'prefix', which determines the command and one or more pieces of binary data.
   Unfortunately, Java uses 2-byte chars and my C++ program 1-byte chars, so most of my processing is done on bytes in the java world.
   
   I have no problem sending byte arrays over to C++ and they get parsed and exectued correctly on that end. During a certain command (Simulation.Step) my C++ program will respond with multiple commands. While my parser on the Java side works fine if the socket's buffer only has one command in it, it fails miserably on multiple on them, because my extension of the Socket class does not deal with multiple commands being in the Buffer when the "read" function is called.
   
   I was initially using a BufferedInputStream as my reader, but am thinking that the PushbackInputStream may be the better alternative, since I want to only return a single command on every "read" invocation.
   
   Getting to the point: My commands are variable length and I am calling InputStream.read() with a buffer that is slightly larger than the longest command that I have currently implemented. Since I want to only read one command, I would like to push back everything that I did not read onto the stream. I parse commands using a static function of my OpnetCommand class: OpnetCommand fromNetwork(ByteBuffer b) which is overloaded to correctly instantiate a subclass that is equivalent of the received command.
   
   I was considering to advance the "position" pointer and then run "compact" on the buffer in oder to push back the whole thing....
   
   ok, I think I just figured out how to do it... ooops.
   
   
   (gülcan, Falkland Islands (Malvinas))


2. Amazing how talking to noone solves a problem :p
   
   
   (RECEP, Oman)


3. I see that you've found a solution, but I'll post a little on this anyway.
   
   In my experience, only two techniques exist to delimit messages in a network byte stream:
   
   1. End-delimited: Examine the data and eventually find a byte that tells you you've hit the end. Some protocols use newlines for this, perhaps for others the 10th tab character terminates the message.
   
   2. Header-based: You can sensibly separate the message into two parts, a header (which contains explicit or implicit length info) and a remainder. Once you've processed the header, you can calculate the length of the remainder.
   
   I suppose you could create a hybrid scheme, with the undesirable characteristics of both.
   
   For the first, a correct solution would examine each byte in sequence and (conceptuatlly) move it to another buffer. When you identify the final byte, stop processing. Typically you'll need to maintain a little bit of state.
   
   For the second, you'll need to isolate your header. Typically it will be fixed-length or delimited. After this, calculate the length of the remainder and transfer those bytes.
   
   After you have a correct solution, you can start looking at faster solutions which use bulk transfers rather than single-byte transfers.
   
   Also remember that you may receive only part of a message. If your software assumes that your buffer always contains at least one complete message then it contains a bug which is difficult to test for.
   
   
   (gökyüzü, Dominican Republic)


4. yes, sometimes just writing a problem down offers a new perspective on it, solving it. :)
   
   
   (&@esma@, Christmas Island)


5. Quote:

   Originally Posted by MonkeyShave I see that you've found a solution, but I'll post a little on this anyway. In my experience, only two techniques exist to delimit messages in a network byte stream:

   it's not the delimiting that is the problem it is retaining the extra messages for the next invocation of the "read" function. And unfortunately, I have yet to solve my problem :/
   
   

   Quote:

   Originally Posted by MonkeyShave 1. End-delimited: Examine the data and eventually find a byte that tells you you've hit the end. Some protocols use newlines for this, perhaps for others the 10th tab character terminates the message.

   That is what I am doing each of my command ends in the character _ followed by 0x0A (newline I think) and then 0x00.
   

   Quote:

   Originally Posted by MonkeyShave 2. Header-based: You can sensibly separate the message into two parts, a header (which contains explicit or implicit length info) and a remainder. Once you've processed the header, you can calculate the length of the remainder.

   The first two characters of the incoming command uniquely determine its size as well.
   
   

   Quote:

   Originally Posted by MonkeyShave For the first, a correct solution would examine each byte in sequence and (conceptuatlly) move it to another buffer. When you identify the final byte, stop processing. Typically you'll need to maintain a little bit of state.

   that is pretty much what I am doing in C++, but I am having a hard time with Java due to the lack of pointers. I guess my OO programming isn't good enough. In C++ I can return a Pointer to a sub-set of a buffer. I guess I should look into the "slice" operation on ByteBuffers.
   
   

   Quote:

   Also remember that you may receive only part of a message. If your software assumes that your buffer always contains at least one complete message then it contains a bug which is difficult to test for.

   thanks for re-reminding me of that. It's something I need to triple check now.
   
   
   (DESPERADOOOOO, Chad)

Related Topics ... (or search in 1.720.883 topics !)