First steps to NOM-1

pieterhpieterh wrote on 14 Jun 2011 12:35

lab.png

NOM-1 is a lightweight messaging unprotocol for #zeromq over UDP. The name stands for "NOM-oriented messaging". There's a good reason for the cuteness. If I gave it a confidence-inspiring name like "Message-oriented Datagram Protocol", people might use it without further thought. The silly name will make life and public acceptance hard for NOM-1. So it's going to have work really hard to succeed.

NOM-1 provides the basic support for 0MQ socket patterns, over UDP. It's the first in a series of unprotocols I want to build. They all have the same philosophy: to extend the 0MQ transport family with new, unusual, experimental, and hopefully useful protocols.

If you've used UDP you'll know it's quite fun. No connections, you just send to a hostname:port, and if there's someone to catch the ball, a message arrives. If not, it gets dropped. And UDP will drop messages randomly anyhow. And it limits them to about 500 bytes. But it's fast, simple, doesn't do weird stuff behind the scenes.

It turns out that 0MQ and UDP fit together neatly. Like UDP, 0MQ doesn't really have connections. They kind of come and go invisibly. It also drops messages when it gets bored. So, building some simple 0MQ-friendly semantics on top of UDP was fun.

What you get is not like TCP at all. Here is what NOM-1 does already:

  • It has a simple framing for commands and messages.
  • It allows peers to establish 'peerings' (like connections but with fewer overloaded semantics and letters to type)
  • It does heartbeating so that peers can destroy or disconnect peerings rapidly.
  • It does broadcast peerings (this is fun, you connect to a host called '*' and the first that replies becomes the endpoint for the peering).
  • It does asynchronous message transfer (single parts, no reliability)
  • It does peering reconnection as peers go away and come back.

All this is implemented in the VTX UDP driver, which is the reference implementation for NOM-1. I'll probably make a stand-alone stack as a learning tool, because VTX is kind of abstract (thus complex) in itself.

Next up will be basic request-reply retries, needed since UDP randomly drops packets and this'll kill any 0MQ REQ sockets waiting for an answer. It's still not TCP. For example, pub-sub or push-pull will work at full speed without any acknowledgements or retries. Unreliable pipelines? Why not.

My test case is a simple client that does a broadcast connect, sends 'hello?' and waits for a response. The server does a bind, waits for a request, and replies with 'acknowledge'.

Here's the client trace:

ph@ws200901:~/work/vtx/v3 :master$ vtx_testcli
11-06-14 14:11:33 I: create peering=0x25aee30 - 192.168.55.255:32000
11-06-14 14:11:33 C: hello?
11-06-14 14:11:34 I: send [OHAI] - 20 bytes to 192.168.55.255:32000
11-06-14 14:11:35 I: send [OHAI] - 20 bytes to 192.168.55.255:32000
11-06-14 14:11:36 I: send [OHAI] - 20 bytes to 192.168.55.255:32000
11-06-14 14:11:37 I: send [OHAI] - 20 bytes to 192.168.55.255:32000
11-06-14 14:11:38 C: hello?
11-06-14 14:11:38 I: send [OHAI] - 20 bytes to 192.168.55.255:32000
11-06-14 14:11:38 I: recv [OHAI-OK] - 22 bytes from 192.168.55.156:35860
11-06-14 14:11:38 I: rename peering from 192.168.55.255:32000 to 192.168...
11-06-14 14:11:38 I: bring up peering=0x25aee30
11-06-14 14:11:38 I: send [NOM] - 6 bytes to 192.168.55.156:35860
11-06-14 14:11:38 I: send [NOM] - 6 bytes to 192.168.55.156:35860
11-06-14 14:11:38 I: recv [NOM] - 13 bytes from 192.168.55.156:35860
11-06-14 14:11:38 I: recv [NOM] - 13 bytes from 192.168.55.156:35860
11-06-14 14:11:39 C: hello?
11-06-14 14:11:39 I: send [HUGZ] - 0 bytes to 192.168.55.156:35860
11-06-14 14:11:39 I: send [NOM] - 6 bytes to 192.168.55.156:35860
11-06-14 14:11:39 I: recv [HUGZ] - 2 bytes from 192.168.55.156:35860
11-06-14 14:11:39 I: send [HUGZ-OK] - 0 bytes to 192.168.55.156:35860
11-06-14 14:11:39 I: recv [HUGZ-OK] - 2 bytes from 192.168.55.156:35860
11-06-14 14:11:39 I: recv [NOM] - 13 bytes from 192.168.55.156:35860
11-06-14 14:11:40 C: hello?
11-06-14 14:11:40 I: send [NOM] - 6 bytes to 192.168.55.156:35860
11-06-14 14:11:40 I: recv [NOM] - 13 bytes from 192.168.55.156:35860
11-06-14 14:11:41 C: hello?
11-06-14 14:11:41 I: send [NOM] - 6 bytes to 192.168.55.156:35860
11-06-14 14:11:41 I: recv [NOM] - 13 bytes from 192.168.55.156:35860
11-06-14 14:11:42 C: hello?
11-06-14 14:11:42 I: send [NOM] - 6 bytes to 192.168.55.156:35860
^C11-06-14 14:11:43 I: shutting down driver, 0 errors
11-06-14 14:11:43 I: delete peering=0x25aee30 - 192.168.55.156:35860

And here's the server trace:

ph@ws200901:~/work/vtx/v3 :master$ vtx_testsrv
11-06-14 14:11:38 I: recv [OHAI] - 22 bytes from 192.168.55.156:48680
11-06-14 14:11:38 I: create peering=0xda7f00 - 192.168.55.156:48680
11-06-14 14:11:38 I: send [OHAI-OK] - 20 bytes to 192.168.55.156:48680
11-06-14 14:11:38 I: bring up peering=0xda7f00
11-06-14 14:11:38 I: recv [NOM] - 8 bytes from 192.168.55.156:48680
11-06-14 14:11:38 I: recv [NOM] - 8 bytes from 192.168.55.156:48680
11-06-14 14:11:38 S: acknowledge
11-06-14 14:11:38 I: send [NOM] - 11 bytes to 192.168.55.156:48680
11-06-14 14:11:38 S: acknowledge
11-06-14 14:11:38 I: send [NOM] - 11 bytes to 192.168.55.156:48680
11-06-14 14:11:39 I: send [HUGZ] - 0 bytes to 192.168.55.156:48680
11-06-14 14:11:39 I: recv [HUGZ] - 2 bytes from 192.168.55.156:48680
11-06-14 14:11:39 I: send [HUGZ-OK] - 0 bytes to 192.168.55.156:48680
11-06-14 14:11:39 I: recv [NOM] - 8 bytes from 192.168.55.156:48680
11-06-14 14:11:39 I: recv [HUGZ-OK] - 2 bytes from 192.168.55.156:48680
11-06-14 14:11:39 S: acknowledge
11-06-14 14:11:39 I: send [NOM] - 11 bytes to 192.168.55.156:48680
11-06-14 14:11:40 I: recv [NOM] - 8 bytes from 192.168.55.156:48680
11-06-14 14:11:40 S: acknowledge
11-06-14 14:11:40 I: send [NOM] - 11 bytes to 192.168.55.156:48680
11-06-14 14:11:41 I: recv [NOM] - 8 bytes from 192.168.55.156:48680
11-06-14 14:11:41 S: acknowledge
11-06-14 14:11:41 I: send [NOM] - 11 bytes to 192.168.55.156:48680
^C11-06-14 14:11:42 I: shutting down driver, 0 errors
11-06-14 14:11:42 I: delete peering=0xda7f00 - 192.168.55.156:48680

I'll have a draft spec up real soon now.

Comments

Add a New Comment
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License