All Articles

Pieter Hintjens is a writer, programmer and thinker who has spent decades building large software systems and on-line communities, which he describes as "Living Systems". He is an expert in distributed computing, having written over 30 protocols and distributed software systems. He designed AMQP in 2004, and founded the ZeroMQ free software project in 2007.

He is the author of the O'Reilly ZeroMQ book, "Culture and Empire", "The Psychopath Code", "Social Architecture", and "Confessions of a Necromancer." In April 2016 he was diagnosed with terminal metastasis of a previous cancer.


C4.1 - an Exothermic Process
In February 2012 the ZeroMQ community said "adieu" to its long time maintainers, and set sail in a new direction. Using a new process called C4.1 as its compass, we headed off into unknown territory. Looking back three and a half years later, what is the result? Did ZeroMQ lose its way, as many predicted and feared? Or did the new process work as planned?

date.png16 Sep 2015 20:49 | comments.png 2 Comments | 0

A Protocol for REST over ZeroMQ
In this article I'll descibe a popular way of making really awful protocols, using "RPC" or Remote Procedure Calls. My goal is to teach you to never use RPC, ever again. As a safer and saner alternative I'll explain how to do RESTful work over ZeroMQ.

date.png09 Feb 2015 10:51 | comments.png 2 Comments | 0

The End of Software Versions
Software version numbers are the crack cocaine of change management. They are an easy and attractive path to serious long term stress and pain. The worst ever distress in the ZeroMQ community came from allowing library version numbers to define compatibility. The reasons are real, yet subtle and often counter-intuitive. In this article I'll explain in detail, and propose a replacement for software versioning.

date.png06 Feb 2015 14:20 | comments.png 0 Comments | 0

Revisting ZeroMQ Multipart
Around FOSDEM 2015, a few core ZeroMQ maintainers decided to experiment with a simpler messaging API, and decided it'd be profitable to explore rethinking multipart messages from ZeroMQ. In this article I'll explain the why's and how's of this potential shift.

date.png03 Feb 2015 09:57 | comments.png 3 Comments | 0

Solving the Makefile Problem
The ZeroMQ community started many years ago as a group of low-latency fanatics who focused their "make it cheaper to build distributed systems" mission on a single library, libzmq. The mission hasn't changed, yet the focus is wider than ever. In this article I will talk about a young project, zproject, which makes it cheaper to build other projects that use ZeroMQ.

date.png14 Nov 2014 11:16 | comments.png 0 Comments | 5

Generating Servers with zproto
While ZeroMQ gives you a powerful communications engine to use in many different ways, building a conventional server is still fairly solid work. We use the ROUTER socket for that, managing the state for each individual client connection. Today I'll present a new tool — zproto — that generates whole servers, in C, from state machine models.

date.png29 Mar 2014 13:12 | comments.png 5 Comments | 0

Cluster load-balancing using ZeroMQ over TIPC
Erik Hugne writes, "I've cooked up another example with real code illustrating load-balancing using the new TIPC transport for ZeroMQ. This demo uses overlapping port names." The code is available to download.

date.png06 Nov 2013 12:56 | comments.png 0 Comments | 0

The New TIPC Transport in ZeroMQ
ZeroMQ now supports TIPC, the cluster IPC protocol developed by Ericsson. This will be part of the next major 4.1 release. In this article I'll interview Ericsson's Erik Hugne, the engineer who added this transport to ZeroMQ.

date.png04 Nov 2013 15:08 | comments.png 0 Comments | 0

ZeroMQ Certificates, Design Iteration 1
In the ongoing search for a new certificate format for ZeroMQ, I kicked off a nice lively thread on zeromq-dev with some to and for, little agreement, and lots of ideas. In this article I'll present Design Iteration 1, a strawman that people can beat up. The format is inspired by the SSH2 public key format.

date.png14 Oct 2013 12:20 | comments.png 1 Comments | 0

Securing ZeroMQ: Soul of a New Certificate
While wrapping ZeroMQ's new security API up in the high-level C binding, I accidentally a certificate format. With all respect to X.509 and existing PKI standards, the reason I built CURVE for ZeroMQ was to get simple and foolproof security. Using a complex legacy certificate format would spoil the soup. I've no idea what the right format is, but to start, I'm going to try to collect requirements.

date.png01 Oct 2013 07:54 | comments.png 6 Comments | 0

Using ZeroMQ Security (part 2)
In the previous article I gave an overview of how and why ZeroMQ's security layers work. In this article I'll develop a simple secure application, step by step. We'll use a simple example of a server PUSH socket sending "Hello" to a client PULL socket. We'll work through the ZeroMQ NULL, PLAIN, and CURVE security mechanisms, up to full authentication. The examples are in C but the principles apply to all languages.

date.png19 Sep 2013 21:07 | comments.png 15 Comments | 0

Using ZeroMQ Security (part 1)
In this series of articles I'll explain how to use the new ZeroMQ security layers in your applications. The example we'll make is a chat application that provides unbreakable strong security. In this first article, I'll explain more about ZeroMQ's security technology, how it works, and why we chose it. (Read part 2.)

date.png12 Sep 2013 09:17 | comments.png 0 Comments | 0

Charlie and the X-ray Factory
Last week I had the pleasure of visiting not one major European physics research facility, but two. My first stop was at ESRF in Grenoble, then I went to visit CERN in Geneva. Both these organizations are moving to use ZeroMQ for their control systems. In this article I'll explain what that means.

date.png07 Jul 2013 12:26 | comments.png 0 Comments | 0

Securing ZeroMQ: Circus Time
Thanks to the quiet but persistent work of Martin Hurton, the master branch of libzmq, the ZeroMQ core library, now "does security". In this last article in the mini-series, "Securing ZeroMQ", I'll explain what we built, and why, and how this can work for your ZeroMQ applications.

date.png04 Jul 2013 16:59 | comments.png 1 Comments | 0

A Web Server in 30 Lines of C
There are developers who never built distributed systems. Then there are developers who have done this before, using a messaging system. Then there are those who've done it the hard way, using BSD sockets. Privately, I call these the Good, the Bad, and the Ugly, because of how they respond when they meet ZeroMQ and read the Guide. Today, "Getting Started with ZeroMQ for Uglies". With a "Hello, World" web server in —40— 30 lines of C, just for fun.

date.png29 Apr 2013 09:57 | comments.png 6 Comments | 0

Securing ZeroMQ: draft ZMTP v3.0 Protocol
In the previous article in this series, "Securing ZeroMQ", I showed a proof-of-concept for CurveZMQ. Now we're moving that into the ZeroMQ protocol, ZMTP. It's not a small change. You can't just sprinkle security over a protocol like chocolate chips onto pancakes. It means a new protocol, and this gives us a chance to address other problems with ZMTP. The result is ZMTP v3.0. In this article I'll explain ZMTP v3.0. If you never read the 1.0 or 2.0 spec, don't worry, I'll cover those briefly too.

date.png14 Apr 2013 11:15 | comments.png 0 Comments | 0

The PUB-SUB Hacks: Census
In the first article on PUB-SUB hacks, we looked at the Meerkat pattern. In this second article I'll walk through a pattern—Census—that's loosely inspired by the "Surveyor" socket type from the defunct Crossroads fork of ZeroMQ. It's a fairly simple pattern. The code for this article is in a GitHub gist.

date.png26 Mar 2013 19:28 | comments.png 1 Comments | 0

The PUB-SUB Hacks: Meerkat
While the ØMQ pub-sub pattern is well-known for being a one-way pattern, it does in fact support a two-way flow of information. SUB sockets (or more accurately, XSUB sockets) can send messages to XPUB sockets. This week, I'll post a series of short articles that cover different patterns—Meerkat, Census, and Repeater—we can build using this functionality. As usual, I'll demonstrate each pattern with code you can download and run.

date.png26 Mar 2013 18:44 | comments.png 1 Comments | 0

Securing ZeroMQ: CurveZMQ protocol and implementation
This week, we flesh out the basics of our CurveCP-derived protocol, with an implementation in C that I'm calling CurveZMQ. In this article I'll explain what this simple but powerful security protocol looks like. The code here will already work over 0MQ sockets but our next stage is to move this into libzmq itself, for all sockets over tcp:// and ipc://. So stay tuned!

date.png22 Mar 2013 22:19 | comments.png 0 Comments | 0

Securing ZeroMQ: the Sodium Library
Marc Falzon (@falzm) pointed me to libsodium, aka Sodium, a repackaging of the NaCl library. The Sodium README says that it's, "tested on a variety of compilers and operating systems, including Windows, iOS and Android," and "tests and benchmarks will be performed at run-time, so that the same binary package can still run everywhere." This fixes the biggest problem with NaCl, which is that it optimizes at compile-time, so you can't ship it in binaries. Today I tried Sodium, and here are the results of my tests.

date.png12 Mar 2013 10:48 | comments.png 0 Comments | 0

Securing ZeroMQ: CurveCP and NaCl
One of the biggest user requests for ØMQ is a good security layer. Mainstream options like TLS/SSL are complex, slow and designed for web browsing, not high-speed messaging. In this article I'll present CurveCP, one of the most exciting security developments in recent years. It's part of the NaCl networking and cryptography library and looks perfectly suited to ØMQ. To demonstrate this, I've made a "Hello World" proof of concept that shows an authenticated, confidential connection from one client to one server, over ØMQ.

date.png11 Mar 2013 21:29 | comments.png 2 Comments | 0

Solving the Discovery Problem
One long-standing question for 0MQ developers is how to discover services on the network. A lot of people have built answers to this, such as ZeroConf and UPnP, or even DNS, but they tend to be over-complex and unfriendly to application developers. In this article I'll explain zbeacon, which is a new module in the CZMQ binding for 0MQ that does service discovery.

date.png26 Feb 2013 15:05 | comments.png 0 Comments | 0

Credit-based Flow Control
Using network buffers - such as 0MQ's queues - has a disadvantage when you are sending data to many readers. Your writer will block when the buffer is full. To avoid this, you can use non-blocking writes, and poll for sockets that are 'ready for writing'. But here's an alternative that dispenses with high-water marks and blocking writes. It's called "credit-based flow control".

date.png19 Jul 2011 14:58 | comments.png 6 Comments | 0

MOPED - a Message-Oriented Pattern for Elastic Design
People learning to build distributed systems with 0MQ for the first time (there's a protocol joke in there somewhere) can make some awful mistakes. One cannot just convert design approaches like "let's make a structure of classes" into "let's make a structure of tasks". So, here's a simple design pattern, that I call MOPED, for building distributed systems. Surprisingly, MOPED works, and is fun.

date.png16 Jun 2011 15:45 | comments.png 0 Comments | 0

Clouds and Unprotocols
The future seems clear. Mobile applications, clouds, and unprotocols. My previous gig was CEO of Wikidot, which runs this site. It's a cloud. Cheap, scalable, instantly available to anyone who needs it. My next gig will be portable applications. Always available, smart, geolocated, social. Put the two together and you have something magic, as Google knows.

date.png01 Jun 2011 02:45 | comments.png 0 Comments | 0

Smallest Stack Ever
@dln_eintr (Daniel Lundin) broke two or three records with this one: zmq_push() { m=$(cat) && echo -e $(printf '\\x01\\x00\\x%02x\\x00%s' $((1 + ${#m})) "$m") | nc -q1 $@; }

date.png06 May 2011 08:35 | comments.png 0 Comments | 0
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License