All Articles

Pieter Hintjens is a writer, programmer and thinker who has spent decades building large software systems and on-line communities, which he describes as "Living Systems". He is an expert in distributed computing, having written over 30 protocols and distributed software systems. He designed AMQP in 2004, and founded the ZeroMQ free software project in 2007.

He is the author of the O'Reilly ZeroMQ book, "Culture and Empire", "The Psychopath Code", "Social Architecture", and "Confessions of a Necromancer." In April 2016 he was diagnosed with terminal metastasis of a previous cancer.


Confessions of a Necromancer
Thirty-five years I've written code, a necromancer weaving spells to bring the dead to life. Hardware and electronics never held any charm for me. I've no love for chips and cables and solder. Give me a keyboard, a screen, and a language, and you have my attention. Thirty-five years produced a lot of work. So I thought, maybe time to talk about some of those projects.

date.png20 Sep 2016 20:38 | comments.png 23 Comments | 5

Conferences and Kids
I've taken my daughter, now 13, to FOSDEM in Brussels every year that I had slots there. She isn't a geek, yet enjoys the crowds and the freebies. When I could, I also took my kids to other events, where I was speaking. In this post I'd like to capture my feelings about why children should be part of conferences, and what conferences can do to make this easier.

date.png24 Aug 2016 19:39 | comments.png 4 Comments | 4.5

Fighting Cancer
There are no easy conversations when it comes to dying. Especially when it comes to a disease like cancer, which eats us up from the inside, a betrayal by our own cells. "Fight it," people still tell me. "Don't give up! We need you!" This notion that cancer is a fight… it's one I want to break down, and then rebuild, in this article. I've come to believe that death can be a positive social experience. Let me explain…

date.png18 Aug 2016 10:08 | comments.png 19 Comments | 5

Pack Hunters of the Silicon Savannas
People seem to like my book "The Psychopath Code". Part of it is, I think, that the underlying models are solid. Once we see psychopaths as social predators, we can decode much of their behavior. It all makes sense. In this article I'm going to take the predator model in another direction, to think about the future of AI, and of humanity itself.

date.png18 Jul 2016 11:49 | comments.png 4 Comments | 5

Living, in Limbo
I've written a lot in my life. This is perhaps the most difficult piece I've ever done. In April, with a diagnosis of terminal cancer, I prepared to die. It's now July, and I'm still not dead. Instead, I'm in an in-between state, neither healthy nor obviously sick. Limbo is a strange land.

date.png04 Jul 2016 09:40 | comments.png 8 Comments | 5

Social Architecture FAQ
Kevin Meredith asked me an interesting question on Twitter. What's the best way to answer this?, I wondered. A tweet is so short. A full blog post so clumsy. So here's the Social Architecture FAQ, which I will update with other questions if/when people ask them. LET F=1.

date.png23 May 2016 17:15 | comments.png 8 Comments | 0

So Far, So Good
It's been two weeks since my last update. People ask me, every day, "how are you doing?" so I figured it's time to sketch this out in more detail.

date.png19 May 2016 14:57 | comments.png 9 Comments | 0

Review of the Astrohaus Freewrite
At the end of February I put down $449.00 (plus VAT and shipping to a total of $578) on a Freewrite. It was a stupid impulse buy on Kickstarter of all places. Yet I've been searching for a portable daylight readable writing tool for years. Decades, even. Yes, laptops are great. Yet it takes extra effort to zone out of all the possibles (ooh, let me check Twitter!), and into the writing. And today a package arrived. So, obviously, a review. Written on my steampunkwriter and unedited from here on.

date.png17 May 2016 12:02 | comments.png 1 Comments | 0

Building Online Communities
This article packages decades of my experience and practice in building online communities. It is a discipline I call "Social Architecture." This text originates from my books Culture and Empire, chapter 2, and ZeroMQ - The Guide, chapter 6. This is a long article. Note: this was an early draft of my new book, "Social Architecture".

date.png05 May 2016 06:43 | comments.png 1 Comments | 0

Planned Death
On Monday I got home from the hospital. It was nice, as hospitals go. My sisters and mother came to get me, and after hours of paper work and last minute urine tests, the doctor gave me my papers and told me I was free to leave. Without oxygen, I sat quietly during the taxi drive home and then got home.

date.png04 May 2016 05:10 | comments.png 10 Comments | 4.5

A Protocol for Dying
Time for my last article (as it turns out, not really). I could probably write more, yet there are times for everything and after this, my attention will be focused on the most comfortable position for my bed, the schedule for pain killers, and the people around me.

date.png22 Apr 2016 03:43 | comments.png 116 Comments | 5

Why I've Quit Twitter
I've loved Twitter, the format and the transparency. Yet over the last year, and months, I've become increasingly concerned about it. We've all seen the divisive arguments over race and gender. These aren't just your usual Internet arguments. These are becoming a civil war and I'm stepping out of it.

date.png26 Mar 2016 13:22 | comments.png 12 Comments | 0

Trademarks: a Brief
Trademarks. What are they, do you need them, and how much do they cost? These are questions that often crop up when we build open source projects. Trademarks can be key to protecting a project from bad actors. Yet there is little advice on line. So here is my guide to using trademarks in open source. This is practical advice, IANAL, and certainly not your lawyer.

date.png24 Mar 2016 13:22 | comments.png 2 Comments | 0

Requiem for Nanomsg
Projects and companies die for different reasons. Sometimes it's money, and sometimes it's pride. In this post I'm going to expand on Drew Crawford's "nanomsg postmortem and other stories".

date.png08 Feb 2016 12:47 | comments.png 4 Comments | 0

Requiem for GitHub
Since its birth in 2008, GitHub redefined how software developers worked together. The firm was famous for several reasons: it had no middle management, it had a strong remote working culture, it made exactly what we needed, no more or less, and it was always profitable. These are interrelated. Today, GitHub has 500 employees, is valued at $2bn, and I think it is dying. Here is why.

date.png07 Feb 2016 12:51 | comments.png 8 Comments | 0

Scalable C
At the end of 2015 I asked people what my next book should be about. The overwhelming choice was for a book on distributed C.

date.png03 Jan 2016 20:58 | comments.png 7 Comments | 0

Five Years, Five Wishes
On December 10th 2010, I didn't start to die. Instead, a surgeon opened me up, and removed the cancer that was spreading rapidly through my bile duct, to my lymph nodes and pancreas. I survived the surgery, a resistant infection, and chemotherapy. So far, so good. Today I celebrate five years of extended life with a personal article.

date.png10 Dec 2015 11:35 | comments.png 4 Comments | 0

Codes of Misconduct
Whenever people gather together, there's a risk of misbehavior. In tech conferences we are trying to solve this by declaring and enforcing a "Code of Conduct." I've had and seen harassment multiple times, over the years. It seems to me that the Codes of Conduct are not working. In this article I'll try to explain the problems, and propose a fix.

date.png04 Dec 2015 16:50 | comments.png 20 Comments | 0

Ten Steps to Better Public Speaking
Speaking to an audience can be difficult for many of us. For the audience, it can be painfully boring. Yet when done well, a talk becomes a magical moment. I've had a few of these, among a lot of disasters. I'm going to explain my strategy for becoming a better public speaker. Some people are more gifted than others. No matter: the key is not talent, it is patient effort in the right direction.

date.png03 Dec 2015 22:36 | comments.png 7 Comments | 0

Why Optimistic Merging Works Better
I spoke at DomCode in November 2015 (excellent conference, small and beautiful city!) explaining my top rules for building open source communities. One person asked me later to explain why I recommend to merge quickly, without waiting for Continuous Integration testing to finish, and without review of the code. I'm going to call this strategy Optimistic Merging or OM. Here's the reasoning behind OM.

date.png16 Nov 2015 15:14 | comments.png 10 Comments | 0

Ten Myths About Harassment
Today I watched a revealing video of Yale students with a professor. The mob insult and harangue someone with decades of experience defending free speech. It goes on far too long and leaves us disturbed. These young people act like a pampered, idiot mob. And yet you cannot deny their deep anger. Who is the harasser, and who the harassed, in this video?

date.png10 Nov 2015 19:56 | comments.png 2 Comments | 0

How the Book Works
The Psychopath Code has eight chapters, each telling part of the story. You can read these in any order. I'd suggest you skim the text rapidly, then read it carefully a few times. Then discuss with people you trust, and let your new knowledge sink in slowly. There is a lot to learn, and it will take you time, maybe years, to digest it all.

date.png26 Oct 2015 13:21 | comments.png 1 Comments | 0

Break Glass in Case of Emergency
If you're reading The Psychopath Code to get a grip on problems in your personal life, or work, start here. I'm going to explain the key lessons in a short summary. Read this and feel how it applies to you.

date.png26 Oct 2015 13:19 | comments.png 0 Comments | 0

The Bug Formula
In this article we'll see how estimate the number of bugs in a code base. This article was originally written by Leif Svalgaard and myself, around 1990. It's old yet still relevant to many projects that haven't adopted a zero-defect approach as the ZeroMQ community has done.

date.png25 Oct 2015 16:21 | comments.png 1 Comments | 0

Ten Tips for Young Programmers
Writing code is like doing magic. Just say the right words, and amazing stuff happens. Or, horrible things happen. Becoming a great programmer isn't easy. It takes time. For young coders, I'm going to give ten tips to help you along that path.

date.png25 Oct 2015 16:02 | comments.png 1 Comments | 0

Amdahl to Zipf: Ten Laws of the Physics of People
When we make software, laws of physics apply. I call this the "physics of people." When we ignore these laws, our work collapses like a badly designed bridge. Maybe you didn't realize Einstein's Equivalency Principle applied to you? Then read on, and be enlightened…

date.png08 Oct 2015 09:34 | comments.png 4 Comments | 0

Ten Steps Towards Happiness
Today, I'm going to explain ten simple steps you can take towards being happier, and making those around you happier. It takes no money and no magic. Just a shift in how you see the world, yourself, and others. As the 14th Dalai Lama said, "Happiness… comes from your own actions."

date.png29 Sep 2015 09:57 | comments.png 0 Comments | 0

Ten Habits of a Good Programmer
Good programmers are born and then made. You need talent and passion, and you need experience. I was lucky to learn from the best, and have lots of opportunity to practice. I'm going to summarize ten of the top habits I've always applied to my work. Cause or correlation? You tell me.

date.png28 Sep 2015 17:01 | comments.png 0 Comments | 0

Ten Signs of a Psychopath
One in 25 people is a (sub-clinical) psychopath. You may meet them at work, on dating sites, in the street. If psychopaths have one goal, it is to take from others and give nothing back. If they have one supreme talent, it is not getting caught. Everyone likes a psychopath. Until it gets personal, and then your life starts to bend. I'll list ten ways a psychopath (let's call him or her "Mallory") touches your life. If you recognize all of these, beware…

date.png25 Sep 2015 09:30 | comments.png 0 Comments | 0

Ten Rules for Good Code
It's one of my interview questions: "what is Good Code?" Surprisingly, almost no-one gets it right. It's not about speed, elegance, language, or style. Good Code is code that solves real problems for real people, in an effective way. Let me list the top 10 rules for writing good code.

date.png22 Sep 2015 15:55 | comments.png 0 Comments | 0

Ten Rules for Open Source Success
Everyone wants it, lots of people try it, yet doing it is mostly painful and irritating. I'm speaking about free software aka open source. Today I'm going to summarize 30 years of coding experience in ten management-proof bullet points.

date.png22 Sep 2015 09:40 | comments.png 2 Comments | 5

Ten Rules for Good API Design
Every software developer uses APIs and most of us make them. The design of a "good" API is a black art. You know one when you see one. And yet how many of us could explain why some APIs are complex and hard to learn, while others are clean, simple, and a joy to use. It's a question I'll answer in this article, and provide ten rules for good API design.

date.png17 Sep 2015 10:43 | comments.png 0 Comments | 0

C4.1 - an Exothermic Process
In February 2012 the ZeroMQ community said "adieu" to its long time maintainers, and set sail in a new direction. Using a new process called C4.1 as its compass, we headed off into unknown territory. Looking back three and a half years later, what is the result? Did ZeroMQ lose its way, as many predicted and feared? Or did the new process work as planned?

date.png16 Sep 2015 20:49 | comments.png 2 Comments | 0

Belling the Cats
We come to the last article in this series before I wrap the lot up as a book. I've still got a lot to talk about. So I'm going to conduct an interview with myself. Think of this as The Transcript of The Podcast of The Conclusion of The Book.

date.png12 Jun 2015 10:34 | comments.png 3 Comments | 0

Machines of Chaos
When we meet others suffering from abusive relationships, our first response is often disbelief. "Just leave!" we say. Just change jobs. Just divorce. Just move out. Yet that is like saying to a sick person, "just get better!" Victims of psychopaths find themselves tied down by a web of lies, promises, and threats that can make escape impossible. In this article I'll describe some of these mechanisms.

date.png07 Jun 2015 08:36 | comments.png 9 Comments | 0

Dancing With the Wolf
This is part of my continuing series on psychopaths. In my last article I explained where, and how, psychopaths hunt. Now, I'll explain how psychopaths catch and entangle their targets.

date.png15 Mar 2015 18:02 | comments.png 4 Comments | 0

How Psychopaths Hunt
In my continuing series on psychopaths, I'm going to explain how they control other individuals. In this article, I'll cover where and how psychopaths hunt. There is a fair bit of sex and violence in this text, plus lots of informed speculation, so if you don't like that sort of stuff, please skip it. This material comes from forums and websites, from psychopaths themselves, and from my own experience.

date.png11 Mar 2015 17:01 | comments.png 0 Comments | 0

Tales from the Trenches
Analysis of psychopaths can be a little overwhelming. In this article I'm going to bring it down to earth by telling some stories, inspired real events, of conflicts with a psychopath at their heart. These dramas play millions of times every day across the globe, in the endless conflict between the two natures of humanity.

date.png18 Feb 2015 20:23 | comments.png 3 Comments | 0

The Vampire Slayer
This is my third article on psychopaths. I've explained where psychopathy came from, and how to recognize it in others. These are difficult ideas to untangle and understand. In this article I'll take the hardest story of all: how to deal with psychopaths you are already entangled with.

date.png16 Feb 2015 17:21 | comments.png 0 Comments | 0

A Protocol for REST over ZeroMQ
In this article I'll descibe a popular way of making really awful protocols, using "RPC" or Remote Procedure Calls. My goal is to teach you to never use RPC, ever again. As a safer and saner alternative I'll explain how to do RESTful work over ZeroMQ.

date.png09 Feb 2015 10:51 | comments.png 2 Comments | 0

The End of Software Versions
Software version numbers are the crack cocaine of change management. They are an easy and attractive path to serious long term stress and pain. The worst ever distress in the ZeroMQ community came from allowing library version numbers to define compatibility. The reasons are real, yet subtle and often counter-intuitive. In this article I'll explain in detail, and propose a replacement for software versioning.

date.png06 Feb 2015 14:20 | comments.png 0 Comments | 0

Revisting ZeroMQ Multipart
Around FOSDEM 2015, a few core ZeroMQ maintainers decided to experiment with a simpler messaging API, and decided it'd be profitable to explore rethinking multipart messages from ZeroMQ. In this article I'll explain the why's and how's of this potential shift.

date.png03 Feb 2015 09:57 | comments.png 3 Comments | 0

The Long Red Line
We celebrate the close of 2014 with a 500-page executive summary of the CIA Torture Report. "This is democracy, promise!" says the state, as it casually tortures, for profit and amusement. Society is defined by how well, or badly, we treat our weakest members. I want to comment.. no, that's wrong. I want to find the people responsible, and I want to hurt them. Instead, I'm republishing a short story I wrote in 2008, about torture.

date.png12 Dec 2014 11:04 | comments.png 0 Comments | 0

From Dusk till Dawn
In my first article on psychopaths, I explained my Two Actor hypothesis of how human psychopathy evolved, over millions of years, in a predator-prey arms race between collaborative and cheater behaviors. I also postulated that human culture emerged from that arms race. In this article I'll answer more pragmatic questions: how can we tell if someone is a psychopath, and what can we do about it?

date.png10 Dec 2014 22:38 | comments.png 8 Comments | 0

The Cretan Method
The search for truth is an ancient, and difficult pursuit. Modern society still swims in a sea of lies, in politics, business, and especially the software industry, where grand lies breed like cults. My work is to cut through the lies to develop better theories of the truth. It turns out that lies cause pain, and as we approach truth, we also become happier. In this essay I'll explain how I do this, and provide a series of tools and lessons for your benefit.

date.png28 Nov 2014 23:57 | comments.png 5 Comments | 0

My 4-year old is a better programmer than you (or me)
At three, my youngest son Gregor picked up a mouse and started playing Minecraft, like his older brother and sister. At the age of four he is already a better programmer than you, or me. This is of course an insane claim. So let me explain…

date.png17 Nov 2014 23:22 | comments.png 0 Comments | 0

Solving the Makefile Problem
The ZeroMQ community started many years ago as a group of low-latency fanatics who focused their "make it cheaper to build distributed systems" mission on a single library, libzmq. The mission hasn't changed, yet the focus is wider than ever. In this article I will talk about a young project, zproject, which makes it cheaper to build other projects that use ZeroMQ.

date.png14 Nov 2014 11:16 | comments.png 0 Comments | 5

Children of the Fight
Professionally, and personally, we all deal with bad actors from time to time. Mostly we don't realize it until much later, when the depression and PTSD has passed, and we look back. Estimates of professional bad actors (those who live by deceit and abuse) vary from 1% to 4% of the population. We call them psychopaths, sociopaths, malignant narcissists, trolls. The conventional view is that they are parasitic or even mentally ill, In this article I'll propose a new, very different analytical framework.

date.png18 Oct 2014 09:47 | comments.png 10 Comments | 0

The Life FAQ
Who's the purpose of Life? Who's in charge? How much does it cost, and what colors does it come in? Today, I take all these and other Frequently Asked Questions about Life and answer them in a single easy-to-hate blog post.

date.png04 Oct 2014 13:46 | comments.png 2 Comments | 0

Hacking the Edges
On October 30 and 31, Marvin Arnold and myself will be organizing a hackathon in Brussels, with the goal: produce a prototype for a fully decentralized Internet. The idea came after Marvin, the guy behind Unplugged and I were talking about ways to kick off something together.

date.png23 Sep 2014 17:34 | comments.png 2 Comments | 0

Generating Servers with zproto
While ZeroMQ gives you a powerful communications engine to use in many different ways, building a conventional server is still fairly solid work. We use the ROUTER socket for that, managing the state for each individual client connection. Today I'll present a new tool — zproto — that generates whole servers, in C, from state machine models.

date.png29 Mar 2014 13:12 | comments.png 5 Comments | 0

The Power of Living Systems
A "Living System" is one that grows into its environment, by self-organizing around opportunities. Living systems can last for a long time, adapt well to change, and thus be highly successful. By contrast, "Planned Systems" tend to be fragile, poor at coping with change, and thus short-lived. In this article I'll explain Living Systems, of software and people, and how to grow them.

date.png27 Jan 2014 08:22 | comments.png 0 Comments | 0

Sex in Title, and Other Stories
First Code Mesh in London, then Built Stuff in Vilnius: two great events during the first half of December 2013, where I spoke, and despite that, nothing bad happened. Indeed, meeting so many amazing and stimulating people didn't just take the discussions out of the box. It shredded the box, burnt the remains, and chucked the ashes into the toilet. There were so many amazing little stories that came out of these two trips. I want to capture some of them and share them with you.

date.png15 Dec 2013 21:07 | comments.png 5 Comments | 0

The Digital Revolution Roadshow: Vilnius
On 9th December 2013 I'll be giving the opening keynote at Build Stuff '13, in Vilnius, Lithuania. I'll also be giving a technical talk in the conference itself, so this is a rare occasion to come and heckle me not once, but twice in a day. This is an awesome conference, with over 50 speakers in three days. If you bring a copy of any of my books, I'll sign them for you. Here is the abstract for my keynote…

date.png18 Nov 2013 15:23 | comments.png 0 Comments | 0

Cluster load-balancing using ZeroMQ over TIPC
Erik Hugne writes, "I've cooked up another example with real code illustrating load-balancing using the new TIPC transport for ZeroMQ. This demo uses overlapping port names." The code is available to download.

date.png06 Nov 2013 12:56 | comments.png 0 Comments | 0

The New TIPC Transport in ZeroMQ
ZeroMQ now supports TIPC, the cluster IPC protocol developed by Ericsson. This will be part of the next major 4.1 release. In this article I'll interview Ericsson's Erik Hugne, the engineer who added this transport to ZeroMQ.

date.png04 Nov 2013 15:08 | comments.png 0 Comments | 0

A Beautiful, Minimal Publishing Platform
I've just published my book, Culture & Empire, using GitHub pages and reveal.js. The results are a beautiful and minimal web-based ebook. Take a look. In this article I'll explain how I made this.

date.png24 Oct 2013 05:09 | comments.png 0 Comments | 0

How To Capture an Open Source Project
Ars Technica has an interesting article on how Google is closing off Android piece by piece. It is a classic game of "capture the flag", played against an open source community. I'm going to explain how this capture works, and how to prevent it.

date.png21 Oct 2013 12:08 | comments.png 0 Comments | 0

ZeroMQ Certificates, Design Iteration 1
In the ongoing search for a new certificate format for ZeroMQ, I kicked off a nice lively thread on zeromq-dev with some to and for, little agreement, and lots of ideas. In this article I'll present Design Iteration 1, a strawman that people can beat up. The format is inspired by the SSH2 public key format.

date.png14 Oct 2013 12:20 | comments.png 1 Comments | 0

Securing ZeroMQ: Soul of a New Certificate
While wrapping ZeroMQ's new security API up in the high-level C binding, I accidentally a certificate format. With all respect to X.509 and existing PKI standards, the reason I built CURVE for ZeroMQ was to get simple and foolproof security. Using a complex legacy certificate format would spoil the soup. I've no idea what the right format is, but to start, I'm going to try to collect requirements.

date.png01 Oct 2013 07:54 | comments.png 6 Comments | 0

Using ZeroMQ Security (part 2)
In the previous article I gave an overview of how and why ZeroMQ's security layers work. In this article I'll develop a simple secure application, step by step. We'll use a simple example of a server PUSH socket sending "Hello" to a client PULL socket. We'll work through the ZeroMQ NULL, PLAIN, and CURVE security mechanisms, up to full authentication. The examples are in C but the principles apply to all languages.

date.png19 Sep 2013 21:07 | comments.png 15 Comments | 0

Using ZeroMQ Security (part 1)
In this series of articles I'll explain how to use the new ZeroMQ security layers in your applications. The example we'll make is a chat application that provides unbreakable strong security. In this first article, I'll explain more about ZeroMQ's security technology, how it works, and why we chose it. (Read part 2.)

date.png12 Sep 2013 09:17 | comments.png 0 Comments | 0

The Castle and the City
One of the topics for my upcoming book "Culture and Empire" is how to build online communities. I'm going to argue that there are two general layouts for a large organization — the Castle and the City — and compare these. Is your project a Castle, or a City?

date.png29 Jul 2013 16:59 | comments.png 0 Comments | 0

Charlie and the X-ray Factory
Last week I had the pleasure of visiting not one major European physics research facility, but two. My first stop was at ESRF in Grenoble, then I went to visit CERN in Geneva. Both these organizations are moving to use ZeroMQ for their control systems. In this article I'll explain what that means.

date.png07 Jul 2013 12:26 | comments.png 0 Comments | 0

Securing ZeroMQ: Circus Time
Thanks to the quiet but persistent work of Martin Hurton, the master branch of libzmq, the ZeroMQ core library, now "does security". In this last article in the mini-series, "Securing ZeroMQ", I'll explain what we built, and why, and how this can work for your ZeroMQ applications.

date.png04 Jul 2013 16:59 | comments.png 1 Comments | 0

The Gender Gap in Coding
My daughter, nine, comes with me to tech conferences like FOSDEM, in Brussels. She's been using a Linux PC since she was two, self-taught and self-motivating. I'd like to teach her to code, and perhaps make a profession in the software industry. But the chances she'll succeed at that are getting lower each year. I think there's a reason for the so-called "gender gap" but it's not any of the usual explanations.

date.png29 Apr 2013 21:33 | comments.png 2 Comments | 0

10 Tips for an Awesome Open Source Project
In this article, ten top tips for making your open source project awesome! We've collected these tips by paying scientists to study the most awesome open source projects in the world. Whether you're starting a new open source project, or open sourcing your company's kind of old codebase, these tips will help you. Enjoy and retweet!

date.png29 Apr 2013 13:53 | comments.png 0 Comments | 0

A Web Server in 30 Lines of C
There are developers who never built distributed systems. Then there are developers who have done this before, using a messaging system. Then there are those who've done it the hard way, using BSD sockets. Privately, I call these the Good, the Bad, and the Ugly, because of how they respond when they meet ZeroMQ and read the Guide. Today, "Getting Started with ZeroMQ for Uglies". With a "Hello, World" web server in —40— 30 lines of C, just for fun.

date.png29 Apr 2013 09:57 | comments.png 6 Comments | 0

Licenses for Protocols
A few years back, when we set-up the Digital Standards Organization, one of our goals was to create a simpler framework for protocol development. At the time, licensing a protocol spec as open source was unheard of, and offensive to many people. Today it's just radical. In this article, I'll look at why and how to license a protocol spec.

date.png17 Apr 2013 13:51 | comments.png 0 Comments | 0

Doing Stuff You're Bad at is Good For You
One of the tricks I use to not burn out on a project is to work as hard on learning new things as I do on doing what I already know. Last December I bought a piano and started teaching myself to play. The results… well, my daughter likes them and that's good enough for me.

date.png15 Apr 2013 18:32 | comments.png 0 Comments | 0

Securing ZeroMQ: draft ZMTP v3.0 Protocol
In the previous article in this series, "Securing ZeroMQ", I showed a proof-of-concept for CurveZMQ. Now we're moving that into the ZeroMQ protocol, ZMTP. It's not a small change. You can't just sprinkle security over a protocol like chocolate chips onto pancakes. It means a new protocol, and this gives us a chance to address other problems with ZMTP. The result is ZMTP v3.0. In this article I'll explain ZMTP v3.0. If you never read the 1.0 or 2.0 spec, don't worry, I'll cover those briefly too.

date.png14 Apr 2013 11:15 | comments.png 0 Comments | 0

The PUB-SUB Hacks: Census
In the first article on PUB-SUB hacks, we looked at the Meerkat pattern. In this second article I'll walk through a pattern—Census—that's loosely inspired by the "Surveyor" socket type from the defunct Crossroads fork of ZeroMQ. It's a fairly simple pattern. The code for this article is in a GitHub gist.

date.png26 Mar 2013 19:28 | comments.png 1 Comments | 0

The PUB-SUB Hacks: Meerkat
While the ØMQ pub-sub pattern is well-known for being a one-way pattern, it does in fact support a two-way flow of information. SUB sockets (or more accurately, XSUB sockets) can send messages to XPUB sockets. This week, I'll post a series of short articles that cover different patterns—Meerkat, Census, and Repeater—we can build using this functionality. As usual, I'll demonstrate each pattern with code you can download and run.

date.png26 Mar 2013 18:44 | comments.png 1 Comments | 0

Securing ZeroMQ: CurveZMQ protocol and implementation
This week, we flesh out the basics of our CurveCP-derived protocol, with an implementation in C that I'm calling CurveZMQ. In this article I'll explain what this simple but powerful security protocol looks like. The code here will already work over 0MQ sockets but our next stage is to move this into libzmq itself, for all sockets over tcp:// and ipc://. So stay tuned!

date.png22 Mar 2013 22:19 | comments.png 0 Comments | 0

Securing ZeroMQ: the Sodium Library
Marc Falzon (@falzm) pointed me to libsodium, aka Sodium, a repackaging of the NaCl library. The Sodium README says that it's, "tested on a variety of compilers and operating systems, including Windows, iOS and Android," and "tests and benchmarks will be performed at run-time, so that the same binary package can still run everywhere." This fixes the biggest problem with NaCl, which is that it optimizes at compile-time, so you can't ship it in binaries. Today I tried Sodium, and here are the results of my tests.

date.png12 Mar 2013 10:48 | comments.png 0 Comments | 0

Securing ZeroMQ: CurveCP and NaCl
One of the biggest user requests for ØMQ is a good security layer. Mainstream options like TLS/SSL are complex, slow and designed for web browsing, not high-speed messaging. In this article I'll present CurveCP, one of the most exciting security developments in recent years. It's part of the NaCl networking and cryptography library and looks perfectly suited to ØMQ. To demonstrate this, I've made a "Hello World" proof of concept that shows an authenticated, confidential connection from one client to one server, over ØMQ.

date.png11 Mar 2013 21:29 | comments.png 2 Comments | 0

Making Music on Linux
I've always loved music, and have always been a terrible musician. Recently I found myself playing a cheap Casio keyboard and graduated to a rather nicer Kawai electronic piano. The Kawai has midi outputs, so as an experiment I've been recording some of my rambling piano compositions. I'll explain how I got these into a nice digital format using just a dead cat and a length of garden string.

date.png27 Feb 2013 15:01 | comments.png 0 Comments | 0

Solving the Discovery Problem
One long-standing question for 0MQ developers is how to discover services on the network. A lot of people have built answers to this, such as ZeroConf and UPnP, or even DNS, but they tend to be over-complex and unfriendly to application developers. In this article I'll explain zbeacon, which is a new module in the CZMQ binding for 0MQ that does service discovery.

date.png26 Feb 2013 15:05 | comments.png 0 Comments | 0

Patents Considered Evil
The concept of "intellectual property" dates from the French revolution, and patents in their modern form were solidified in the 1850s. At that time, Europe debated the pros and cons of patents exhaustively. The Economist newspaper was founded as an anti-patent, pro-free trade paper. Most familiar arguments for and against patents date to this era. The patent system was ended in several countries (Switzerland, Netherlands, Germany). However, free-trade economists lost power after a serious economic crisis, and the patent system came back into force by the 1870s. The patent system remained largely unquestioned until the 21st century, when it started to seriously affect the software industry.

date.png07 Feb 2013 14:54 | comments.png 6 Comments | 0

Code Connected Volume 1
The title of this book for programmers, "Code Connected", is a homage to Steve McConnell's "Code Complete", which was easily one of the best books on software development ever written. Code Connected focuses on the basic skill of producing working code, in minimal form, to answer a wide diversity of problems. All the problems are related one way or another to distributed applications, and the answers all, in one way or another, use ZeroMQ.

date.png07 Feb 2013 14:31 | comments.png 2 Comments | 0

Measuring Cost Gravity
The other day I bought a little black and white laser printer for my office. It cost about 50 Euro, and it prints very nicely, and rapidly. The first comparable consumer lasers came from HP in 1985 and cost about $4,000. They were huge, and slow. I wondered, could we use these two data points to compute the "cost gravity" of laser printers?

date.png21 Jan 2013 10:04 | comments.png 0 Comments | 0

Burning Down the House: AMQP revisited
When the Wikipedia page for a contentious topic shows no real argument, you know someone with time and money has something to hide. And AMQP is certainly contentious. In my experience, and because I'm observant, rather than sensitive, the AMQP/1.0 authors are simple bullies who believe might makes right. But one learns to fight back against bullies, because, karma. I'm going to coin a nickname for AMQP/1.0, which is the "Burning Down The House" release.

date.png22 Oct 2012 11:25 | comments.png 2 Comments | 0

How to Make Money from Open Source
There are, it has been said, two ways to make really large-scale software. Option One is to throw massive amounts of money and problems at empires of smart people, and hope that what emerges is not yet another career killer. If you're very lucky, and are building on lots of experience, and have kept your teams solid, and are not aiming for technical brilliance, and are furthermore incredibly lucky, it works.

date.png18 Sep 2012 11:02 | comments.png 0 Comments | 0

The End of Stable Releases?
The C4 process we adopted some time back for ZeroMQ (except our dependency on Jira), CZMQ, and some other projects, looks like it's working as planned. But the drama of science lies in the extremes. If we really can reduce change latency to almost zero using C4, how does this affect how we deliver stable releases?

date.png25 May 2012 17:01 | comments.png 0 Comments | 0

The Myth of Intelligent Design
The dominant theory of design is that you take smart, creative people and money, and produce amazing products. The smarter the people, the better the results. I'm going to claim that theory is bogus and based on a quasi-religious model of the "inventor" and "invention" as a function of individual minds. As an alternative I'll present the Theory of Heuristic Innovation, which states roughly that we do not invent solutions, we discover them, and that discovery process can be highly automated.

date.png10 May 2012 21:10 | comments.png 0 Comments | 0

Git Branches Considered Harmful
One of git's great features is how easy it makes branches. Almost all git projects use branches, and the selection of the "best" branching strategy is like a rite of passage for an open source project. Vincent Driessen's git-flow is maybe the best known. It has 'base' branches (master, develop), 'feature' branches, 'release' branches, 'hotfix' branches, and 'support' branches. Many teams have adopted git-flow, which even has git extensions to support it. However, in this article I'll argue that public git branches are harmful, based on experience and evidence, and propose a branch-free approach, based on forks.

date.png09 May 2012 21:49 | comments.png 9 Comments | 0

Good, Cheap, and Fast - PC3
The Pedantic Code Construction Contract (PC3) is an evolution of the GitHub Fork + Pull Model, and the ZeroMQ C4 process, aimed at providing an optimal collaboration model for commercial software projects. PC3 helps an organization build consistently good software, cheaply, and rapidly.

date.png29 Mar 2012 16:12 | comments.png 0 Comments | 0

The Lazy Perfectionist and other Patterns
In this article I'm presenting a series of patterns for success in software engineering. These patterns aim to capture the essence of what divides glorious success from tragic failure. They were described as "religious maniacal dogma" by a manager, and "anything else would be fucking insane" by a colleague, in a single day. For me, they are science, the results of decades of trial by error. Treat the Lazy Perfectionist and others as tools to use, sharpen, and throw away if something better comes along.

date.png01 Mar 2012 20:00 | comments.png 2 Comments | 0

Social Engineering 101
Social architecture is the discipline of designing and building large-scale, successful on-line communities. An underlying toolkit is the more focused skill of social engineering, or making friends and influencing people. It's often confused with social hacking but is quite different. In this article I'll explain the basics. As a case study I'll tell the story of how I talked myself into seat 2A in first class on United UA 973, Brussels to Chicago.

date.png08 Feb 2012 19:23 | comments.png 1 Comments | 0

Diversity is Joy
If you're into community building, you will inevitably bump against individuals who cost you sleepless nights, who draw you into endless email threads debating the most inane details, and who generally turn what should be a pleasant win-win experience into a war of words. In this short article I'll explain how to deal with such people.

date.png04 Feb 2012 19:50 | comments.png 0 Comments | 0

How to Design Perfect (Software) Products
My tweet "Still amazed by the power of engineers to over-design. Complexity is easy, folks, it's simplicity that is hard" got over 50 retweets. Clearly I touched a nerve in a world swimming in hopeless complexity. But talk is easy. How do we design for simplicity? Well, I've got a process, which I will explain. I call this process "Simplicity Oriented Design", or SOD.

date.png29 Jan 2012 12:33 | comments.png 0 Comments | 0

The Economics of Evil
In 2008 I'd been president of the FFII for two years and we worked hard to get people involved in the fight against software patents. We developed the rule of thumb that positive campaigns don't work. Every campaign needs a bad guy. So as I moved back to building open source communities, I wondered how this rule could translate to different kinds of community.

date.png26 Jan 2012 01:15 | comments.png 0 Comments | 0

Testing Considered Evil
Here's a provocation: the more you test software, the worse it will be. To understand why, I need to explain how we (as a profession or industry) actually make good software. Very few people understand this, and we use techniques like unit tests for support, not illumination.

date.png04 Oct 2011 12:53 | comments.png 0 Comments | 0

A Tale of Two Bridges
Two old engineers were talking of their lives and boasting of their greatest projects. One of the engineers explained how he had designed the largest bridge ever made.

date.png03 Aug 2011 22:09 | comments.png 0 Comments | 0

Credit-based Flow Control
Using network buffers - such as 0MQ's queues - has a disadvantage when you are sending data to many readers. Your writer will block when the buffer is full. To avoid this, you can use non-blocking writes, and poll for sockets that are 'ready for writing'. But here's an alternative that dispenses with high-water marks and blocking writes. It's called "credit-based flow control".

date.png19 Jul 2011 14:58 | comments.png 6 Comments | 0

How to Recognise and Prevent Burnout
Any organisation or community that relies on pro-bono efforts from its members runs the risk of burnout. In this article I'll explain what causes burnout, how to recognise it, how to prevent it, and (if it happens) how to treat it. Disclaimer: I'm not a psychiatrist and this article is based on my own experiences of working in pro-bono contexts for the last 20 years, including free software projects, and NGOs such as the FFII.

date.png23 Jun 2011 13:53 | comments.png 0 Comments | 0

MOPED - a Message-Oriented Pattern for Elastic Design
People learning to build distributed systems with 0MQ for the first time (there's a protocol joke in there somewhere) can make some awful mistakes. One cannot just convert design approaches like "let's make a structure of classes" into "let's make a structure of tasks". So, here's a simple design pattern, that I call MOPED, for building distributed systems. Surprisingly, MOPED works, and is fun.

date.png16 Jun 2011 15:45 | comments.png 0 Comments | 0

NOM-1 grammar and notes
Time to get formal. Even the simplest contracts have to be robust against our charming but ineradicable stupidity. Perhaps the simplest formal language for an unprotocol is ABNF. So here is a first draft grammar for NOM-1, with notes. This page will morph into a real unprotocol draft spec.

date.png15 Jun 2011 13:57 | comments.png 0 Comments | 0

First steps to NOM-1
NOM-1 is a lightweight messaging unprotocol for #zeromq over UDP. The name stands for "NOM-oriented messaging". There's a good reason for the cuteness. If I gave it a confidence-inspiring name like "Message-oriented Datagram Protocol", people might use it without further thought. The silly name will make life and public acceptance hard for NOM-1. So it's going to have work really hard to succeed.

date.png14 Jun 2011 12:35 | comments.png 0 Comments | 0

How to Grow a Community
There are a few ways people define open source or free software. One is by the license: "yes, we can see the code". Two, is by process: "yes, anyone can contribute". Three is by community: "yes, it's built by all of us". For me, only the last measure counts, and steps one and two are milestones. But they're not sufficient, as many failed projects show. I'll present fifteen measures that I've extracted from years of trying (and sometimes managing) to build self-steering, sustainable communities. You tell me if I'm on the right track here or not.

date.png07 Jun 2011 14:50 | comments.png 0 Comments | 0

Cheap and Nasty, two essential patterns
Unprotocol enlightenment comes one step at a time. While it can be tempting to try to use generic solutions, it's often wiser to divide problems into classes, each with an optimal solution. Let's look at Cheap and Nasty, two essential patterns for unprotocol design.

date.png06 Jun 2011 19:14 | comments.png 0 Comments | 0

Psychological Elements of Software Architecture
Dirkjan Ochtman pointed me to Wikipedia's definition of Software Architecture as "the set of structures needed to reason about the system, which comprise software elements, relations among them, and properties of both". It's a good example of how miserably little we understand about what actually makes a successful large scale software architecture.

date.png06 Jun 2011 09:20 | comments.png 0 Comments | 0

Clouds and Unprotocols
The future seems clear. Mobile applications, clouds, and unprotocols. My previous gig was CEO of Wikidot, which runs this site. It's a cloud. Cheap, scalable, instantly available to anyone who needs it. My next gig will be portable applications. Always available, smart, geolocated, social. Put the two together and you have something magic, as Google knows.

date.png01 Jun 2011 02:45 | comments.png 0 Comments | 0

Smallest Stack Ever
@dln_eintr (Daniel Lundin) broke two or three records with this one: zmq_push() { m=$(cat) && echo -e $(printf '\\x01\\x00\\x%02x\\x00%s' $((1 + ${#m})) "$m") | nc -q1 $@; }

date.png06 May 2011 08:35 | comments.png 0 Comments | 0

Majordomo and Titanic in PHP
The Majordomo Protocol and Titanic Service Procotol are two neat little unprotocols built on top of ØMQ. Ian Barber (@ianbarber) has just made PHP implementations of these.

date.png05 May 2011 15:26 | comments.png 0 Comments | 0

Using a RESTful Transport Layer
Couple of us are developing a new mobile concept, which I'll write about later somewhere else. The little thing talks to a web service, and for this we're using RestTL, an unprotocol that "specifies standard rules for representing resources, and standard mechanisms for working with them in a RESTful fashion over a plain HTTP client-server network."

date.png02 May 2011 10:13 | comments.png 0 Comments | 0

Finally, the ØMQ wire level protocol!
ZMTP/1.0 defines the wire level protocol used by ØMQ (versions 2.x) over TCP. It has a framing layer, a connection layer, and a content layer. It's a neat little unprotocol that is free to remix, lightweight, and easy to understand.

date.png30 Apr 2011 15:59 | comments.png 0 Comments | 0

Keywords for Unprotocols
Up to now, software protocols have been designed with levels of formality that tend to exclude facile participation from the fat belly of developers. tl;dr: protocols in suits bore the pants off most of us. We propose the concept of the "unprotocol", which is a lightweight spec that's easy to read, fast to implement, and obvious to get right.

date.png30 Apr 2011 15:50 | comments.png 0 Comments | 0

Elegant Little Pieces
Unprotocols are elegant little pieces that we remix to create order and sanity in the swirling chaos that is the software industry. There are, IMO, three essential properties of an unprotocol. One, it's immune from capture. Two, it's lightweight. Three, it uses natural semantics. Said @leastfixedpoint: It's a mystery to me why AMQP isn't yet specified in little elegant pieces like this:

date.png30 Apr 2011 15:30 | comments.png 0 Comments | 0
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License