It's been more than a year that I started working on Walrus, Sealgram's email server. For performance reasons, I decided to write it in Haskell, a language that's unusual in many ways. Overall, that was a great choice, in ways that I couldn't even begin to anticipate at the time. Learning Haskell is both an empowering and offsetting experience, one that I want to share.
Ok, first a little bit of context. Sealgram is the result of the question "how could one make email security easy enough that people would actually use?" not getting out of my head for a log time. What I got was a nice general purpose two way communication system that is mainly SMTP, but enables all kinds of data sharing, privately and controlled by a flexible capability system.
Now I'm writing it. I'd like to show some use cases, but I can't... Everybody says that working on email is a fool's errand, and I have an year of work with a barely working server to show for it . But I can be a persistent fool, and expect to have something to show soon. Anyway, that's enough context. This is about Haskell, not Sealgram.
Before Walrus, I had created some small scripts in Haskell, enough that I could have an idea of its performance benefits, and didn't feel completely lost on its syntax. Overall, I felt that I needed just to get used to it, and a better hold onto those high order types...
What a delusion! That's like saying, "I mostly know Lisp, I just need to learn those macros", or "I mostly know C, it's just those pointers that I can't get yet". But despite me not getting the entire point of the language, it did already look good enough to start a project.
Anyway, the performance was there, the top of line implementation of green threads in GHC made it so the first, naive, implementation of my SMTP server was able to handle 700K simultaneous connection on a small VPS instance. Something that I discovered soon, because a lack of grasp of laziness made the server leak connections like mad. And thus, with that test, the language question was settled for good.
Also, pattern matching happened to be a perfect fit for a protocol as complex as SMTP .
Anyway, back at Haskell. At the beginning, learning it feels like any other language.
The beginning: a time of wonders
Like in "I wonder what this crazy type means".
Haskell syntax is alien. There are lots and lots of operators, with precedence rules that, well, "unusual" does not describe it, because it implies there are some rules that would be usual. In fact, the most common operators of Haskell don't appear on any other language that I know. A couple of which are used mainly for changing precedence, having no other effect on the code.
There's also the types.
And the wondering does not stop at syntax. Semantics is also completely alien. While there are plenty of objects oriented languages, Haskell has the unusual feature of being type oriented. Although types are not first class features, they have deep impact on the semantics of a program. There's also laziness, purity, immutability, and a laundry list of unusual concepts to get you by surprise. But nothing made me wonder more than high order types - have no doubt, they are simple, just completely unusual.
But this stage is not unique to Haskell. Maybe not as many, but all languages have unique features. It all passes once you've seen them all, and one's productivity starts to grow.
When struggling to learn a new language, there comes a time when you realize you aren't struggling anymore. By this time you have memorized all the most common elements, and code starts flowing.
After that, there's a smooth productivity gain while you learn new things. Any good feature comes with a bliss, any bad feature with a shriek. That's the time to learn how to make best use of the good features, and avoid hitting the bad ones. That's when one starts to care about writing idiomatic code.
For most languages, that subdues fast. Most of the bad features are on the language's core, most of the earth shattering good ones too. A language core is something small and simple. As a consequence, productivity gains come fast at the beginning, but soon things start to settle. Also, bad experiences come in a huge batch at the beginning, and then mostly don't come anymore.
Haskell, of course, is an exception. Or better, it's either that, or the "beginning" is long enough to gain fluency on most other languages. I can't really know, odds are I'm still a beginner. In fact, it's very likely that an year later, I'm still a beginner.
And the bad experiences come in a huge batch at the beginning.
It's slowing me down!
Shortly after the language starts to flow, Haskell's unusual nature starts to hit.
Laziness is still not a powerful tool that lets you encapsulate the world; instead, it's a powerful source of bugs. Bugs with strange, sometimes even unrecognizable faces, that will lurk around waiting to get you when they have the biggest impact.
Purity is still not an "artistic editor" that constrains you into getting powerful, yet simple and clear abstractions. It's a constraint that requires your code to be organized in an unnatural, confusing, hard to write and maintain structure. At this stage purity will require you to labor away writing big and fragile code, that will break soon, and often.
And finally, there are the types. Haskell type system is strict. Yes, that's also a powerful tool - the most powerful of the Haskell arsenal. The type system will not only check your code for correctness and help you treat all (not most, all) corner cases, it will also make your code smaller, easier to read, and more flexible. But that is all in the future. At this stage, the type system will make your code more verbose, repetitive, harder to refactor, harder to abstract, tightly coupled, and a kitten killer. Ok, maybe not the last one, but hopelessly watching the computer killing kittens might have been less painful than adapting my early code into the Haskell type system.
But things are not all bad.
High level functions are just great to work with, from the beginning. Also, I've never used another language where I would define a function, in a restricted scope, to call exactly once later, just because it was easier than inlining all the code there; and this is idiomatic Haskell, everybody does this, all the time. People aren't kidding when they say it's easy to create functions in Haskell.
Types do make entire classes of bugs go away. It took several months for me learn how to create automatic tests in Haskell, because I didn't see a gain for Walrus. That happens since the beginning, and only gets better with time.
Finally, there is the performance. In fact, if my alternative wasn't implementing some non-blocking asynchronous server on another language, with all the problems that come with it, I'd have dropped Haskell at this stage. And then, I would be here talking about how awful a language, and how unfit to real world it is.
Instead, I'm here telling how this passes. It takes effort, and perseverance. The problems come one at a time, for a long time. When you learn how to deal with one, bang, there is another one. But keep learning, and when you less expect it, you'll notice there are no problems there anymore.
A New Perspective - In Everything
Keep learning, and when you less expect it, you'll look around and notice there are no problems there anymore. Also, you'll notice you don't recognize any of the buildings, nor the skyline. But it's a beautiful new skyline!
All of the problems of the previous section happened because I was trying to fit square programming ideas into round Haskell roles. No exception.
Haskell is amazing. Unfortunately I don't think I'm able to explain it to the one year ago version of me. The best I can say is, this XKCD is about Python:
And yes, I did feel this way when I first learned Python; and when I first used a library with metaprograming magic; and when I first created a library with metaprograming magic. The feeling lasted while I learned to apply an amazing new feature, and then I got back to ground.
Haskell is the one language that brought me that feeling again. Except that before I learn to fully apply a concept, I find another one bringing the same amazement. Yes, in part it is because Haskell concepts are harder to learn, but they also are more powerful. Much more powerful.
I'm sure I'll get down to ground again eventually, but it is indeed taking some time.
|||Ok, it's not that bad. I'm writing it during nights and weekends, when I don't have more pressing tasks, and when I'm not too tired. Also, I'm learning a new language for it, a hard to learn one. Those things slow one down. But, one year is a long time; Walrus is a 10k lines of code (yes, 10k lines of Haskell) monster, and is still only serving traditional email.|
|||Whoever put that "S" in the protocol name! I'm dropping it! I need a new URI format, thus I need to change it anyway.|