Functional design has historically been somewhat difficult to teach, and even harder to teach concisely; there aren't many resources providing holistic approaches to the domain, and those of us who have learned how have mostly done so through trial and error, scrounging like vultures through blog posts, mentorships, and our own experience. For everyone else, there's basically pithy axioms and received wisdom: make as much of your logic pure as possible; make invalid states unrepresentable; make flat hierarchies with composition; parse, don't validate.
I'm working on that comprehensive resource. But in the meantime, here's something I've been mulling over, another pithy axiom to add to the pile:
Permissive, then restrictive.
Every Haskeller has, at some point or another, realized that some function deep in the stack needs some data from waaaaay higher in the callstack. You forgot some sort of config parameter, some global-ish data that you didn't think you were going to need. You think to yourself, do I really have to add an extra parameter to every single function that calls this? It's a huge pain, having to thread an extra value through 5, 10, 50 function definitions, a value that those functions don't even care about, just so I can pass it to this one utility. Can't I just use some sort of global state? Isn't there a better way to do this?
You may also have run into situations where you can't figure out how to construct some data because of IO. You want to construct a timestamp based on the current date, but with the time of day set to a specific time. So you try to create a value of type UTCTime, but the only functions you have that give you the current time give you back IO UTCTime. So you desperately try to figure out how to go from the latter to the former. Or maybe you need to construct some data containing an MVar or a TVar, but you end up in the same problem: I only have functions of type IO MVar! What do I do?! The experienced Haskeller already knows at a glance that doing that is impossible1, but what's someone new to the language going to do? Bang their head against the impossibility of the type trying to find a direct solution to the problem, and eventually ask: Isn't there a better way to do this?
Okay, one more example. You've learned about designing your types to make invalid states unrepresentable. So you, trying to be a good Haskeller, create a bunch of domain types before writing a single line of domain logic. You start implementing, and eventually you realize that one of the fields in your data is the wrong type. Maybe you're working with some external service or API and you didn't know that that field could be nullable. You change the data definition and ugh — do I really have to change 20-something usage sites? Plus, some of these functions were written with the assumption that that field would always be there; now I have to refactor a quarter of my program, just because this one field turned out to be nullable? Isn't there a better way to do this?
What's with all these situations where beginners run into a brick wall of having to completely rethink their program design? Conceptually what you want to do seems pretty simple. "This would be a one-line change in $PROGRAMMING_LANGUAGE. I'm not trying to do anything hard. Why is Haskell so dumb?"
What all these situations have in common is that our hypothetical beginner has begun by writing restrictive code first, code that prevents them from doing things. They write pure code by default, which results in them not being able to easily add global state, or not being able to do 'basic' things like get the current time. They try to make their types as precise as possible right from the start, which results in overspecifying and not being able to deal with unforeseen situations.
Once they get themselves into this situation, it's hard to see their way out. The "correct" solution — fundamentally redesigning their code to make it more permissive — is extremely indirect in nature. Without experience, who would think of that first?
In my experience, it's much easier to go the other direction: to start from writing overly permissive code, and gradually refactor to allow the code to do exactly as much as it needs to.
Depending on your level of experience, there are lots of different techniques for moving up or down this "permissiveness scale." The next post in this miniseries takes a look at the most common situations. But for the rest of this article, let's take a look at things from the beginner perspective. What if beginners learned to write programs as permissive, then restrictive? Why do they so commonly end up painting themselves into a corner? And what techniques can they apply now based on this philosophy to ease the learning curve?
Haskell as a language is set up to enable exactly this kind of restrictiveness, in contrast to what the vast, vast majority of developers are used to. That restrictiveness is a strength, giving you way more confidence about your code. But that same restrictiveness exacerbates the problem for beginners.
In an imperative language, what kind of code do you end up writing when you just start out? Maximally permissive code. Every function can do whatever it wants; you always have an escape hatch. So if you realize that you're missing some data or functionality at this or that point in your program, you can just shove it in. Need a global variable? Just declare it. Need to log something? Just stick in a logging statement wherever you want. Need to run a DB query or call a third-party API in your domain logic? Just do it.
Over time, you learn why doing that is a bad idea, and you start to write around it. All your functions can technically still do whatever they want, but you make a conscious effort to separate your logic from your persistence, to make as much of your code pure as possible, to not use global variables, to not use mutation. Though the language will still let you do whatever you want, gradually you evolve the way you code to be more and more restrictive.
On the flip side, what kind of code do you start out writing in Haskell? All the tutorials start you off writing simple, pure recursive functions: Implement factorial! Implement map! Implement foldl! All the documentation and propaganda emphasizes restrictiveness: Look at all this purity! Look at how precisely these types model the domain! Look at all these bugs that are impossible! So when you leave the warm, comforting embrace of Learn You a Haskell and strike out to do something on your own, naturally you gravitate towards writing code in that style.
But as we've seen already, that kind of code is extremely rigid. It's unforgiving. Our hypothetical beginner either gets the design correct from the start, or they're going to have to slice apart wide swaths of code once they realize they messed something up. Sure, you or I know that compared to doing the same amount of refactoring in another language, the process is easy because it requires zero brainpower, even if it takes time. Sure, you or I know that when we mess up our dataflow or side-effect permissioning, we need to restructure our program, maybe do some more I/O at the boundaries, maybe make a bunch more functions monadic. But a beginner is not going to think that way. They're going to look for a direct solution. 'Okay, I need this API data right here. How can I get it right here, inside this pure function?' And they'll bash their head against that specific problem, not make any progress, and get frustrated.
So if starting off writing restrictive code leads to hitting a brick wall... what about starting off writing permissive code?
Let's look at those example situations at the beginning one more time.
- I forgot to pass some important data or config down into my core logic.
- I need to construct an '
a', but I only have an 'IO a'. - I need to work with some external service, and I messed up modelling the data they give me.
You forgot to pass some important data or config down into your core logic
Okay, so you started off restrictive, writing nice, pure core logic, but forgot a parameter; now you have to modify dozens of other functions to thread that value through.
What would it look like if you had started off making all your functions maximally permissive instead?
Let's say you had started off having all your functions return IO from the very beginning.
-- our old, crusty pure function
needsSomeConfig :: Int -> Text
needsSomeConfig x = ...
-- our new, shiny IO-ified function
needsSomeConfig :: Int -> IO Text
needsSomeConfig x = do
...What does that even buy us? Well, now instead of having to modify needsSomeConfig and every other function that calls it, you can pass it a value with just a few lines:
import Data.IORef
import System.IO.Unsafe ( unsafePerformIO )
-- as a toplevel declaration
-- don't worry, this is safe
{-# NOINLINE someConfigString #-}
someConfigString :: IORef Text
someConfigString = unsafePerformIO (newIORef "")
-- set the value in main or some other function
-- higher up in the stack...
main :: IO ()
main = do
writeIORef someConfigString "hahahaukeru"
... the rest of your program ...
-- ...and then read it inside `needsSomeConfig'
needsSomeConfig :: Int -> IO Text
needsSomeConfig x = do
cfgStr <- readIORef someConfigString
...By making our functions more permissive (putting them into IO) we're able to get the data we need with much less effort. Is this an ugly hack? Absolutely. But that's the entire point: when learning other languages, you make lots of ugly hacks because you don't know what you're doing at first. The learning curve is gentler because you always have that escape hatch to get something working now, and slowly learn why you shouldn't do that later. In Haskell, even if you want something working now, you might get totally stuck. We're just putting that escape hatch back in.
You need to construct an 'a', but you only have an 'IO a'
Presumably you're attempting to do this as part of some pure function (i.e. that doesn't return IO or some other monad). Once again, what would it look like if you had started off making all your functions maximally permissive?
Just like the last example, let's say you had started off having all your functions return IO from the very beginning.
import Control.Concurrent.MVar
data ProgramState = ProgramState { pendingRequest :: MVar Text, ... }
-- our old function
wantAnMVar :: Int -> ProgramState
wantAnMVar x =
... -- how do I call newEmptyMVar here?!
-- our new function
wantAnMVar :: Int -> IO ProgramState
wantAnMVar x = do
pending <- newEmptyMVar
...
pure (ProgramState { pendingRequest = pending, ... })Tada, nice and easy.
You need to work with some external service and messed up modelling the data
So you've followed all the advice and tried to be a good Haskeller by writing your own datatypes to model the data payload. We'll assume that the data comes to you in JSON format, but the general idea works equally well for other formats like key-value pairs or XML or what have you.
What would it look like if instead of starting off trying to restrict your types, you made them permissive instead?
Instead of creating new types, you can pass around raw JSON Values or Objects instead. Parse the payload into one of these, then access nested data by either pulling out the fields you need on demand using custom parsers (simpler) or using lens-aeson (harder, but less boilerplate).
import Data.Aeson ( Object )
-- old datatype declaration; annoying to change
-- hypothetical: turns out API can send all non-ID fields as nullable
data APIReturn = APIReturn
{ apiUserID :: Int
, apiUserEmail :: Text
, ...
}
-- new way: just make the payload an opaque type
type APIReturn = Object
-- or since user ID can't be null:
data APIReturn = APIReturn
{ apiUserID :: Int
, apiPayload :: Object -- email and all other fields inside here
}Payload can be in multiple formats? Weird key structure? Doesn't matter, since we can now store any possible JSON object. We lose some guarantees, since now we have to do runtime checks for any key we want to use, but hey, all that means is that we're approximately as good as JavaScript or Python. Better even, since we don't have to leave our code like this. As you gain both experience with Haskell and knowledge of the API or domain, you can slowly refactor fields out of the opaque JSON blob and into your type and get those guarantees back.
Putting everything in IO, not encoding hard-won domain knowledge in the types; for a polished or production codebase these would be ugly hacks that degrade the maintainability of the program. Does someone learning the language have to do these forever? No, of course not. Instead, it's a conscious choice of where to start learning, an alternative to the traditional way of learning by trying to lock down your code as much as possible.
Now, now we have a way to learn the language that doesn't result in slamming straight into a brick wall. But remember, this is just a starting point. To reiterate the theme of this post: permissive, then restrictive. We've made our code maximally permissive; what steps do we now take to get more compile-time safety, more bug-impossibility, more guarantees, back? If this is the starting point, where do you go?
Remember, we've only looked at very simple solutions to these problems, ones that don't require too much understanding of the language. As you gain experience, you'll be able to pick and use more sophisticated solutions to the same problems. You'll pick choices closer to the optimal level of restrictiveness for your domain right from the start. The next post in this miniseries talks concretely about what situations tend to be solved by making your code more permissive, what those choices of solutions are, and how you then reclaim the guarantees you initially gave up. But to give a high level overview: just start asking questions of your code. First, move everything into IO. Use more general types, pass around primitives at first. Then examine and reflect: do I need to have IO here? Can I move some of the side effects up the stack, pass down some data as parameters, and make more of my core logic pure? Do I need an IORef here, or can I use a Reader or just pass function parameters? Oh, I know more about the domain constraints now; can I make a custom type here to give me more compile-time safety?
Keep this philosophy in mind: make your code permissive first, restrictive later.
Next: Permissive, then restrictive: concrete solutions and examples
Do you have any memorable anecdotes of situations where you needed to completely restructure your Haskell? Have a problem that you think fits this post, but not sure how to approach it? Found this useful, or otherwise have comments or questions? Talk to me!
Thanks to Christopher Nies and Jacob Stanley for providing feedback on drafts of this article!
Before you close that tab...
Want to write practical, production-ready Haskell? Tired of broken libraries, barebones documentation, and endless type-theory papers only a postdoc could understand? I want to help. Subscribe below and you'll get useful techniques for writing real, useful programs straight in your inbox.
Absolutely no spam, ever. I respect your email privacy. Unsubscribe anytime.
Footnotes
↥1 Impossible to do safely, at least.
Okay, impossible to do safely without a lot of careful scrutiny, as well as a solid understanding of lazy evaluation, and even then it’s still suspect.