# Demystifying `MonadBaseControl`

`MonadBaseControl`

from the `monad-control`

package is a confusing typeclass, and its methods have complicated types. For many people, it’s nothing more than scary, impossible-to-understand magic that is, for some reason, needed when lifting certain kinds of operations. Few resources exist that adequately explain how, why, and when it works, which sadly seems to have resulted in some FUD about its use.

There’s no doubt that the machinery of `MonadBaseControl`

is complex, and the role it plays in practice is often subtle. However, its essence is actually much simpler than it appears, and I promise it can be understood by mere mortals. In this blog post, I hope to provide a complete survey of `MonadBaseControl`

—how it works, how it’s designed, and how it can go wrong—in a way that is accessible to anyone with a firm grasp of monads and monad transformers. To start, we’ll motivate `MonadBaseControl`

by reinventing it ourselves.

## The higher-order action problem

Say we have a function with the following type:^{1}

`foo :: IO a -> IO a`

If we have an action built from a transformer stack like

`bar :: StateT X IO Y`

then we might wish to apply `foo`

to `bar`

, but that is ill-typed, since `IO`

is not the same as `StateT X IO`

. In cases like these, we often use `lift`

, but it’s not good enough here: `lift`

*adds* a new monad transformer to an action, but here we need to *remove* a transformer. So we need a function with a type like this:

`unliftState :: StateT X IO Y -> IO Y`

However, if you think about that type just a little bit, it’s clear something’s wrong: it throws away information, namely the state. You may remember that a `StateT X IO Y`

action is equivalent to a function of type `X -> IO (Y, X)`

, so our hypothetical `unliftState`

function has two problems:

We have no

`X`

to use as the initial state.We’ll lose any modifications

`bar`

made to the state, since the result type is just`Y`

, not`(Y, X)`

.

Clearly, we’ll need something more sophisticated, but what?

## A naïve solution

Given that `foo`

doesn’t know anything about the state, we can’t easily thread it through `foo`

itself. However, by using `runStateT`

explicitly, we could do some of the state management ourselves:

```
foo' :: StateT s IO a -> StateT s IO a
foo' m = do
s <- get
(v, s') <- lift $ foo (runStateT m s)
put s'
pure v
```

Do you see what’s going on there? It’s not actually very complicated: we get the current state, then pass it as the initial state to `runStateT`

. This produces an action `IO (a, s)`

that has *closed over* the current state. We can pass that action to `foo`

without issue, since `foo`

is polymorphic in the action’s return type. Finally, all we have to do is `put`

the modified state back into the enclosing `StateT`

computation, and we can get on with our business.

That strategy works okay when we only have one monad transformer, but it gets hairy quickly as soon as we have two or more. For example, if we had `baz :: ExceptT X (StateT Y IO) Z`

, then we *could* do the same trick by getting the underlying

`Y -> IO (Either X Z, Y)`

function, closing over the state, restoring it, and doing the appropriate case analysis to re-raise any `ExceptT`

errors, but that’s a lot of work to do for every single function! What we’d like to do instead is somehow abstract over the pattern we used to write `foo'`

in a way that scales to arbitrary monad transformers.

## The essence of `MonadBaseControl`

To build a more general solution for “unlifting” arbitrary monad transformers, we need to start thinking about monad transformer state. The technique we used to implement `foo'`

operated on the following process:

Capture the action’s input state and close over it.

Package up the action’s output state with its result and run it.

Restore the action’s output state into the enclosing transformer.

Return the action’s result.

For `StateT s`

, it turns out that the input state and output state are both `s`

, but other monad transformers have state, too. Consider the input and output state for the following common monad transformers:

transformer | representation | input state | output state |
---|---|---|---|

`StateT s m a` |
`s -> m (a, s)` |
`s` |
`s` |

`ReaderT r m a` |
`r -> m a` |
`r` |
`()` |

`WriterT w m a` |
`m (a, w)` |
`()` |
`w` |

Notice how the input state is whatever is to the left of the `->`

, while the output state is whatever extra information gets produced alongside the result. Using the same reasoning, we can also deduce the input and output state for compositions of multiple monad transformers, such as the following:

transformer | representation | input state | output state |
---|---|---|---|

`ReaderT r (WriterT w m) a` |
`r -> m (a, w)` |
`r` |
`w` |

`StateT s (ReaderT r m) a` |
`r -> s -> m (a, s)` |
`(r, s)` |
`s` |

`WriterT w (StateT s m) a` |
`s -> m ((a, w), s)` |
`s` |
`(w, s)` |

Notice that when monad transformers are composed, their states are composed, too. This is useful to keep in mind, since our goal is to capture the four steps above in a typeclass, polymorphic in the state of the monad transformers we need to lift through. At minimum, we need two new operations: one to capture the input state and close over it (step 1) and one to restore the output state (step 3). One class we might come up with could look like this:

```
class MonadBase b m => MonadBaseControl b m | m -> b where
type InputState m
type OutputState m
captureInputState :: m (InputState m)
closeOverInputState :: m a -> InputState m -> b (a, OutputState m)
restoreOutputState :: OutputState m -> m ()
```

If we can write instances of that typeclass for various transformers, we can use the class’s operations to implement `foo'`

in a generic way that works with any combination of them:

```
foo' :: MonadBaseControl IO m => m a -> m a
foo' m = do
s <- captureInputState
let m' = closeOverInputState m s
(v, s') <- liftBase $ foo m'
restoreOutputState s'
pure v
```

So how do we implement those instances? Let’s start with `IO`

, since that’s the base case:

```
instance MonadBaseControl IO IO where
type InputState IO = ()
type OutputState IO = ()
captureInputState = pure ()
closeOverInputState m () = m <&> (, ())
restoreOutputState () = pure ()
```

Not very exciting. The `StateT s`

instance, on the other hand, is significantly more interesting:

```
instance MonadBaseControl b m => MonadBaseControl b (StateT s m) where
type InputState (StateT s m) = (s, InputState m)
type OutputState (StateT s m) = (s, OutputState m)
captureInputState = (,) <$> get <*> lift captureInputState
closeOverInputState m (s, ss) = do
((v, s'), ss') <- closeOverInputState (runStateT m s) ss
pure (v, (s', ss'))
restoreOutputState (s, ss) = lift (restoreOutputState ss) *> put s
```

**This instance alone includes most of the key ideas behind MonadBaseControl.** There’s a lot going on, so let’s break it down, step by step:

Start by examining the definitions of

`InputState`

and`OutputState`

. Are they what you expected? You’d be forgiven for expecting the following:`type InputState (StateT s m) = s type OutputState (StateT s m) = s`

After all, that’s what we wrote in the table, isn’t it?

However, if you give it a try, you’ll find it doesn’t work.

`InputState`

and`OutputState`

must capture the state of the*entire*monad, not just a single transformer layer, so we have to combine the`StateT s`

state with the state of the underlying monad. In the simplest case we get`InputState (StateT s IO) = (s, ())`

which is boring, but in a more complex case, we need to get something like this:

`InputState (StateT s (ReaderT IO)) = (s, (r, ()))`

Therefore,

`InputState (StateT s m)`

combines`s`

with`InputState m`

in a tuple, and`OutputState`

does the same.Moving on, take a look at

`captureInputState`

and`closeOverInputState`

. Just as`InputState`

and`OutputState`

capture the state of the entire monad, these functions need to be inductive in the same way.`captureInputState`

acquires the current state using`get`

, and it combines it with the remaining monadic state using`lift captureInputState`

.`closeOverInputState`

uses the captured state to peel off the outermost`StateT`

layer, then calls`closeOverInputState`

recursively to peel off the rest of them.Finally,

`restoreOutputState`

restores the state of the underlying monad stack, then restores the`StateT`

state, ensuring everything ends up back the way it’s supposed to be.

Take the time to digest all that—work through it yourself if you need to—as it’s a dense piece of code. Once you feel comfortable with it, take a look at the instances for `ReaderT`

and `WriterT`

as well:

```
instance MonadBaseControl b m => MonadBaseControl b (ReaderT r m) where
type InputState (ReaderT r m) = (r, InputState m)
type OutputState (ReaderT r m) = OutputState m
captureInputState = (,) <$> ask <*> lift captureInputState
closeOverInputState m (s, ss) = closeOverInputState (runReaderT m s) ss
restoreOutputState ss = lift (restoreOutputState ss)
instance (MonadBaseControl b m, Monoid w) => MonadBaseControl b (WriterT w m) where
type InputState (WriterT w m) = InputState m
type OutputState (WriterT w m) = (w, OutputState m)
captureInputState = lift captureInputState
closeOverInputState m ss = do
((v, s'), ss') <- closeOverInputState (runWriterT m) ss
pure (v, (s', ss'))
restoreOutputState (s, ss) = lift (restoreOutputState ss) *> tell s
```

Make sure you understand these instances, too. It should be easier this time, since they share most of their structure with the `StateT`

instance, but note the asymmetry that arises from the differing input and output states. (It may even help to try and write these instances yourself, focusing on the types whenever you get stuck.)

If you feel alright with them, then congratulations: you’re already well on your way to grokking `MonadBaseControl`

!

### Hiding the input state

So far, our implementation of `MonadBaseControl`

works, but it’s actually slightly more complicated than it needs to be. As it happens, all valid uses of `MonadBaseControl`

will always end up performing the following pattern:

```
s <- captureInputState
let m' = closeOverInputState m s
```

That is, we close over the input state as soon as we capture it. We can therefore combine `captureInputState`

and `closeOverInputState`

into a single function:

`captureAndCloseOverInputState :: m a -> m (b (a, OutputState m))`

What’s more, we no longer need the `InputState`

associated type at all! This is an improvement, since it simplifies the API and removes the possibility for any misuse of the input state, since it’s never directly exposed. On the other hand, it has a more complicated type: it produces a monadic action *that returns another monadic action*. This can be a little more difficult to grok, which is why I presented the original version first, but it may help to consider how the above type arises naturally from the following definition:

`captureAndCloseOverInputState m = closeOverInputState m <$> captureInputState`

Let’s update the `MonadBaseControl`

class to incorporate this simplification:

```
class MonadBase b m => MonadBaseControl b m | m -> b where
type OutputState m
captureAndCloseOverInputState :: m a -> m (b (a, OutputState m))
restoreOutputState :: OutputState m -> m ()
```

We can then update all the instances to use the simpler API by simply fusing the definitions of `captureInputState`

and `closeOverInputState`

together:

```
instance MonadBaseControl IO IO where
type OutputState IO = ()
captureAndCloseOverInputState m = pure (m <&> (, ()))
restoreOutputState () = pure ()
instance MonadBaseControl b m => MonadBaseControl b (StateT s m) where
type OutputState (StateT s m) = (s, OutputState m)
captureAndCloseOverInputState m = do
s <- get
m' <- lift $ captureAndCloseOverInputState (runStateT m s)
pure $ do
((v, s'), ss') <- m'
pure (v, (s', ss'))
restoreOutputState (s, ss) = lift (restoreOutputState ss) *> put s
instance MonadBaseControl b m => MonadBaseControl b (ReaderT r m) where
type OutputState (ReaderT r m) = OutputState m
captureAndCloseOverInputState m = do
s <- ask
lift $ captureAndCloseOverInputState (runReaderT m s)
restoreOutputState ss = lift (restoreOutputState ss)
instance (MonadBaseControl b m, Monoid w) => MonadBaseControl b (WriterT w m) where
type OutputState (WriterT w m) = (w, OutputState m)
captureAndCloseOverInputState m = do
m' <- lift $ captureAndCloseOverInputState (runWriterT m)
pure $ do
((v, s'), ss') <- m'
pure (v, (s', ss'))
restoreOutputState (s, ss) = lift (restoreOutputState ss) *> tell s
```

This is already very close to a full `MonadBaseControl`

implementation. The `captureAndCloseOverInputState`

implementations are getting a little out of hand, but bear with me—they’ll get simpler before this blog post is over.

### Coping with partiality

Our `MonadBaseControl`

class now works with `StateT`

, `ReaderT`

, and `WriterT`

, but one transformer we haven’t considered is `ExceptT`

. Let’s try to extend our table from before with a row for `ExceptT`

:

transformer | representation | input state | output state |
---|---|---|---|

`ExceptT e m a` |
`m (Either e a)` |
`()` |
`???` |

Hmm… what *is* the output state for `ExceptT`

?

The answer can’t be `e`

, since we might not end up with an `e`

—the computation might not fail. `Maybe e`

would be closer… could that work?

Well, let’s try it. Let’s write a `MonadBaseControl`

instance for `ExceptT`

:

```
instance MonadBaseControl b m => MonadBaseControl b (ExceptT e m) where
type OutputState (ExceptT e m) = (Maybe e, OutputState m)
captureAndCloseOverInputState m = do
m' <- lift $ captureAndCloseOverInputState (runExceptT m)
pure $ do
((v, s'), ss') <- m'
pure (v, (s', ss'))
restoreOutputState (s, ss) = lift (restoreOutputState ss) *> case s of
Just e -> throwError e
Nothing -> pure ()
```

Sadly, the above implementation doesn’t typecheck; it is rejected with the following type error:

```
• Couldn't match type ‘Either e a’ with ‘(a, Maybe e)’
Expected type: m (b ((a, Maybe e), OutputState m))
Actual type: m (b (Either e a, OutputState m))
• In the second argument of ‘($)’, namely
‘captureAndCloseOverInputState (runExceptT m)’
In a stmt of a 'do' block:
m' <- lift $ captureAndCloseOverInputState (runExceptT m)
In the expression:
do m' <- lift $ captureAndCloseOverInputState (runExceptT m)
return do ((v, s'), ss') <- m'
pure (v, (s', ss'))
```

We promised a `(a, Maybe e)`

, but we have an `Either e a`

, and there’s certainly no way to get the former from the latter. Are we stuck? (If you’d like, take a moment to think about how you’d solve this type error before moving on, as it may be helpful for understanding the following solution.)

The fundamental problem here is *partiality*. The type of the `captureAndCloseOverInputState`

method always produces an action in the base monad that includes an `a`

*in addition* to some other output state. But `ExceptT`

is different: when it an error is raised, it doesn’t produce an `a`

at all—it only produces an `e`

. Therefore, as written, it’s impossible to give `ExceptT`

a `MonadBaseControl`

instance.

Of course, we’d very much *like* to give `ExceptT`

a `MonadBaseControl`

instance, so that isn’t very satisfying. Somehow, we need to change `captureAndCloseOverInputState`

so that it doesn’t always need to produce an `a`

. There are a few ways we could accomplish that, but an elegant way to do it is this:

```
class MonadBase b m => MonadBaseControl b m | m -> b where
type WithOutputState m a
captureAndCloseOverInputState :: m a -> m (b (WithOutputState m a))
restoreOutputState :: WithOutputState m a -> m a
```

We’ve replaced the old `OutputState`

associated type with a new `WithOutputState`

type, and the key difference between them is that `WithOutputState`

describes the type of a *combination* of the result (of type `a`

) and the output state, rather than describing the type of the output state alone. For total monad transformers like `StateT`

, `ReaderT`

, and `WriterT`

, `WithOutputState m a`

will just be a tuple of the result value and the output state, the same as before. For example, here’s an updated `MonadBaseControl`

instance for `StateT`

:

```
instance MonadBaseControl b m => MonadBaseControl b (StateT s m) where
type WithOutputState (StateT s m) a = WithOutputState m (a, s)
captureAndCloseOverInputState m = do
s <- get
lift $ captureAndCloseOverInputState (runStateT m s)
restoreOutputState ss = do
(a, s) <- lift $ restoreOutputState ss
put s
pure a
```

Before we consider how this helps us with `ExceptT`

, let’s pause for a moment and examine the revised `StateT`

instance in detail, as there are some new things going on here:

Take a close look at the definition of

`WithOutputState (StateT s m) a`

. Note that we’ve defined it to be`WithOutputState m (a, s)`

,*not*`(WithOutputState m a, s)`

. Consider, for a moment, the difference between these types. Can you see why we used the former, not the latter?If it’s unclear to you, that’s okay—let’s illustrate the difference with an example. Consider two similar monad transformer stacks:

`m1 :: StateT s (ExceptT e IO) a m2 :: ExceptT e (StateT s IO) a`

Both these stacks contain

`StateT`

and`ExceptT`

, but they are layered in a different order. What’s the difference? Well, consider what`m1`

and`m2`

return once fully unwrapped:`runExceptT (runStateT m1 s) :: m (Either e (a, s)) runStateT (runExceptT m2) s :: m (Either e a, s)`

These results are meaningfully different: in

`m1`

, the state is discarded if an error is raised, but in`m2`

, the final state is always returned, even if the computation is aborted. What does this mean for`WithOutputState`

?Here’s the important detail:

**the state is discarded when**This can be counterintuitive, since the`ExceptT`

is “inside”`StateT`

, not the other way around.`s`

ends up*inside*the`Either`

when the`StateT`

constructor is on the*outside*and vice versa. This is really just a property of how monad transformers compose, not anything specific to`MonadBaseControl`

, so an explanation of why this happens is outside the scope of this blog post, but the relevant insight is that the`m`

in`StateT s m a`

controls the eventual action’s output state.If we had defined

`WithOutputState (StateT s m) a`

to be`(WithOutputState m a, s)`

, we’d be in a pickle, since`m`

would be unable to influence the presence of`s`

in the output state. Therefore, we have no choice but to use`WithOutputState m (a, s)`

. (If you are still confused by this, try it yourself; you’ll find that there’s no way to make the other definition typecheck.)Now that we’ve developed an intuitive understanding of why

`WithOutputState`

must be defined the way it is, let’s look at things from another perspective. Consider the type of`runStateT`

once more:`runStateT :: StateT s m a -> s -> m (a, s)`

Note that the result type is

`m (a, s)`

, with the`m`

on the outside. As it happens, this correspondence simplifies the definition of`captureAndCloseOverInputState`

, since we no longer have to do any fiddling with its result—it’s already in the proper shape, so we can just return it directly.Finally, this instance illustrates an interesting change to

`restoreOutputState`

. Since the`a`

is now packed inside the`WithOutputState m a`

value, the caller of`captureAndCloseOverInputState`

needs some way to get the`a`

back out! Conveniently,`restoreOutputState`

can play that role, both restoring the output state and unpacking the result.Even ignoring partial transformers like

`ExceptT`

, this is an improvement over the old API, as it conveniently prevents the programmer from forgetting to call`restoreOutputState`

. However, as we’ll see shortly, it is much more than a convenience: once`ExceptT`

comes into play, it is essential!

With those details addressed, let’s return to `ExceptT`

. Using the new interface, writing an instance for `ExceptT`

is not only possible, it’s actually rather easy:

```
instance MonadBaseControl b m => MonadBaseControl b (ExceptT e m) where
type WithOutputState (ExceptT e m) a = WithOutputState m (Either e a)
captureAndCloseOverInputState m =
lift $ captureAndCloseOverInputState (runExceptT m)
restoreOutputState ss =
either throwError pure =<< lift (restoreOutputState ss)
```

This instance illustrates why it’s so crucial that `restoreOutputState`

have the aforementioned dual role: it must handle the case where no `a`

exists at all! In the case of `ExceptT`

, it restores the state in the enclosing monad by re-raising an error.

Now all that’s left to do is update the other instances:

```
instance MonadBaseControl IO IO where
type WithOutputState IO a = a
captureAndCloseOverInputState = pure
restoreOutputState = pure
instance MonadBaseControl b m => MonadBaseControl b (ReaderT r m) where
type WithOutputState (ReaderT r m) a = WithOutputState m a
captureAndCloseOverInputState m = do
s <- ask
lift $ captureAndCloseOverInputState (runReaderT m s)
restoreOutputState ss = lift $ restoreOutputState ss
instance (MonadBaseControl b m, Monoid w) => MonadBaseControl b (WriterT w m) where
type WithOutputState (WriterT w m) a = WithOutputState m (a, w)
captureAndCloseOverInputState m =
lift $ captureAndCloseOverInputState (runWriterT m)
restoreOutputState ss = do
(a, s) <- lift $ restoreOutputState ss
tell s
pure a
```

Finally, we can update our lifted variant of `foo`

to use the new interface so it will work with transformer stacks that include `ExceptT`

:

```
foo' :: MonadBaseControl IO m => m a -> m a
foo' m = do
m' <- captureAndCloseOverInputState m
restoreOutputState =<< liftBase (foo m')
```

At this point, it’s worth considering something: although getting the `MonadBaseControl`

class and instances right was a lot of work, the resulting `foo'`

implementation is actually incredibly simple. That’s a good sign, since we only have to write the `MonadBaseControl`

instances once (in a library), but we have to write functions like `foo'`

quite often.

## Scaling to the real `MonadBaseControl`

The `MonadBaseControl`

class we implemented in the previous section is complete. It is a working, useful class that is equivalent in power to the “real” `MonadBaseControl`

class in the `monad-control`

library. However, if you compare the two, you’ll notice that the version in `monad-control`

looks a little bit different. What gives?

Let’s compare the two classes side by side:

```
-- ours
class MonadBase b m => MonadBaseControl b m | m -> b where
type WithOutputState m a
captureAndCloseOverInputState :: m a -> m (b (WithOutputState m a))
restoreOutputState :: WithOutputState m a -> m a
-- theirs
class MonadBase b m => MonadBaseControl b m | m -> b where
type StM m a
liftBaseWith :: (RunInBase m b -> b a) -> m a
restoreM :: StM m a -> m a
```

Let’s start with the similarities, since those are easy:

Our

`WithOutputState`

associated type is precisely equivalent to their`StM`

associated type, they just use a (considerably) shorter name.Likewise, our

`restoreOutputState`

method is precisely equivalent to their`restoreM`

method, simply under a different name.

That leaves `captureAndCloseOverInputState`

and `liftBaseWith`

. Those two methods both do similar things, but they aren’t identical, and that’s where all the differences lie. To understand `liftBaseWith`

, let’s start by inlining the definition of the `RunInBase`

type alias so we can see the fully-expanded type:

```
liftBaseWith
:: MonadBaseControl b m
=> ((forall c. m c -> b (StM m c)) -> b a)
-> m a
```

That type is complicated! However, if we break it down, hopefully you’ll find it’s not as scary as it first appears. Let’s reimplement the `foo'`

example from before using `liftBaseWith`

to show how this version of `MonadBaseControl`

works:

```
foo' :: MonadBaseControl IO m => m a -> m a
foo' m = do
s <- liftBaseWith $ \runInBase -> foo (runInBase m)
restoreM s
```

This is, in some ways, superficially similar to the version we wrote using our version of `MonadBaseControl`

. Just like in our version, we capture the input state, apply `foo`

in the `IO`

monad, then restore the state. But what exactly is doing the state capturing, and what is `runInBase`

?

Let’s start by adding a type annotation to `runInBase`

to help make it a little clearer what’s going on:

```
foo' :: forall m a. MonadBaseControl IO m => m a -> m a
foo' m = do
s <- liftBaseWith $ \(runInBase :: forall b. m b -> IO (StM m b)) ->
foo (runInBase m)
restoreM s
```

That type should look sort of recognizable. If we replace `StM`

with `WithOutputState`

, then we get a type that looks very similar to that of our original `closeOverInputState`

function, except it doesn’t need to take the input state as an argument. How does that work?

Here’s the trick: `liftBaseWith`

starts by capturing the input state, just as before. However, it then builds a function, `runInBase`

, which is like `closeOverInputState`

partially-applied to the input state it captured. It hands that function to us, and we’re free to apply it to `m`

, which produces the `IO (StM m a)`

action we need, and we can now pass that action to `foo`

. The result is returned in the outer monad, and we restore the state using `restoreM`

.

### Sharing the input state

At first, this might seem needlessly complicated. When we first started, we separated capturing the input state and closing over it into two separate operations (`captureInputState`

and `closeOverInputState`

), but we eventually combined them so that we could keep the input state hidden. Why does `monad-control`

split them back into two operations again?

As it turns out, when lifting `foo`

, there’s no advantage to the more complicated API of `monad-control`

. In fact, we could implement our `captureAndCloseOverInputState`

operation in terms of `liftBaseWith`

, and we could use that to implement `foo'`

the same way we did before:

```
captureAndCloseOverInputState :: MonadBaseControl b m => m a -> m (b (StM m a))
captureAndCloseOverInputState m = liftBaseWith $ \runInBase -> pure (runInBase m)
foo' :: MonadBaseControl IO m => m a -> m a
foo' m = do
m' <- captureAndCloseOverInputState m
restoreM =<< liftBase (foo m')
```

However, that approach has a downside once we need to lift more complicated functions. `foo`

is exceptionally simple, as it only accepts a single input argument, but what if we wanted to lift a more complicated function that took *two* monadic arguments, such as this one:

`bar :: IO a -> IO a -> IO a`

We could implement that by calling `captureAndCloseOverInputState`

twice, like this:

```
bar' :: MonadBaseControl IO m => m a -> m a -> m a
bar' ma mb = do
ma' <- captureAndCloseOverInputState ma
mb' <- captureAndCloseOverInputState mb
restoreM =<< liftBase (bar ma' mb')
```

However, that would capture the monadic state twice, which is rather inefficient. By using `liftBaseWith`

, the state capturing is done just once, and it’s shared between all calls to `runInBase`

:

```
bar' :: MonadBaseControl IO m => m a -> m a -> m a
bar' ma mb = do
s <- liftBaseWith $ \runInBase ->
bar (runInBase ma) (runInBase mb)
restoreM s
```

By providing a “running” function (`runInBase`

) instead of direct access to the input state, `liftBaseWith`

allows sharing the captured input state between multiple actions without exposing it directly.

### Sidebar: continuation-passing and impredicativity

One last point before we move on: although the above explains why `captureAndCloseOverInputState`

is insufficient, you may be left wondering why `liftBaseWith`

can’t just *return* `runInBase`

. Why does it need to be given a continuation? After all, it would be nicer if we could just write this:

```
bar' :: MonadBaseControl IO m => m a -> m a -> m a
bar' ma mb = do
runInBase <- askRunInBase
restoreM =<< liftBase (bar (runInBase ma) (runInBase mb))
```

To understand the problem with a hypothetical `askRunInBase`

function, remember that the type of `runInBase`

is polymorphic:

`runInBase :: forall a. m a -> b (StM m a)`

This is important, since if you need to lift a function with a type like

`baz :: IO b -> IO c -> IO (Either b c)`

then you’ll want to instantiate that `a`

variable with two different types. We’d need to retain that power in `askRunInBase`

, so it would need to have the following type:

`askRunInBase :: MonadBaseControl b m => m (forall a. m a -> b (StM m a))`

Sadly, that type is illegal in Haskell. Type constructors must be applied to monomorphic types, but in the above type signature, `m`

is applied to a polymorphic type.^{2} The `RankNTypes`

GHC extension introduces a single exception: the `(->)`

type constructor is special and may be applied to polymorphic types. That’s why `liftBaseWith`

is legal, but `askRunInBase`

is not: since `liftBaseWith`

is passed a higher-order function that receives `runInBase`

as an argument, the polymorphic type appears immediately under an application of `(->)`

, which is allowed.

The aforementioned restriction means we’re basically out of luck, but if you *really* want `askRunInBase`

, there is a workaround. GHC is perfectly alright with a field of a datatype being polymorphic, so we can define a newtype that wraps a suitably-polymorphic function:

`newtype RunInBase b m = RunInBase (forall a. m a -> b (StM m a))`

We can now alter `askRunInBase`

to return our newtype, and we can implement it in terms of `liftBaseWith`

:^{3}

```
askRunInBase :: MonadBaseControl b m => m (RunInBase b m)
askRunInBase = liftBaseWith $ \runInBase -> pure $ RunInBase runInBase
```

To use `askRunInBase`

, we have to pattern match on the `RunInBase`

constructor, but it isn’t very noisy, since we can do it directly in a `do`

binding. For example, we could implement a lifted version of `baz`

this way:

```
baz' :: MonadBaseControl IO m => m a -> m b -> m (Either a b)
baz' ma mb = do
RunInBase runInBase <- askRunInBase
s <- liftBase (baz (runInBase ma) (runInBase mb))
bitraverse restoreM restoreM s
```

As of version 1.0.2.3, `monad-control`

does not provide a newtype like `RunInBase`

, so it also doesn’t provide a function like `askRunInBase`

. For now, you’ll have to use `liftBaseWith`

, but it might be a useful future addition to the library.

## Pitfalls

At this point in the blog post, we’ve covered the essentials of `MonadBaseControl`

: how it works, how it’s designed, and how you might go about using it. However, so far, we’ve only considered situations where `MonadBaseControl`

works well, and I’ve intentionally avoided examples where the technique breaks down. In this section, we’re going to take a look at the pitfalls and drawbacks of `MonadBaseControl`

, plus some ways they can be mitigated.

### No polymorphism, no lifting

All of the pitfalls of `MonadBaseControl`

stem from the same root problem, and that’s the particular technique it uses to save and restore monadic state. We’ll start by considering one of the simplest ways that technique is thwarted, and that’s monomorphism. Consider the following two functions:

```
poly :: IO a -> IO a
mono :: IO X -> IO X
```

Even after all we’ve covered, it may surprise you to learn that although `poly`

can be easily lifted to `MonadBaseControl IO m => m a -> m a`

, it’s *impossible* to lift `mono`

to `MonadBaseControl IO m => m X -> m X`

. It’s a little unintuitive, as we often think of polymorphic types as being more complicated (so surely lifting polymorphic functions ought to be harder), but in fact, it’s the flexibility of polymorphism that allows `MonadBaseControl`

to work in the first place.

To understand the problem, remember that when we lift a function of type `forall a. b a -> b a`

using `MonadBaseControl`

, we actually instantiate `a`

to `(StM m c)`

. That produces a function of type `b (StM m c) -> b (StM m c)`

, which is isomorphic to the `m c -> m c`

type we want. The instantiation step is easily overlooked, but it’s crucial, since otherwise we have no way to thread the state through the otherwise opaque function we’re trying to lift!

In the case of `mono`

, that’s exactly the problem we’re faced with. `mono`

will not accept an `IO (StM m X)`

as an argument, only precisely an `IO X`

, so we can’t pass along the monadic state. For all its machinery, `MonadBaseControl`

is no help at all if no polymorphism is involved. Trying to generalize `mono`

without modifying its implementation is a lost cause.

### The dangers of discarded state

Our inability to lift `mono`

is frustrating, but at least it’s conclusively impossible. In practice, however, many functions lie in an insidious in-between: polymorphic enough to be lifted, but not without compromises. The simplest of these functions have types such as the following:

`sideEffect :: IO a -> IO ()`

Unlike `mono`

, it’s entirely possible to lift `sideEffect`

:

```
sideEffect' :: MonadBaseControl IO m => m a -> m ()
sideEffect' m = liftBaseWith $ \runInBase -> sideEffect (runInBase m)
```

This definition typechecks, but you may very well prefer it didn’t, since it has a serious problem: any changes made by `m`

to the monadic state are completely discarded once `sideEffect'`

returns! Since `sideEffect'`

never calls `restoreM`

, there’s no way the state of `m`

can be any different from the original state, but it’s impossible to call `restoreM`

since we don’t actually get an `StM m ()`

result from `sideEffect`

.

Sometimes this may be acceptable, since some monad transformers don’t actually have any output state anyway, such as `ReaderT r`

. In other cases, however, `sideEffect'`

could be a bug waiting to happen. One way to make `sideEffect'`

safe would be to add a `StM m a ~ a`

constraint to its context, since that guarantees the monad transformers being lifted through are stateless, and nothing is actually being discarded. Of course, that significantly restricts the set of monad transformers that can be lifted through.

#### Rewindable state

One scenario where state discarding can actually be useful is operations with so-called rewindable or transactional state. The most common example of such an operation is `catch`

:

`catch :: Exception e => IO a -> (e -> IO a) -> IO a`

When lifted, state changes from the action *or* from the exception handler will be “committed,” but never both. If an exception is raised during the computation, those state changes are discarded (“rewound”), giving `catch`

a kind of backtracking semantics. This behavior arises naturally from the way a lifted version of `catch`

must be implemented:

```
catch' :: (Exception e, MonadBaseControl IO m) => m a -> (e -> m a) -> m a
catch' m f = do
s <- liftBaseWith $ \runInBase ->
catch (runInBase m) (runInBase . f)
restoreM s
```

If `m`

raises an exception, it will never return an `StM m a`

value, so there’s no way to get ahold of any of the state changes that happened before the exception. Therefore, the only option is to discard that state.

This behavior is actually quite useful, and it’s definitely not unreasonable. However, useful or not, it’s inconsistent with state changes to mutable values like `IORef`

s or `MVar`

s (they stay modified whether an exception is raised or not), so it can still be a gotcha. Either way, it’s worth being aware of.

#### Partially discarded state

The next function we’re going to examine is `finally`

:

`finally :: IO a -> IO b -> IO a`

This function has a similar type to `catch`

, and it even has similar semantics. Like `catch`

, `finally`

can be lifted, but unlike `catch`

, its state *can’t* be given any satisfying treatment. The only way to implement a lifted version is

```
finally' :: MonadBaseControl IO m => m a -> m b -> m a
finally' ma mb = do
s <- liftBaseWith $ \runInBase ->
finally (runInBase ma) (runInBase mb)
restoreM s
```

which always discards all state changes made by the second argument. This is clear just from looking at `finally`

’s type: since `b`

doesn’t appear anywhere in the return type, there’s simply no way to access that action’s result, and therefore no way to access its modified state.

However, don’t despair: there actually *is* a way to produce a lifted version of `finally`

that preserves all state changes. It can’t be done by lifting `finally`

directly, but if we reimplement `finally`

in terms of simpler lifted functions that are more amenable to lifting, we can produce a lifted version of `finally`

that preserves all the state:^{4}

```
finally' :: MonadBaseControl IO m => m a -> m b -> m a
finally' ma mb = mask' $ \restore -> do
a <- liftBaseWith $ \runInBase ->
try (runInBase (restore ma))
case a of
Left e -> mb *> liftBase (throwIO (e :: SomeException))
Right s -> restoreM s <* mb
```

This illustrates an important (and interesting) point about `MonadBaseControl`

: whether or not an operation can be made state-preserving is not a fundamental property of the operation’s type, but rather a property of the types of the exposed primitives. There is sometimes a way to implement a state-preserving variant of operations that might otherwise seem unliftable given the right primitives and a bit of cleverness.

#### Forking state

As a final example, I want to provide an example where the state may not actually be discarded *per se*, just inaccessible. Consider the type of `forkIO`

:

`forkIO :: IO () -> IO ThreadId`

Although `forkIO`

isn’t actually polymorphic in its argument, we can convert *any* `IO`

action to one that produces `()`

via `void`

, so it might as well be. Therefore, we can lift `forkIO`

in much the same way we did with `sideEffect`

:

```
forkIO' :: MonadBaseControl IO m => m () -> m ThreadId
forkIO' m = liftBaseWith $ \runInBase -> forkIO (void $ runInBase m)
```

As with `sideEffect`

, we can’t recover the output state, but in this case, there’s a fundamental reason that goes deeper than the types: we’ve forked off a concurrent computation! We’ve therefore split the state in two, which might be what we want… but it also might not. `forkIO`

is yet another illustration that it’s important to think about the state-preservation semantics when using `MonadBaseControl`

, or you may end up with a bug!

`MonadBaseControl`

in context

Congratulations: you’ve made it through most of this blog post. If you’ve followed everything so far, you now understand `MonadBaseControl`

. All the tricky parts are over. However, before wrapping up, I’d like to add a little extra information about how `MonadBaseControl`

relates to various other parts of the Haskell ecosystem. In practice, that information can be as important as understanding `MonadBaseControl`

itself.

### The remainder of `monad-control`

If you look at the documentation for `monad-control`

, you’ll find that it provides more than just the `MonadBaseControl`

typeclass. I’m not going to cover everything else in detail in this blog post, but I do want to touch upon it briefly.

First off, you should definitely take a look at the handful of helper functions provided by `monad-control`

, such as `control`

and `liftBaseOp_`

. These functions provide support for lifting common function types without having to use `liftBaseWith`

directly. It’s useful to understand `liftBaseWith`

, since it’s the most general way to use `MonadBaseControl`

, but in practice, it is simpler and more readable to use the more specialized functions wherever possible. Many of the examples in this very blog post could be simplified using them, and I only stuck to `liftBaseWith`

to introduce as few new concepts at a time as possible.

Second, I’d like to mention the related `MonadTransControl`

typeclass. You hopefully remember from earlier in the blog post how we defined `MonadBaseControl`

instances inductively so that we could lift all the way down to the base monad. `MonadTransControl`

is like `MonadBaseControl`

if it intentionally did *not* do that—it allows lifting through a single transformer at a time, rather than through all of them at once.

Usually, `MonadTransControl`

is not terribly useful to use directly (though I did use it once in a previous blog post of mine to help derive instances of mtl-style classes), but it *is* useful for implementing `MonadBaseControl`

instances for your own transformers. If you define a `MonadTransControl`

instance for your monad transformer, you can get a `MonadBaseControl`

implementation for free using the provided `ComposeSt`

, `defaultLiftBaseWith`

, and `defaultRestoreM`

bindings; see the documentation for more details.

`lifted-base`

and `lifted-async`

If you’re going to use `MonadBaseControl`

, the `lifted-base`

and `lifted-async`

packages are good to know about. As their names imply, they provide lifted versions of bindings in the `base`

and `async`

packages, so you can use them directly without needing to lift them yourself. For example, if you needed a lifted version of `mask`

from `Control.Exception`

, you could swap it for the `mask`

export from `Control.Exception.Lifted`

, and everything would mostly just work (though always be sure to check the documentation for any caveats on state discarding).

### Relationship to `MonadUnliftIO`

Recently, FP Complete has developed the `unliftio`

package as an alternative to `monad-control`

. It provides the `MonadUnliftIO`

typeclass, which is similar in spirit to `MonadBaseControl`

, but heavily restricted: it is specialized to `IO`

as the base monad, and it *only* allows instances for stateless monads, such as `ReaderT`

. This is designed to encourage the so-called `ReaderT`

design pattern, which avoids ever using stateful monads like `ExceptT`

or `StateT`

over `IO`

, encouraging the use of `IO`

exceptions and mutable variables (e.g. `MVar`

s or `TVar`

s) instead.

I should be clear: I really like most of what FP Complete has done—to this day, I still use `stack`

as my Haskell build tool of choice—and I think the suggestions given in the aforementioned “`ReaderT`

design pattern” blog post have real weight to them. I have a deep respect for Michael Snoyman’s commitment to opinionated, user-friendly tools and libraries. But truthfully, I can’t stand `MonadUnliftIO`

.

`MonadUnliftIO`

is designed to avoid all the complexity around state discarding that `MonadBaseControl`

introduces, and on its own, that’s a noble goal. Safety first, after all. The problem is that `MonadUnliftIO`

really is extremely limiting, and what’s more, it can actually be trivially encoded in terms of `MonadBaseControl`

as follows:

`type MonadUnliftIO m = (MonadBaseControl IO m, forall a. StM m a ~ a)`

This alias can be used to define safe, lifted functions that never discard state while still allowing functions that *can* be safely lifted through stateful transformers to do so. Indeed, the `Control.Concurrent.Async.Lifted.Safe`

module from `lifted-async`

does exactly that (albeit with a slightly different formulation than the above alias).

To be fair, the `unliftio`

README does address this in its comparison section:

`monad-control`

allows us to unlift both styles. In theory, we could write a variant of`lifted-base`

that never does state discards […] In other words, this is an advantage of`monad-control`

over`MonadUnliftIO`

. We've avoided providing any such extra typeclass in this package though, for two reasons:

`MonadUnliftIO`

is a simple typeclass, easy to explain. We don't want to complicated [sic] matters […]Having this kind of split would be confusing in user code, when suddenly [certain operations are] not available to us.

In other words, the authors of `unliftio`

felt that `MonadBaseControl`

was simply not worth the complexity, and they could get away with `MonadUnliftIO`

. Frankly, if you feel the same way, by all means, use `unliftio`

. I just found it too limiting given the way I write Haskell, plain and simple.

## Recap

So ends another long blog post. As often seems the case, I set out to write something short, but I ended up writing well over 5,000 words. I suppose that means I learned something from this experience, too: `MonadBaseControl`

is more complicated than I had anticipated! Maybe there’s something to take away from that.

In any case, it’s over now, so I’d like to briefly summarize what we’ve covered:

`MonadBaseControl`

allows us to lift higher-order monadic operations.It operates by capturing the current monadic state and explicitly threading it through the action in the base monad before restoring it.

That technique works well for polymorphic operations for the type

`forall a. b a -> b a`

, but it can be tricky or even impossible for more complex operations, sometimes leading to discarded state.This can sometimes be mitigated by restricting certain operations to stateless monads using a

`StM m a ~ a`

constraint, or by reimplementing the operation in terms of simpler primitives.The

`lifted-base`

and`lifted-async`

packages provide lifted versions of existing operations, avoiding the need to lift them yourself.

As with many abstractions in Haskell, don’t worry too much if you don’t have a completely firm grasp of `MonadBaseControl`

at first. Insight often comes with repeated experience, and `monad-control`

can still be used in useful ways even without a perfect understanding. My hope is that this blog post has helped you build intuitions about `MonadBaseControl`

even if some of the underlying machinery remains a little fuzzy, and I hope it can also serve as a reference for those who want or need to understand (or just be reminded of) all the little details.

Finally, I’ll admit `MonadBaseControl`

isn’t especially elegant or beautiful as Haskell abstractions go. In fact, in many ways, it’s a bit of a kludge! Perhaps, in time, effect systems will evolve and mature so that it and its ilk are no longer necessary, and they may become distant relics of an inferior past. But in the meantime, it’s here, it’s useful, and I think it’s worth embracing. If you’ve shied away from it in the past, I hope I’ve illuminated it enough to make you consider giving it another try.

One example of a function with that type is

`mask_`

. ↩Types with polymorphic types under type constructors are called

*impredicative*. GHC technically has limited support for impredicativity via the`ImpredicativeTypes`

language extension, but as of GHC 8.8, it has been fairly broken for some time. A fix is apparently being worked on, but even if that effort is successful, I don’t know what impact it will have on type inference. ↩Note that

`askRunInBase = liftBaseWith (pure . RunInBase)`

does*not*typecheck, as it would require impredicative polymorphism: it would require instantiating the type of`(.)`

with polymorphic types. The version using`($)`

works because GHC actually has special typechecking rules for`($)`

! Effectively,`f $ x`

is really syntax in GHC. ↩Assume that

`mask'`

is a suitably lifted version of`mask`

(which can in fact be made state-preserving). ↩