Higher Kinds in C# with language-ext [Part 10- ReaderT monad transformer]
![Higher Kinds in C# with language-ext [Part 10- ReaderT monad transformer]](/content/images/size/w960/2024/06/readert.png)
This is a multi-part series of articles – and you'll need to know much of what's in the earlier parts to follow this – so it's worth going back to the earlier episodes if you haven't read them already:
- Part 1 - Introduction, traits, semigroups, and monoids
- Part 2 - Functors
- Part 3 - Foldables
- Part 4 - Applicatives
- Part 5 - Validation
- Part 6 - Traversables
- Part 7 - Monads
- Part 8 - Monads (continued)
- Part 9 - Monad Transformers
In the last article we started looking into monad transformers. We mostly focused on the MaybeT
transformer (OptionT
in language-ext), lifting the IO
monad into it to create an optional side-effect.
Later in the article I did a quick overview of a project (that I am working on) that creates 'domain monads' by wrapping monad-transformer stacks into new types that are then further composed with free-monads to create a small application architecture.
There is never a 'right way' to structure an application and I think it's risky to use blogs, or other short-form methods to share coding advice on the 'one true way', because it doesn't exist. However, I think when you see how domains, layers, components, and structure in-general, evolves out of compositional types (like monads and monad-transformers) then that light-bulb moment will happen where you can suddenly 'see' the structure of your architecture from 10,000 feet.
These moments of clarity were always missing for me in my former OOP life, whereas with the pure functional programming approach I feel I have total clarity, but also it feels like the architecture is fluid and malleable (it can change with requirements) in a way that just isn't there in the object-oriented world. This may well be a personal thing, so please don't take this as me saying "there's exactly one way to write code" – because all self-appointed internet experts really should be taken with a handful of salt!
Even though the architecture example (from the last article) was a small example, I hope that it whetted your appetite when thinking about how to use pure functional programming techniques to create structure in your programs!
To really benefit the most from this approach we should aim to make our entire code-base pure and getting the architecture right is part of that story. What I find valuable about this approach is that the architecture itself becomes composable. That's quite profound when you think about it and it certainly changes the way I write code now.
It was quite a lot to take in, of course, and I certainly wouldn't expect novice FP devs to get it straight-away. But, one of the biggest issues I've had with building language-ext has been how to put control of building functional types into the hands of the users of the library. Before the higher-kinds approach (demonstrated in this series) there was no way to write code that worked for all monads, or all functors, etc. Now you can.
And before we had monad-transformers there was no way to compose the monads that already existed into new 'super monads', which meant I was stuck writing OptionAsync
, EitherAsync
, TryOption
, and other combination-monads. And you were stuck waiting for me to build various pre-combined monads. This issue no longer exists.
These abstractions now exist. And many of the core types needed for you to build your own functional stacks now exist. The days of waiting for me to buildValidationAsync<F, A>
are over, just use theValidationT<F, IO, A>
monad transformer stack.
It may take a little while for the implications of the changes in v5
of language-ext to really materialise as opportunities for you. It's a style of programming and an approach to composition that just hasn't existed in C# to-date. So, unless you've come from a Haskell background, this might not seem to be quite as profound as it really is.
C# isn't Haskell of course, but the reason for all of this isn't to make C# into Haskell, it's to make the inertia of the C# developer track in the direction of pure functional programming. And it just so happens that Haskell (as a pure functional programming language) has solved many patterns we need and so the inspiration often comes from that language.
Let's take a look at another monad-transformer...
ReaderT<Env, M, A>
The ReaderT
monad-transformer adds a static environment, Env
, to a given monad M
. We briefly covered the Reader<Env, A>
monad in the Monads Continued article – this generalises the idea into a monad-transformer. So, for example, let's say we had some application configuration in a simple record type:
public record MyConfig(string SourceFolder);
We want to write pure code, so we can't just stick this in a global static
variable.
One option is to pass it as an argument to every function that needs the configuration settings:
public static IO<string> readSourceText(string file, MyConfig config)
{
var path = Path.Combine(config.SourceFolder, file);
return IO.liftAsync(e => File.ReadAllTextAsync(path, e.Token));
}
This is perfectly valid, but it can start to get a bit messy and annoying. And, aside from the name of the type, it's not very declarative. If MyConfig
had a less descriptive name then it might make the interface to the function more confusing.
Instead, we drop the MyConfig
argument and use the ReaderT
monad-transformer. We can stack it with the IO
monad to get both a static environment and side-effects.
ReaderT
will automatically thread the MyConfig
value through the computation for us and do so declaratively:
public static ReaderT<MyConfig, IO, string> readSourceText(string file) =>
from config in ReaderT.ask<IO, MyConfig>()
let path = Path.Combine(config.SourceFolder, file)
from text in IO.liftAsync(e => File.ReadAllTextAsync(path, e.Token))
select text;
Notice how there isn't a MyConfig
value passed as an argument to readSourceText
. It somehow magics the configuration out of thin air!
The key to this is on the first line in the LINQ expression:
ReaderT.ask<IO, MyConfig>()
This static
function: ask
, somehow gets the MyConfig
structure and loads it into the config
variable. How can a static
function retrieve a non-global state?
If we look at the definition of ReaderT<Env, M, A>
we'll get some clues:
public record ReaderT<Env, M, A>(Func<Env, K<M, A>> runReader) : K<ReaderT<Env, M>, A>
where M : Monad<M>, SemiAlternative<M>
{
public K<M, A> Run(Env env) =>
this.runReader(env);
...
}
Internally, ReaderT
is a Func<Env, K<M, A>>
. If we provide an Env
then we get a K<M, A>
back (which is constrained to be a Monad<M>
).
There's also a SemiAlternative<M>
constraint. Ignore that for now, I'll cover it in a later article.
This follows the transformer pattern. Except this is a little different to MaybeT
and OptionT
in that K<M, A>
is the inner type, whereas with MaybeT
and OptionT
it was the outer type: K<M, Option<A>>
.
This difference in implementation is all to do with what is best for the particular transformer. It makes sense to provide an environment for the inner monad to run in. Whereas, for the optional types, it makes sense for the result to be optional! In general, optional types are wrapped by the lifted monad, and stateful-computation types wrap the lifted monad.
When we come to run the ReaderT
transformer, we must pass the configuration value to it, so it can be passed through the computation:
// Create the application configuration object.
// Typically you would do this once
var configObj = new MyConfig("c:\\folder");
// Run the ReaderT. This returns an IO<string>
var configTextIO = readSourceText("test.cs").Run(configObj);
The result is an IO<string>
. The IO
monad being that which was lifted into the transformer.
If we run the IO
value then we get the raw text:
var configText = configTextIO.Run();
Reminder: You shouldn't exit the IO
monad unless you're at the very edge of your application.
That doesn't explain how ask
works from earlier. It somehow magicked the configuration out of thin air. Remember, we're working with a wrapped function here, which means we can defer the gathering of the configuration (make it lazy):
new ReaderT<Env, M, Env>(env => M.Pure(env));
So, we don't need to know the environment to implement ask
, we just need to know we will be given it at some point. Here we take the env
input and simply lift it into the M
monad which exposes it as the bound value. Extremely simple, but also extremely effective.
It can be simplified further to:
new ReaderT<Env, M, Env>(M.Pure);
And even further when it's part of a function that can infer the return type:
public static ReaderT<Env, M, Env> ask<Env, M>()
where M : Monad<M>, SemiAlternative<M> =>
new (M.Pure);
Everything in ReaderT
therefore is lazy. We know the Run
method will provide an Env
value which we can use to invoke the encapsulated delegate and realise a concrete result. But, every other (non-Run
) method on ReaderT
must use standard function composition to achieve its behaviours.
For example, the Bind
function:
public ReaderT<Env, M, B> Bind<B>(Func<A, ReaderT<Env, M, B>> f) =>
new(env => M.Bind(runReader(env), x => f(x).runReader(env)));
We create a new Func
that provides us with an Env env
, we invoke our runReader
function with the env
value. It gives us the wrapped K<M, A>
which we can M.Bind
because it's constrained to be a Monad<M>
. That gets us the A
value from within the M
monad (assigned to x
). We pass the A
to the f
function provided. It returns a new ReaderT
which we then invoke (with the same env
). It returns a K<M, B>
which is what M.Bind
expects as a result!
This is how the environment is threaded through the monad-transformer and how it can be made available for use in the LINQ expression. The env
is always available because every ReaderT
has it given when invoked.
Readable
So, that's all awesome and everything, but when we use transformers it's often good to wrap up the stack into a new type for encapsulation reasons. The eagle eyed among you may realise that ReaderT<Env, IO, A>
is essentially Eff<RT, A>
from previous versions of language-ext. The Eff
monad was designed to be an IO monad with a 'runtime' that allowed for injectable side-effects.
So, let's wrap it up and create our own simplified Eff
monad. First, let's create the Eff<RT, A>
type:
public record Eff<RT, A>(ReaderT<RT, IO, A> runEff) : K<Eff<RT>, A>;
This simply wraps the ReaderT<RT, IO, A>
monad transformer.
Next, let's make Eff<RT, A>
into a monad by implementing the Monad
trait:
public class Eff<RT> : Monad<Eff<RT>>
{
public static K<Eff<RT>, B> Bind<A, B>(K<Eff<RT>, A> ma, Func<A, K<Eff<RT>, B>> f) =>
new Eff<RT, B>(ma.As().runEff.Bind(x => f(x).As().runEff));
public static K<Eff<RT>, B> Map<A, B>(Func<A, B> f, K<Eff<RT>, A> ma) =>
new Eff<RT, B>(ma.As().runEff.Map(f));
public static K<Eff<RT>, A> Pure<A>(A value) =>
new Eff<RT, A>(ReaderT.Pure<RT, IO, A>(value));
public static K<Eff<RT>, B> Apply<A, B>(K<Eff<RT>, Func<A, B>> mf, K<Eff<RT>, A> ma) =>
new Eff<RT, B>(mf.As().runEff.Apply(ma.As().runEff).As());
public static K<Eff<RT>, A> LiftIO<A>(IO<A> ma) =>
new Eff<RT, A>(ReaderT.liftIO<RT, IO, A>(ma));
}
Note how every method is just a wrapper around the calls to the ReaderT
implementations (runEff
is a ReaderT<RT, IO, A>
). Also note that we have implemented LiftIO
which isn't enforced (there's a default implementation) but this allows us to use IO
operations in our LINQ expressions directly (we covered this in the last article).
Next, create our As
and Run
extensions and add some constructors:
public static class Eff
{
public static Eff<RT, A> As<RT, A>(this K<Eff<RT>, A> ma) =>
(Eff<RT, A>)ma;
public static IO<A> Run<RT, A>(this K<Eff<RT>, A> ma, RT runtime) =>
ma.As().runEff.Run(runtime).As();
public static Eff<RT, A> Pure<RT, A>(A value) =>
new (ReaderT.Pure<RT, IO, A>(value));
public static Eff<RT, A> Fail<RT, A>(Error error) =>
new (ReaderT.liftIO<RT, IO, A>(IO.Fail<A>(error)));
}
And that's it!
Of course, the real Eff
monad comes with lots of helper functions that make working with effects easier. But pretty much everything is just lifting the existing functionality from the transformers into the Eff
type. For example, to access the runtime is simply a ReaderT.ask
:
public static Eff<RT, RT> runtime<RT>() =>
new (ReaderT.ask<IO, RT>());
You should be getting the idea now that this is just a wrapper around the transformer stack. We could extend the capabilities by stacking more transformers if necessary.
The real reason I wanted to demonstrate this is because there is another trait-type especially for readers. It's called: Readable
and any type that implements it becomes 'readable':
public interface Readable<M, Env>
where M : Readable<M, Env>
{
public static abstract K<M, A> Asks<A>(Func<Env, A> f);
public static virtual K<M, Env> Ask =>
M.Asks(Prelude.identity);
public static abstract K<M, A> Local<A>(
Func<Env, Env> f,
K<M, A> ma);
}
For those who know Haskell, this is essentiallyMonadReader
without inheriting theMonad
constraint!
We can add it to the traits that Eff<RT>
implements:
public class Eff<RT> : Monad<Eff<RT>>, Readable<Eff<RT>, RT>
{
// ... implementations from earlier ...
public static K<Eff<RT>, A> Asks<A>(Func<RT, A> f) =>
new Eff<RT, A>(ReaderT<RT, IO, A>.Asks(f));
public static K<Eff<RT>, A> Local<A>(Func<RT, RT> f, K<Eff<RT>, A> ma) =>
new Eff<RT, A>(ma.As().runEff.Local(f));
}
Readable
just abstracts the idea of accessing a static environment away from the monad. It means we can write code that accepts any 'readable' type, whether it's Eff
, ReaderT
, Reader
, or whatever bespoke type we have built.
Readeable
also comes with some static functions that makes readable types easy to work with:
public static class Readable
{
public static K<M, Env> ask<M, Env>()
where M : Readable<M, Env> =>
M.Ask;
public static K<M, A> asks<M, Env, A>(Func<Env, A> f)
where M : Readable<M, Env> =>
M.Asks(f);
public static K<M, A> asksM<M, Env, A>(Func<Env, K<M, A>> f)
where M : Readable<M, Env>, Monad<M> =>
M.Flatten(M.Asks(f));
public static K<M, A> local<M, Env, A>(Func<Env, Env> f, K<M, A> ma)
where M : Readable<M, Env> =>
M.Local(f, ma);
}
Let's say our 'runtime' was just an int
and we want to write a function that takes two Eff
monads with int
as their bound-values and then sum those values with the int
that's in the runtime (the environment).
public static K<Eff<int>, int> addEff(Eff<int, int> mx, Eff<int, int> my) =>
from x in mx
from y in my
from z in Eff.runtime<int>()
select x + y + z;
This works just fine, we can access the runtime value via the Eff.runtime
function. However, this function only works with one monadic type: Eff
.
We can generalise it even further:
public static K<M, int> addRdr<M>(K<M, int> mx, K<M, int> my)
where M : Readable<M, int>, Monad<M> =>
from x in mx
from y in my
from z in Readable.ask<M, int>()
select x + y + z;
Note how M
is not only constrained to be a Monad<M>
but it is also constrained to be a Readable<M, int>
. So, it's bindable and readable! Let's try running it with some Eff
monads:
var compute = addRdr(Eff.Pure<int, int>(100), Eff.Pure<int, int>(200));
var result = compute.Run(300).Run(); // 600
addRdr
has no idea what monad we're running and it has no idea how we're managing our static environment. It is a complete abstraction away from the internals. The Eff
monad supports IO side-effects, parallelism, resource tracking and auto-cleanup, error tracking, retries, repeats, ..., and a static environment. All of which could happen in the addRdr
function (because mx
and my
are computations) – but the implementation doesn't care, it is abstracted away. All it cares about is that the values are bindable and readable!
This is the path to true abstraction – where everything is pure. We can safely abstract, because we know all of the components are playing by the rules!
You may think this kind of abstraction isn't that valuable. But, if you remember the real-world domain-monads example from the previous article – where I created a Db
monad, a Service
monad, and an Api
monad – then it's not a massive stretch to imagine writing some very general support functions that work with all of those monads (using the Monad
constraint) and accessing their configuration (using the Readable
constraint).
For example, you could imagine a User
value being in the environment and a getLoggedInUser
function to access it from any monad that supports it:
public record User(string Id);
public record Session(User LoggedIn, DateTime LastAccess);
public static K<M, Session> getSession<M>()
where M : ReaderM<M, Session> =>
ReaderM.ask<M, Session>();
public static K<M, User> getLoggedInUser<M>()
where M : ReaderM<M, Session> =>
ReaderM.asks<M, Session, User>(s => s.LoggedIn);
We could expand that out to embed security roles and rights so we could do inline assertions throughout our code-base:
// Simple User type with list of roles it's a member of
public record User(string Id, Seq<Role> Memberships);
// Union type of rights
public abstract record Right;
public record CanViewDemographic : Right;
public record CanSendInvoice : Right;
// Role with its allowed rights
public record Role(Seq<Right> Rights);
Next we can write the function exactly once to deal with access-rights:
public static K<M, Unit> assertHasRight<M, R>()
where R : Right
where M : Readable<M, Session>, Functor<M> =>
getLoggedInUser<M>().Map(
u => u.Memberships.Exists(
role => role.Rights.Exists(
right => right is R))
? Prelude.unit
: throw new SecurityException("Access denied"));
Then at the start of an API call, for example, you could setup the User
in a Session
environment (pre-populating the rights and roles once per request). That Session
can then be passed to the computation that represents the API request (via Run(session)
) which will then automatically thread the session, user, and rights through your monadic stack.
The (read-only) user, security rights, and roles will automatically propagate through your code. Each monad that wants to have access to security credentials could just expose them via a Readable
trait implementation.
We can then write our security assertions code in any monad that supports them:
public static Eff<Session, Unit> sendInvoice() =>
from _ in assertHasRight<Eff<Session>, CanSendInvoice>()
from r in doSendInvoice()
select r;
Being able to write security code once and keep it tightly audited is the mark of a professional approach to software development. So these abstractions are not just academic examples of what can be done with pure functional programming, they're very real examples of how you can make your code more robust and more secure. You'll be the auditors best friend – although I'm not sure that's a good thing!
Local environments
Having a fixed static environment for your entire code-base is of course possible, but that will tend to expose more 'environment' than is necessary for any subsystem. And so, it would be good if we could swap out our larger environments for something smaller on-the-fly. This is where ReaderT.local(f, ma)
and ReaderT.with(f, ma)
comes in:
local
maps anEnv
to anEnv
– this allows simple mutation of the environment. Thema
computation is then run in that locally mutated environment. By the end of the call the environment is reset.with
maps anEnv
to an alternative type which can be any other value you like, but often it is derived fromEnv
. For example, you may have anAppConfig
which contains aDbConfig
– it could make sense to only expose theDbConfig
to your data-layer by mapping fromAppConfig -> DbConfig
.
Note thatwith
is not available in theReadable
trait and so you must useReader.with
andReaderT.with
to map the environment to a different type (meaning we can't fully generalise the environment mapping). For any type that encapsulatesReaderT
orReader
it is worth adding your own equivalent ofwith
to do environment mapping (if that makes sense for your type, some wrappers might have a fixed environment). This is a limitation brought about by how the higher-kinds are implemented (theEnv
is baked into the trait implementation) – I haven't thought of an easy solution to this yet!
Conclusion
I often show (and will continue to show) the IO
monad being the monad that is lifted into the transformer. I'm mostly doing this because it's easier to see tangible examples of what it means to compose monads with transformers. Just remember that any monad can be lifted into ReaderT
– it could be a Validation<F, A>
to have an environment for your validators (for example).
One other thing to note is that ReaderT
would usually be the outermost type in the transformer stack. This is usually because you want the inner monad(s) in the transformer stack to have access to the environment. This isn't a rule, it's just usually the most effective position.
The ReaderT
transformer really is the first 'control' transformer that shows how capabilities can be 'bolted on' to existing monads in a way that starts to break away from the relatively 'small thinking' of optional types and other common FP topics. Being able to carry a – pure – environment through your code really is the first part of building a pure architecture.
In previous versions of language-ext we only have Reader
which was just an environment carrying monad – nothing else, it didn't compose with anything and that meant it was probably never used (I certainly never used it). Now we have the ability to augment with readable traits and that is extremely powerful!
Next up will be the StateT
transformer. Very much like the ReaderT
transformer except the environment can be modified. Pure mutation? You bet!