Introduction

Presented approach is nothing new but just well-known patterns used in a cohesive way. In a sense it tries to compete with free monads or more modern Eff monad without going crazy too much. Those techniques are much more complicated and consequently more powerful, but they require a lot of boilerplate (even when using some libraries born to reduce it, like Freek or Liberator for free monads) and refactoring.

On the other hand, monadic abstractions approach in a sense is a less radical step forward to pure functional programming. It's much simpler and can be more easily applied to existing code that uses scala.concurrent.Futures directly, without re-writing it from scratch.

Step by step guide by example

So, let's pretend we're writing a microservice that uses Scala futures for asynchronicity, Slick 3 for relational database access, and something like akka-http to make REST calls to other microservices. First thing we do is...

Abstracting interfaces

Let's say we have a service layer and a repository layer. The first trick is to abstract away from concrete "monad" used as a methods result types wrapper, e.g. Future or DBIO, in interfaces:

Currently dominant style is to use scala.concurrent.Future as M in service trait and scala.concurrent.Future or DBIO as M in repository traits. There are two downsides with such approach:

• using futures in repositories would force to evaluate your DBIO actions too early (before leaving repository layer) effectively disallowing a transaction to span across multiple repository calls;
• using DBIO as M in repository interfaces would leak repository layer implementation details to upper layers, adding `Slick* stuff to services classpath.

So, by abstracting it out, we want to be able to run cross-repository calls transactions and fix the implementation details leak. And we don't really want to have Slick classes in service layer auto-complete!

We use higher order type parameter M, saying that we return a "container" of something, but we're not explicit about its type. It can be Scala Future, Monix Task, Slick's DBIO, cats' Eval, or even Function0!

Implementing repositories

Next we implement repositories by using DBIO to substitute abstract Ms:

Implementation details aren't relevant here and therefore omitted for brevity. More interesting is how we implement the service.

Implementing services

Notice that service implementation should not depend on repository implementation but only on interfaces. We can say that each layer (e.g. repository, service, etc) has at least two parts, and consequently build artifacts: the api part and one or more impl parts. So each layer above only depends on api artifact of a layer below. Explicitly:

• service api doesn't depend on repository artifacts, cause it would make them available to layers above - something we what to disallow;
• service impl depends on repository api (not impl!) so that repository implementation could be swapped without changing service code (ideally);
• there is an app layer on the very top, where all the wiring takes place, it depends on both service and repository layers impl artifacts (and apis transitively) and provides a concrete repository layer implementation to the service layer.

But how the service code looks then? Here comes the most interesting part:

Let's iterate it one thing at a time.

Still generic M[_]

Notice that service code is still generic in the sense that it's detached from a concrete monad implementation, we're still using M[_]. And it's a good sign!

DbEffect type parameter

Additionally, our service implementation declares a type parameter to abstract over a concrete "monad" (or "effect", or "container") of the repository layer. We could call it just D but I like the DbEffect name as more self-explanatory here. We also parameterize our repositories dependencies with DbEffect effectively saying that we depend on repositories that produce values within a generic container called DbEffect - and that's all we know about them so far, but... stay tuned!

And yes - we're using constructor parameters for dependency injection.

evalDb argument

We've also got a fancy looking evalDb: DbEffect ~> M constructor parameter. It can be written more explicitly like this:

In cats' code it's just a type alias for FunctionK trait:

This thing is called a natural transformation and all it can do is to convert a value of one generic type F[_] to another generic type G[_]. And that's exactly what we need to "run" our underlying DbEffects: whatever monad service code uses for its results, we want to transform repository effects into it. For DBIO actions, when we pass evalDb implementation to our service in the app layer, we are going to execute them transactionally and get back futures (if we decide to use them as service's M implementation).

There is one aspect of DbEffect and M that gives compile-time guarantee of not having long-running transactions (what is probably a good idea). If you decide to do something like making REST call when transaction is running, you'll get a result of type DbEffect[M[DbEffect[T]]]. And you won't be able to escape out of it. Strictly speaking, there is a typeclass called Traverse that brings the ability to "traverse" one context with another, i.e. go from F[G[T]] to G[F[T]], and since we can't have it for DbEffect and M (at least when DbEffect is DBIO and M is some asynchronous boundary like Future), - we cannot traverse it!

Now to the...

Implicit parameters

To recap, we require the following implicits when creating the service:

In case you know what is a Monad (e.g. from my previous article) but never saw the MonadError, you can think about it like a Monad with built-in error handling. You can use standard Scala Future methods intuition, like Future.failed, Future.recover, or Future.recoverWith. They allow to create a failed value and to recover from a failure. Same for the MonadError typeclass but in much more abstract purely functional setting.

Why we require MonadError for M but Monad for DbEffect? It's just a rule of least power: I'm not going to use error handling for DbEffect values in the method implementation so I just stick with the Monad instance for DbEffect but it could definitely be MonadError[DbEffect, Throwable] as well - no trickery here!

So, by having this implicit parameters we declare that:

• M is a monad (with error handling) and consequently we can map/flatMap/recover values of type M - same way as one would use Futures in for comprehension;
• DbEffect is a monad, so we can chain DbEffect values as well, before transforming them to M using evalDb natural transformation (you can think about it like about transaction).

Wiring it all together

Before actually implementing our service method, let's see how the components can be wired in the top app level. Here are some common Slick helpers to be used:

I won't go into much detail here, this is something similar to what many teams come up with when using Slick. The most important thing is to be able to define ~> instance that can evaluate DBIO values. Notice that we're using same "abstracting interfaces" approach for shared common code as for the components above.

Now, wiring can look similar to this:

For simplicity I'm not going to define web layer, but it might use akka-http and require services dependencies as securityService: SecurityService[Future]-like constructor parameters (or even securityService: SecurityService[M] if you decide to generify your controllers/resources/whatever-you-call-it as well, though there's less benefit from doing that in comparison to service code).

But what about monad instances?

An astute reader might notice that something is missing when creating the SecurityServiceGenericImpl above - namely the implicit Monad and MonadError instances:

• MonadError[Future];
• Monad[DBIO].

The first is defined in cats and can be put in scope by adding import cats.implicits._ import.

As for the second one, there is a project on Github called slick-cats that defines MonadError and some other instances for DBIO. They can be brought in scope by introducing a dependency on slick-cats and adding import com.rms.miu.slickcats.DBIOInstances._ import statement.

Finally we're ready to implement our example service method.

Service method implementation

Method logic is following:

1. get user's password last changed date;
2. independently, get user's password validity period setting;
3. if password is expired, call some other microservice to send a notification email to the user;
4. if an error occurs during the notification sending, recover error to a specific SendNotificationResult ADT value.

Even in this simplified example we have multiple things to consider, namely:

• we want to make first two calls independently on each other, i.e. no flatMaping;
• execute both calls in a single transaction (it might be more important in more real-world scenario than here);
• return a ErrorWhileSending ADT value if an error occurs during notification sending (error handling);
• though we imply that repository calls cannot fail in our example, we might want to provide MonadError instance for DbEffect and handle DB errors gracefully (not covered here).

Here is the implementation split to several methods:

Steps 1 and 2 are implemented in isPasswordExpired method. Notice that it doesn't leave DbEffect monad yet, effectively causing both repository calls to execute in a single transaction, when run later by evalDb. This method uses applicative builder syntax to leverage the fact that Monad is also an Applicative functor. In a nutshell, Applicative is a less powerful abstraction that allows to, in this case, apply a function (A, B) => C to F[A] and F[B] to get F[C]. First, we use the fact that DbEffect is an Applicative, then the fact that Option is also an Applicative (typeclass instance for Applicative[Option] provided by cats).

The main difference between Monad and Applicative is that the former allows for computations chaining making one computation to depend on another, while the latter doesn't allow chaining and considers computations simultaneously, even with ability to run them in parallel when looking at how Applicative[Future] instance works. See cats documentation for more information.

In sendNotificationIfPasswordExpired method we evalDb the isPasswordExpired method result and, since it has DbEffect[Option[Boolean]] type, we can apply OptionT monad transformer to it. Monad transformers (hence the T in OptionT) allow to stack one monad on top of another and work with them without nesting. Again, google for cats monads transformers if needed. In semiflatMap method lambda we're already in M "domain" and we call sendExpiredPasswordEmail method, mapping it to NotificationSent ADT on success and recovering to ErrorWhileSending ADT on an error. In the end of the method, to get out of OptionT transformer, we call getOrElse method and return UserNotFound ADT if we were unable to get user data out of our repositories (any of two calls).

sendExpiredPasswordEmail method implementation is supposed to make email sender microservice call, here we pretend it does it successfully and use pure monadic operation to wrap Unit value into monadic context.

Observations

As you can see, we are very abstract about what monads we're using in the service implementation. This allows to decouple application layers and change our monads without touching the service code. If one day you decide to go for Monix' Task instead of Scala Future, you won't need to change the service implementation at all. Just provide the required Monad/MonadError instances (and yes - Monix does have them!) and all' keep working.

Applications to unit testing

Last thing to touch upon is how this approach affects unit testing. Instead of dealing with asynchronous results in unit tests (even though ScalaTest has a decent support for them), you can instantiate the service by substituting cats.Eval as your M and DbEffect monads.

Notice that cats have MonadError instance for Eval in the master branch at the moment of writing, but it's not released yet.

Using ScalaTest and Mockito, you can do something like this:

I use FunctionK.id[Eval] that's just a "higher order identity"(i.e. like Scala identity but applied to M[_]) to "evaluate" DbEffect values. So, same Eval monad for both service and repository layers in tests.

It's worth saying that this test reveals another problem with our implementation: invoking side-effective methods like LocalDateTime.now() in the service code. Instead, some date/time provider component should be passed to the service constructor to make the code more purely functional and consequently testable.

Wrapping blocking APIs

To wrap blocking APIs - those that return immediate values and do IO during their computation - one can define a transformation from Eval to whatever monad you use, e.g. Future:

And then use it to transform blocking calls into Ms:

Conclusion

I hope you came away with something applicable to the real-world from the presented monadic approach to abstract and structure (are these two synonyms in a sense?) the code. You can imagine having more "effects" like DbEffect in your code and that's completely fine if you have required natural transformations machinery for them in place.

The thing I like the most about this approach is that it is simple. If you looked at free monads or Eff, you know what I mean. It doesn't force you to rewrite your code from scratch and can be adapted iteratively: first you apply it to the commons code, like DatabaseSupport in the above examples, then to app's bottom layers, then move higher to services, and so on.

Next logical step to investigate is how to abstract streaming - very important technique nowadays. It would be cumbersome to have akka-http, or Monix, or Reactive Streams classes along with M and DbEffect. But that's the topic of another article.

Monad is a trait with the following operations:

Having such trait implementation for a type M makes it a monad instance.

Sometimes, pure is called return or point, whereas bind might instead be named flatMap (preferred in Scala) or >>= (Haskell's way). The method names are not so important, what's important is their signatures and what they do.

pure and bind implementations must obey some easily google-able mathematical laws omitted here to make things less boring.

M type parameter

Type parameter of higher kind M requires a monad instance to be parameterized by a type with a single type parameter, e.g. List, Future, Option, ({ type λ[A] = Either[Throwable, A] })#λ, or any type similar to:

pure operation

In a nutshell, pure gives a way to wrap a value of type A into type M (sometimes called monadic context). E.g.: List(a), Future { a }, Some(a), Right[Throwable, A](a). Put clearly: pure is A => M[A], and that's it.

bind operation

The essence of this operation is to allow flattening of M[M[A]] to M[A]. This might be not clear immediately, given the above Monad trait definition, but that's easy to understand: having a value ma: M[A] and a function f: A => M[B], to apply this function to ma, one needs a way to apply f to an a: A that's inside the context M, and the result will be of type M[M[B]]. So, bind allows to apply a function to monadic values and to flatten nested monadic contexts, while pure - to add new ones. For example, List(a).flatMap(a => List(a)), Future { a } flatMap { a => Future { a } }, Some(a).flatMap(a => Some(a)). These are artificial examples, but it's easy to imagine some nested conditional logic in the supplied f function.

Different perspective on Monad trait

It turns out, Monad trait might be formulated in a different way that is closer to the flattening intuition above:

Here, join (or flatten) does exactly that. Also, we've got a map operation which is needed to fill the 'gap' from loosing the full power of bind. Those two definitions are equivalent: bind can be implemented in terms of join and map:

or map and join in terms of bind and pure:

Different perspective on map and bind

There is also an alternative view on map and bind methods signatures. Let's call 'em lift and liftM accordingly:

Notice how similar they are. Both take a function, either ordinary A => B in case of lift or a monadic one A => M[B] (sometimes called Kleisli arrow) in case of liftM, and lift it to a function that works on monadic values. And these definitions are easily expressible in terms of original map/bind:

So, from this perspective, a monad can also be defined using pure and liftM or pure, join, and lift.

The essence of monads

From the dry definition above it might be not clear how monads can be useful, why they exist in the first place. So, to put it simply, monads give a way to chain computations (of type A => M[B]) while abstracting some details of how to exactly do that (extract values out of monadic context and flatten nested contexts) into the monad itself. For example, for Listss it performs flattening, for Options it stops on first None, for Futures - allows to make another asynchronous computation from within asynchronous computation, for some more esoteric state monad - to emulate mutable state in pure functional setting. This is what actually happens in Scala when using for comprehension syntax (though it doesn't force to implement any kind of trait, like the Monad above: just follow the method signatures, and all' be fine):

And it's just a syntax sugar for:

Unlike the Monad trait definition above, monadic operations here are written in object-oriented style, but that doesn't change their meaning.

Some monad instances

A monad's essence is in how chaining (bind operation) is implemented. So, here some monads instances to make it all clearer:

Notice how partial type application is used for Either which has two type parameters while monad requires just one. In general, for types having more than one type parameter, one needs to fix all the parameters but one to make an, in fact, a new type that could potentially be a monad instance.

Conclusion

Don't expect to understand the nature of monads just in one seat if you are new to them: you definitely need to play with various monads, see how they behave, implement some of them on your own. But the underlying idea is simple and is just a common programming practice: separate concerns and, as a consequence, prevent code duplication, by moving common stuff into a single easily maintainable and comprehensible place.