Structuring real-world code using monadic abstractions
In this article I present a simple relatively boilerplate-free approach to structuring Scala code using monadic abstractions. I use cats but it can be done using any other library, like Scalaz.
Introduction
Presented approach is nothing new but just well-known patterns used in a cohesive way. In a sense it tries to compete with free monads or more modern Eff monad without going crazy too much. Those techniques are much more complicated and consequently more powerful, but they require a lot of boilerplate (even when using some libraries born to reduce it, like Freek or Liberator for free monads) and refactoring.
On the other hand, monadic abstractions approach in a sense is a less radical step forward to pure functional programming. It's much simpler and can be more easily applied to existing code that uses scala.concurrent.Future
s directly, without re-writing it from scratch.
Step by step guide by example
So, let's pretend we're writing a microservice that uses Scala futures for asynchronicity, Slick 3 for relational database access, and something like akka-http
to make REST calls to other microservices. First thing we do is...
Abstracting interfaces
Let's say we have a service layer and a repository layer. The first trick is to abstract away from concrete "monad" used as a methods result types wrapper, e.g. Future
or DBIO
, in interfaces:
Currently dominant style is to use scala.concurrent.Future
as M
in service trait and scala.concurrent.Future
or DBIO
as M
in repository traits. There are two downsides with such approach:
- using futures in repositories would force to evaluate your
DBIO
actions too early (before leaving repository layer) effectively disallowing a transaction to span across multiple repository calls; - using
DBIO
asM
in repository interfaces would leak repository layer implementation details to upper layers, adding `Slick* stuff to services classpath.
So, by abstracting it out, we want to be able to run cross-repository calls transactions and fix the implementation details leak. And we don't really want to have Slick classes in service layer auto-complete!
We use higher order type parameter M
, saying that we return a "container" of something, but we're not explicit about its type. It can be Scala Future
, Monix Task
, Slick's DBIO
, cats' Eval
, or even Function0
!
Implementing repositories
Next we implement repositories by using DBIO
to substitute abstract M
s:
Implementation details aren't relevant here and therefore omitted for brevity. More interesting is how we implement the service.
Implementing services
Notice that service implementation should not depend on repository implementation but only on interfaces. We can say that each layer (e.g. repository, service, etc) has at least two parts, and consequently build artifacts: the api
part and one or more impl
parts. So each layer above only depends on api
artifact of a layer below. Explicitly:
- service
api
doesn't depend on repository artifacts, cause it would make them available to layers above - something we what to disallow; - service
impl
depends on repositoryapi
(notimpl
!) so that repository implementation could be swapped without changing service code (ideally); - there is an app layer on the very top, where all the wiring takes place, it depends on both service and repository layers
impl
artifacts (andapi
s transitively) and provides a concrete repository layer implementation to the service layer.
But how the service code looks then? Here comes the most interesting part:
Let's iterate it one thing at a time.
Still generic M[_]
Notice that service code is still generic in the sense that it's detached from a concrete monad implementation, we're still using M[_]
. And it's a good sign!
DbEffect
type parameter
Additionally, our service implementation declares a type parameter to abstract over a concrete "monad" (or "effect", or "container") of the repository layer. We could call it just D
but I like the DbEffect
name as more self-explanatory here. We also parameterize our repositories dependencies with DbEffect
effectively saying that we depend on repositories that produce values within a generic container called DbEffect
- and that's all we know about them so far, but... stay tuned!
And yes - we're using constructor parameters for dependency injection.
evalDb
argument
We've also got a fancy looking evalDb: DbEffect ~> M
constructor parameter. It can be written more explicitly like this:
In cats' code it's just a type alias for FunctionK
trait:
This thing is called a natural transformation and all it can do is to convert a value of one generic type F[_]
to another generic type G[_]
. And that's exactly what we need to "run" our underlying DbEffect
s: whatever monad service code uses for its results, we want to transform repository effects into it. For DBIO
actions, when we pass evalDb
implementation to our service in the app layer, we are going to execute them transactionally and get back futures (if we decide to use them as service's M
implementation).
There is one aspect of DbEffect
and M
that gives compile-time guarantee of not having long-running transactions (what is probably a good idea). If you decide to do something like making REST call when transaction is running, you'll get a result of type DbEffect[M[DbEffect[T]]]
. And you won't be able to escape out of it. Strictly speaking, there is a typeclass called Traverse
that brings the ability to "traverse" one context with another, i.e. go from F[G[T]]
to G[F[T]]
, and since we can't have it for DbEffect
and M
(at least when DbEffect
is DBIO
and M
is some asynchronous boundary like Future
), - we cannot traverse it!
Now to the...
Implicit parameters
To recap, we require the following implicits when creating the service:
In case you know what is a Monad
(e.g. from my previous article) but never saw the MonadError
, you can think about it like a Monad
with built-in error handling. You can use standard Scala Future
methods intuition, like Future.failed
, Future.recover
, or Future.recoverWith
. They allow to create a failed value and to recover from a failure. Same for the MonadError
typeclass but in much more abstract purely functional setting.
Why we require MonadError
for M
but Monad
for DbEffect
? It's just a rule of least power: I'm not going to use error handling for DbEffect
values in the method implementation so I just stick with the Monad
instance for DbEffect
but it could definitely be MonadError[DbEffect, Throwable]
as well - no trickery here!
So, by having this implicit parameters we declare that:
M
is a monad (with error handling) and consequently we canmap
/flatMap
/recover
values of typeM
- same way as one would useFuture
s in for comprehension;DbEffect
is a monad, so we can chainDbEffect
values as well, before transforming them toM
usingevalDb
natural transformation (you can think about it like about transaction).
Wiring it all together
Before actually implementing our service method, let's see how the components can be wired in the top app level. Here are some common Slick helpers to be used:
I won't go into much detail here, this is something similar to what many teams come up with when using Slick. The most important thing is to be able to define ~>
instance that can evaluate DBIO
values. Notice that we're using same "abstracting interfaces" approach for shared common code as for the components above.
Now, wiring can look similar to this:
For simplicity I'm not going to define web layer, but it might use akka-http
and require services dependencies as securityService: SecurityService[Future]
-like constructor parameters (or even securityService: SecurityService[M]
if you decide to generify your controllers/resources/whatever-you-call-it as well, though there's less benefit from doing that in comparison to service code).
But what about monad instances?
An astute reader might notice that something is missing when creating the SecurityServiceGenericImpl
above - namely the implicit Monad
and MonadError
instances:
MonadError[Future]
;Monad[DBIO]
.
The first is defined in cats and can be put in scope by adding import cats.implicits._
import.
As for the second one, there is a project on Github called slick-cats that defines MonadError
and some other instances for DBIO
. They can be brought in scope by introducing a dependency on slick-cats
and adding import com.rms.miu.slickcats.DBIOInstances._
import statement.
Finally we're ready to implement our example service method.
Service method implementation
Method logic is following:
- get user's password last changed date;
- independently, get user's password validity period setting;
- if password is expired, call some other microservice to send a notification email to the user;
- if an error occurs during the notification sending, recover error to a specific
SendNotificationResult
ADT value.
Even in this simplified example we have multiple things to consider, namely:
- we want to make first two calls independently on each other, i.e. no
flatMap
ing; - execute both calls in a single transaction (it might be more important in more real-world scenario than here);
- return a
ErrorWhileSending
ADT value if an error occurs during notification sending (error handling); - though we imply that repository calls cannot fail in our example, we might want to provide
MonadError
instance forDbEffect
and handle DB errors gracefully (not covered here).
Here is the implementation split to several methods:
Steps 1 and 2 are implemented in isPasswordExpired
method. Notice that it doesn't leave DbEffect
monad yet, effectively causing both repository calls to execute in a single transaction, when run later by evalDb
. This method uses applicative builder syntax to leverage the fact that Monad
is also an Applicative
functor. In a nutshell, Applicative
is a less powerful abstraction that allows to, in this case, apply a function (A, B) => C
to F[A]
and F[B]
to get F[C]
. First, we use the fact that DbEffect
is an Applicative
, then the fact that Option
is also an Applicative
(typeclass instance for Applicative[Option]
provided by cats).
The main difference between Monad
and Applicative
is that the former allows for computations chaining making one computation to depend on another, while the latter doesn't allow chaining and considers computations simultaneously, even with ability to run them in parallel when looking at how Applicative[Future]
instance works. See cats documentation for more information.
In sendNotificationIfPasswordExpired
method we evalDb
the isPasswordExpired
method result and, since it has DbEffect[Option[Boolean]]
type, we can apply OptionT
monad transformer to it. Monad transformers (hence the T
in OptionT
) allow to stack one monad on top of another and work with them without nesting. Again, google for cats monads transformers if needed. In semiflatMap
method lambda we're already in M
"domain" and we call sendExpiredPasswordEmail
method, mapping it to NotificationSent
ADT on success and recovering to ErrorWhileSending
ADT on an error. In the end of the method, to get out of OptionT
transformer, we call getOrElse
method and return UserNotFound
ADT if we were unable to get user data out of our repositories (any of two calls).
sendExpiredPasswordEmail
method implementation is supposed to make email sender microservice call, here we pretend it does it successfully and use pure
monadic operation to wrap Unit
value into monadic context.
Observations
As you can see, we are very abstract about what monads we're using in the service implementation. This allows to decouple application layers and change our monads without touching the service code. If one day you decide to go for Monix' Task
instead of Scala Future
, you won't need to change the service implementation at all. Just provide the required Monad
/MonadError
instances (and yes - Monix does have them!) and all' keep working.
Applications to unit testing
Last thing to touch upon is how this approach affects unit testing. Instead of dealing with asynchronous results in unit tests (even though ScalaTest has a decent support for them), you can instantiate the service by substituting cats.Eval
as your M
and DbEffect
monads.
Notice that cats have MonadError
instance for Eval
in the master branch at the moment of writing, but it's not released yet.
Using ScalaTest and Mockito, you can do something like this:
I use FunctionK.id[Eval]
that's just a "higher order identity"(i.e. like Scala identity
but applied to M[_]
) to "evaluate" DbEffect
values. So, same Eval
monad for both service and repository layers in tests.
It's worth saying that this test reveals another problem with our implementation: invoking side-effective methods like LocalDateTime.now()
in the service code. Instead, some date/time provider component should be passed to the service constructor to make the code more purely functional and consequently testable.
Wrapping blocking APIs
To wrap blocking APIs - those that return immediate values and do IO during their computation - one can define a transformation from Eval
to whatever monad you use, e.g. Future
:
And then use it to transform blocking calls into M
s:
Conclusion
I hope you came away with something applicable to the real-world from the presented monadic approach to abstract and structure (are these two synonyms in a sense?) the code. You can imagine having more "effects" like DbEffect
in your code and that's completely fine if you have required natural transformations machinery for them in place.
The thing I like the most about this approach is that it is simple. If you looked at free monads or Eff, you know what I mean. It doesn't force you to rewrite your code from scratch and can be adapted iteratively: first you apply it to the commons code, like DatabaseSupport
in the above examples, then to app's bottom layers, then move higher to services, and so on.
Next logical step to investigate is how to abstract streaming - very important technique nowadays. It would be cumbersome to have akka-http, or Monix, or Reactive Streams classes along with M
and DbEffect
. But that's the topic of another article.
You might also be interested in these articles...