In this post we discuss Kubernetes and its emergence as a tool of choice for infrastructure orchestration. We specifically focus on operators, a pattern for extending the Kubernetes to manage arbitrary custom resources, and discuss how they enable infrastructure management of complex hybrid environments. We look at how to implement operators in a robust, generic way for any resource that itself has a management API consisting of CRUD (create, read, update and delete) operations, and introduce a library created to enable this.

In this blog we look at how to use Redux effectively in a React App. The unidirectional circular data flow in the Redux pattern is helpful in simplifying the reasoning around a UI App. However the benefits are frequently discarded by not using Redux effectively, and result a compromised user experience or performance and/or the need for complex workarounds. We discuss how to use the Redux effectively to avoid common pitfalls, and delivery a responsive, flicker-free user experience with minimal spinners.

In this post I discuss my journey as a backend developer getting to grips with front end development using the React framework, using Redux for state management, and higher order components. The outcome is some principles for doing front end development that lead to a good separation of concerns, and make React easier write and understand and more enjoyable to work with.

In this post, following on from Free monad event sourcing interpreters, we complete the series on event sourcing with free monads by discussing playback. We demonstrate by example, using the doobie data access library and the FS2 streaming library to playback an event log of operations recorded and serialised as Json in a SQL database. There isn’t that much to say about it, but we’ve included this post for the sake of completeness. Recording is only half of the story, in this case the more difficult half, but it’s still worth discussing playback. If you are not familiar with FS2, it will also serve as a good motivating example for FS2 and its usage.

This post follows from the post on Free monads and event sourcing architecture. Here we develop a generic free monad based event sourcing framework that can capture events from any free algebra and has correct effect processing semantics. We describe a concrete example instance that uses the doobie library to persist the events to a SQL database. Our implementation is typesafe, performant and involves very little boilerplate code.

In this post we look at free monads, a widely applicable functional programming technique that is rapidly gaining traction amongst Scala developers, and show that they are ideally suited to implementing an event sourcing data architecture. We discuss some of the restrictions on a free monad API that are required for event sourcing to work optimally, and review some related best practices.

As a backend developer, you are going to want to expose your efforts through an API such as a REST API. Your API may be publicly exposed, which means you can’t even trust that the caller has good intentions. Good validation of the incoming parameters is essential. However it is not that easy to achieve with conventional imperative code, at least not in a way that is not very messy and doesn’t obscure the intent of the code itself. We investigate a functional paradigm for validation, step-by-step, using the Scalaz ValidationNel applicative functor. As an example, we apply it to a Play REST endpoint.

This post follows from the post on Creating composable data pipelines. Wrapping our Spark processes in a reader monad provides a great deal of flexibility, and this includes how we conduct execution of a a Spark program across a cluster. Here we look at how this can be done and take advantage of Akka to coordinate low latency Spark jobs, providing resilience and supervision.

We investigate a pattern of functional programming that we can apply to Apache Spark to help create clean and elegant code for data transformations, and avoiding state management headaches. We do so by separating the definition of data transformation processes from their execution.

One of the topics that is often glossed over in the first pass of learning Scala is the topic of variance. Variance is the rules that determine how the inheritance relationship between two types is transferred to objects parameterised by those types. We look at the rules regarding blood donation for the different blood groups as an analogy for variance.