Kubernetes operators for resource management

November 25, 2019

In this post we discuss Kubernetes and its emergence as a tool of choice for infrastructure orchestration. We specifically focus on operators, a pattern for extending the Kubernetes to manage arbitrary custom resources, and discuss how they enable infrastructure management of complex hybrid environments. We look at how to implement operators in a robust, generic way for any resource that itself has a management API consisting of CRUD (create, read, update and delete) operations, and introduce a library created to enable this.

Read More

Redux patterns and principles

November 10, 2018

In this blog we look at how to use Redux effectively in a React App. The unidirectional circular data flow in the Redux pattern is helpful in simplifying the reasoning around a UI App. However the benefits are frequently discarded by not using Redux effectively, and result a compromised user experience or performance and/or the need for complex workarounds. We discuss how to use the Redux effectively to avoid common pitfalls, and delivery a responsive, flicker-free user experience with minimal spinners.

Read More

React, redux and higher order components

June 25, 2018

In this post I discuss my journey as a backend developer getting to grips with front end development using the React framework, using Redux for state management, and higher order components. The outcome is some principles for doing front end development that lead to a good separation of concerns, and make React easier write and understand and more enjoyable to work with.

Read More

Free monad event sourcing - playback and wrap-up

March 30, 2017

In this post, following on from Free monad event sourcing interpreters, we complete the series on event sourcing with free monads by discussing playback. We demonstrate by example, using the doobie data access library and the FS2 streaming library to playback an event log of operations recorded and serialised as Json in a SQL database. There isn’t that much to say about it, but we’ve included this post for the sake of completeness. Recording is only half of the story, in this case the more difficult half, but it’s still worth discussing playback. If you are not familiar with FS2, it will also serve as a good motivating example for FS2 and its usage.

Read More

Interpreters for event sourcing free monads

December 18, 2016

This post follows from the post on Free monads and event sourcing architecture. Here we develop a generic free monad based event sourcing framework that can capture events from any free algebra and has correct effect processing semantics. We describe a concrete example instance that uses the doobie library to persist the events to a SQL database. Our implementation is typesafe, performant and involves very little boilerplate code.

Read More

Free monads and event sourcing architecture

December 8, 2016

In this post we look at free monads, a widely applicable functional programming technique that is rapidly gaining traction amongst Scala developers, and show that they are ideally suited to implementing an event sourcing data architecture. We discuss some of the restrictions on a free monad API that are required for event sourcing to work optimally, and review some related best practices.

Read More

Bulletproof functional validation using Scalaz

April 29, 2016

As a backend developer, you are going to want to expose your efforts through an API such as a REST API. Your API may be publicly exposed, which means you can’t even trust that the caller has good intentions. Good validation of the incoming parameters is essential. However it is not that easy to achieve with conventional imperative code, at least not in a way that is not very messy and doesn’t obscure the intent of the code itself. We investigate a functional paradigm for validation, step-by-step, using the Scalaz ValidationNel applicative functor. As an example, we apply it to a Play REST endpoint.

Read More

Spark cluster execution with Akka

April 17, 2016

This post follows from the post on Creating composable data pipelines. Wrapping our Spark processes in a reader monad provides a great deal of flexibility, and this includes how we conduct execution of a a Spark program across a cluster. Here we look at how this can be done and take advantage of Akka to coordinate low latency Spark jobs, providing resilience and supervision.

Read More

Composing Spark data pipelines

April 5, 2016

We investigate a pattern of functional programming that we can apply to Apache Spark to help create clean and elegant code for data transformations, and avoiding state management headaches. We do so by separating the definition of data transformation processes from their execution.

Read More

Scala variance and blood groups

April 2, 2016

One of the topics that is often glossed over in the first pass of learning Scala is the topic of variance. Variance is the rules that determine how the inheritance relationship between two types is transferred to objects parameterised by those types. We look at the rules regarding blood donation for the different blood groups as an analogy for variance.

Read More