Hi, I'm Felipe. I'm a software developer who builds scalable and performant data systems. I've worked primarily with Kafka, Postgres, Flink, Go, Java, and Kubernetes. I'm low-latency, fault-tolerant, durably logged, and well-hydrated.
If you want to reach me, try prowling around some Toronto-area softball diamonds. I'll turn up sooner or later. Alternatively, shoot an email to "inquiries" at the domain of this website.
Please enjoy some blog posts! 👇
Let's talk a bit about event-driven microservices. Event streaming is a really powerful substrate for microservices, so it sees a lot of use, but it comes with caveats that are worth exploring before you fully commit to the event streaming model.
An event-driven microservice subscribes to messages several topics in a distributed log (probably Kafka) and publishes to several other topics. It processes data asynchronously and batches its I/O. A Kafka broker gives you service discovery, load balancing, horizontal scaling, fault tolerance, guaranteed delivery, and best-in-class throughput; it's a pretty good deal.
All these benefits come bundled together, however, which makes event streaming contagious, in the way that async functions are contagious. In order for an event-driven service to processes messages at speed, it can't perform a separate set of network round-trips for each one. It shouldn't be calling APIs or querying a database; rather, it should be ingesting streamed versions of those things. Any synchronous operations will block the pipeline and throw the service's performance off a cliff. The service's output also has to be asynchronous, which propagates all these constraints to downstream services, too.
A couple notes on software testing:
1. A test suite should be a legible description of the system under test.
Each test should clearly describe one piece of behaviour with as few independent factors and implementation details as possible. Of course, factors and details like those are the hallmarks of interesting corner cases, so this level of clarity can be difficult to implement in practice without sacrificing rigour. Still, it's astonishing how often I come across tests that say nothing explicit about the system they're meant to describe, beyond "when you run it with these inputs, you'll get those outputs."
This is the worst type of test suite: a baroque aggregation of arbitrary scenarios. Suites like this are millstones around your neck. A change in any part of the system can break hundreds of tests, and it's often unclear whether a broken test indicates a bug or simply that the system has changed.
A perfect test suite uses randomized property tests. Each test generates a thousand random inputs and validates that a particular property holds true under any possible circumstance. It's highly decoupled from implementation details and one step away from a formal proof of correctness. Not every domain is amenable to property testing, but it reaches corner cases without sacrificing clarity, so it's worth the trouble when possible.
On the other hand, I have a lot of respect for snapshot testing, even though it's about as far from the property-testing ideal as you can get. Snapshot tests (a.k.a. golden files) embrace the need to update test expectations all the time by providing automated tools for reviewing and updating changes to expectations. Perhaps this sort of test struggles to legibly describe a system, but it's great at describing the impact of a change to the system, and I like that. It feels like a pragmatic compromise for situations where you can't write perfect tests.