2022
An alternative approach to custom partition assignment strategy for Kafka consumers with Karafka
Recently, in BookingSync, we were performing a migration from Karafka 1.4 to 2.0, which we use for communication with Kafka. One of the great features available in version 1.4 was a custom partition assignment strategy for consumers. It was particularly useful for us as we’ve had some topics that had a way higher throughput than the other ones, so just a round-robin strategy with even distribution of topics for consumers was not a suitable choice as we needed dedicated consumers for these specific topics/partitions. Unfortunately, custom partition assignment strategy for consumers is no longer available in Karafka 2.0. Nevertheless, we managed to perform the migration and replaced the custom partition assignment strategy with a more straightforward and robust solution.
Read on →The inherent unreliability of after_commit callbacks and most service objects' implementation
Service objects and/or after_commit callbacks are ubiquitous in most real-world Rails applications. Whether it’s a good idea or not (ActiveRecord callbacks - I’m looking at you) is a different story, but one thing that is notoriously overlooked in the application design is reliability. And yes, the service objects are equally bad as after_commit callbacks in that regard.
2021
The Story of a Critical Issue With Kafka
Recently, I’ve had an unfortunate opportunity to deal with a very unexpected issue with Kafka that had quite terrible consequences. The exact origin of the issue is yet to be discovered; nevertheless, the process leading to the final solution to the problem and the fix itself were interesting enough that I decided to write a quick blog post about it as it might be potentially valuable to someone who also encounters a similar problem.
Read on →Multiple databases in a single query in your Rails apps - Postgres Foreign Data Wrappers to the rescue
Imagine that you are building a separate application for your e-commerce system dedicated to business intelligence. In other words, you want to calculate some stats for the orders. So you are going to create some new model, like OrderStat(s), and have a separate Postgres database for a new app. Sounds trivial so far.
Read on →Kafka for Rubyists Mini Course: Advanced Karafka
This is the fourth and also the last part of the Kafka For Rubyists mini course. If you are interested in other parts of the course, please check them out here.
Read on →Kafka for Rubyists Mini Course: Advanced Kafka
This is the third part of the Kafka For Rubyists mini course. If you are interested in other parts of the course, please check them out here.
Read on →2020
Kafka for Rubyists Mini Course: Consuming and Performance
This is the second part of the Kafka For Rubyists mini course. If you are interested in other parts of the course, please check them out here.
Read on →Kafka for Rubyists Mini Course: Fundamentals
This is the first part of the Kafka For Rubyists mini course. If you are interested in other parts of the course, please check them out here.
Read on →Kafka for Rubyists Mini Course
I’ve been recently planning to record a couple of tutorials for my team in BookingSync, focusing mostly on monitoring and some other more advanced aspects of Kafka and Karafka. But if I was already recording something, then why not make it available publicly so that more developers could benefit from it? So, in the end, I decided to make a bit more effort and recorded this mini-course, which will be available for everyone, for free ;).
Read on →Tracking All Paper Trail Version From A Single Request With Correlation UUIDs
If you’ve ever had a need to implement an audit log to track all the changes that get persisted for all or at least some models in your application, there is a good chance that you’ve encountered PaperTrail gem that makes it trivial to track all the changes - it might be as easy as adding has_paper_trail to the desired models.
Race Conditions on Rails
Imagine that you are implementing a payment processing system for the e-commerce system and discover that multiple customers were charged twice for exactly the same order. Ouch… Sounds like a real nightmare to deal with! And the next day, you see that something is not right with the credits system in which users were able to pay using special credits instead of using credit card - somehow instead of getting charged $100 in total for two orders of $25 and $75, they were charged just 25$! And to make it even more epic, it turned out that the uniqueness validation you added didn’t work at all, and now you have three users with the same email address!
Read on →A Secret Weapon For Your Rails Apps - RPC with RabbitMQ and Hutch
Communication between two or more applications is often everyday stuff, and it might seem that there is not too much to add there as this subject has been covered pretty well in the last years. Thanks to that, multiple patterns and standards have emerged. You no longer need to think about how the response format should look like for your REST API (go with JSONAPI and stick to the conventions) or figure out the authentication/authorization protocol (go with OAuth and the security headaches won’t bother you).
Read on →2018
The Problems With Validating ActiveRecord Models And Why State Validation Is a Bad Idea
In the typical Rails application, you can find the most of the validations in the ActiveRecord models, which is nothing surprising - ActiveRecord models are used for multiple things. Whether it is a good thing, or a bad thing (in most cases it’s the latter) deserves a separate book or at least blog post-series as it’s not a simple problem, there is one specific thing that can cause a lot of issues that are difficult to solve and go beyond design decisions and ease of maintenance of the application, something that impacts the behavior of the model - the validations.
Read on →Indexes on Rails: How to make the most of your Postgres database
Optimizing database queries is arguably one of the fastest ways to improve the performance of the Rails applications. There are multiple ways how you can approach it, depending on the kind of a problem. N+1 queries seem to be a pretty common issue, which is, fortunately, easy to address. However, sometimes you have some relatively simple-looking queries that seem to take way longer than they should be, indicating that they might require some optimization. The best way to improve such queries is adding a proper index.
Read on →2017
Traps on Rails - Overriding boolean methods in models
One very useful feature of ActiveRecord is automatically defining attribute readers and writers for all the columns for given tables. For the ones with boolean type, however, there is one more addition - defining an alias of the method with a question mark. Sometimes it might be useful to override this method and add some extra requirements for a given condition. However, this might not be such a good idea.
Read on →5 years of professional Ruby and Rails development - My Reflections
As hard as it is for me to believe, I already have over 5 years of professional experience in Ruby and Rails. Throughout all these years my attitude towards Rails has been fluctuating between going from blind love to harsh critic (ActiveRecord, I’m looking at you) ending with a bit more balanced but certainly a positive approach. Such time is long enough to have a meaningful opinion about the overall experience using any framework, so here are few points about Rails that I would particularly like to focus on in my reflections.
Read on →2016
Decoding Rails Magic: How Does ActiveJob work?
Executing background jobs is quite a common feature in many of the web applications. Switching between different background processing frameworks used to be quite painful as most of them had different API for enqueuing jobs, enqueuing mailers and scheduling jobs. One of the great addition in Rails 4.2 was a solution to this problem: ActiveJob, which provides extra layer on top of background jobs framework and unifies the API regardless of the queue adapter you use. But how exactly does it work? What are the requirements for adding new queue adapters? What kind of API does ActiveJob provide? Let’s dive deep into the codebase and answer these and some other questions.
Read on →Little-known but useful Rails features: ActiveRecord.extending
Every now and then I discover some features in Rails that are not that (arguably) commonly used, but there are some use cases when they turn out to be super useful and the best tool for the job. One of them would definitely be a nice addition to ActiveRecord::QueryMethods - extending method. Let’s see how it could be used in the Rails apps.
Decoding Rails Magic: How Does Calling Class Methods On Mailers Work
Have you ever wondered how it is possible that calling class methods on mailers work in Rails, even though you only define some instance methods in those classes? It seems like it’s quite common question, especially when you see the mailers in action for the first time. Apparently, there is some Ruby "magic" involved here, so let’s try to decode it and check what happens under the hood.
Read on →Scaling Up Rails Applications With PostgreSQL Table Partitioning - Part 3
After publishing recent blog posts about table partitioning - its SQL basics part and how to use in in Rails application I was asked quite a few times what is the real performance gain when using table partitioning. This is a great question, so let's answer it by performing some benchmarks.
Read on →Scaling Up Rails Applications With PostgreSQL Table Partitioning - Part 2
In the previous blog post we learned some basics about table partitioning: how it works and what kind of problems it solves. So far we've been discussing mostly basic concepts with raw SQL examples. But the essential question in our case would be: how to make it work inside Rails application then? Let's see what we can do about it.
Read on →Scaling Up Rails Applications With PostgreSQL Table Partitioning - Part 1
You've probably heard many times that the database is the bottleneck of many web applications. This isn't necessarily true. Often it happens that some heavy queries can be substiantially optimized, making them really efficient and fast. As the time passes by, however, the data can remarkably grow in size, especially in several years time which can indeed make the database a bottleneck - the tables are huge and don't fit into memory any longer, the indexes are even bigger making queries much slower and you can't really optimize further any query. In many cases deleting old records that are no longer used is not an option - they still can have some value, e.g. for data analysis or statystical purposes. Fortunately, it's not a dead end. You can make your database performance great again with a new secret weapon: table partitioning.
Read on →Security On Rails: Hacking Sessions With Insecure Secret Key Base
I was recently asked what is secret key base used for in Rails applications and why not secure value of it (or even worse - the public one!) creates a security issue. That was a really good question, I remember how it was a serious threat years ago, especially before introducing secrets.yml in Rails 4.1 - at that time by default secret_token initializer was generated and the secret key was directly stored there. The result was that in many open source projects secret key was publicly available creating a great security risk. Let's take a look how exposed secret key base could be exploited.
How To Handle Non-CRUD Logic In Your API
If you happen to develop API for non-trivial app with complex business logic beyond CRUD directly mapped to the database tables (i.e. typical Active Record pattern) you were probably wondering many times how to handle these cases and what's the best way to do it. There are quite a few solutions to this problem: you could add another endpoint for handling given use case, add non-RESTful action to already existing endpoint or you can add a magic param to the payload that would force non-standard scenario in the API. Let's take a closer look at the these solutions and discuss some advantages and disadvantages of each of them.
Read on →Refactoring Tips: Trade Conditionals For Type Delegation
Having some kind of type attribute in your models is a pretty common thing, especially in applications with more complex domain. How do you handle such cases? Is it with multiple conditionals / case statements in every method dealing with the type attribute? If you've been struggling with organizing and maintaing code for such use cases then say hello to your new friend: type delegation.
2015
Embarking on Programming Journey
Recently I've been repeatedly asked how to get started with programming, especially Ruby and Rails. To keep things DRY and make sure I always include all great resources for learning, I decided to write this blog post. Most of people who asked me had never programmed before or had done some coding before in other languages, but are not (yet) proficient developers, so if you are a total beginner or just started learning Ruby / Rails and you are not sure what the next step is then you came to the right place.
Read on →2014
You don't (necessarily) need gem for that feature
You are working currently on that awesome app and just started thinking about implementing new feature, let's call it feature X. What's the first thing you do? Rolling your own solution or... maybe checking if there's a magical gem that can help you solve that problem? Ok, it turns out there's already a gem Y that does what you expect. Also, it does tons of other things and is really complex. After some time your app breaks, something is definitively not working and it seems that gem Y is responsible for that. So you read all the issues on Github, pull requests and even read the source code and finally find a little bug. You managed to do some monkeypatching first and then send pull request for a small fix and solved a problem, which took you a few hours. Looks like a problem is solved. And then, you try to update Rails to2 the current version. Seems like there's a dependency problem - gem Y depends on previous version of Rails...
Read on →Test Driven Rails - Part 2
Last time, in part 1, I was giving some advice about testing - why to test at all, which tests are valuable and which are not, when to write acceptance tests and in what cases aim for the maximum code coverage. It brought about some serious discussion about testing ideas and if you haven't read it yet, you should probably check (it) it out. Giving some general point of view about such broad topic like Test Driven Development / Behavior Driven Development is definetely not enough so I will try to apply these techniques by implementing a concrete feature. I wanted to choose some popular usecase so that most developers will have an opinion how they would approach it. In most applications you will probably need:
Read on →Test Driven Rails – Part 1
Testing is still one of the hottest topics when it comes to developing maintainable and business-critical applications. Ruby on Rails community embraces the importance of writings tests, yet there are so little resources about the Test-Driven Development or Behavior-Driven Development in Rails applications from a wider perspective. How should we test our application? Which tests are valuable and which don’t provide any value? What scenarios should be included in acceptance tests?
Read on →2013
Structuring Rails applications
There’ve been a lot of discussions recently about applying Object Oriented Programming in Rails applications, how ActiveRecord callbacks make testing painful and how Rails makes it hard to do OOP the right way. Is it really true? Rails makes everything easy - you can easily write terrible code, which will be a maintenance nightmare, but is also easy to apply good practices, especially with available gems. What is the good way then to extract logic in Rails applications and the best place to put it?
Read on →CentOS 6.4 server setup with Ruby On Rails, Nginx and PostgreSQL
Server setup with the entire environment for Rails applications can be quite tricky, especially when you do it for the first time. Here is step by step guide how to setup CentOS 6.4 server with a basic environment for deploying Rails applications. I encourage you to choose CentOS Linux - it is a reliable distro (well, based on Red Hat Enterprise Linux), easy to handle and doesn't require advanced Unix knowledge like Gentoo (especially while updating system related stuff).
Read on →