Software evolves - An experience guided by results

Mon, 15 Nov 2021 00:00:00 +0000

Systems, languages, and applications come and go as people progress in their own careers, problems that had not so obvious solutions in the beginning, become easier to tackle as application and team evolves. In this post, I would like to share a bit about my experience on migrating and scaling a system that I worked with, by observing and trying to understand the problem before thinking about changing the technology completely.

There are multiple factors that influences the team to use a language, it can go through speed constraints, team familiarity to the language or how fast people expect things to be done. However I believe no language is perfect, no language will allow you and your team to scale 10, 20, 100 times because no language does that but software architecture does. Software changes, sometimes completely. Its like writing a post or a book sometimes you start with a small idea and in the end one might end with something completely different. So as you might imagine its quite hard to get everything right from the beginning.

While working at Delivery Hero, I had the opportunity to work in a team that was reponsible for migrating the system to a new one. At that point, it was decided to keep the same language but avoiding obvious bottlenecks.

One of multiple problems, that prevented the initial solution from scaling nicely, was a loop that checks every single delivery and rider applying or not actions based on configurations the app allows.

Unfortunately I cannot share the exact same code, but I created something that represents the problem pretty well. Along the this post I will try to guide you through the same process the team went through talking about the past, present and what was my idea about the future.

The past

Imagine you are building a software that keeps track of a mobile device’s position. As goal, the application has to keep track of a state, and after a certain period it should check a bunch of rules to take or not actions based on the state.

To handle that, the initial solution loops through a list of devices checking if the latest location was received recently and if it doesnt the app changes the device status to out_of_range. Besides that in case the device is already out_of_range for two periods in a row it marks the device as offline.

filter_online.each do |person|
  device.move_to('offline') if device.is?('out_of_range')
  device.move_to('out_of_range') if device.no_location_received?
end

The code above is just a small part of the solution, for the full version please take a look here

By executing the function above every minute the problem is solved, right?

Hold on, what if each check takes one second? As you might notice, we have a O(N) here and according to Rob Bell

“O(N) describes an algorithm whose performance will grow linearly and in direct proportion to the size of the input data set.”

Taking that into consideration, once the app have 61 devices it no longer able to run the solution in one minute and our solution will be invalid. Furthermore, based on my experience I can enumerate a few others issues like:

If a single device returns an error or the app crashes, the devices which were not checked yet will have wait until the next period to be processed;
Given that the solution sequential, its impossible to distribuite the load across different instances, pods or threads;
The interval & number of checks applied to each device might become quite inconsistent, if the interval decreases considerably from 1 minute to 30 seconds for example;

I might be missing others but I think the idea its pretty clear here, it doesnt scale because we cannot divide and conquer, its quite fragile and error prone. How can we make it better?

The present

One of the things I’ve been experimenting along these years is Elixir. From a Ruby developer perspective, the language is quite familiar and it has the awesome and battleproven OTP providing great actor model system based on GenServer. However as with every technology, it takes time and a huge effort for companies and people to start considering it. Well, as Ruby and Rails was already familiar for most of the team members and the risk had to be minimized, the second version was written in Ruby using Rails as web framework. Giving us a quite mature solution that offers also stable foundation for 90% of the web related problems. So how was it implemented?

The problems with the initial solution are scaling and making each check independent from each other. So what if each device could be checked individually with its own “process”, not affecting any others without changing the language? That would be pretty cool, right?

The proposed solution would look like a recursive call using Sidekiq to treat errors, something like the following

class CheckWorker
  include Sidekiq::Worker

  def perform(index)
    device = DEVICES[index]
    Sidekiq.logger.info(device)

    if device.is?('out_of_range')
      return device.move_to('offline')
    end

    device.move_to('out_of_range') if device.no_location_received?
    CheckWorker.perform_in(10, index)
  end
end

DEVICES.each_with_index do |_device, index|
  CheckWorker.perform_in(10, index)
end

For the full version please take a look here

Lets now go back to the problems we had initially to validate if new solution is better or worse.

About the first problem, now the execution time is only O(1) per job since now each device is managed by its own job individually. Of course it also adds some overhead for each job, but it can no longer affect any other device or even stop the process, so thats a win. Now jobs go to redis and can easily distributed across nodes, besides that Kubernetes HPA can be used to increase the number of pods, as the number of jobs increases, by checking cpu levels, cool right? The service we rewrote this approach pretty much everywhere within its core.

Based on the experience I had writing this app, I can tell you that the solution above scales pretty well, after adopting the Sidekiq Pro and it’s reliable_scheduler it got even better. Features that were impossible before like decreasing intervals to 10 seconds or even customising the processing flow, started becoming quite easy and straightforward. The company grew a lot, it went from 800 thousand orders daily to something like 5 or 6 million, in our biggest country we have the following numbers:

40 million jobs every day in a single country not too shabby

Even though the current approach scales pretty well, it doesnt scale infinitely, at least not in the way we have right now. As you might know Postgres has a limit in number of connections and with that as we add more and more pods to handle the increasing number of jobs at some point it will be just unbearable to the DB.

Wait, there is a way to solve that, the solution we found was using PGBouncer in transaction mode, so each query to the DB executes quite fast creating like a ConnectionPool for all the pods.

Cool, all problems solved right? Temporarely, this solution scales until a certain point due to DB constraints, at some point if for some reason any slow query is introduced or too many things are executed in parallel we noticed the following behaviour.

Too many clients start pilling up on PGBouncer slowing things down affecting the app performance.

The app is currently able to handle 5x the load ever had in production but things have to be improved in the future.

Things like cache and further query optimizations might minimize round-trips to the DB tend to improve the general performance but the DB will always be the limiting factor here.

And here we are, the present. How can we make things better?

The desired future

The problem faced now its not about the language, so changing to Go or insert-your-fancy-technology-here wont solve the problem because it lies in the approach itself, remember the actor model? Well if the app didnt have to search things in the DB for every single job the problem would be avoided. How would that look like?

While migrating the application to the new approach, we had the opportunity to write a load-test app in elixir to understand better what are the pain points of using this tool, unfortunately I believe I wont have time to work on it but I know that would be pretty viable. The solution using GenServers is not perfect, though, it eliminates the round-trips to the DB but it might increase the memory consumption considerably. Another point is that the state would have to be pretty well managed to avoid unexpected bugs or problems but I still believe that’s a best solution compared to the Sidekiq recursive approach.

Conclusion

As Developers, Software engineers or whatever name you might use, we like to learn new languages and sometimes to use newest one, however just changing the language by itself doesnt improve things magically.

Most of apps that I worked on so far are heavily limited by external calls (HTTP, DB, Network) so in most of the cases if you are running a web app, it doesnt matter much how fast the language is, if you are doing things right it should be enough. Not everybody is the Google. For me, it matters way more features like how easy to maintain, onboard new members, how fast to deploy, and run specs allowing cycles to run faster and that doesnt depend on the language itself.

So before considering moving to other language and wasting time and money, try to improve the project you are working on, MEASURE, observe your system, know the weeknesses and the strenghts, try to talk to other people and share your ideas/problems with others. If you want to take one thing from this post just consider applying the Scientific Method that according to Sedgewick & Wayne is:

“Observe some feature of the natural world, generally with precise measurements;
Hypothesize a model that is consistent with the observations;
Predict events using the hypothesis;
Verify the preditions by making further observations;
Validate by repeating until the hypothesis and observations agree; “

And iterate over it!

Once one apply this, the difference will be noticiable, that I can tell you for sure, these things are game changers, apply it and be happy. I strongly believe that programming languages are tools in our utility belt and even though we as developers we have our preferences (me included, I ♡ Ruby) we should always be accontable to use the best tool to the job.

Special Thanks

I’m really happy I’ve contribuited to a such nice project and happy to have worked with my team mates. Thanks for your understanding and patience along the way. I’m really proud on what we together as a team have done :)

References

https://algs4.cs.princeton.edu/home/
https://rob-bell.net/2009/06/a-beginners-guide-to-big-o-notation

Cover Photo by Chris Lawton on Unsplash

IoT Saga - Part 5 - The two way communication bridge: Presenting the Bitfrost

Tue, 08 Aug 2017 00:00:00 +0000

Hello, today I will present what I’ve been thinking about this application. Well, first of all, I have to explain the reason behind this application. Along the previous posts I’ve shown how I build the base to my dashboard panel using Hanami, Websockets and a MQTT broker, now it’s time to change the subject to start to talk about the application responsible for connect the Arduino world (Mygard) to the Dashboard world (Usgard).

There is a diagram that explains the main idea:

As you can see above, the Bitfrost is a deamon application that runs threads to send and receive messages from MQTT and RXTX. There are many ways to make it possible, I thought that the best way is to use Convention over Configuration and just follow a guide to publish and subscribe channels without configure all of them. I don’t want to change my application.yml everytime that I need to add a new sensor or an actuator, then I think is a good way to solve my problem, at least for now.

After this little introduction (I hope to explain again on the next posts), let’s see something about the input and output of this application.

MQTT Broker

I’m using mosquitto as MQTT broker. This broker is capable to deal with the message’s flux using channels, these channels are like routes or paths where we can put some information (messages). Take a look at the example below.

# Incomming message
MQTT Topic: /actuator/2/send

MQTT message headers: {mqtt_retained=false, mqtt_qos=0, id=78b2dcd1-21d1-9eb1-0bc1-c74c5d238e3b, mqtt_topic=actuator/2/send, mqtt_duplicate=false, timestamp=1502160412777}

MQTT message payload: on

It sounds familiar, isn’t it? Last week I’ve read a post about MQTT and it gave me the basic knowledge to start to think about this application. For now I don’t want to be worried about permissions and abilities for a specific user because, at least for now, it’s only my hobby project. However I think it’s very easy to add this kind of functionality because I can use the user as a level for a channel like ‘user/123/sensor/1’

That’s the way I will handle MQTT messages and channels so let’s take a look how RXTX works.

RXTX

The first question is what and why RXTX. The RXTX it’s a possible way to send and receive bytes through serial or a USB port. I think that’s the main point, but wait why do I intend to use it? I wish to build the whole thing by myself besides that I don’t have a Arduino capable to connect on the Internet by itself, summing up, from my point of view it’s the better way (at least that I found out).

When we open the arduino IDE something like the picture below appears

The’/dev/ttyACM0’ is the port responsible for send data between our arduino and the computer.

I will not cover all RXTX installation because it’s different depending the Operational System that you are using, but there are a lot of materials covering this installation on Internet. I’m used to use Linux so I’ve used the following tutorial:

http://rxtx.qbang.org/wiki/index.php/Installation_on_Linux

Sometimes the USB port it’s not available for the RXTX application then there is a ‘workaround’ to solve it:

sudo ln -s /dev/ttyACM0 /dev/ttyS81

This command creates a link to the ‘ACM0’ port and makes it always available for the application.

I will stop now, but I hope to write about how connect the Java application to serial on the next post, as soon as possible.

Thanks!

References

http://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices
http://rxtx.qbang.org/wiki/index.php/Installation_on_Linux
https://mosquitto.org/

IoT Saga - Part 4 - Publishing and Subscribing through MQTT: Displaying data on dashboard

Mon, 03 Jul 2017 00:00:00 +0000

Hello everyone, Today I will talk about how I’m using the LiteCable/Anycable and MQTT to display data on my dashboard. I made a huge number of changes on my views changing it to use the Materializecss instead bootstrap, however I not going to talk about it, not yet. Well I’ve shown how can we use anycable, but it’s only a battle to reach the illumination. Well, today I’m going to show how I connected the Websocket with the MQTT, If I could tell something now I think would be that it was the easiest part so far, but I can share some details with you. Let’s start with a map and here it is.

As you could see above there are not only many moving parts but also bidirectional communication ways… Ok I know, but c’mon I added just one part, the MQTT. In the previous post I wrote regarding what is and showed how to use LiteCable, so if you don’t saw the previous post please check it out before continue :).

Now we have a broker that is responsible to handle all connections. The MQTT broker is basically currently my single point of failure, but I’m not worried about it yet, here are some docker stats:

It’s not a very precise test but I ran a ruby publisher using a sleep time of 0.001 and works well. I tests were made only on my machine

To connect on MQTT I’m using this gem(ruby-mqtt). I simply added the following line to my Gemfile.

gem 'mqtt'

After that we have to build not only the code to subscribe all topics needed but also to broadcast messages through websockets to the related channel, I’m using the code below.

require "rack"
require "anycable"
require "litecable"
require "mqtt"

# we must enable the compatibility mode as was made in anycable server file.
LiteCable.anycable!

# Here we have the mqtt connection
client = MQTT::Client.connect(host: 'mqtt', port: 1883)

# Subscribing topics needed
client.subscribe("sensor/#")
client.subscribe("actuator/#")

client.get do |topic, message|
  topic = topic.gsub('/', '_')

  # Broadcasts to the websockets in a specific channel
  LiteCable.broadcast topic, user: 'external', message: message, sid: Random.rand(1..99999)
end

Well, when we execute ‘client.get’ the execution thread will loop getting packets forever. How I know it? I searched into the gem and discovered how the method ‘get’ works. The code is very well documented then thanks @njh, great work! I printed some pieces of code, so here it is.

And how it gets incoming packets from MQTT topics

As I mentioned previously I’m using docker-compose to separate the responsabilities in little containers avoiding tools like god or systemd on docker. The subscribe script runs as a daemon inside the container, summing up we have to add a new container to docker-compose.yml.

// More code here...
  subscriber:
    build: .
    command: bundle exec ruby subscriber
    volumes:
      - .:/usgard
    env_file:
      - .env.development
    environment:
      - ANYCABLE_REDIS_URL=redis://redis:6379/0
    depends_on:
      - redis
      - db
      - mqtt

Now it works!

Ok, Now we receive data from MQTT on dashboard but on the actuator’s page we have it the following feature:

We have a console, using it we have to send data to actuators

So, how can we send data to an mqtt topic? It’s just send to ‘actuators/1’, isn’t it? Almost there padawan because we don’t want to see the same data that we send. So, when we send data to MQTT we have to use a different topic, for example ‘actuators/1/send’. I already show how connect the websocket with the panel using the litecable channel then we have to change the Actuators channel to send data to MQTT, something like the code below.

# frozen_string_literal: true

module Usgard
  module Ws
    class Channel::Actuator < Usgard::Ws::Channel
      identifier :actuator

      def subscribed
        reject unless actuator_id
        stream_from "actuator_#{actuator_id}"
      end

      def speak(data)
        Hanami::Logger.new.info "#{@user} connected"
        LiteCable.broadcast "actuator_#{actuator_id}", user: user, message: data["message"], sid: sid

        send_to_mqtt(data['message'])
      end

      private

      def send_to_mqtt(message)
        # creates the client connection
        @client ||= MQTT::Client.connect(host: 'mqtt', port: 1883)

        # publishes on actuators output topic
        @client.publish("actuator/#{actuator_id}/send", message)
      end

      def actuator_id
        @actuator_id ||= params.fetch("id")
      end
    end
  end
end

To conclude now the Usgard dashboard is able to receive and send data through MQTT using Websockets. On the next post I will start to talk about both the Bridge, the Bitfrost project, and also about Java, Optionals, Threads and RXTX. I will talk again about Usgard project and Hanami later, don’t worry :) (I maybe write about Vue.js and Rodauth). Thank you, and please give your opinion about it, it makes me happy :).

References

http://materializecss.com
https://github.com/njh/ruby-mqtt

IoT Saga - Part 3 - Websockets! Connecting LiteCable to Hanami

Fri, 26 May 2017 00:00:00 +0000

Hello, Today I’d like to talk about how I’m using Websockets in Hanami. Well, when I was starting I added the following line inside my application.rb but after that I was worried about it.

security.content_security_policy %{
  connect-src: ws: 'self';
}

I was using PahoJS to connect to MQTT directly, but I think it isn’t a good option because credentials can be exposed. And, above all I have to control who is receiving and sending data through MQTT in future.

I started to search on the Internet a good option when suddenly I saw this page:

That sounds good, then why not?!

How it works

Vladimir Dementyev(@palkan_tula) recently wrote a post about it, From his point of view Ruby and Rails aren’t the best option for websockets based his experience and benchmarks, a decision has been made then they (Anycable.io) decided to extract the WebSocket responsability to another language, in this case, the language selected was Go. Anycable-Go deals with the websocket management and many other things without know of any business logic.

To deal with this layer, we need to create our classes to manage our rules, however how does AnyCable WebSocket(Go) connect to a Ruby Application?

They solved this problem using a gRPC client connected to another ruby process like the following picture.

Extracted from: https://evilmartians.com/chronicles/anycable-actioncable-on-steroids

Well, They explain all pieces in this post, it’s very interesting, please check it out.

I chose Hanami as Framework, I was looking for anyone that already made the connection between Hanami and Anycable but I didn’t find anything. That’s is reason why I decided to do it by myself and I will share my experience along this post. Fortunately, Anycable already has a example using Sinatra, I basically followed these steps changing some pieces, let’s start!

Adding pieces to setup

Firstly, we need a script to start our RPC server. I used the following code to start the Anycable RPC server and load all Hanami dependencies.

require "rack"
require "anycable"
require "litecable"
require_relative './config/boot'

LiteCable.anycable!

Anycable.configure do |config|
  config.connection_factory = Usgard::Ws::Connection
end

Anycable::Server.start

This server is a rack application then rack is required to run it, and also the ‘config/boot.rb’, which will load all Hanami components using ‘Hanami.boot’. After that the line ‘LiteCable.anycable!’ will enable the anycable compatibility mode. We must configure the class responsible to handle the connections, in this case ‘Usgard::Ws::Connection’. In the end, the server is started, then ‘Anycable::Server.start’ do it.

In sinatra example they’ve shown how start anycable-go and the RPC server using hivemind to start all processes. I use docker-compose, then I added the following lines to my compose file.

services:
  #More stuff here
  rpc:
    build: .
    command: bundle exec ruby anycable
    volumes:
      - .:/usgard
    env_file:
      - .env.development
    environment:
      - ANYCABLE_REDIS_URL=redis://redis:6379/0
      - ANYCABLE_RPC_HOST=0.0.0.0:50051
    depends_on:
      - redis
      - db
  anycable:
    image: 'anycable/anycable-go:0.3'
    ports:
      - "8080:8080"
    environment:
      - ADDR=0.0.0.0:8080
      - REDIS=redis://redis:6379/0
      - RPC=rpc:50051
    depends_on:
      - redis
      - rpc

Ps:. You can check this file here

RPC server starts running ‘bundle exec ruby anycable’ and the Anycable-Go image will start automatically, Some environment variables are required, though. Anycable uses Redis to manage the connections and broadcasts.

Ps:. The variable ‘DATABASE_URL’ must contains the connection string when ‘Hanami.boot’ is executed!

Now, the basic infrastructure is prepared to handle all websocket connections. Finally we can start to add the business logic.

Creating Channels and Connections

LiteCable is a ActionCable implementation, I think Rails defines whole concepts behind it very well. The paragraph below has been extracted from Rails doc.

For every WebSocket accepted by the server, a connection object is instantiated. […] The connection itself does not deal with any specific application logic beyond authentication and authorization. - Rails ActionCable Overview

So, we need to create a connection class to deal with this layer.

module Usgard
  module Ws
    class Connection < LiteCable::Connection::Base
      identified_by :user, :sid

      def connect
        #Ps:. I don't have authentication in this project, yet.
        @user = 'usgard' #cookies["user"]
        @sid = request.params["sid"]
        reject_unauthorized_connection unless @user
        Hanami::Logger.new.info "#{@user} connected"
      end

      def disconnect
        Hanami::Logger.new.info "#{@user} disconnected"
      end
    end
  end
end

Rails defines channels as “a logical unit of work, similar to what a controller does in a regular MVC setup.” So, a Channel class is required, I used the following class for my actuators.

module Usgard
  module Ws
    class Channel::Actuator < Usgard::Ws::Channel
      identifier :actuator

      def subscribed
        reject unless actuator_id
        stream_from "actuator_#{actuator_id}"
      end

      def speak(data)
        Hanami::Logger.new.info "#{@user} connected"
        LiteCable.broadcast "actuator_#{actuator_id}", user: user, message: data["message"], sid: sid
      end

      private

      def actuator_id
        @actuator_id ||= params.fetch("id")
      end
    end
  end
end

We already have all backend structure, besides we have to build the consumers at frontend.

Consuming websockets

Well, In Anycable page they mention:

“AnyCable uses ActionCable protocol, so you can use ActionCable JavaScript client without any monkey-patching.” - Anycable

I used the same JS of Rails, this JS is available here, and it handles the communication and keeps the websocket connection alive:

We can create our JS abstraction, but not now. I would rather use the cable.js, then I downloaded the JS and added to my application.html.slim

  == javascript 'cable'
  == javascript 'channel'

But, wait a moment, What’s that channel.js? This JS is responsible to create the channel, I’m using the Revealing Module pattern on my JS, from my point of view it’s a good JS pattern and I guess it can be changed easily later.

App.channel = (function() {
  function init(configuration) {
    return configureCable(configuration);
  }

  // configure and create cable using identifier and functions
  function configureCable(configuration) {
    return createCable().subscriptions
             .create(configuration.identifiers,
                     configuration.functions);
  }

  function createCable() {
    return ActionCable
             .createConsumer('ws://localhost:8080/cable'
                             + '?sid=' + socketId());
  }

  // Unique identifier for a connection
  function socketId() {
    return Date.now + generateRandomNumber();
  }

  function generateRandomNumber() {...}

  return {
    init: init
  }
}());

After that we have to create the JS deal with the incomming messages and send them to the WebSocket. I used the following code. I wanna build something like a terminal, which one I have to send messages to actuators channel and receive it.

PS:. I omitted some javascript of examples to turn easier to understand, however this code is available here

App.sensor = (function() {
  var config = { container: "display_box", channel: "actuator",
  user: "usgard", socket: null };

  function init(configuration) {
    config =  Object.assign({}, config, configuration);

    config.socket = App.channel
                       .init({identifiers: identifier(),
                              functions: subscriptionFunctions()});

    addListeners();
  }

  function identifier() {
    return {
      channel: config.channel, id: config.identifier
    };
  }

  function subscriptionFunctions() {
    return { connected: onConnected, disconnected:  onDisconnected,
             received: onReceive }
  }

  // These functions will be evaluated when cable trigger the subscriptions
  function onDisconnected() {
    appendMessageToBox({ user: 'system', message: "Connection Lost" });
  }

  function onReceive(data) {
    appendMessageToBox(data);
  }

  // Similar to onReceive function
  function onConnected() {
    // { ... }
  }

  // This function will handle the message when enter is typed
  function addListeners() {
    return getConsoleInput().addEventListener("keydown", function (event) {
      if (event.which == 13 || event.keyCode == 13) {
        onEnter();
        return false;
      }
      return true;
    });
  }

  // Sends to ActuatorChannel
  function onEnter() {
    config.socket.perform('speak',
                          { message: getMessageFromConsoleInput() });
  }

  // Create HTML elements
  function getMessageFromConsoleInput() {
    // { ... }
  }

  // Some other functions { ... }

  return {
    init: init
  }
}());

In the end, we have the HTML - in this case I used slim. So, here it is.

div
  h1 #{actuator.name}

  p id='actuatorid' style='display: none' #{actuator.id}

  dl.dl-horizontal
    dt id='mqtt_topic' #{actuator.mqtt_topic}
    dd #{actuator.description}

  div.col-md-7
    label Message
  div.col-md-5
    div.display_box id='display_box'
    div.col-xs-offset-0.form-group
      input.form-control id='console' type="text" name="msg"

  div.row
    button.btn.btn-secondary id="status" Status
    button.btn.btn-danger id="delete" Destroy

  == javascript 'actuator'

  javascript:
    App.sensor.init({identifier: "#"});

And it works! All changes that I’ve been made can be found in this pull request and also the Repository is available, please feel free to check it here.

Well, Do you like this post? Please feel leave your comments and share it with your friends, Thanks!

References

http://anycable.io
https://evilmartians.com/chronicles/anycable-actioncable-on-steroids
http://guides.rubyonrails.org/action_cable_overview.html
https://addyosmani.com/resources/essentialjsdesignpatterns/book/#revealingmodulepatternjavascript
https://github.com/palkan/litecable/tree/master/examples/sinatra

IoT Saga - Part 2 - Creating the dashboard: My little overview about hanami

Mon, 17 Apr 2017 00:00:00 +0000

Hello, Today I will talk about I’ve been learning. My first language at college was Java, from my point of view this language enforces you to learn many things. By the time, I remember how many hours I spent learning Software Engineering and Design Patterns to build all requested homework. Even through different than a many developers that I met, I enjoyed so much my course.

When I decided to learn Ruby I also started with Rails (I’m still using it everyday) the most common choice. Rails is great, this incredible framework allows us to build applications very fast. It uses Convention over Configuration and combine a very powerful toolkit to improve our productivity. On the other hand, some applications that I worked on could be better with using a different mindset. Don’t get me wrong, I like Rails but we can try another framework, language or paradigm.

I decided to experiment Hanami, because I saw a good talk of Marcello Rocha on RubyConf 2016 and also I can learn new approaches. I will share my throughts about Hanami along this post.

About the project

I intend to build a kind of dashboard where I can manage and send commands my arduino sensors and actuators. To reach my goal I’m going to create three applications (Usgard, Mygard and Bitfrost).

The first one will be the dashboard, Usgard. The first part of this saga mentions SpaceWing as project name but I decided to rename it. I chose this name because makes me remember Asgard, where Nordic Gods live. From there Gods can send commands and collect informations from worlds like Midgard. The second one, called Mygard, will receive commands and send data to RXTX. The third one will be the Bitfrost, the application to connect Mygard and Usgard.

There is a diagram to explain my basic idea (I intend to change the architeture):

Well, as you’ve seen above, I intend combine different technologies to create this dashboard. I want to combine this Stack because Java sounds familiar to me and a currently I know how I can combine them. My main aim is build a sandbox where I can test and create new things.

Usgard uses Hanami as WebFramework and PahoJS as WebSocket connector. I didn’t find something cool to my JS yet, I would use React, Redux or something that I don’t know. :) Suggestions?

I created two entities to manage my data, I called the first one Sensor and the another one Actuator. On the show page we can see the Sensor/Actuator information and send messages in actuator’s page. I didn’t finish the first version yet, but I learnt some new things using Hanami and I will share it with you.

The bright side

DISCLAIMER

First of all, this post represents only my opinion about the topics discussed, I do not intent to hurt anyone or something else. Think by yourself and get your own conclusions. I like to learn new ways to build applications and it can push me forward.

1. Decreases the coupling between domain and persistence layer

Well, I usually saw examples like the following diagram at college:

Extracted from: https://martinfowler.com/eaaCatalog/repository.html

Now, I can remember how was boring to write all Repository implementations using pure JDBC only to populate my POJO classes, but I was starting. By the time I didn’t know any technology capable to it by itself. After that I learnt a little of SpringJPA then when I was learning it, I saw something like the following code:

public interface UserRepository extends CrudRepository<User, Long> {
  Long countByLastname(String lastname);
}

This example made me think. Why do they use Repository to search instead use an ORM like us (Ruby on Rails programmers)? Although we use ActiveRecord because is natural for us and usually solve our problems very well, we can try something different.

Using Hanami models we have:

“Entity - A model domain object defined by it’s identity. Repository - An object that mediates between the entities and the persistence layer.” - Hanami documentation

We can create models and repositories using a command or create it by ourselves. Running the command below, Hanami creates the entity, the repository to manipulate data and the migration file. Inside it, we can describe the table columns and types to represent our Entity.

~/projects/ruby/usgard(dev ✔) bundle exec hanami generate model blargh

      create  lib/usgard/entities/blargh.rb
      create  lib/usgard/repositories/blargh_repository.rb
      create  db/migrations/20170402190609_create_blarghs.rb
      create  spec/usgard/entities/blargh_spec.rb
      create  spec/usgard/repositories/blargh_repository_spec.rb

From my point of view, the Repository pattern is good because removes the responsibility to manage database from models. Moreover this pattern can keep our models simpler and guide us to the Single Responsibility principle.

2. Views

“The Rails framework provides a large number of helpers for working with assets, dates, forms, numbers and model objects, to name a few. These helpers are available to ALL TEMPLATES by default.

In addition to using the standard template helpers provided, creating custom helpers to extract complicated logic or reusable functionality is strongly encouraged.” - Rails documentation

I usually saw Rails applications using Helpers to create anything sharing these custom methods along the application. I know Rails Helpers is different to the concept of Hanami Views but view layer is a good addiction to encapsulates only the necessary code. Well, the Hanami definition for Views is that:

“A view is an object that encapsulates the presentation logic of a page. A template is a file that defines the semantic and visual elements of a page. In order to show a result to a user, a template must be rendered by a view.” - Hanami documentation

3.Env files

According to Twelve-Factor, config files as database.yml in Rails, is a big improvement instead constants along application. On the other hand, it’s easy to send these files mistakenly to code repo and also every language adopt a different format each other. The Twelve-Factor suggest to us, to use Environment variables instead config/*.yml to decrease the chance to send to credentials to the code repo. Furthermore, it’s a “language- and OS-agnostic standard”

At part one, I showed how I made my application setup using Docker. Using env-vars instead config files turned it easier. Besides that, if you think about Environment automatization and Virtualization this concept would be good.

4. Controllers

I strongly believe it’s the best point of Hanami. Hanami uses Actions instead Controllers. The Action Layer allows us to:

Create whitelisting: to permit only trusted data;
Create custom validations: Hanami::Validation uses the power of Dry-Validation as dependency then we can build our validations
Create custom exception management: We can change the status or execute an method, for example;
Expose our variables using exposures: to grant encapsulation;
Use callbacks: I tend to avoid it, but some times we need to execute a method before Action#call;

And many other things, you can check it at project page.

I created an action in my project and I used the following structure to all actions, at least now.

module Web::Controllers::Sensors
  class Create
    include Web::Action # Action mixin (Composition over Inheritance)

    # Dry-AutoInject \o/, I used but it isn't default
    include ::AutoInject['commands.sensor.create']


    # Input validation using the power of Dry-Validation
    params do
      required(:sensor).schema do
        required(:name).filled(:str?)
        required(:mqtt_topic).filled(:str?)
      end
    end

    # Custom status returned when an Exception is raised
    handle_exception ArgumentError => 422


    # Single method to execute the action
    def call(params)
      if params.valid?

        # Similar to Interactions, they use something similar at Dnsimple, you can check at: https://blog.dnsimple.com/2017/03/why-we-ended-up-not-using-rails-for-our-new-json-api/
        sensor = create.(params.get(:sensor))

        redirect_to routes.sensor_url(id: sensor.id)
      else
        raise ::ArgumentError, params.errors
      end
    end
  end
end

The use of Commands or Interations mixed with the possibility to customize my actions before send it to Repository made me very happy. I made my model responsible only to store state like the PORO concept.

5. Umbrella applications

Sometimes we need to create application that shares the same common code. Hanami allows us to create applications inside the our application using the Umbrella concept. This concept allows us to create many applications sharing entities and repositories.

We know that the set of features that we’re going to introduce doesn’t belong to our main UI (Web). On the other hand, it’s too early for us to implement a microservices architecture, only for the purpose of helping our users reset their password. - Hanami Guide

The Premature Optimization problem can be applied here. Usually working at some startup or a company project wants to be released as fast as we can. Although we prefer to build isolated application, usually we don’t have time enough to create microservices. Using umbrella applications would be a good start to solve this problem.

The not so bright side

Well, when I was building my first test application (Usgard is my second one), I faced two questions. The first one was How can I run sidekiq? It was suffice a quick look at the project issues and I saw my answer, I was using the version 0.9. A little time later, on 1.0.0beta1 it was solved because Hanami added the config/boot.rb as default.

The second one was a question about routes. I have three applications and I mounted to routes file as below.

mount Sidekiq::Web,     at: '/sidekiq'
mount Api::Application, at: '/api'
mount Web::Application, at: '/'

It was my first contact and I mistakenly changed the order as the following example.

mount Web::Application, at: '/'
mount Sidekiq::Web,     at: '/sidekiq'
mount Api::Application, at: '/api'

After that, when I visited ‘/sidekiq’ route it redirects to my ‘/’. I wasted at least two hours to figure out what I did wrong.

Using Hanami maybe you can feel alone or cannot find your answer on internet, in these cases you can go to Hanami Chat to request help or take a look at the codebase.

I think Rails is great and Hanami too. Hanami released the 1.0.0 and a new core team was created then I really believe on this project.

Thanks @Hanamirb and @Rails teams and contribuitors!

References

https://docs.spring.io/spring-data/jpa/docs/current/reference/html/
https://martinfowler.com/eaaCatalog/repository.html
http://hanamirb.org/guides
https://12factor.net/config
http://api.rubyonrails.org/classes/ActionController/Helpers.html
http://hanamirb.org/guides/architecture/overview/
https://martinfowler.com/bliki/MonolithFirst.html
https://github.com/hanami/hanami
https://mkdev.me/en/posts/a-couple-of-words-about-interactors-in-rails
https://www.toptal.com/software/single-responsibility-principle
https://github.com/hanami/view
https://github.com/hanami/controller
https://blog.dnsimple.com/2017/03/why-we-ended-up-not-using-rails-for-our-new-json-api/
http://blog.steveklabnik.com/posts/2011-09-06-the-secret-to-rails-oo-design

IoT Saga - Part 1 - My first (for development) setup for a hanami application

Tue, 14 Feb 2017 12:00:00 +0000

Hello, Today I’m gonna talk about Hanami, I really wish to use it, then I decided to create a project to combine everything that I want to learn. I intend to do a saga covering my journey to complete it. I intend to use some technologies like MQTT and RXTX to do the communication with my Arduino and after that an application to execute some commands on my board. This saga is inspired by this post (made by a friend of mine), but I decided to use different tools, languages or frameworks to get some new knowledge.

The first post is about the high level application, responsible to send and receive all commands like a kind of dashboard where I can send commands, indicators and other things.

Introduction

Basically, I’m going to show how I made my setup basic development to a new Hanami application, but first what’s Hanami? My first definition is Hanami, based on website, is a modular web framework that allows you to do applications decoupled based on Clean Architeture and Monolith First.

Ok, now I have the framework, but I don’t need to install all dependencies at my machine then I asked myself. “How can I simplify my setup?” One possibility is Docker. I decided to use it because I like :) and it turns easier to do the setup and to run applications. With Docker we can encapsulate all environment inside a container.

Another good tip, gave by a colleague, is Scripts to rule them all, it’s convenient because following these rules we can use it like a convention to run projects in different languages and frameworks keeping on mind we need only to run a script inside a folder to test, run and build.

Starting

0. Creating an Hanami application

After the hanami installation (running ‘gem install hanami’), the first thing to do is create an application that we want to run, we can do it running the command below.

hanami new <project-name>

# In my case I used
hanami new space_wing --database=postgresql

Ps:. You can pass options to it, for example database, test framework and etc, you can check all options at the code. or on site

1. Running on Docker

To run Docker first we need to install it, we want to install docker and docker-compose, but I won’t cover installation here to simplify this post, but you can check how do it at the docker website.

After installation, we can check it.

~/projects/space_wing(dev ✔) docker --version
Docker version 1.12.1, build 23cf638

Ps:. Currently it’s stable version but you can use superior versions.

We need the file called ‘Dockerfile’ to specify all steps to build our application container, so we can start creating it.

~/projects/space_wing(dev ✔) touch Dockerfile

#Dockerfile content
FROM ruby:2.3.3

RUN mkdir /space_wing
WORKDIR /space_wing

ADD Gemfile .
ADD Gemfile.lock .

RUN bundle install

ADD . /space_wing

After that, we can use the command ‘docker build .’ to create a container with my application inside it. We also need to build all dependencies like the database and link between all containers, to do it we need to create the file called ‘docker-compose.yml’. This file is responsible to build all dependencies and the network between all containers, then let’s create it.

~/projects/space_wing(dev ✔) touch docker-compose.yml

#docker-compose.yml content
version: '2'

services:
  db:
    image: postgres
    environment:
      POSTGRES_DB: spacewing_development
      POSTGRES_USER: spacewing
      POSTGRES_PASSWORD: inicial1234
  web:
    build: .
    env_file:
      - .env
    command: bundle exec hanami s --host '0.0.0.0'
    volumes:
      - .:/space_wing
    ports:
      - "2300:2300"
    depends_on:
      - db

And the .env file to store our environment variables, in our case a database string connection.

~/projects/space_wing(dev ✔) touch .env

#.env content
DATABASE_URL=postgres://spacewing:inicial1234@db/spacewing_development

And when we run ‘docker-compose up’, we got it:

It’s cool, right?

2. Adding some scripts

In docker-compose we added this line:

command: bundle exec hanami s --host '0.0.0.0'

This command is responsible to run the app. If I’m at phoenix application for example we need to write a command like below.

command: mix phoenix.server

However we need to remember “how can I run my server?”, “how can I execute tests?”, “how can I do all setup to my application?”. If I have some migrations, I need to create the database and run migrations. Afterwards, I can forgot to initialize git submodules If I have any. I need to remember what command I need to run to install my dependencies and so on.

Now, imagine when we have a codebase with many languages, we can spend much time only remembering how do the setup for the application before start some task. From my point of view it’s a caos.

One of possible solutions comes from Script rule them all. It has a mission to show some tips to automate some common tasks in your project, we can use it for any language or framework keeping on mind only the main idea behind that.

2.1 Script to run tests

Let’s create a folder to store all scripts, and then we need to add two scripts, one is responsible to install the dependencies and the another one is responsible to run our tests.

~/projects/space_wing(dev ✔) mkdir script
~/projects/space_wing(dev ✔) touch ./script/bootstrap
~/projects/space_wing(dev ✔) touch ./script/test

# bootstrap script content
#!/bin/sh

# script/bootstrap: Resolve all dependencies that the application requires to
#                   run.

echo "==> Building and solving dependencies…"

if command -v docker-compose >/dev/null 2>&1; then
    docker-compose build
else
    echo "==> Please install docker first"
fi

# test script content
#!/bin/sh
# script/test: Run test suite for application. Optionally pass in a path to an
#              individual test file to run a single test.

echo "==> Running tests…"

./script/bootstrap

if [ -n "$1" ]; then
  docker-compose run web rake test "$1"
else
  docker-compose run web rake test
fi

The script ‘test’ is resposible to run tests, to execute it, we need to run ‘./script/test’ or ‘./script/test ' to run a single test.

2.2 Script to run server

As the same example above, now, we can create a script to run the server. We can use the script below to execute it inside of container.

~/projects/space_wing(dev ✔) touch ./script/server

# server script content
#!/bin/sh
# script/server: Launch the application and any extra required processes
#                locally.
set -e
cd "$(dirname "$0")/.."

echo ">>> Starting application..."

if command -v docker-compose >/dev/null 2>&1; then
  docker-compose up
else
  echo "==> Please install docker first"
fi

We can add many other scripts to different purposes trying to remember the rules, I don’t wanna to cover all scripts but you can check it at the project page.

That’s all folks! We can add ci like travis or wercker but I’m going to stop here. I really wish you like it, and if you like it, please share with your friends.

References

https://martinfowler.com/bliki/MonolithFirst.html
https://8thlight.com/blog/uncle-bob/2012/08/13/the-clean-architecture.html
https://github.com/github/scripts-to-rule-them-all
https://github.com/hanami/hanami/blob/master/lib/hanami/commands/new/app.rb
http://hanamirb.org
https://www.sergioaugrod.com.br/iot-arduino-serial-e-mqtt/
https://docs.docker.com/engine/installation/

OO programmers, Monads may be useful to us too

Thu, 29 Dec 2016 00:00:00 +0000

Hello, I saw the “Monad” concept at first time at TDC(São Paulo) 2016 at the functional programming Track. I didn’t get any use possibility in speech (maybe because I don’t know Haskell), but three months ago, when I started to work in a Java project with two collegues, a senior job colleague convinced me to use java.util.Optional in some cases along this project “It’s cool”, he said. He already worked with Scala using some monadics built-in tools, then he knows the monads advantages.

This project has many sequencial steps, the code majority has to validate something, convert to another type, save in database, send to a queue and so on. After many sprints interactions, when it started to take form, the code did everything needed keeping all things simple basically composing a lot of Functions applied in Java Optional Objects.

After that, when I came back to Ruby, I started to search some gem with the similar behavior, I googled it and I found Dry-Monads inspired by Kleisli. When I visited the Kleisli page I saw this phrase “You can use Haskell-like function composition with F and the familiar”, I thought “Oh my Gosh, I got Monads now”. Moments later, I found a cool post in Quora website about Monads, it’s different(and cool) because don’t envolves a huge number of mathematical formulas to explain the Monads concept, you can take a look here. He uses Haskell in examples, I don’t know Haskell but when if you compare to Dry-monads you will understand too.

Introduction

Everyone that already programmed in Java, at least once, got an NullPointerException. This error occurs when someone call a function to null reference.

null.toString();

java.lang.NullPointerException

If we use Ruby:

2.3.1 :004 > nil.to_s
 => ""
 2.3.1 :005 > nil.methods
  => [:&, :^, :|, :===, :inspect, :to_a, :to_s, :to_i, :to_f, :nil?, ... , :instance_eval, :instance_exec, :__id__]

2.3.1 :006 > nil.bla
NoMethodError: undefined method `bla' for nil:NilClass
  from (irb):6
    from /home/gabriel/.rvm/rubies/ruby-2.3.1/bin/irb:11:in `<main>'

Ruby return an exception when we try to call an inexistent attribute or method, but when we call ‘.to_s’ it returns a blank string. If we use Java, when we call any method to null, it will return an “NullPointerException”.

Since Java 8, we have the java.util.Optional, it can be useful in this case, to return an empty string when we use a null reference. Let’s see the example below:

java.util.Optional.ofNullable(null).map(Object::toString).orElse("");

//And it returns
""

It’s extremely useful to avoid Java NullPointerException. Another cool thing is the chain methods possibility, we can do it using some functions as ‘#map’ and ‘#flatMap’. We can also use the Optional behavior to avoid ‘if’ sequences and we can to turn our code more readable using the same text way. In Elixir we can chain methods using the pipe operator ‘|>’, let’s check it.

"Some String" |> String.upcase

//And it returns
"SOME STRING"

We can do it in Java too.

java.util.Optional.ofNullable("Some String")
                    .map(String::toUpperCase).get();

//And it returns
"SOME STRING"

Starting

We used this behavior many times in our project than, in this post, I will try show a similar concept using Dry-Monads in a simple ‘Order’ example.

Ok, let’s start, Our very simple use case is:

/*
     We have an Order, this Order has a Products, every Product has one price, but it can be nil to gifts, for example. The products sum can't be negative. An Order has a discount. It can be informed or no. Our task is calculate the order total.

     Order
       has -> products
       calc -> total - discount

     Product
       price
*/

First, we need an Order class

class Order
  def total
    @products.pluck(:price).reduce(:+) - discount
  end

  def add_product(product)
    @products = Array(@products) << product
  end

  def discount=(value)
    @discount = value
  end

  def discount
    @discount
  end

  def products=(value)
    @products = value
  end

  def products
    @products
  end
end

Ps:. I wrote all getter and setters to turn easier to understand.

And the Product class

class Product
  def initialize(price:)
    @price = price
  end

  def price
    @price
  end
end

Now, let’s check if works creating an Order and some products

2.3.1 :001 > order = Order.new
=> #<Order:0x000000031aa0d8>
2.3.1 :002 > products = []
=> []
2.3.1 :003 > 10.times { products << Product.new(price: Random.rand * 10) }
=> 10
2.3.1 :004 > order.products = products
=> [#<Product:0x000000046b2a70 @price=6.490346125032528>, [...More products here], #<Product:0x000000046b24f8 @price=4.088797576323275>, #<Product:0x000000046b2430 @price=5.311290981668898>]
2.3.1 :005 > order.discount= 10
 => 10
 2.3.1 :006 > order.total
  => 34.6909868839012

Awesome, it works! But if we set discount as nil…

2.3.1 :009 > order.discount = nil
 => nil
 2.3.1 :010 > order.total
 TypeError: nil can't be coerced into Float
  from /home/gabriel/projects/ruby/blog/app/models/order.rb:3:in '-'

Oh :( it’s bad. Do you know what happen if at least one product has a nil price?

2.3.1 :011 > order.add_product(Product.new(price: nil))
 => [#<Product:0x000000046b2a70 @price=6.490346125032528>, [...More products here], #<Product:0x000000046b2430 @price=5.311290981668898>, #<Product:0x000000045fc108 @price=nil>]
 2.3.1 :012 > order.total
 TypeError: nil can't be coerced into Float
  from /home/gabriel/projects/ruby/blog/app/models/order.rb:3:in '+'

Is it easy to correct, right? We can add the code below to avoid these exceptions.

class Product
...
  def price
    @price || 0
  end
end

class Order
 ...
  def discount
    @discount || 0
  end
end

2.3.1 :001 > order = Order.new
=> #<Order:0x00000004d27a68>
2.3.1 :002 > products = []
=> []
2.3.1 :003 > 10.times { products << Product.new(price: Random.rand * 10) }
=> 10
2.3.1 :004 > order.products = products
=> [#<Product:0x00000004d09248 @price=1.7050223472417392>, [...More products here], #<Product:0x00000004d08b90 @price=1.9066992998913168>, #<Product:0x00000004d08b18 @price=3.178449375930751>]
2.3.1 :005 > order.total
=> 41.55253855960062
2.3.1 :006 > order.add_product(Product.new(price: nil))
=> [#<Product:0x00000004d09248 @price=1.7050223472417392>, [...More products here], #<Product:0x00000004d08b18 @price=3.178449375930751>, #<Product:0x00000004cd1b40 @price=nil>]
2.3.1 :007 > order.total
=> 41.55253855960062

Magic, it’s works! but could we call an order without products?

2.3.1 :008 > order.products = nil
 => nil
 2.3.1 :009 > order.total
 NoMethodError: undefined method 'map' for nil:NilClass

Oh damn it, we can use the ‘||’ operator too or Array(value), but we can try something different using dry-monads. Let’s change the code!

class Product
  def initialize(price:)
    @price = price
  end

  def price
    Dry::Monads::Maybe(@price)
  end
end

// And the order class

class Order
  def total
    products.bind(-> (products) { products.map { |p| p.price.value_or(0) }.reduce(:+) - discount.value_of(0) })
  end

  def add_product(product)
    @products = Array(@products) << product
  end

  def discount=(value)
    @discount = value
  end

  def discount
    Dry::Monads::Maybe(@discount)
  end

  def products=(value)
    @products = value
  end

  def products
    Dry::Monads::Maybe(@products)
  end
end

We used Maybe in this case to get an Optional behavior. It will return None if we pass null or Some in non null cases as code below.

2.3.1 :003 > Dry::Monads::Maybe(nil)
=> None
2.3.1 :004 > Dry::Monads::Maybe(1)
=> Some(1)

The bind function was used to apply a function in object. We can use it to do an Upcase like previously.

2.3.1 :005 > Dry::Monads::Maybe("Bla").bind(->(str) { str.upcase })
=> "BLA"
2.3.1 :006 >Dry::Monads::Maybe("Bla").bind(&:upcase)
=> "BLA"

Then let’s try the new version

2.3.1 :009 > order = Order.new
=> #<Order:0x00000004d11e70>
2.3.1 :010 > products = []
=> []
2.3.1 :011 > 10.times { products << Product.new(price: Random.rand * 10) }
=> 10
2.3.1 :012 > order.products = products
=> [#<Product:0x00000004cf1788 @price=8.306382864273731>, [...More products here], #<Product:0x00000004cf1080 @price=0.4676475154069548>]
2.3.1 :013 > order.total
=> 38.75444516036125
2.3.1 :014 > order.products = nil
=> nil
2.3.1 :015 > order.total
=> None

It’s works, but the total function isn’t readable, we can improve it, but first, take a look:

  def total
    products
      .bind(-> (products) { products.map { |p| p.price.value_or(0) }.reduce(:+) - discount.value_of(0) })
  end

The total function has three responsabilities, it collects all products price, sum it and subtract the discount. We can break it into 3 separated functions, something like the code below.

class Order
  def total
    products
      .fmap(collect_prices)
      .fmap(sum)
      .bind(subtract_discount)
  end

  def add_product(product)
    @products = Array(@products) << product
  end

  def discount=(value)
    @discount = value
  end

  def discount
    Dry::Monads::Maybe(@discount)
  end

  def products=(value)
    @products = value
  end

  def products
    Dry::Monads::Maybe(@products)
  end

  private

  def collect_prices
    -> (products) { products.map { |p| p.price.value_of(0) } }
  end

  def sum
    -> (prices) { prices.reduce(:+) }
  end

  def subtract_discount
    ->(total) { discount.fmap(-> (discount) { total - discount }).or(total) }
  end
end

The code above used a #fmap function, it works like the bind function but it wrap the result into Some. The #fmap function returns None if applied in a None object. It works like the example below.

2.3.1 :003 > Dry::Monads::Some("Bla").fmap(&:upcase)
 => Some("BLA")
 2.3.1 :005 > Dry::Monads::Maybe(nil).fmap(&:upcase)
 => None

It’s cool, but now we need to add another feature. If our order doesn’t has products, we don’t need to calculate the total in these cases, then we can add this behavior using the ‘Either mixin’. Let’s see how we can use it. We will create a new command class ‘Commands::Order::Total’ and use it to calculate the order price.

class Order
  def add_product(product)
    @products = Array(@products) << product
  end

  def discount=(value)
    @discount = value
  end

  def discount
    Dry::Monads::Maybe(@discount)
  end

  def products=(value)
    @products = value
  end

  def products
    Dry::Monads::Maybe(@products)
  end
end

class Commands::Order::Total
  include Dry::Monads::Either::Mixin

  def call(order)
     order.products
      .bind(collect_prices)
      .fmap(sum)
      .bind(subtract_discount(order))
  end

  private

  def collect_prices
    -> (products) {
      return Left('You should add products') if products.empty?
      Right(products.map { |p| p.price.value_or(0) })
    }
  end

  def sum
    -> (prices) { prices.reduce(:+) }
  end

  def subtract_discount(order)
    ->  (total) { order.discount.fmap(-> (discount) { total - discount }).or(total) }
  end
end

/* Some examples */

2.3.1 :001 > order = Order.new
=> #<Order:0x00000004eea670>
2.3.1 :002 > products = []
=> []
2.3.1 :003 > 10.times { products << Product.new(price: Random.rand * 10) }
=> 10
2.3.1 :004 > calc = Commands::Order::Total.new
=> #<Commands::Order::Total:0x00000004e96c28>
2.3.1 :005 > calc.call order
=> None
2.3.1 :006 > order.products = []
=> []
2.3.1 :007 > calc.call order
=> Left("You should add products to calculate the total")
2.3.1 :008 > order.products = products
=> [#<Product:0x00000004ed4af0 @price=3.8706171589310223>, [...More products here], #<Product:0x00000004ed4668 @price=3.8299514817332634>, #<Product:0x00000004ed45f0 @price=0.595434488114438>]
2.3.1 :009 > calc.call order
=> 37.45434887890415
2.3.1 :010 > order.discount = 10
=> 10
2.3.1 :011 > calc.call order
=> Some(27.45434887890415)

The Dry-Monads page describe Left and right as “The Right can be thought of as “everything went right” and the Left is used when “something has gone wrong.”. Left is something like an exception, then when we have a function that returns Left the next chain function isn’t executed. Right is used when we have an expected result, then it can pass to next chain function.

It works! but the code returns None, Some or Left in some cases. If we use only Left and right we can check if we had success or no. Another thing is the sum of prices can be negative, we can return an failure if it occurs. Let’s make some changes.

class Commands::Order::Total
  include Dry::Monads::Either::Mixin

  def call(order)
    order.products
    .bind(collect_prices)
    .bind(sum)
    .bind(subtract_discount(order))
  end

  private

  def collect_prices
    -> (products) {
      return Left('You should add products') if products.empty?
      Right(products.map { |p| p.price.value_or(0) })
    }
  end

  def sum
    -> (prices) {
      total = prices.reduce(:+)
      return Left('The total can\'t be negative') if total < 0
      Right(total)
    }
  end

  def subtract_discount(order)
    ->(total) {
      order.discount
        .bind(-> (discount) { total - discount })
        .bind(-> (total) { Right(total) } )
        .or(Right(total))
    }
  end
end

2.3.1 :039 > order.products = products
=> [#<Product:0x000000043358e8 @price=0.3621462392006869>, [...More products here], #<Product:0x000000044ea0a8 @price=50>, #<Product:0x000000044da630 @price=-120>]
2.3.1 :040 > calc.call order
=> Left("The total can't be negative")
2.3.1 :041 > order.products = []
=> []
2.3.1 :042 > calc.call order
=> Left("You should add products")
2.3.1 :043 > order.add_product(Product.new(price: 300))
=> [#<Product:0x0000000414a088 @price=300>]
  2.3.1 :044 > calc.call order
=> Right(300)
  2.3.1 :045 > order.add_product(Product.new(price: 300))
  => [#<Product:0x0000000414a088 @price=300>, #<Product:0x0000000410ded0 @price=300>]
  2.3.1 :046 > calc.call order
=> Right(600)
2.3.1 :058 > order.discount = 10
 => 10
 2.3.1 :059 > calc.call order
=> Right(590)
2.3.1 :017 > Commands::Order::Total.new.call order
 => Right(51.08628673669591)
2.3.1 :014 > 10.times { products << Product.new({price: Random.rand * 10}) }
 => 10
 2.3.1 :015 > products
 => [#<Product:0x000000052ff4b8 @price=1.8427640303941961>, [...More products here], #<Product:0x000000052feef0 @price=9.708438235799372>]
 2.3.1 :016 > order.products = products
 => [#<Product:0x000000052ff4b8 @price=1.8427640303941961>, [...More products here], #<Product:0x000000052feef0 @price=9.708438235799372>]
2.3.1 :018 > r=_
=> Right(51.08628673669591)
2.3.1 :019 > r.success?
=> true
2.3.1 :020 > products << Product.new({price: -60})
=> [#<Product:0x000000052ff4b8 @price=1.8427640303941961>, [... More products here], #<Product:0x00000005252fd8 @price=-60>]
2.3.1 :021 > order.products = products
=> [#<Product:0x000000052ff4b8 @price=1.8427640303941961>, [...More products here] #<Product:0x00000005252fd8 @price=-60>]
2.3.1 :022 > Commands::Order::Total.new.call order
=> Left("The total can't be negative")
2.3.1 :023 > r=_
=> Left("The total can't be negative")
2.3.1 :024 > r.success?
=> false

That’s all, Is it cool, right? If you like it you can check the page project here, there are many amazing features.

Thanks!

References

http://dry-rb.org/gems/dry-monads/
https://medium.com/@sinisalouc/demystifying-the-monad-in-scala-cc716bb6f534#.g2jyjrme3
https://www.quora.com/What-are-monads-in-functional-programming-and-why-are-they-useful
https://github.com/txus/kleisli

Playing with circuit-breakers

Thu, 08 Dec 2016 00:00:00 +0000

At this year I went to QCon São Paulo, I saw many cool things and many new technologies. At the conference, some speakers talked about how we can make resilient systems. Kolton Andrus (slides) has spoken about of Netflix Toolset, he mentioned Spinnaker, Chaos Kong and Hystrix. I like so much of Hystrix, because the concept behind this mechanism sounds very simple and can be useful in many situations. In this post, I will try to talk about it.

Oh, what is Hystrix? The Netflix describes Hystrix as “[…] a latency and fault tolerance library designed to isolate points of access to remote systems, services and 3rd party libraries, stop cascading failure and enable resilience in complex distributed systems where failure is inevitable.”. Behind all advantages that it gives to us, this library has a lot of patterns, however in this post I will write about only one, called of Circuit-Breaker.

Let’s think about circuit-breaker. Let me see… Oh Circuit-Breaker pattern works like a home circuit-breaker, Obvious. But, how it works? You can see it here. In IT world, this mechanism can be applied to some cases to prevent errors or decrease latency. Imagine that you have a call to an external service, in this scenario many things can fail, you can get network error, some server can be out, or something unexpected can occurs. If you want to make your system resilient, you have to assume that your system can fail.

To avoid that problem, you can respond a default response or with cache. Using a circuit-breaker, when something goes wrong N times, the circuit-breaker stops to call failing method and responds only with a fallback response. After a time period, the circuit-breaker try to call a method again, it can works or not.

In this post, I will try to show some examples with the same app that I created at the previous post. You can download it here(please use version 0.0.2). I added Docker only to turn easier the environment setup, but you can use only ruby(>=2.2) and rails(>=5) to run.

I will show only one the stoplight gem, if you search at RubyToolbox site you can found many similar solutions like circuitbox or resilient, through.

Ok, how it works? Well, we will see it together along this post. So, it’s easier, at least for me, understand with examples.

Ps: To run with docker you can use ‘docker-compose up’ and after that ‘docker exec -it <image-name/id> bash’ to enter in bash. In the end we have to type ‘rails c’.

First of all, we have to add a new gem in this project, so the following line solves it.

gem 'stoplight'

Ps: Everytime that we add a new gem in Gemfile, we need to run ‘docker-compose build’

At the project folder, we need to type the command below to start our server.

 ~/projects/ruby/blog  docker-compose up
 Building web
 Step 1 : FROM ruby:2.3.0
 ---> 7ca70eb2dfea
 Step 2 : RUN apt-get update -qq && apt-get install -y build-essential libpq-dev nodejs
 ---> Using cache
 ---> 9f5de80da16a
...

...
 web_1                | * Version 3.6.0 (ruby 2.3.0-p0), codename: Sleepy Sunday Serenity
 web_1                | * Min threads: 5, max threads: 5
 web_1                | * Environment: development
 web_1                | * Listening on tcp://0.0.0.0:3000

The command above build the image from Dockerfile, starts all dependencies that we added in docker-compose and executes a command specified in docker-compose (in our case ‘bundle exec puma’).

~/projects/ruby/blog  docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
68f52b94aed6        blog_web            "bash -c 'bundle exec"   5 minutes ago       Up 3 minutes        0.0.0.0:3000->3000/tcp   blog_web_1
7dfa02f6c2de        redis:3.0.7         "docker-entrypoint.sh"   8 minutes ago       Up 3 minutes        0.0.0.0:6379->6379/tcp   blog_experiences_redis_1

After that, we have to enter in rails console, so type ‘docker exec -it -it bash" and 'rails c'.

~/projects/ruby/blog  docker exec -it blob_web_1 bash
root@68f52b94aed6:/var/www/experiences# rails c
Running via Spring preloader in process 67
Loading development environment (Rails 5.0.0.1)

The gem Stoplight is very illustrative, it works like a stoplight (Mr. Obvious attacks again! kkk). Please look at the diagram below.

Ps: I tried to extract the flux above from Stoplight code.

Now, let’s play with this gem and, besides that we will try to use it inside our simple ‘blog’ code.

root@e0e62e64b3ec:/var/www/experiences# rails c
Running via Spring preloader in process 49
Loading development environment (Rails 5.0.0.1)
irb(main):001:0> function = -> { puts "Hello World!" }
=> #<Proc:0x00560f121bc6b0@(irb):1 (lambda)>
irb(main):002:0> light = Stoplight('example') { function.call }
=> #<Stoplight::Light:0x00560f1219a498 @name="example", @code=#<Proc:0x00560f1219a4c0@(irb):2>, @cool_off_time=60.0, @data_store=#<Stoplight::DataStore::Memory:0x00560f118b6570 @failures=#<Concurrent::Map:0x00560f118b6548 entries=0 default_proc=#<Proc:0x00560f118b64f8@/usr/local/bundle/gems/stoplight-2.1.0/lib/stoplight/data_store/memory.rb:10>>, @states=#<Concurrent::Map:0x00560f118b6480 entries=0 default_proc=#<Proc:0x00560f118b6458@/usr/local/bundle/gems/stoplight-2.1.0/lib/stoplight/data_store/memory.rb:11>>, @lock=#<Monitor:0x00560f118b63e0 @mon_owner=nil, @mon_count=0, @mon_mutex=#<Thread::Mutex:0x00560f118b6390>>>, @error_handler=#<Proc:0x00560f118b6340@/usr/local/bundle/gems/stoplight-2.1.0/lib/stoplight/default.rb:9 (lambda)>, @error_notifier=#<Proc:0x00560f118b6318@/usr/local/bundle/gems/stoplight-2.1.0/lib/stoplight/default.rb:11 (lambda)>, @fallback=nil, @notifiers=[#<Stoplight::Notifier::IO:0x00560f118b62c8 @object=#<IO:<STDERR>>, @formatter=#<Proc:0x00560f118b62f0@/usr/local/bundle/gems/stoplight-2.1.0/lib/stoplight/default.rb:15 (lambda)>>], @threshold=3>
irb(main):003:0> light.run
Hello World!
=> nil
irb(main):004:0> light.run
Hello World!
=> nil
irb(main):005:0> light.color
=> "green"

The code above shows how Stoplight works, to create a light we must set an identifier, at this case ‘example’. We have to pass a code block that will be executed, in this case a lambda called ‘function’.

When we have a light, we can run it and the result will be evaluated and also get the light color.

Ok, if code works everytime when called this gem is unuseful, but if we have a code that can fail, we can use it to avoid some error scenarios. Let’s take a look when something goes wrong.

irb(main):010:0> function = -> { Blargh.new }
irb(main):011:0> light = Stoplight('error') { function.call }.with_fallback{|error| puts error; puts 'Fallback' }.with_cool_off_time(3)
=> #<Stoplight::Light:0x00560f11ed41a0 @name="error", @code=#<Proc:0x00560f11ed4240@(irb):11>, @cool_off_time=3, @data_store=#<Stoplight::DataStore::Memory:0x00560f118b6570 @failures=#<Concurrent::Map:0x00560f118b6548 entries=1 default_proc=#<Proc:0x00560f118b64f8@/usr/local/bundle/gems/stoplight-2.1.0/lib/stoplight/data_store/memory.rb:10>>, @states=#<Concurrent::Map:0x00560f118b6480 entries=0 default_proc=#<Proc:0x00560f118b6458@/usr/local/bundle/gems/stoplight-2.1.0/lib/stoplight/data_store/memory.rb:11>>, @lock=#<Monitor:0x00560f118b63e0 @mon_owner=nil, @mon_count=0, @mon_mutex=#<Thread::Mutex:0x00560f118b6390>>>, @error_handler=#<Proc:0x00560f118b6340@/usr/local/bundle/gems/stoplight-2.1.0/lib/stoplight/default.rb:9 (lambda)>, @error_notifier=#<Proc:0x00560f118b6318@/usr/local/bundle/gems/stoplight-2.1.0/lib/stoplight/default.rb:11 (lambda)>, @fallback=#<Proc:0x00560f11ed4178@(irb):11>, @notifiers=[#<Stoplight::Notifier::IO:0x00560f118b62c8 @object=#<IO:<STDERR>>, @formatter=#<Proc:0x00560f118b62f0@/usr/local/bundle/gems/stoplight-2.1.0/lib/stoplight/default.rb:15 (lambda)>>], @threshold=3>
irb(main):012:0> light.run
uninitialized constant Blargh
Fallback
=> nil
irb(main):013:0> light.run

Fallback
=> nil
irb(main):014:0> light.run

Fallback
=> nil
irb(main):015:0> light.run
uninitialized constant Blargh
Fallback
=> nil
irb(main):016:0> light.run

Fallback
=> nil
irb(main):017:0> light.color
=> "yellow"
irb(main):018:0> light.run
uninitialized constant Blargh
Fallback
=> nil
irb(main):019:0> light.color
=> "red"
irb(main):020:0> sleep(3)
=> 3
irb(main):021:0> light.color
=> "yellow"
irb(main):022:0> light.run
uninitialized constant Blargh
Fallback
=> nil
irb(main):023:0> light.color
=> "red"

The code above is similar than previous one, we added a function that always fail(at least since we define Blargh class), we also added some customizations like ‘with_fallback’ and ‘with_cool_of_time’.

The function ‘with_fallback’ defines a custom fallback, this code will be evaluated when something goes wrong.

The function ‘with_cool_of_time’ defines a timer to turn to yellow and then the next call will define if light will be green or red again. Don’t be afraid with this, Stoplight has a nice documentation and you can see many examples at project page.

When we have a huge application with multiple instances this code don’t work well because Stoplight uses memory by default to store all light statuses. This gem already has a solution to it, we can use Redis to share lights statuses between the app instances. So, we have to create a file at config/initializers and add the following code into it.

 ~/projects/ruby/blog touch config/initializers/stoplight.rb

require 'stoplight'
require 'redis'

redis = Redis.new(host: ENV["REDIS"])

Stoplight::Light.default_data_store = Stoplight::DataStore::Redis.new(redis)

Now, we will use Stoplight in article#show route to responds with a fallback when something goes wrong. So, we have to add the possibility to fail then a code that fail can be useful. We also have to create a command. You can use all code directly on Article.find, but I will use a command to keep same pattern.

Create a file into lib/commands/article folder, this command will be responsible to search an article and to encapsulate a circuit breaker behavior. Add the following lines into created file.

module Commands
  module Article
    class Find
      def call(id)
        ::Article.find(id)
      end
    end
  end
end

After that, we need to register a new command inside a Blog::Container.

class Blog::Container
  extend Dry::Container::Mixin

  register('commands.article.create') do
    Commands::Article::Create.new
  end

  register('commands.article.find') do
    Commands::Article::Find.new
  end
end

And finally change the code responsible to find Article inside a ArticlesController to call our new Command.

class ArticlesController < ApplicationController
  include AutoInject[create_article: 'commands.article.create',
                     find_article: 'commands.article.find'] #Adds a new dependency

  ...

  private
    def set_article
      @article = find_article.(params[:id]) # the . calls function #call inside a Commands::Article::Find
    end
  ...
end

Run ‘docker-compose up’ or ‘rails s’, create a article and see if everything works as expected.

Now we are ready to add the stoplight inside our project. First of all, we have to add the fail possibility. We will use the following code to determine if it will fail or no.

def fail?
  Random.rand < 0.5
end

After that, we need to add to code responsible responds by fallback. In our case, when something goes wrong fallback will responds with a fake Article.

def fallback(id)
  ::Article.new(id: id, name: "Fallback", description: "Error happened Fallback response")
end

And add the code responsible to fail or search Article in database.

def find(id)
  raise 'Something is wrong' if fail?
  ::Article.find(id)
end

When you search for a inexistent Article, the method #find will throw a error.

irb(main):001:0> Article.find 239882
  Article Load (0.3ms)  SELECT  "articles".\* FROM "articles" WHERE "articles"."id" = ? LIMIT ?  [["id", 239882], ["LIMIT", 1]]
  ActiveRecord::RecordNotFound: Couldn't find Article with 'id'=239882

But you don’t need to responds with a fallback in this case. To around this problem we have to add the code below.

def custom_error_handler(error, handle)
  raise error if error.is_a?(ActiveRecord::RecordNotFound)
  handle.call(error)
end

And finally the code to get together all things.

def call(id)
  Stoplight('article.find') do
    find(id)
  end.with_fallback do |error|
    fallback(id)
  end.with_error_handler do |error, handle|
    custom_error_handler(error, handle)
  end.run
end

All code will look like this:

module Commands
  module Article
    class Find
      def call(id)
        Stoplight('article.find') do
          find(id)
        end.with_fallback do |error|
          fallback(id)
        end.with_error_handler do |error, handle|
          custom_error_handler(error, handle)
        end.run
      end

      private
      def fail?
        Random.rand < 0.5
      end

      def fallback(id)
        ::Article.new(id: id, name: "Fallback", description: "Error happened Fallback response")
      end

      def find(id)
        raise 'Something is wrong' if fail?
        ::Article.find(id)
      end

      def custom_error_handler(error, handle)
        raise error if error.is_a?(ActiveRecord::RecordNotFound)
        handle.call(error)
      end
    end
  end
end

Now, we can run our server again, and access a Article#show route. When you get a error the server will respond with a fallback, like the image below.

If you check your log, you will see something like this:

We can add a custom notifier to something that you want when light changes the status from red to green or vice versa.

Bonus: We can add a custom notifier in our project, so take a look at the following example:

touch lib/notifiers/custom.rb

module Notifiers
  class Custom < Stoplight::Notifier::Base
    def notify(light, from_color, to_color, error)
      puts("[Custom] - Light: #{light.inspect} - From color: #{from_color} - To color #{to_color} - #{error}")
    end
  end
end

We only need to keep the same method signature of notify and add the custom notifier in stoplight configuration. To configure stoplight, we need to add the line below inside config/initializers/stoplight.rb.

Stoplight::Light.default_notifiers += Array(Notifiers::Custom.new)

When we call the route again until an error occurs. The log will look like this:

web_1                | Switching article.find from green to red because RuntimeError Something is going wrong
web_1                | [Custom] - Light: #<Stoplight::Light:0x0056395bbf8368 @name="article.find", @code=#<Proc:0x0056395bbf84f8@/var/www/experiences/lib/commands/article/find.rb:5>, @cool_off_time=60.0, @data_store=#<Stoplight::DataStore::Redis:0x0056395be41de0 @redis=#<Redis client v3.3.2 for redis://redis_experiences:6379/0>>, @error_handler=#<Proc:0x0056395bbf8278@/var/www/experiences/lib/commands/article/find.rb:9>, @error_notifier=#<Proc:0x0056395b7d8e18@/usr/local/bundle/gems/stoplight-2.1.0/lib/stoplight/default.rb:11 (lambda)>, @fallback=#<Proc:0x0056395bbf82f0@/var/www/experiences/lib/commands/article/find.rb:7>, @notifiers=[#<Stoplight::Notifier::IO:0x0056395b7d8dc8 @object=#<IO:<STDERR>>, @formatter=#<Proc:0x0056395b7d8df0@/usr/local/bundle/gems/stoplight-2.1.0/lib/stoplight/default.rb:15 (lambda)>>, #<Notifiers::Custom:0x0056395be297e0>], @threshold=3> - From color: green - To color red - Something is going wrong

Stoplight has a panel to displays all light statuses and to do some actions like lock light status. If you are interested in it, please take a look at the project page.

You can download all code showed from here. That’s all. If you have any doubt please post it in commentaries section.

Thanks!

References

https://github.com/Netflix/Hystrix
http://electronics.howstuffworks.com/circuit-breaker2.htm
https://github.com/orgsync/stoplight
https://github.com/jnunemaker/resilient
https://github.com/yammer/circuitbox
https://www.infoq.com/br/presentations/exercising-failure-at-netflix#downloadPdf

Using dry-auto_inject with rails

Thu, 17 Nov 2016 12:00:00 +0000

Around of 2 or 3 months ago, I saw dry-rb at the first time, I thought: “Oh, that’s awesome, I have to experiment”. Today is the day! If you are thinking: “Oh, sorry what is Dry-rb?”. I will explain… Well, Dry-rb is a bunch of tools to simplify and help us to make some code improvements.

Today in this post, I’m going to show how we can use dry-auto_inject with rails. First, There is a simple question, what is dry-auto_inject? According with dry-rb.org, it’s a “Container-agnostic constructor injection mixin”, if you already programmed in languages like Java (Spring @Autowired, says: ‘Hello’), C#, or some language or framework with dependency-injection support you saw the amazing ‘magic’ of Dependency Injection(DI).

If you are thinking: “Oh, my Gosh but I never saw nothing about dependency injection in my life”. Relax, I didn’t forget you. The Internet has a lot of materials about dependency injection. If you read something about SOLID, you already saw this concept. The D in SOLID, represents “Dependency inversion principle”, this principle can be made with an creational pattern, a factory method or a DEPENDENCY INJECTION framework, take a look below.

Extracted from: https://dzone.com/articles/dependency-injection-0

I won’t explain all details behind this concept, because many people made it as you see below:

solnic;
martin-fowler.

If you saw something like the code below, you already had a possibility to use Dry-Auto_Inject:

class CreateArticle
  def initialize(repository = ArticleRepository.new)
    self.repository = repository
  end

  def call(article)
    repository.call(article)
  end
end

In this case the class CreateArticle receives a external dependency and calls the repository method(#call). The class CreateArticle believes that ArticleRepository implements a method #call, it doesn’t have any details about repository. If you have some tool to inject the dependency automatically, this code can be more uncoupled and the responsability to know where was implemented, can be delegated to another part of code.

Starting

We have to install the development environment. We will use current stable rails version 5.0.0.1 and ruby 2.3.0. If you don’t have Ruby and Rails installed, check how install in RVM or Rbenv sites, it’s very simple ;).

 ~/projects/ruby  ruby -v
ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]

~/projects/ruby  rails -v
Rails 5.0.0.1

After that, we can start! To put it into action, we need a Rails application. To create a new App, type the command below. If you already has a Rails app, you can ignore this step.

~/projects/ruby  rails new blog
create
create  README.md
create  Rakefile
create  config.ru
create  .gitignore
create  Gemfile
create  app
...

After create app, we need to install dry-auto_inject. To install, add this line in your Gemfile.

gem 'dry-auto_inject'

And run

bundle install

/projects/ruby/blog  bundle list | grep dry
  * dry-auto_inject (0.4.1)
  * dry-configurable (0.1.7)
  * dry-container (0.5.0)

Dry-auto_inject depends of dry-configurable and dry-container. The dry-configurable is “a simple mixin to add thread-safe configuration behaviour to your classes”, you can check here. The dry-container is the core of auto-injection mechanism.

Let’s see how it works.

~/projects/ruby/blog  bundle exec rails c
Running via Spring preloader in process 5167
Loading development environment (Rails 5.0.0.1)
2.3.1 :001 >

2.3.1 :002 > container = Dry::Container.new
 => #<Dry::Container:0x000000038bb200 @_container={}>
 2.3.1 :003 > container.register(:article_repository) { Class.new }
  => #<Dry::Container:0x000000038bb200 @_container={"article_repository"=>#<Dry::Container::Item:0x000000034d0640 @item=#<Proc:0x000000034d06e0@(irb):43>, @options={:call=>true}>}>

When someone call Dry::Container#register with the identifier(:article_repository), the dependency is stored inside a proc. After that, when someone call Dry::Container#resolve with identifier(:article_repository) the proc stored previosly is executed or returned. The code inside proc can be executed immediately when resolve the reference or no, let’s see how it works with the code below.

2.3.1 :001 > container = Dry::Container.new
=> #<Dry::Container:0x000000042db460 @_container={}>
2.3.1 :002 > container.register(:lazy_hello, -> { puts "Hello Lazy" }, call: false)
=> #<Dry::Container:0x000000042db460 @_container={"lazy_hello"=>#<Dry::Container::Item:0x00000004342a98 @item=#<Proc:0x000000043432e0@(irb):2 (lambda)>, @options={:call=>false}>}>
2.3.1 :003 > container.resolve(:lazy_hello)
=> #<Proc:0x000000043432e0@(irb):2 (lambda)>
2.3.1 :004 > container.resolve(:lazy_hello).call
Hello Lazy
=> nil
2.3.1 :005 > container.register(:hello, -> { puts "Hello" }, call: true)
=> #<Dry::Container:0x000000042db460 @_container={"lazy_hello"=>#<Dry::Container::Item:0x00000004342a98 @item=#<Proc:0x000000043432e0@(irb):2 (lambda)>, @options={:call=>false}>, "hello"=>#<Dry::Container::Item:0x000000043814f0 @item=#<Proc:0x000000043815b8@(irb):5 (lambda)>, @options={:call=>true}>}>
2.3.1 :006 > container.resolve(:hello)
Hello
=> nil

And we can store namespaced identifiers.

2.3.1 :060 > container = Dry::Container.new
 => #<Dry::Container:0x00000003cebc08 @_container={}>
 2.3.1 :061 > container.namespace(:services) do
 2.3.1 :062 >     namespace(:article) do
 2.3.1 :063 >       register(:repository) { Class.new }
 2.3.1 :064?>     end
 2.3.1 :065?>   end
  => #<Dry::Container:0x00000003cebc08 @_container={"services.article.repository"=>#<Dry::Container::Item:0x00000003c898a0 @item=#<Proc:0x00000003c89ad0@(irb):63>, @options={:call=>true}>}>
  2.3.1 :066 > container.resolve('services.article.repository')
   => #<Class:0x00000003c66490>

For more details, you could check this documentation.

Now, we need to create something to use dry-auto_inject. We will use scaffold to generate the Article model.

 ~/projects/ruby/blog  rails g scaffold Article name:string description:string
 Running via Spring preloader in process 6093
 invoke  active_record
 create    db/migrate/20161115123949_create_articles.rb
 create    app/models/article.rb
 invoke    test_unit
 create      test/models/article_test.rb
 ...

Now, we can see the created routes typing ‘rake routes’.

 ~/projects/ruby/blog  rake routes
 Prefix Verb   URI Pattern                  Controller#Action
 articles GET    /articles(.:format)          articles#index
 POST   /articles(.:format)          articles#create
 new_article GET    /articles/new(.:format)      articles#new
 edit_article GET    /articles/:id/edit(.:format) articles#edit
 article GET    /articles/:id(.:format)      articles#show
 PATCH  /articles/:id(.:format)      articles#update
 PUT    /articles/:id(.:format)      articles#update
 DELETE /articles/:id(.:format)      articles#destroy

Before run application, we need to create the table to store article entries.

 ~/projects/ruby/blog  rake routes
 ~/projects/ruby/blog  rake db:migrate
  == 20161115123949 CreateArticles: migrating ===================================
-- create_table(:articles)
  -> 0.0015s
  == 20161115123949 CreateArticles: migrated (0.0016s) ==========================

We will change only the article#create route, but be free to modify everything you want. You can create an article to test if everything work as expected, So, open your browser and access http://localhost:3000/articles.

After tests, we can change the code. Take a look at articles_controller.rb.

#app/controllers/articles_controller.rb
class ArticlesController < ApplicationController
 ...
  def create
    @article = Article.new(article_params) # we will act only here
    ...
  end
end

I will create a command class only for illustration proposes. The command concept here, is very similar of Trailblazer::Operation. If you don’t like it, you can use Article#create to create directly, don’t worry :).

Now, create the folder /commands/article under lib directory, after that, create file create.rb with lines below.

#lib/commands/article/create.rb
module Commands
  module Article
    class Create
      def call(params)
        ::Article.create(params)
      end
    end
  end
end

If you run rails console and call Commands::Article::Create#call you will get an error, it’s because we need to add the line responsible to load the created structure at Blog::Application class.

#config/application
module Blog
  class Application < Rails::Application
    config.eager_load_paths += Dir["#{Rails.root}/lib"] # add this line

Now you can run rails console and call Commands::Article::Create#call with params. An article is created.

 ~/projects/ruby/blog  rails console
 Running via Spring preloader in process 10039
 Loading development environment (Rails 5.0.0.1)
 2.3.1 :001 > Commands::Article::Create.new.call(name: "AutoInject", description: "How can i use DryAutoInject")
    (0.1ms)  begin transaction
      SQL (0.5ms)  INSERT INTO "articles" ("name", "description", "created_at", "updated_at") VALUES (?, ?, ?, ?)  [["name", "AutoInject"], ["description", "How can i use DryAutoInject"], ["created_at", 2016-11-15 14:03:42 UTC], ["updated_at", 2016-11-15 14:03:42 UTC]]
         (7.9ms)  commit transaction
          => true

We have to define the container to register all dependencies needed. To reach it, we have to create a file under config/initializers to register dependencies.

#config/initializers/auto_inject.rb
class Blog::Container
  extend Dry::Container::Mixin

  register('commands.article.create') do
    Commands::Article::Create.new
  end
end

AutoInject = Dry::AutoInject(Blog::Container)

Inside this file the constant AutoInject was defined, it will be useful inside ArticlesController to inject dependencies.

To use the registered command, we have to include a reference to the registered dependency. The code will look as below:

#app/controllers/articles_controller.rb
class ArticlesController < ApplicationController
  # The alias create_article it's a reference
  # to 'commands.article.create'
  include AutoInject[create_article: 'commands.article.create']
  ...

  def create
    @article = create_article.call(article_params)

    respond_to do |format|
      if @article.persisted?

Finally, you can test again to see DI working. To see how add tests and many cool things please check the site.

All code is available on my github.

Thanks!

References

http://dry-rb.org/;
http://trailblazer.to/gems/operation/1.1/;
http://www.martinfowler.com/articles/injection.html;
http://solnic.eu/2013/12/17/the-world-needs-another-post-about-dependency-injection-in-ruby.html;
https://en.wikipedia.org/wiki/Dependency_inversion_principle.