Crystal Cluster class plus SO_REUSEPORT to serve millions of HTTP requests per second

Crystal lang is a fast language that generate LLVM IR. With this Cluster class and SO_REUSEPORT socket option, Crystal can serve millions of HTTP requests per second. To do this, we need to compile Crystal with SO_REUSEPORT. On Linux, this socket option allows multiple sockets on the same host to bind to the same port : it improves the performance of a multithreaded network app on a multicore system. This is available on Linux > 3.9 .

Compile Crystal with SO_REUSEPORT

To compile Crystal with SO_REUSEPORT, you can follow our tutorial

Create the Cluster class

Now, create a file named cluster.cr and add this :

class Cluster

    def self.fork (env : Hash)
        env["FORKED"] = "1"
        Process.fork { Process.run(PROGRAM_NAME, nil, env, true, false, true, true, true, nil ) }
    end

    def self.master?
     (ENV["FORKED"]? || "0") == "0"
    end
    def self.worker?
     (ENV["FORKED"]? || "0") == "1"
    end

end

Use the Cluster class with the Http server

Create a file http.cr and add in it :

require "http/server"
require "./cluster.cr"

if Cluster.master?
        # IN MASTER
        # Init service
        nbr = 4 # Cluster number
        while(nbr > 0)
                Cluster.fork ({"id" => nbr.to_s})
                nbr -= 1
        end
        sleep
else
        # IN WORKER
        # start http server
        server = HTTP::Server.new(8080) do |context|
                context.response.content_type = "text/plain"
                context.response.print "Hello world! The time is #{Time.now}"
        end
        puts "Listening on http://127.0.0.1:8080"
        server.listen
end

To adjust the number of thread, just change the value of nbr

Benchmark

To benchmark you clustered Http server, you can use the Apache tool ab or wrk. You can install ab on Debian/Ubuntu with this command :

apt-get install apache2-utils

Now, you can benchmark your clustered server :

ab -n 1000 -c 5 http://127.0.0.1:8080/

Command line detailled :

  • -n 1000: ab will send 1000 requests to server
  • -c 5 : 5 is the concurrency number, ab will send 5 number of multiple requests to perform at a time

Problem with the Cluster class

If you have problems or difficulties with the Cluster class, don't hesitate to say it in comments, I'll try to help you.