Wednesday, April 8, 2015

Installing Redis 3.0 in a Linux machine

Hi Friends, lets see the way to install Redis 3.0 in Linux machines.

Basic Installation:
  1. Firstly download the package in Linux machine
  2. Extract it to your favorite directory 
  3. Browse to the directory and pass the command make and press enter
  4. If make command version is perfect, it will start logging some output and you will be able to run the command successfully
  5. After that type make test  and wait for some time till the test gets completed
Advanced Installation (Cluster Mode):

Mode 1 : Installing the cluster setup in one machine

In this mode we will look in to mode of installing the full setup (3 Master, 3 Salves) in one machine, this is non production setup, never try to run production from one machine.
Also the setup process we are discussing here is basically useful for Linux based machines, and not Windows. 

Here are the steps.

  1. Create a folder /root/redis    $ mkdir /root/redis/
        $ cd /root/redis 
  2. Downloading the build from Redis Website
        You can download the build from here, (if not available please visit
        $ wget 
  3. Extract to a folder of your choice, for this doc lets consider we had extracted the redis.tgz  in /root/redis/
        $ tar -xvf redis-3.0.0.tar.gz
    this command will extract the files and create a folder named redis-3.0.0 inside the folder /root/redis, if you have downloaded the file in/root/redis, and extracted inside of it.
  4. Go to that newly created folder
        $ cd redis-3.0.0
  5. Now issue the command make inside of the folder
        $ make
  6. Testing the build
        $ make test
  7. Clean the build
        $ make distclean
  8. Now the Redis is ready to run in cluster or in a normal mode, depending upon the mode you selected.
  9. Now lets make a prerequisites for the cluster setup of Redis in one machine, for this setup let us consider that we are going to work with 6 instances of Redis(3 Masters, 3 Slaves)
  10. Since we are running in 1 instance we will create 6 directories and each directory will have a different configuration
  11. So lets suppose we are going to run the Redis in 7000-7005 port of machine, so create 6 directories naming 7000, 7001, 7002, 7003, 7004, 7005under /root/redis/cluster-test
  12. Here is the configuration template of basic cluster setup, for port make sure to keep perfect number, same of the directory, and cluster-config-file as the nodes0.conf and so on (attach with the last number of the port), and keep it in specific directory.

  1. Redis Config Template
    port 7000
    cluster-enabled yes
    cluster-config-file nodes0.conf
    cluster-node-timeout 5000
    appendonly yes
    So now the directory structure would look something like this,
        --- redis.conf
        --- redis.conf
        --- redis.conf
        --- redis.conf
        --- redis.conf
        --- redis.conf
  2. Now its time to start all the instances of Redis, for that lets start by starting the redis-server with these configurations
       $ /root/redis/redis-3.0.0/src/redis-server /root/redis/cluster-test/7000/redis.conf &
       $ /root/redis/redis-3.0.0/src/redis-server /root/redis/cluster-test/7001/redis.conf &
       $ /root/redis/redis-3.0.0/src/redis-server /root/redis/cluster-test/7002/redis.conf &
       $ /root/redis/redis-3.0.0/src/redis-server /root/redis/cluster-test/7003/redis.conf &
       $ /root/redis/redis-3.0.0/src/redis-server /root/redis/cluster-test/7004/redis.conf &
       $ /root/redis/redis-3.0.0/src/redis-server /root/redis/cluster-test/7005/redis.conf &
  3. If everything goes well it will show some log messages along with message telling "listening port on 7000" and so on.
        [82462] 26 Nov 11:56:55.329 * No cluster configuration found, I'm 97a3a64667477371c4479**32083***8db5858b1`
    This may be different for different machines.
  4. Now we have to create our cluster setup, till now we have started the Redis instances but didn't attached those in our cluster. So for that we will use the redis utility named redis-trib.rb   $ /root/redis/redis-3.0.0/src/redis-trib.rb create --replicas 1
    This command is telling to create a cluster with 1 replica each, and rest are the <ip:port> configurations to tell redis , what machines to use for the cluster.
  5. After this Redis will ask you for the confirmation, say yes and wait for redis to complete the setup, if everything goes well, you will get a log something like this:
    [OK] All 16384 slots covered
  6. Now we will test the setup:
       $ redis-cli -c -p 7000
          redis> set foo bar
          -> Redirected to slot [12182] located at
          redis> set hello world
          -> Redirected to slot [866] located at
          redis> get foo
          -> Redirected to slot [12182] located at
          redis> get hello
          -> Redirected to slot [866] located at
FYI: There may be chance that you will get some different log messages since its up-to redis, how it actually handles the data and pushes it into the cluster.
The parameters used in template are as follows
  1. port
    Tells the redis on which port to start to
  2. cluster-enabled     Tells redis to start on cluster mode
  3. cluster-config-file     Tells redis to use which file to look up for saving the info of the instance. This is not an user editable configuration file, but the file where a Redis Cluster node automatically persists the cluster configuration (the state, basically) every time there is a change, in order to be able to re-read it at start up.
  4. cluster-node-timeout     The maximum amount of time a Redis Cluster node can be unavailable, without it being considered as failing.
  5. appendonly
    This tells redis to append the log which is fully-durable strategy for Redis. So if your computer running Redis stops, your power line fails, or you accidentally kill -9 your instance, the latest data written on Redis will get lost. While this may not be a big deal for some applications, there are use cases for full durability, and in these cases Redis was not a viable option.
Mode 2 : Installing the cluster setup in different machines
  1. Make sure you have installed the redis in all the machines that are to be used as a part of the cluster.
  2. Make sure the port is open in firewall of AWS(security group) as well as local firewall for all the other machines of the cluster to be able to talk to.
  3. Also we have to open the <port number> + 10000, since the higher port is used for the Cluster bus, that is a node-to-node communication channel using a binary protocol. The Cluster bus is used by nodes for failure detection, configuration update, fail-over authorization and so forth. Clients should never try to communicate with the cluster bus port, but always with the normal Redis command port, however make sure you open both ports in your firewall, otherwise Redis cluster nodes will be not able to communicate.
  4. All the other part remains same as of the steps mentioned in above process.
Some common errors that you might face while installation
  1. make[3]: gcc: Command not found
    1. This basically comes when there is no compiler (gcc) for the make command to use
    1. If your machine supports yum installation you have to check for the latest version of gcc : $ sudo yum list gcc
    2. Then install the latest gcc: $ sudo yum -y install  <version that you want to install, came from last command>
  2. zmalloc.h:50:31: fatal error: jemalloc/jemalloc.h: No such file or directory
    1. While downloading and then building the redis some files gets half written and may start problems which we clean it and then restart the make process again.
    1. Before make, please use make distclean: $  make distclean
  3. You need tcl 8.5 or newer in order to run the Redis test
    1. Internally while testing the build by  make test , it internally uses the tcl package to complete the process 
    1. Firstly check the latest version of tcl available: $ sudo yum list tcl
    2. Then install the version using : $ sudo yum -y install  <version that you want to install, came from last command>
  4. /usr/share/ruby/vendor_ruby/2.0/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- redis (LoadError)
    1. Internally while starting the cluster mode Redis uses ruby, rubygems package so you have to make sure that these are installed beforehand.
    1. Firstly check the latest version of ruby available: $ sudo yum list ruby
    2. Then install ruby, follow this
    3. Check the availability of rubygems: $ sudo yum list rubygems
    4. Then install rubygems, follow this
    5. install the redis gem  : gem install redis
  5. Here is the list of commands that can be used directly to install the redis.
    sudo -i
    sudo yum -y install gcc.noarch
    sudo yum -y install tcl.x86_64
    sudo yum -y install ruby ruby-devel rubygems rpm-build
    gem install redis
    gem install  bundler
    mkdir /root/redis
    cd /root/redis
    tar -xvf redis-3.0.0.tar.gz
    cd redis-3.0.0
    make distclean
    make test
    cd ..
  6. vi redis.conf
    keep the following content in it

    port 7000
    cluster-enabled yes
    cluster-config-file nodes.conf
    cluster-node-timeout 5000
    appendonly yes
    appendfsync everysec
    no-appendfsync-on-rewrite yes
    daemonize yes
    logfile /root/redis/logs/redislog.log
    loglevel notice
  7. Open /etc/sysctl.conf
    vi /etc/sysctl.conf 
    and add this in to that file
    vm.overcommit_memory = 1
    sysctl -w net.core.somaxconn = 65535

    This is useful to make the settings available if the machine restarts but we have to make this settings 
    effective right now:
    sysctl vm.overcommit_memory=1
    sysctl -w net.core.somaxconn=65535
  8. Open /etc/rc.local
    vi /etc/rc.local
    and add this in to file
    # Redis config
    echo never > /sys/kernel/mm/transparent_hugepage/enabled
  9. To start the service:/root/redis/redis-3.0.0/src/redis-server redis.conf &
  10. To create the cluster:
    /root/redis/redis-3.0.0/src/redis-trib.rb create --replicas 1 Master1:IP Master2:IP 

No comments:

Post a Comment