Redis Cluster Setup (Linux on Windows)

There are so many things to share but so little time.. Though started writing about installation and setup of Hadoop on Windows using HyperV, deviating and writing about Redis. Have grand plans to write about Hadoop, Cassandra, Redis and SQL 14 along with my favorite Machine Learning. If only I could be better organized :(…

Coming to Redis, Redis recently got Cluster Support. Below from Redis release notes, Cluster feature was released on April 2015.

https://raw.githubusercontent.com/antirez/redis/3.0/00-RELEASENOTES

image

So, tried setting up Redis Cluster (different from Master / Slave concept) on Linux (Centos 7.0). Below is captured notes / lessons learnt. Follow previous posts to install / configure Linux on Windows machine using HyperV

https://abhyast.wordpress.com/2015/05/01/centos-installation-steps-on-windows-using-hypervisor-part-1/

https://abhyast.wordpress.com/2015/05/02/centos-installation-steps-part-2/

 

Few points, 

  • Installation process below is comprehensive for setting up redis on Linux that is setup from base (thus lengthy)
  • Below process installs Redis as Service and not an simple executable. If Service is not needed skip running install_server.sh  step
  • If you want to avoid following below process Redis provides script to setup cluster.
    • Under “utils” folder there is “create-cluster” script. Running below commands would setup cluster with 3 Masters and 3 Slaves
      • create-cluster start
      • create-cluster create
  • If Ruby is already installed, skips steps for installation of Ruby.

Requirements to install Redis Cluster:

  • Connection to Internet to download setup files (Else download manually and copy for Offline installation)
  • Permissions to install

#####################################################################

From here installation process starts.

#####################################################################

  • Update YUM installer
    • yum update
    • yum install wget

clip_image002

 

  • Goto a folder where redis needs to be installed
    • cd /opt/
  • Download Redis Stable version
  • Unzip File
    • tar xvzf redis-stable.tar.gz
  • Goto unzipped directory
    • cd redis-stable
  • Compile code
    • make
    • make Install
  • To run Redis (Standalone or as a Service) below setup files are needed along with their locations.
  • To run & test redis below files are needed
    • redis.conf (/opt/redis-stable/redis.conf)
    • redis-benchmark (/opt/redis-stable/src/redis-benchmark)
    • redis-server (/opt/redis-stable/src/redis-server)
    • redis-check-aof (/opt/redis-stable/src/redis-check-aof)
    • redis-check-dump(/opt/redis-stable/src/redis-check-dump)
    • redis-cli (/opt/redis-stable/src/redis-cli)
  • To setup cluster multiple instances of Redis need to be running either on same or different machines.
  • On same machine: Each instance should have dedicated configuration file along with data and log directories. Create folder structure as needed. Sample below
  • Create a directory name redis-base
    • mkdir redis-base
  • Copy files from above folders to redis-base folder
    • cp /opt/redis-stable/redis.conf /opt/redis-base
    • cp /opt/redis-stable/src/redis-benchmark /opt/redis-base
    • cp /opt/redis-stable/src/redis-server /opt/redis-base
    • cp /opt/redis-stable/src/redis-check-aof /opt/redis-base
    • cp /opt/redis-stable/src/redis-check-dump /opt/redis-base
    • cp /opt/redis-stable/src/redis-cli /opt/redis-base
  • Based on number of instances of redis to be run, create as many folders.
    • mkdir redis_6379
    • mkdir redis_6380
    • mkdir redis_6381
    • mkdir redis_6382
    • mkdir redis_6383
  • Copy files from Redis-base to each of these folders
    • cp /opt/redis-base/* /opt/redis_6379
    • cp /opt/redis-base/* /opt/redis_6380
    • cp /opt/redis-base/* /opt/redis_6381
    • cp /opt/redis-base/* /opt/redis_6382
    • cp /opt/redis-base/* /opt/redis_6383
  • Starting each instance of redis. Below steps should be followed for each instance and port numbers (command port and connect port should be different for each instance of redis, if redis servers are running on same linux server).
    • cd /opt/redis_6379
    • cp redis.conf redis.conf.default
    • vi redis.conf
    • port <each instance to have its own port number>
      • Redis_6379 port: 6379
      • Redis_6380 port: 6380
      • Redis_6381 port: 6381
      • Redis_6382 port: 6382
      • Redis_6383 port: 6383
  • Scroll down to end where it has cluster configuration

clip_image004

 

  • Change below configurations:
    • Remove “#” before cluster-enabled yes. Should look like
    • cluster-enabled yes
    • cluster-config-file nodes-6379.conf
    • cluster-node-timeout 15000
    • cluster-slave-validity-factor 10
    • Press “ESC” button followed by “:” , “w” and Enter to quit vi editor
  • Install by running below command
    • ./opt/redis-stable/utils/install_server.sh
  • Running install_server.sh prompts for
    • Port Number 
    • Config file
    • Log Directory
    • Data Directory
    • Executable Directory
  • Run install_server.sh command as many times as Redis installation required.
  • Start Redis Service:
    • service redis_6379 start
  • If required to stop Redis Service:
    • service redis_6379 stop
  • To check running services, run chkconfig command
    • chkconfig –list
  • Connect to each instance of Redis and ensure it is running and cluster-enabled is set to 1
    • redis-cli –h 127.0.0.1 –p 6379 <Port Number>
  • To get help from client, redis-cli –help
  • After connected it show IP and Port Number. At the prompt type command “info”
    • 127.0.0.1:6379>info

clip_image005

  • Cluster setup tool is written in RUBY. Ruby needs to be installed. Follow below steps to install Ruby and then configure cluster.
  • install all required packages for ruby installation
    • yum install gcc-c++ patch readline readline-devel zlib zlib-devel
    • yum install libyaml-devel libffi-devel openssl-devel make
    • yum install bzip2 autoconf automake libtool bison iconv-devel
  • Install Ruby Version Manager (RVM)
    • gpg –keyserver hkp://keys.gnupg.net –recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
    • curl -sSL https://get.rvm.io | bash -s stable
      • Optionally add any of these –rails or –ruby or –ruby = 1.9.3
  • Rvm provides shell script to setup RVM environment before installing Ruby
    • source /etc/profile.d/rvm.sh
  • After RVM is setup, install RUBY
    • rvm install 2.1.2
  • Use RVM command to setup default Ruby version
    • rvm use 2.1.2 –default
  • Check current ruby version
    • ruby –version
  • Ruby Script to configure Redis Cluster requires redis. Ruby script references redis. Prior to running Cluster setup script run
    • gem install redis
  • After ruby installation is complete run below command to setup Redis Cluster
    • /opt/redis-stable/src/redis-trib.rb create –replicas 0 127.0.0.1:6379 127.0.0.1:6380 127.0.0.1:6381

This completes “Redis Cluster Installation with default options”.

References:

-Abhyast

Advertisements

3 thoughts on “Redis Cluster Setup (Linux on Windows)

    • A very good post, very clear. Thanks a lot.

      rvm use 2.1.2 –default
      ruby –version

      The above shoud be corrected as,
      rvm use 2.1.2 –default
      ruby –version

      Like

  1. After successful completion of the server , I am able to “set” , “get” data from any of the three master, and the redirection, works nicely.
    Problem started, when I tried to connect from a remote client, I was needed to comment the line # bind 127.0.0.1
    unless I was unable to connect .
    Now The redirection always gets an error.
    10.10.87.91:6379> set 1asds 100
    OK
    10.10.87.91:6379> set dasd1asds 100
    (error) MOVED 10848 127.0.0.1:6380

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s