Redis Cluster Setup (Linux on Windows)

There are so many things to share but so little time.. Though started writing about installation and setup of Hadoop on Windows using HyperV, deviating and writing about Redis. Have grand plans to write about Hadoop, Cassandra, Redis and SQL 14 along with my favorite Machine Learning. If only I could be better organized :(…

Coming to Redis, Redis recently got Cluster Support. Below from Redis release notes, Cluster feature was released on April 2015.


So, tried setting up Redis Cluster (different from Master / Slave concept) on Linux (Centos 7.0). Below is captured notes / lessons learnt. Follow previous posts to install / configure Linux on Windows machine using HyperV


Few points, 

  • Installation process below is comprehensive for setting up redis on Linux that is setup from base (thus lengthy)
  • Below process installs Redis as Service and not an simple executable. If Service is not needed skip running  step
  • If you want to avoid following below process Redis provides script to setup cluster.
    • Under “utils” folder there is “create-cluster” script. Running below commands would setup cluster with 3 Masters and 3 Slaves
      • create-cluster start
      • create-cluster create
  • If Ruby is already installed, skips steps for installation of Ruby.

Requirements to install Redis Cluster:

  • Connection to Internet to download setup files (Else download manually and copy for Offline installation)
  • Permissions to install


From here installation process starts.


  • Update YUM installer
    • yum update
    • yum install wget



  • Goto a folder where redis needs to be installed
    • cd /opt/
  • Download Redis Stable version
  • Unzip File
    • tar xvzf redis-stable.tar.gz
  • Goto unzipped directory
    • cd redis-stable
  • Compile code
    • make
    • make Install
  • To run Redis (Standalone or as a Service) below setup files are needed along with their locations.
  • To run & test redis below files are needed
    • redis.conf (/opt/redis-stable/redis.conf)
    • redis-benchmark (/opt/redis-stable/src/redis-benchmark)
    • redis-server (/opt/redis-stable/src/redis-server)
    • redis-check-aof (/opt/redis-stable/src/redis-check-aof)
    • redis-check-dump(/opt/redis-stable/src/redis-check-dump)
    • redis-cli (/opt/redis-stable/src/redis-cli)
  • To setup cluster multiple instances of Redis need to be running either on same or different machines.
  • On same machine: Each instance should have dedicated configuration file along with data and log directories. Create folder structure as needed. Sample below
  • Create a directory name redis-base
    • mkdir redis-base
  • Copy files from above folders to redis-base folder
    • cp /opt/redis-stable/redis.conf /opt/redis-base
    • cp /opt/redis-stable/src/redis-benchmark /opt/redis-base
    • cp /opt/redis-stable/src/redis-server /opt/redis-base
    • cp /opt/redis-stable/src/redis-check-aof /opt/redis-base
    • cp /opt/redis-stable/src/redis-check-dump /opt/redis-base
    • cp /opt/redis-stable/src/redis-cli /opt/redis-base
  • Based on number of instances of redis to be run, create as many folders.
    • mkdir redis_6379
    • mkdir redis_6380
    • mkdir redis_6381
    • mkdir redis_6382
    • mkdir redis_6383
  • Copy files from Redis-base to each of these folders
    • cp /opt/redis-base/* /opt/redis_6379
    • cp /opt/redis-base/* /opt/redis_6380
    • cp /opt/redis-base/* /opt/redis_6381
    • cp /opt/redis-base/* /opt/redis_6382
    • cp /opt/redis-base/* /opt/redis_6383
  • Starting each instance of redis. Below steps should be followed for each instance and port numbers (command port and connect port should be different for each instance of redis, if redis servers are running on same linux server).
    • cd /opt/redis_6379
    • cp redis.conf redis.conf.default
    • vi redis.conf
    • port <each instance to have its own port number>
      • Redis_6379 port: 6379
      • Redis_6380 port: 6380
      • Redis_6381 port: 6381
      • Redis_6382 port: 6382
      • Redis_6383 port: 6383
  • Scroll down to end where it has cluster configuration



  • Change below configurations:
    • Remove “#” before cluster-enabled yes. Should look like
    • cluster-enabled yes
    • cluster-config-file nodes-6379.conf
    • cluster-node-timeout 15000
    • cluster-slave-validity-factor 10
    • Press “ESC” button followed by “:” , “w” and Enter to quit vi editor
  • Install by running below command
    • ./opt/redis-stable/utils/
  • Running prompts for
    • Port Number 
    • Config file
    • Log Directory
    • Data Directory
    • Executable Directory
  • Run command as many times as Redis installation required.
  • Start Redis Service:
    • service redis_6379 start
  • If required to stop Redis Service:
    • service redis_6379 stop
  • To check running services, run chkconfig command
    • chkconfig –list
  • Connect to each instance of Redis and ensure it is running and cluster-enabled is set to 1
    • redis-cli –h –p 6379 <Port Number>
  • To get help from client, redis-cli –help
  • After connected it show IP and Port Number. At the prompt type command “info”


  • Cluster setup tool is written in RUBY. Ruby needs to be installed. Follow below steps to install Ruby and then configure cluster.
  • install all required packages for ruby installation
    • yum install gcc-c++ patch readline readline-devel zlib zlib-devel
    • yum install libyaml-devel libffi-devel openssl-devel make
    • yum install bzip2 autoconf automake libtool bison iconv-devel
  • Install Ruby Version Manager (RVM)
    • gpg –keyserver hkp:// –recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
    • curl -sSL | bash -s stable
      • Optionally add any of these –rails or –ruby or –ruby = 1.9.3
  • Rvm provides shell script to setup RVM environment before installing Ruby
    • source /etc/profile.d/
  • After RVM is setup, install RUBY
    • rvm install 2.1.2
  • Use RVM command to setup default Ruby version
    • rvm use 2.1.2 –default
  • Check current ruby version
    • ruby –version
  • Ruby Script to configure Redis Cluster requires redis. Ruby script references redis. Prior to running Cluster setup script run
    • gem install redis
  • After ruby installation is complete run below command to setup Redis Cluster
    • /opt/redis-stable/src/redis-trib.rb create –replicas 0

This completes “Redis Cluster Installation with default options”.




CentOS installation steps – Part 2

Proceeding with installation of CentOS on Virtualized environment. Last step in previous post was to click on Connect to launch CentOS bootable image.

1. CentOS 7 bootable image gives option to

  • Install
  • Test
  • Troubleshoot

2. Click on Tabs to move to different options. Select “Install CentOS 7” and  hit enter.


3. Executes bunch of commands. Nothing much to do but wait till execution is complete..


4. After sequence of steps are complete, launches CENTOS 7 INSTALLATION SCREEN

5. Select necessary language and click “Continue” button at bottom right corner.


5. Click on each of options below and configure as required.

  • Date & Time: To configure date and time. Configure Local Time and configure same consistently across all VMs.
  • Keyboard: To configure Keyboard layout.
  • Language Support: As afore detailed, language to be supported by OS
  • Installation Source: Auto detects to ISO image if not can be manually configured.
  • Software Selection: Multiple options (With UI, DevTools and Minimal Install). Used Minimal Install as running on very low Memory.
    • Also noticed UI is not smooth (may be because of Virtual Machine instead of Physical Machine)



6. Clicking on Date & Time opens up map where Country and location can be selected.

7. Also Network Time Protocol (NTP, norm in enterprises) can be configured when connected to network.

8 Choose appropriate Time Zone and formatting details at bottom and click on “DONE” on left top corner below “Date & Time” to return to Setup Configuration screen


7. Clicking on “Installation Source ” is optional as ISO was mounted in earlier step.

8. But good to check options under, click “”Installation Source” to open below window.

9 Nothing much to change, review and click “Done”


10. Click on Software Selection to choose one among various options.

  • For GUI and prebuilt development tools, select GNOME desktop or KDE plasma workspaces
  • GUI can be enabled later as well if required

11. Here “Minimal Install” was selected as we want to manually install software using Linux Command line utilities.

12. After choosing type of  Software Installation, click Done to to move to main installation configuration screen


12. Click on “Installation Destination” to review options.

13. Leave default, post review and click “Done”



14. Click Network and Hostname to open configuration options.

15. If already connected to network and enabled to Virtual Machine Manager (on HOST OS), Guest OS (CentOS) gets IP address.


16. After all configurations are complete, there should be not be any warnings (Red or Orange color ! and text below each button)

17. Click “Begin Installation”, bottom right corner to start installation of CentOS.


18. Parallel to installation running behind the scenes, two tasks need to be completed.

  • Setting  up Root password
  • Creation of User


19. Click on each button and configure as required.

  • Complex Password requirement.
  • Additional admin user is what was created for this setup.


20. After installation is complete below screen is presented that prompts for “root” user login.


This completes Linux Installation (only Minimal Installation). Next post is installing preq-requisites software to run CloudEra Hadoop Installation (CDH5)

Until then..


CentOS installation steps on Windows using HyperVisor – Part 1


For building Hadoop Cluster on Cloudera Hadoop Cluster (CDH 5), first step is to install Linux. Using any of HyperV(Microsoft), VMWare Workstation (VMWare) , Virtual Box (Oracle) Linux installation can be done on a virtual server. This post details step by step installation guide for installing CentOS 7.0 using Hyper-V and Virtual Box.

Before starting installation download required ISO images from CentOS site. Click below link and download appropriate ISO image required.  . For this walk through guide, CentOS-7.0-1406-x86_64-DVD.iso was used.


1. Open Hyper-V Manager on Windows Machine.

2. Right Click and select New -> Virtual Machine

3. A “New Virtual Machine Wizard launches”


  4. Click “Next” on New Virtual Machine Wizard to go to “Specify Name and Location” tab.


5. Enter Name of Virtual machine. HDPNameNode was entered below as I was building Hadoop Cluster.

6. Optionally, if alternative location is required, Select “Store the virtual machine in a different location” and enter path to store VM

7. Click Next to go to tab “Specify Generation”.


8. Selection “Generation 1”

Note: Need to check installation with “Generation 2” for Linux.

9. Click Next to move to “Assign Memory”


10. In “Assign Memory” tab, based on available memory on system and number of VMs that would run, assign memory to Guest OS.


10. In this section provide

  1. 1. Name of Virtual hard disk
  2. 2. Path where to store Virtual Hard disk.
  3. Size of Virtual Hard Disk (16 – 24 GB is good enough).


11. Select bootable image that was downloaded earlier.


12. Review Virtual Machine configuration and click finish to complete “New Virtual Machine” Wizard.


13. On Hyper V manager, new Virtual Machine “HDPNameNode” will be listed. Right click on Virtual Machine name and click on Connect.


This launches Linux bootable image. Next post will detail step by step installation of Linux (CentOS 7.0 version).