Squid cluster on raspberry pis

Creating an active-passive 2-node raspberry pi cluster for highly available squid proxy.

By Kostas Koutsogiannopoulos

Our configuration implements active/passive architecture with shared nfs mount on our network attached storage (NAS).

The NAS hosts our squid disk-cache and squid configuration file in order to have one configuration file for the two nodes.

We need 2 raspberry pis.

We boot the first with raspbian and we follow the steps:

  • Configure hostname as "pi1"
  • Configure a static ip for pi1 (I prefer to do that on my dhcp)
  • Install (sudo apt-get install) squid3, corosync, pacemaker
  • Mount NAS 
  • Configure squid (follow  for details) to use the NAS mount point for cache and configuration file
  • Test the proxy
  • Stop squid3 service and configure not to start automaticaly at boot
  • Shutdown pi1
  • Clone the sd card of pi1
  • Use the clone to boot pi2
  • Configure hostname p2
  • Configure a static ip for pi2
  • Boot pi1

We now have two nodes ready for cluster configuration.

Corosync takes care of our cluster internal communications while Pacemaker manages our cluster resources (start, stop, monitor).

For corosync we are using the following configuration:


compatibility: whitetank
aisexec {
    # Run as root - this is necessary to be able to manage resources with Pacemaker
        user: root
        group: root
totem {
        version: 2
        secauth: on
        threads: 0
        rrp_mode: active
        token: 10000
        transport: udpu
        interface {
                ringnumber: 0
                mcastport: 5405
                ttl: 1
                member {
                        memberaddr: (Node 1)
                member {
                        memberaddr: (Node 2)
logging {
        fileline: off
        to_stderr: no
        to_logfile: yes
        to_syslog: yes
        debug: off
        logfile: /var/log/corosync/corosync.log
        timestamp: on
        logger_subsys {
                subsys: AMF
                debug: off

Our nodes are sharing the same squid configuration file and the squid disk-cache directory.

Therefore nfs mount to NAS need to be one of our managed cluster resources.

Furthermore we need to receive an informational mail when our resources are migrated between nodes.

So we have the following highly available resources for pacemaker to manage:

  • Cluster IP address
  • Proxy server
  • Informational e-Mail
  • Squid cache mount

In our cluster example the ip addresses are:

  • Node 1: pi1
  • Node 2: pi2
  • NAS: (for shared file system)
  • Cluster IP: (up on our active node)

The full pacemaker configuration:

node pi1 \
    attributes standby="off"
node pi2 \
    attributes standby="off"
primitive ClusterIP ocf:heartbeat:IPaddr2 \
    params ip="" cidr_netmask="24" \
    op monitor interval="30"
primitive ProxyRsc1 ocf:heartbeat:Squid \
    params squid_exe="/usr/sbin/squid3" squid_conf="/mnt/a300/squid.conf" squid_pidfile="/var/run/" squid_port="3128" squid_stop_timeout="10" debug_mode="v" debug_log="/var/log/cluster.log" \
    op start interval="0" timeout="5s" \
    op stop interval="0" timeout="10s" \
    op monitor interval="20s" timeout="30s"
primitive p_MailTo ocf:heartbeat:MailTo \
    params email="" \
    op monitor interval="10" timeout="10" depth="0"
primitive share-cache ocf:heartbeat:Filesystem \
    params device="" directory="/mnt/a300" fstype="nfs" options="user,rw,async,vers=3" fast_stop="no" \
    op monitor interval="20s" timeout="40s" \
    op start interval="0" timeout="60s" \
    op stop interval="0" timeout="120s" \
    meta is-managed="true" target-role="started"
group ProxyAndIP ClusterIP share-cache ProxyRsc1 p_MailTo
location prefer-pi1 ProxyRsc1 50: pi1
order SquidAfterIP inf: ClusterIP share-cache ProxyRsc1 p_MailTo
property $id="cib-bootstrap-options" \
    dc-version="1.1.7-ee0730e13d124c3d58f00016c3376a1de5323cff" \
    cluster-infrastructure="openais" \
    expected-quorum-votes="2" \
    stonith-enabled="false" \
    last-lrm-refresh="1441655139" \
rsc_defaults $id="rsc-options" \

You can check Custerlabs documentation for configuration options.

Some of the most useful commands are:

root@pi1 ~# crm status
Last updated: Wed Oct 28 12:29:11 2015
Last change: Sat Oct  3 23:18:50 2015 via crm_attribute on pi1
Stack: openais
Current DC: pi1 - partition with quorum
Version: 1.1.7-ee0730e13d124c3d58f00016c3376a1de5323cff
2 Nodes configured, 2 expected votes
4 Resources configured.

Online: [ pi1 pi2 ]

 Resource Group: ProxyAndIP
     ClusterIP    (ocf::heartbeat:IPaddr2):    Started pi1
     share-cache    (ocf::heartbeat:Filesystem):    Started pi1
     ProxyRsc1    (ocf::heartbeat:Squid):    Started pi1
     p_MailTo    (ocf::heartbeat:MailTo):    Started pi1

root@pi1 ~# crm resource move ProxyRsc1 pi2 # Move resources to the other node for pi1 maintainance

root@pi1 ~# crm resource unmove ProxyRsc1 # Give the control back to the cluster

root@pi1 ~# crm configure show # Show configuration

root@pi1 ~# crm_mon # Live monitoring

root@pi1 ~# crm_resource -P # Cleaning "Failed actions" messages

The last thing you need to do is to configure the cluster IP address as your new proxy server on all your devices.

View epilis's profile on LinkedIn Visit us on facebook Twitter epilis rss feed: Latest articles