{"id":8514,"date":"2013-08-18T17:31:32","date_gmt":"2013-08-18T22:31:32","guid":{"rendered":"http:\/\/jianmingli.com\/wp\/?p=8514"},"modified":"2014-03-18T21:43:48","modified_gmt":"2014-03-19T02:43:48","slug":"layer7-configre-a-gateway-cluster-with-pfsense","status":"publish","type":"post","link":"https:\/\/jianmingli.com\/wp\/?p=8514","title":{"rendered":"Layer7: Configure a Gateway Cluster with pfSense"},"content":{"rendered":"
\n

Contents<\/h2>\n
    \n\t
  1. \n\t\tOverview<\/a>\n\t\t
      \n\t\t\t
    1. \n\t\t\t\tArchitecture<\/a>\n\t\t\t<\/li>\n\t\t\t
    2. \n\t\t\t\tSystem Requirements<\/a>\n\t\t\t<\/li>\n\t\t\t
    3. \n\t\t\t\tOverview of Creating a new Gateway Cluster<\/a>\n\t\t\t<\/li>\n\t\t<\/ol>\n\t
    4. \n\t\tConfigure DB Replication<\/a>\n\t\t
        \n\t\t\t
      1. \n\t\t\t\tSystem Requirements<\/a>\n\t\t\t<\/li>\n\t\t\t
      2. \n\t\t\t\tConfigure Replication<\/a>\n\t\t\t\t
          \n\t\t\t\t\t
        1. \n\t\t\t\t\t\tRun add_slave_user.sh Script<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t\t
        2. \n\t\t\t\t\t\tRun create_slave.sh Script<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t\t
        3. \n\t\t\t\t\t\tVerify replication has started<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t<\/ol>\n\t\t\t
        4. \n\t\t\t\tMonitor Replication Failure<\/a>\n\t\t\t<\/li>\n\t\t\t
        5. \n\t\t\t\tRestart Replication<\/a>\n\t\t\t<\/li>\n\t\t<\/ol>\n\t
        6. \n\t\tConfigure Nodes<\/a>\n\t\t
            \n\t\t\t
          1. \n\t\t\t\tConfigure First Node: layer701<\/a>\n\t\t\t<\/li>\n\t\t\t
          2. \n\t\t\t\tConfigure Additional Node<\/a>\n\t\t\t\t
              \n\t\t\t\t\t
            1. \n\t\t\t\t\t\tlayer702<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t<\/ol>\n\t\t\t
            2. \n\t\t\t\tConfigure Name Resolution<\/a>\n\t\t\t<\/li>\n\t\t\t
            3. \n\t\t\t\tStart Cluster<\/a>\n\t\t\t<\/li>\n\t\t\t
            4. \n\t\t\t\tConfigure a CA Key for the Cluster<\/a>\n\t\t\t<\/li>\n\t\t\t
            5. \n\t\t\t\tDeactivating a Cluster Node<\/a>\n\t\t\t<\/li>\n\t\t<\/ol>\n\t
            6. \n\t\tConfigure Layer 7 Nodes<\/a>\n\t\t
                \n\t\t\t
              1. \n\t\t\t\tConfigure Node 1<\/a>\n\t\t\t<\/li>\n\t\t\t
              2. \n\t\t\t\tConfigure Node 2<\/a>\n\t\t\t<\/li>\n\t\t<\/ol>\n\t
              3. \n\t\tConfigure LB with pfSense<\/a>\n\t\t
                  \n\t\t\t
                1. \n\t\t\t\tInstall pfSense<\/a>\n\t\t\t<\/li>\n\t\t\t
                2. \n\t\t\t\tOverview<\/a>\n\t\t\t<\/li>\n\t\t\t
                3. \n\t\t\t\tTurn on Layer 7 pingServlet.mode<\/a>\n\t\t\t<\/li>\n\t\t\t
                4. \n\t\t\t\tConfig pfSense Load Balancer<\/a>\n\t\t\t\t
                    \n\t\t\t\t\t
                  1. \n\t\t\t\t\t\tAdd Monitors<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t\t
                  2. \n\t\t\t\t\t\tAdd Pools<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t<\/ol>\n\t\t\t
                  3. \n\t\t\t\tAdd Virtual Monitors<\/a>\n\t\t\t<\/li>\n\t\t\t
                  4. \n\t\t\t\tCheck Load Balancer Status<\/a>\n\t\t\t<\/li>\n\t\t\t
                  5. \n\t\t\t\tOpen WAN Firewalls for Port 8080 and 8443<\/a>\n\t\t\t<\/li>\n\t\t\t
                  6. \n\t\t\t\tOpen LAN Firewalls for both Port 8080 and 8443<\/a>\n\t\t\t<\/li>\n\t\t\t
                  7. \n\t\t\t\tCheck that Load Balancer is Listening on both Port 8080 and 8443<\/a>\n\t\t\t<\/li>\n\t\t\t
                  8. \n\t\t\t\tPoint Layer 7 Policy Manager to Load Balance IP<\/a>\n\t\t\t<\/li>\n<\/ol>\n<\/ol>\n<\/ol>\n<\/div>\n
                     <\/div>\n

                    Overview<\/h2><\/span>\n

                    * Gateway cluster nodes share:
                    \n– service policies
                    \n– identity providers
                    \n– configuration settings<\/p>\n

                    Architecture<\/h3><\/span>\n
                    \"\"<\/a><\/h6><\/span>\n

                    System Requirements<\/h3><\/span>\n

                    * Load Balancer device to provide TCP-level load balancing and failover
                    \n* Each node must possess its own host name, IP, and original node address within the LB
                    \n* Cluster must possess a host name and IP in LB
                    \n* Two nodes of the cluster must be installed and configured with the MySQL database with known root user names and root user passwords<\/p>\n

                    Overview of Creating a new Gateway Cluster<\/h3><\/span>\n

                    * Configure db replication on both Gateway db nodes
                    \n* Configure the first node
                    \n* Configure subsequent nodes
                    \n* Start cluster
                    \n* Create CA key for the cluster
                    \n* Install and configure LB on the network<\/p>\n

                    Configure DB Replication<\/h2><\/span>\n

                    * Maximum two MySQL db servers can be configured in a cluster
                    \n* Each peered db unit becomes both a slave and master to the other unit, i.e. master-master replication<\/p>\n

                    System Requirements<\/h3><\/span>\n

                    * Both db servers have host names and IPs in DNS or \/etc\/hosts
                    \n* Both MySQL services are running
                    \n* Both Gateway services stopped
                    \n* Time synchronized among all Gateway nodes<\/p>\n

                    Configure Replication<\/h3><\/span>\n

                    Run add_slave_user.sh Script<\/h4><\/span>\n

                    Run the following script against the local db on each node to add permissions for the users to MySQL:<\/p>\n

                    \r\nservice ssg stop\r\n\/opt\/SecureSpan\/Appliance\/bin\/add_slave_user.sh\r\n<\/pre>\n
                    You need to enter:
                    \n* For layer701:
                    \n– hostname or IP for the : layer702.pfsense.local<\/strong> # Note that this is the other node
                    \n– replication user (defaults to repluser): repluser<\/strong>
                    \n– replication password: Welcome1<\/strong>
                    \n– MySQL root user: root<\/strong>
                    \n– MySQL root password: 7layer<\/strong>
                    \n– Is this the primary (1) or Secondary database node? 1<\/strong><\/p>\n
                    \"\"<\/a><\/h6><\/span>\n
                    * For layer702:
                    \n– hostname or IP for the : layer701.pfsense.local<\/strong> # Note that this is the other node
                    \n– replication user (defaults to repluser): repluser<\/strong>
                    \n– replication password: Welcome1<\/strong>
                    \n– MySQL root user: root<\/strong>
                    \n– MySQL root password: 7layer<\/strong>
                    \n– Is this the primary (1) or Secondary database node? 2<\/strong><\/p>\n
                    \"\"<\/a><\/h6><\/span>\n
                    Run create_slave.sh Script<\/h4><\/span>\n
                    Run the following script against each db node to setup the replication to run between the two databases, using the user configured in the previous add_slave_user.sh<\/em> script. This sets up the other db as master:<\/p>\n
                    \r\n\/opt\/SecureSpan\/Appliance\/bin\/create_slave.sh\r\n<\/pre>\n
                    * For layer701:
                    \n– hostname or IP for the MASTER: layer702.pfsense.local<\/strong>
                    \n– replication user: repluser<\/strong>
                    \n– replication password: Welcome1<\/strong>
                    \n– MySQL root user: root<\/strong>
                    \n– MySQL root pass: 7layer<\/strong>
                    \n– Do you want to clone a database? no<\/strong><\/p>\n
                    \"\"<\/a><\/h6><\/span>\n
                    * For layer702:
                    \n– hostname or IP for the MASTER: layer701.pfsense.local<\/strong>
                    \n– replication user: repluser<\/strong>
                    \n– replication password: Welcome1<\/strong>
                    \n– MySQL root user: root<\/strong>
                    \n– MySQL root pass: 7layer<\/strong>
                    \n– Do you want to clone a database? no<\/strong><\/p>\n
                    \"\"<\/a><\/h6><\/span>\n
                    Verify replication has started<\/h4><\/span>\n
                    \r\nmysql\r\npager less\r\nshow slave status\\G\r\n<\/pre>\n
                    * For layer701:<\/p>\n
                    \"\"<\/a><\/h6><\/span>\n
                    \"\"<\/a><\/h6><\/span>\n
                    * Forlayer702:<\/p>\n
                    \"\"<\/a><\/h6><\/span>\n
                    \"\"<\/a><\/h6><\/span>\n
                    * If not, you can re-create replication process:<\/p>\n
                    \r\n\/opt\/SecureSpan\/Appliance\/bin\/create_slave.sh\r\n<\/pre>\n
                    Monitor Replication Failure<\/h3><\/span>\n
                    * Cluster properties for minitoring of replication delays or failures:
                    \n– db.replicationDelayThreshold<\/em>: threshod before Gateway audits a warning for slow or failed replication. Defaults to 60 sec
                    \n– db.replicationErrorAuditInterval<\/em>: minimum interval between db replication failure audits. Defaults to 60 min
                    \n* Replication events being audited:
                    \n– Replication failure
                    \n– Replication recovery
                    \n– Database failure<\/p>\n
                    Restart Replication<\/h3><\/span>\n
                    * Run against local db on each node:<\/p>\n
                    \r\n\/opt\/SecureSpan\/Appliance\/bin\/restart_replication.sh\r\n<\/pre>\n
                    and enter:
                    \n– hostname or IP for the MASTER: # the other node
                    \n– replication user: repluser<\/strong>
                    \n– replication password: Welcome1<\/strong>
                    \n– MySQL root user: root<\/strong>
                    \n– MySQL root password: 7layer<\/strong><\/p>\n
                    Configure Nodes<\/h2><\/span>\n
                    * Make sure db replication has been configured correctly!
                    \n* Reboot?<\/p>\n
                    Configure First Node: layer701<\/h3><\/span>\n
                    * Login first node as ssgconfig<\/strong>
                    \n* Select option 2 (Display Gateway configuration menu)<\/strong>
                    \n* Select option 2 (Create a new Gateway database)<\/strong>
                    \n* Restart appliance<\/p>\n
                    Configure Additional Node<\/h3><\/span>\n
                    layer702<\/h4><\/span>\n
                    * Login node as ssgconfig<\/strong>
                    \n* Select option 2 (Display Gateway configuration menu)<\/strong>
                    \n* Select option 3 (Configure the Gateway)<\/strong>
                    \n– Database Host: enter hostname for db server 1: layer701.pfsense.local<\/strong>
                    \n– Database Port [3306<\/strong>]:
                    \n– Database Name: use value for the first node: ssg<\/strong>
                    \n– Database username and database password: use values for first node: gateway\/welcome1<\/strong>
                    \n– Cluster password: use value for first node: welcome1<\/strong>
                    \n* Ensure the node is enabled and then press [Enter] at the configuration summary.
                    \n* Restart appliance<\/p>\n
                    Configure Name Resolution<\/h3><\/span>\n
                    * It is expected that each node of the Gateway cluster can resolve the IP address of the cluster and all other nodes by DNS.
                    \n* If DNS is not configured to provide this, then each node must do so via the \u201c\/etc\/hosts\u201d file.<\/p>\n
                    Start Cluster<\/h3><\/span>\n
                    * Start primary node
                    \n* Start other nodes<\/p>\n
                    Configure a CA Key for the Cluster<\/h3><\/span>\n
                    Deactivating a Cluster Node<\/h3><\/span>\n
                    Configure Layer 7 Nodes<\/h2><\/span>\n
                    Configure Node 1<\/h3><\/span>\n
                    IP: 192.168.2.61<\/strong>
                    \nNet mask: 255.255.255.0<\/strong>
                    \nGateway: 192.168.2.1<\/strong>
                    \nName server: 192.168.2.1<\/strong>
                    \nNTP server: pool.ntp.org<\/strong><\/p>\n
                    Configure Node 2<\/h3><\/span>\n
                    IP: 192.168.2.62<\/strong>
                    \nNet mask: 255.255.255.0<\/strong>
                    \nGateway: 192.168.2.1<\/strong>
                    \nName server: 192.168.2.1<\/strong>
                    \nNTP server: pool.ntp.org
                    \n<\/strong><\/p>\n
                    Configure LB with pfSense<\/h2><\/span>\n
                    Install pfSense<\/h3><\/span>\n
                    * See this post<\/a> to install pfSense.<\/p>\n
                    Overview<\/h3><\/span>\n
                    * IP addresses:
                    \n– Cluster IP: 192.168.1.60<\/strong>
                    \n– Node IP addresses: 192.168.2.61, 192.168.2.62<\/strong>
                    \n* Configure the virtual server
                    \n* Configure session persistence
                    \n– Ensure LB session timeout limit is set to 30 min
                    \n* Configure service availability determination
                    \n– Set check frequency to 120 sec
                    \n– Set response timeout to 120 sec
                    \n– For port 8080, set communication type to “Normal”
                    \n– For port 8443, set communication type to “SSL”
                    \n– Configure ports 8080 and 8443 to check the URL “\/ssg\/ping” (defaults to 8443 SSL with HTTP basic credentials in the request)
                    \n– An “OK” message for successful ping
                    \nNote: you need to set pingServlet.mode<\/em> to OPEN<\/em> from Layer 7 Policy Manager in order to get an OK<\/em> message by hitting \/ssg\/ping<\/em><\/p>\n
                    Turn on Layer 7 pingServlet.mode<\/h3><\/span>\n
                    * Log in Layer 7 Policy Manager
                    \n* Go to Tasks > Manage Clust-Wide Properties<\/em>
                    \n* Set
                    \n– pingServlet.mode<\/em> to OPEN<\/strong><\/p>\n
                    Config pfSense Load Balancer<\/h3><\/span>\n
                    * Log in pfSense web console<\/p>\n
                    Add Monitors<\/h4><\/span>\n
                    * Go to: Services > Load Balancer > Monitors<\/em>
                    \n* Add a new monitor:
                    \n– Name: Layer7Mon8080<\/strong>
                    \n– Description: Layer 7 cluster monitor on port 8080.<\/strong>
                    \n– Type: TCP<\/strong><\/p>\n
                    \"\"<\/a><\/h6><\/span>\n
                    * Add another new monitor:
                    \n– Name: Layer7Mon8443<\/strong>
                    \n– Description: Layer 7 cluster monitor on port 8443.<\/strong>
                    \n– Type: TCP<\/strong><\/p>\n
                    \"\"<\/a><\/h6><\/span>\n
                    Add Pools<\/h4><\/span>\n
                    * Go to: Services > Load Balancer > Pools<\/em>
                    \n* Add a new pool
                    \n– Name: Layer7Pool_8080<\/strong>
                    \n– Mode: Load Balance<\/em>
                    \n– Port: 8080<\/strong>
                    \n– Monitor: Layer7Mon8080<\/strong>
                    \n– Members: 192.168.2.61, 192.168.2.62<\/strong><\/p>\n
                    \"\"<\/a><\/h6><\/span>\n
                    * Add another pool
                    \n– Name: Layer7Pool_8443<\/strong>
                    \n– Mode: Load Balance<\/strong>
                    \n– Port: 8443<\/strong>
                    \n– Monitor: Layer7Mon8080<\/em>
                    \n– Members: 192.168.2.61, 192.168.2.62<\/strong><\/p>\n
                    \"\"<\/a><\/h6><\/span>\n
                    Add Virtual Monitors<\/h3><\/span>\n
                    * Go to: Services > Load Balancer > Virtual Servers<\/em>
                    \n* Add a new virtual server:
                    \n– Name: Layer7vs8080<\/strong>
                    \n– Description: Layer 7 virtual srever on port 8080.<\/strong>
                    \n– IP Address: 192.168.1.60<\/strong> (This is the WAN public IP)
                    \n– Port: 8080<\/strong>
                    \n– Virtual Server Pool: Layer7Pool_8080<\/strong><\/p>\n
                    \"\"<\/a><\/h6><\/span>\n
                    * Add another virtual server for port 8443:
                    \n– Name: Layer7vs8443<\/strong>
                    \n– Description: Layer 7 virtual srever on port 8443.<\/strong>
                    \n– IP Address: 192.168.1.60<\/strong> (This is the WAN public IP)
                    \n– Port: 8443<\/strong>
                    \n– Virtual Server Pool: Layer7Pool_8443<\/em><\/p>\n
                    \"\"<\/a><\/h6><\/span>\n
                    Check Load Balancer Status<\/h3><\/span>\n
                    * Login pfSense console
                    \n* Go to: Status > Load Balancer<\/em>
                    \n* Check all pools are green:<\/p>\n
                    \"\"<\/a><\/h6><\/span>\n
                    * Check all virtual servers are green:<\/p>\n
                    \"\"<\/a><\/h6><\/span>\n
                    Open WAN Firewalls for Port 8080 and 8443<\/h3><\/span>\n
                    * Go to Firewall > Rules<\/em>
                    \n* Open port 8080:<\/p>\n
                    \"\"<\/a><\/h6><\/span>\n
                    * Do the same to open port 8443:<\/p>\n
                    \"\"<\/a><\/h6><\/span>\n
                    Open LAN Firewalls for both Port 8080 and 8443<\/h3><\/span>\n
                    \"\"<\/a><\/h6><\/span>\n
                    \"\"<\/a><\/h6><\/span>\n
                    Check that Load Balancer is Listening on both Port 8080 and 8443<\/h3><\/span>\n
                    \r\ntelnet 192.168.1.60 8080\r\ntelnet 192.168.1.60 8443\r\n<\/pre>\n
                    Point Layer 7 Policy Manager to Load Balance IP<\/h3><\/span>\n
                    \"\"<\/a><\/h6><\/span>\n","protected":false},"excerpt":{"rendered":"
                    Overview * Gateway cluster nodes share: – service policies – identity providers – configuration settings Architecture System Requirements * Load Balancer device to provide TCP-level load balancing and failover * Each node must possess its own host name, IP, and … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[229],"tags":[630,426,424],"class_list":["post-8514","post","type-post","status-publish","format-standard","hentry","category-firewall","tag-layer7","tag-loadbalancer","tag-pfsense"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p8cRUO-2dk","_links":{"self":[{"href":"https:\/\/jianmingli.com\/wp\/index.php?rest_route=\/wp\/v2\/posts\/8514","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jianmingli.com\/wp\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jianmingli.com\/wp\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jianmingli.com\/wp\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/jianmingli.com\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=8514"}],"version-history":[{"count":14,"href":"https:\/\/jianmingli.com\/wp\/index.php?rest_route=\/wp\/v2\/posts\/8514\/revisions"}],"predecessor-version":[{"id":9853,"href":"https:\/\/jianmingli.com\/wp\/index.php?rest_route=\/wp\/v2\/posts\/8514\/revisions\/9853"}],"wp:attachment":[{"href":"https:\/\/jianmingli.com\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=8514"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jianmingli.com\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=8514"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jianmingli.com\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=8514"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}