WebLogic 11g Clustering: Whole Server Migration

* During a whole server migration, a migratable server instance, and all of its services, is migrated to a different physical machine upon failure.
* Provides high availability for “pinned services” such as JMS and JTA transaction recovery system.
* Supports both manual and automatic migration at server level.

* Migratable server:
– A clustered server instance that can be migrated as a whole.
– Intended to hos pinned services such as JMS servers and JTA transaction recovery servers.
* Whole server migration: a WebLogic Server instance to be migrated to a different physical machine upon failure, either manually or automatically.
* Service migration:
– Manual service migration.
– Automatic service migration.
* Cluster leader: one server instance in a cluster, elected by a majority of the servers, that is responsible for maintaining the leasing information.
* Cluster master:
– One server instance in a cluster that contains migratable servers acts as the cluster master and orchestrates the process of automatic server migration, in the event of failure.
– Any Managed Server in a cluster can serve as the cluster master, whether it hosts pinned services or not.
* Singleton master:
– A lightweight singleton service that monitors other services that can be migrated automatically.
– The server that currently hosts the singleton master is responsible for starting and stopping the migration tasks associated with each migratable service.
* Candidate machines: a user-defined list of machines within a cluster that can be a potential target for migration.
* Target mahcines: a set of machines that are designated as allowable or preferred hosts for migratable servers.
* Node manager
* Lease table:
– A database table in which migratable servers persist their state,
– Cluster master monitors it to verify the health and liveness of migratable servers.
* Administration Server: Used to
– configure migratable servers and target machines,
– obtain the run-time state of migratable servers,
– orchestrate the manual migration process.
* Floating IP address: an IP address that follows a server from one physical machine to another after migration.

* Issues periodic heartbeats to the other servers in the cluster.
* Periodically reads the lease table to verify that each migratable server has a current lease.
– An expired lease indicates the migratable server should be restarted.
* Upon determining that a migratable server’s lease is expired, waits for period specified by the FencingGracePeriodMillis on the ClusterMBean, and then tries to invoke the Node Manager process on the machine that hosts the migratable server whose lease is expired, to restart the migratable server.
* If unable to retsart a migratable server, cluster master selects a target machine:
– chooses a machine on a list of preferred destination machines for the migratable server, in order.
– otherwise, chooses a mahcine on the list of those configured as available for hosting migratable servers in the cluster.
– A list of machines that can host migratable servers can be configured at two levels: for the cluster as a whole, and for an individual migratable server. You can define a machine list at both levels. You must define a machine list on at least one level.
*To accomplish the migration of a server instance to a new machine, the cluster master invokes the Node Manager process on the target machine to create a process for the server instance.
* The time required to perform the migration depends on the server configuration and startup time.
– The maximum time taken for the cluster master to restart the migratable server is (HealthCheckPeriodsUntilFencing * HealthCheckIntervalMillis) + FencingGracePeriodMillis.
– The total time before the server becomes available for client requests depends on the server startup time and the application deployment time.

* Leasing is the process WebLogic Server uses to manage services that are required to run on only one member of a cluster at a time.
* Leasing ensures exclusive ownership of a cluster-wide entity.
* Within a cluster, there is a single owner of a lease.
* Additionally, leases can failover in case of server or cluster failure. This helps to avoid having a single point of failure.

* Automatic Whole Server Migration:
– uses leasing to elect a cluster master.
– ensures cluster master is always running on one and only one server at a time within a cluster.
* Automatic Service Migration:
– Migratable targets use leasing to accomplish automatic service migration
* Singleton Services:
– uses leasing to make sure singleton service is available on only one cluster member at a time.
* Job Scheduler:
– The Job Scheduler is a persistent timer that is used with in a cluster.
– The Job Scheduler uses the timer master to load balance the timer across a cluster.

* High-availability database leasing.
– Requires a high-availability database to store leasing information.
– No need to use Node Manager to manage servers.
* Non-database consensus leasing.
– Stores leasing information in memory within a cluster member.
– Requires all servers in the cluster are started by Node Manager.

* Verify that whole server migration is supported on your platform. Check here.
* Each Managed Server uses the same subnet mask (so unicat and multicast communication among servers can work).
* All servers hosting migratable servers are time-synchronized.
* All OS versions support identical functionality for ifconfig.
* The primary interface names used by migratable servers are the same.
* Verify your database support automatic server migration.
* You cannot create channels/network access points that have a different listen address on a migratable server.
* There is no built-in mechanism for transferring files that a server depends on between machines. Using a disk that is accessible from all machines is the preferred way to ensure file availability. If you cannot share disks between servers, you must ensure that the contents of domain_dir/bin are copied to each machine.
* Ensure that the Node Manager security files are copied to each machine using the nmEnroll() WLST command.
* Use high availability storage for state data (e.g. SAN).

* Obtain floating IP addresses for each Managed Server that will have migration enabled.
– Also set AutoMigrationEnabled to true.
* Configure Node Manager. See here for more details.
* If using database leasing,
– Configure database.
– Run leasing.ddl script to reset leasing table in dev environment when needed.
– Setup and migrate data source. Set DataSourceForAutomaticMigration to this data source in each cluster configuration.
* Grant superuser privileges to the following scripts in the WL_HOME/common/bin directory (the scripts are used to transfer IP addresses from one machine to another during migration)
– wlsifconfig.sh script (on UNIX)
– or the wlsifconfig.cmd script (on Windows)
* Put in machine PATH:
– WL_HOME/common/bin/wlsifconfig.sh (UNIX) or wlsifconfig.cmd (Windows)
– WL_HOME/common/bin/wlscontrol.sh (UNIX)
– WL_HOME/common/nodemanager/nodemanager.domains
* If using SSH version of Node Manager on Unix:
– Setup SSH trust among migratable servers.
* Set the candidate machines for server migration. Each server can have a different set of candidate machines, or they can all have the same set.
* Restart the Administration Server.

* Whole Server Migration