Our approach is to dynamically distribute requests based on current system state information. All servers in cluster report state information to front node and front node uses this information to distribute individual client requests (each TCP connection) coming to cluster among servers in cluster intelligently. Front node reports aggregated cluster load information to DNS like a single node of high capacity. Which once again uses this information to resolve IP address of a cluster for queries from client to provide them better service in terms of perceived delay. Collecting only server state information is not sufficient, servers also collect number of requests coming from each IP address and send to front node which aggregates this information and reports IP addresses of clients having very high request rate to DNS. It has been found that more than 75% requests can come from 10% of networks. Using request rate information, DNS asks few clusters to collect cluster to client proximity information only for those clients. Cluster to client proximity is found by sending ICMP Echo request messages to clients. Additional messages sent among entities are shown in Figure 3.2.
For client request distribution inside each cluster, IP packet forwarding by changing destination IP address of request packets only can be used as shown in Figure 3.3. Every IP packet that reaches at front node for HTTP connection is diverted at IP layer before delivery to TCP layer. A program running at the front node selects server for this client based on client IP address and server load information, address of selected server is filled in destination address field of IP packet and packet is re-injected back on network, so it reaches the selected server. At server this packet is once again diverted and destination address is set back to IP address of front node and is re-injected in TCP/IP stack of server node. Each server has secondary IP address (ARP disabled) same as IP address of front node, so HTTP server accepts this packet and response packets directly go to client from selected server without doing any additional modification or delay. This results in additional delay of about one millisecond for each incoming packet, if servers and front node are on same LAN. Since this packet forwarding can be done at application layer, it was chosen for emulation, however in actual system, packet forwarding inside kernel using MAC address can be done or dedicated hardware can be used for more efficient dispatching.
