Thread: Changeing "Default Route" interface.

Hi,

I few days ago I ran into a shortcoming in the network configuration flexibility when using Kuzu to deploy/configure a cluster.

All the nodes in the cluster has 2x interfaces, eth0 and eth1.

eth0 - This is the cluster provisioning and private network.
eth1 - This interface is connected to the public LAN.

I want eth1 to be the interface to the default router. eth1 is defined by using netedit, and it is configured to use the dhcp server on my public LAN to have ip-address and other IP options assigned.

A drawing of the seup:

Code:

===============================================================
                      Ethernet Switches.                    
         Configured for Windows VLAN 1, and Linux VLAN 2.
===============================================================
        ^                       ^          ^ ^ ^          ^
      V |                     V |        V | | |        V |
      L |                     L |        L | | |        L |
      A |                     A |        A | | |        A |
      N |                     N |        N | | |        N |
        |                       |          | | |          |
      1 |                     2 |        2 | | |        1 |
        |                       |          | | |          |
  ______|eth0              _____|eth0   __________  __________ 
 [_..._....°]             [_..._....°] [_|||||||_°][_|||||||_°]
 [_..._....°]             [_..._....°] [_|||||||_°][_|||||||_°]
 [_..._....°]             [_..._....°] [_|||||||_°][_|||||||_°]
 [_..._....°]             [_..._....°]      NAS Filer
 [_|||||||_°]             [_|||||||_°]                     
 [_|||||||_°]             [_|||||||_°]\
 [_|||||||_°]             [_|||||||_°] \
 [_________°]             [_________°]  \
 [_________°]eth1     eth1[_________°]   \
 [_________°]\           /[_________°]  __v_________   
 [___....__°] \         / [___....__°] |_ooooooooo_°|  
    Rack1      \       /     Rack2       Infiniband
Windows Nodes   \     /   Linux Nodes      Switch  
                 v   v
              .-,(  ),-.    
           .-(          )-. 
          (   Public LAN   )
           '-(          ).-'
               '-.( ).-'    
===============================================================

My Problem:

It is possible to remove "Gateway" from the provisioning network on the kuzu (OCS 5.3) installer, but after doing this and then running addhost dhcpd.conf is broken, and dhcpd will not start.

The problem seem to be that setting "empty" gateway in netedit will leave the string "option routers ;" in dhcpd.conf. Normally this would be "option routers 10.10.10.1;". When the option exist, but no IP address is entered dhcpd will not start.


To solve the problem I ended up writing an init script that I distribute to all nodes via cfm. This script will change the routeing table for compute nodes as they boot up. This is _NOT_ the optimal solution...


IMO it is a design flaw that you cannot simply select what interface to get default gateway information from in netedit.

It is possible to remove "Gateway" from the provisioning network on the kuzu (OCS 5.3) installer, but after doing this and then running addhost dhcpd.conf is broken, and dhcpd will not start.

Is this during the installation screen during installation?

-Liming

Kinda ran into similar problem. I have an installer node and the default gateway is the public vlan, node has 4 NIC ports. Added a compute node with 4 NIC ports on it and defined it in the lsf master list. This node is also a public login node, but kusu designed a default gateway back to installer node. Ended up removeing the route and adding my public route via /etc/rc.local. Also not so cool but it works.

The other suggestion I have, it that in netedit, it would be nice to be able to add an eth specification of a single IP, that with increment value of 0. On our public vlan IPs are hardcoded and in inventory, so you just can not pick the next public IP for the next login node.

-Henk

-Henk

Henk,

Thanks for your feedback.

Certainly, we will want to improve how network settings are being applied on compute node. We do see some deficiencies in Kusu especially setting in public ip (whether dhcp or static) on other interfaces on the compute nodes.

Also, if you are on support contract with OCS/PCM, raise a ticket to the support folks to express interest in getting this fixed. This will greatly help speed up the process.

-Liming