VCS Troubleshooting

Linux Log Files

/var/log/messages : General message and system related stuff
/var/log/auth.log : Authenication logs
/var/log/kern.log : Kernel logs
/var/log/cron.log : Crond logs (cron job)
/var/log/maillog : Mail server logs
/var/log/qmail/ : Qmail log directory (more files inside this directory)
/var/log/httpd/ : Apache access and error logs directory
/var/log/lighttpd/ : Lighttpd access and error logs directory
/var/log/boot.log : System boot log
/var/log/mysqld.log : MySQL database server log file
/var/log/secure or /var/log/auth.log : Authentication log
/var/log/utmp or /var/log/wtmp : Login records file
/var/log/yum.log : Yum command log file.

LLT Port Information

[root@vcs1 /]# lltstat -p
LLT port information:
    Port    Usage        Cookie
      0     GAB_Control  0x0
          opens:     0 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
          connects:  0 1
      7     gab          0x7
          opens:     0 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
          connects:  0 1
     31     gab          0x1F
          opens:     0 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
          connects:  0 1
[root@vcs1 /]#


[root@vcs2 /]# lltstat -p
LLT port information:
    Port    Usage        Cookie
      0     GAB_Control  0x0
          opens:     1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
          connects:  0 1
      7     gab          0x7
          opens:     1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
          connects:  0 1
     31     gab          0x1F
          opens:     1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
          connects:  0 1
[root@vcs2 /]#

So the LLT is configured on 3 links running on the port 0, 7 and 31.

Add a link to LLT

[root@vcs1 /]# lltconfig -t eth2 -d eth2
LLT lltconfig INFO V-14-2-15653 No link type is specified, defaulting to link type "ether"
[root@vcs1 /]# lltconfig -a list
Link 0 (eth0):
  Node   0 vcs1       :   00:0C:29:DE:BA:35  permanent
  Node   1 vcs2       :   00:0C:29:52:9B:FB

Link 1 (eth1):
  Node   0 vcs1       :   00:0C:29:DE:BA:3F  permanent
  Node   1 vcs2       :   00:0C:29:52:9B:05

Link 2 (eth2):
  Node   0 vcs1       :   00:0C:29:DE:BA:49  permanent

[root@vcs1 /]#

[root@vcs2 /]# lltconfig -t eth2 -d eth2
LLT lltconfig INFO V-14-2-15653 No link type is specified, defaulting to link type "ether"
[root@vcs2 /]#

[root@vcs1 /]# lltconfig -a list
Link 0 (eth0):
  Node   0 vcs1       :   00:0C:29:DE:BA:35  permanent
  Node   1 vcs2       :   00:0C:29:52:9B:FB

Link 1 (eth1):
  Node   0 vcs1       :   00:0C:29:DE:BA:3F  permanent
  Node   1 vcs2       :   00:0C:29:52:9B:05

Link 2 (eth2):
  Node   0 vcs1       :   00:0C:29:DE:BA:49  permanent
  Node   1 vcs2       :   00:0C:29:52:9B:0F

[root@vcs1 /]#

Removing a link from LLT

[root@vcs1 /]# lltconfig -u eth2----------> Command to remove a link from LLT Layer. (Need to execute the command on both the nodes)

[root@vcs2 /]# lltconfig -u eth2

This command unconfigures the LLT link, but does not permanently remove the link from configuration files.

Upon reboot, or restart of LLT, the cluster node will use the link again.

To remove the link permanently from the configuration, on each cluster node modify the /etc/llttab file and remove the line for device "eth2".

[root@vcs1 /]# lltconfig -a list
Link 0 (eth0):
  Node   0 vcs1       :   00:0C:29:DE:BA:35  permanent
  Node   1 vcs2       :   00:0C:29:52:9B:FB

Link 1 (eth1):
  Node   0 vcs1       :   00:0C:29:DE:BA:3F  permanent
  Node   1 vcs2       :   00:0C:29:52:9B:05

[root@vcs1 /]#


LLT node information:
    Node                 State    Link  Status  Address
   * 0 vcs1              OPEN
                                  eth0   UP      00:0C:29:DE:BA:35
                                  eth1   UP      00:0C:29:DE:BA:3F
     1 vcs2              OPEN
                                  eth0   UP      00:0C:29:52:9B:FB
                                  eth1   UP      00:0C:29:52:9B:05


[root@vcs1 /]# lltstat -l
LLT link information:
link 0  eth0 on ether hipri
        mtu 1500, sap 0xcafe, broadcast FF:FF:FF:FF:FF:FF, addrlen 6
        txpkts 3429  txbytes 370955
        rxpkts 3180  rxbytes 232432
        latehb 0  badcksum 0  errors 0
link 1  eth1 on ether hipri
        mtu 1500, sap 0xcafe, broadcast FF:FF:FF:FF:FF:FF, addrlen 6
        txpkts 3450  txbytes 349877
        rxpkts 3216  rxbytes 227532
        latehb 0  badcksum 0  errors 0
[root@vcs1 /]#