天天干在线观看,狠狠噜噜久久,中文在线视频,成人久久久久久

在Oracle Clusterware需要從集群中逐出特定節點或一組節點的情況時，基于服務器權重的節點驅逐可充當斷裂機制，通過服務器上的負載附加信息來識別要驅逐的節點或節點組。其中所有節點被驅逐的可能性是一樣的。

本課程視頻，點擊閱讀原文即可下載

基于權重的集群驅逐（Server Weight-Based Node Eviction）是一項Oracle 12.2版本引入的一項新特性。在此之前，Oracle集群在處理腦裂問題時，通過判斷子群(各自獨立)的規模情況，來決定在腦裂問題發生時，終止哪個子群，判斷的依據如下：

• 如果子群規模（包含節點數量）不等，則大的子群獲勝，例如，{1}  {2,3,4}后者獲勝，子群{1}被驅逐；

• 如果子群規模（包含節點數量）相等，則包含最小節點號的子群獲勝，例如,{1,4}{2,3}, 子群{1,4}獲勝，子群{2,3}被驅逐。

而在Oracle 12.2版本上引入的這項新特性，一定程度上增加了我們對于集群的控制。這種控制使得我們能避免由于一些規則原來的限定，而減少對大范圍的應用產生影響。

注意：

以下針對weight以及權重表述的是同一個意思。

使用前提條件

• 權重(Weight)分配只能在admin-managed的節點生效

• 可以對于server或者對于注冊到集群的應用進行權重(Weight)分配

使用說明

• 權重分配給server

使用crsctl set server css_critical yes命令

• 權重分配給數據庫實例或服務

在srvctl add/modify databse 或 srvctladd/modify service命令中帶上“-css_critical yes”參數

• 權重分配給非ora.*資源

在crsctl add/modify resource命令中，加入-attr "CSS_CRITICAL=yes"參數

注意：

有些權重的分配需要重啟集群才能生效，而有些資源的分配，則不需要重啟資源。

根據目前發現，非ora.*資源并非所有都能直接加入屬性直接修改，但是屬性中可以看到有CSS_CRITICAL屬性，可能是當前版本未開發修改接口。

實驗環境說明

• 使用軟件環境說明：

OS:MacOS 10.11.6

VirtualBox:v5.1.30 r118389 (Qt5.6.3)

• 虛擬機環境說明：

OEL(OracleEnterprise Linux) 6.5 , x86_64

Oracle 12.2.0.1 (2-node RAC)

實驗前的準備工作

VirtualBox虛擬機中的網卡情況如下：

其中網卡1為Public網絡，網卡2為心跳網絡分別對應OEL6.5中的eth0網卡和eth1網卡。

為了模擬心跳網絡中斷，我們在系統中編寫了如下兩個腳本去模擬心跳網絡中斷以及恢復：

Milo-Mac:lab milo$ ls -l

total 16

-rwxr--r--  1 milo  staff  111  1 24 21:38 interconnect_down.sh

-rwxr--r--  1 milo  staff  109  1 24 21:39 interconnect_up.sh

Milo-Mac:lab milo$ 

#### 模擬心跳網絡中斷：

Milo-Mac:lab milo$ sh interconnect_down.sh 

查看虛擬機的網卡的連通情況：

[root@rac122a ~]# mii-tool eth0

eth0: no autonegotiation, 100baseTx-FD, link ok

[root@rac122a ~]# mii-tool eth1

eth1: autonegotiation restarted, no link

[root@rac122b ~]# mii-tool eth0

eth0: no autonegotiation, 100baseTx-FD, link ok

[root@rac122b ~]# mii-tool eth1

eth1: autonegotiation restarted, no link

可以看到在腳本運行后，私網網卡eth1顯示為no link表示網線沒有連接到網卡中，即我們認為心跳網絡出現故障。

Milo-Mac:lab milo$ ls -l

total 16

-rwxr--r--  1 milo  staff  111  1 24 21:38 interconnect_down.sh

-rwxr--r--  1 milo  staff  109  1 24 21:39 interconnect_up.sh

Milo-Mac:lab milo$ 

#### 恢復心跳網絡：

查看虛擬機的網卡的連通情況：

[root@rac122a ~]# mii-tool eth0

eth0: no autonegotiation, 100baseTx-FD, link ok

[root@rac122a ~]# mii-tool eth1

eth1: no autonegotiation, 100baseTx-FD, link ok

[root@rac122b ~]# mii-tool eth0

eth0: no autonegotiation, 100baseTx-FD, link ok

[root@rac122b ~]# mii-tool eth1

可以看到在腳本運行后，私網網卡eth1顯示為link ok表示網線連接到網卡中，即我們認為心跳網絡恢復，當然實際情況是我們還需要重新禁用和啟用eth1網卡，才能使其完全恢復，因此需要在兩個節點都執行以下腳本：

[root@rac122a ~]# sh recover_interconnect.sh

Device state: 3 (disconnected)

Active connection state: activated

上述相關腳本如下：

Milo-Mac:lab milo$ cat interconnect_down.sh 

VBoxManage controlvm "12c_rac_node1" setlinkstate2 off

VBoxManage controlvm "12c_rac_node2" setlinkstate2 off

Milo-Mac:lab milo$ cat interconnect_up.sh 

VBoxManage controlvm "12c_rac_node1" setlinkstate2 on

VBoxManage controlvm "12c_rac_node2" setlinkstate2 on

[root@rac122a ~]# cat recover_interconnect.sh

ifdown eth1 && ifup eth1

經過上述測試，我們已經可以模擬心跳網絡故障以及恢復心跳網絡故障。

以下測試場景不再贅述，上述故障模擬以及故障恢復過程。

測試場景

[oracle@rac122a ~]$ crsctl get server css_critical

CRS-5092: Current value of the server attribute CSS_CRITICAL is no.

[oracle@rac122b ~]$ crsctl get server css_critical

CRS-5092: Current value of the server attribute CSS_CRITICAL is no.

 當前系統上存在的服務也為設置：

[oracle@rac122b ~]$ crsctl stat res ora.milodb.milodb_srv1.svc -f | grep CSS

CSS_CRITICAL=no

[oracle@rac122b ~]$ crsctl stat res ora.milodb.milodb_srv2.svc -f | grep CSS

CSS_CRITICAL=no

模擬心跳網絡故障

Milo-Mac:lab milo$ sh interconnect_down.sh

節點1集群的alert日志：

2018-01-25 08:17:33.423 [OCSSD(3896)]CRS-1612: Network communication with node rac122b (2) missing for 50% of timeout interval.  Removal of this node from cluster in 14.610 seconds

2018-01-25 08:17:41.435 [OCSSD(3896)]CRS-1611: Network communication with node rac122b (2) missing for 75% of timeout interval.  Removal of this node from cluster in 6.600 seconds

2018-01-25 08:17:45.438 [OCSSD(3896)]CRS-1610: Network communication with node rac122b (2) missing for 90% of timeout interval.  Removal of this node from cluster in 2.600 seconds

2018-01-25 08:17:49.554 [OCSSD(3896)]CRS-1607: Node rac122b is being evicted  in cluster incarnation 412297497; details at (:CSSNM00007:) in /u01/app/grid/diag/crs/rac122a/crs/trace/ocssd.trc.

節點2集群的alert日志：

2018-01-25 08:17:34.128 [OCSSD(25756)]CRS-1612: Network communication with node rac122a (1) missing for 50% of timeout interval.  Removal of this node from cluster in 14.450 seconds

2018-01-25 08:17:41.140 [OCSSD(25756)]CRS-1611: Network communication with node rac122a (1) missing for 75% of timeout interval.  Removal of this node from cluster in 7.440 seconds

2018-01-25 08:17:46.193 [OCSSD(25756)]CRS-1610: Network communication with node rac122a (1) missing for 90% of timeout interval.  Removal of this node from cluster in 2.380 seconds

…省略部分信息

從上述信息來看，根據之前的集群腦裂出現的規則，在同等規模的子群中，節點1節點號小，因而存活，節點2被驅逐出集群。

將心跳網絡恢復以及集群狀態恢復正常后，我們將對server級別的權重進行設置。

[root@rac122a ~]# crsctl get server css_critical

CRS-5092: Current value of the server attribute CSS_CRITICAL is no.

[root@rac122b ~]# crsctl get server css_critical

CRS-5092: Current value of the server attribute CSS_CRITICAL is no.

[root@rac122b ~]# crsctl set server css_critical yes

CRS-4416: Server attribute 'CSS_CRITICAL' successfully changed. Restart Oracle High Availability Services for new value to take effect. 

設置完成后，提示需要重啟OHAS才能生效。

$ srvctl stop instance -d milodb -i milodb2

# crsctl stop crs

# crsctl start crs

模擬網路故障：

Milo-Mac:lab milo$ sh interconnect_down.sh

此時，我們來觀察集群的一些日志情況：

節點1集群的alert日志：

2018-01-25 09:44:03.671 [OCSSD(3717)]CRS-1612: Network communication with node rac122b (2) missing for 50% of timeout interval.  Removal of this node from cluster in 14.890 seconds

2018-01-25 09:44:11.731 [OCSSD(3717)]CRS-1611: Network communication with node rac122b (2) missing for 75% of timeout interval.  Removal of this node from cluster in 6.830 seconds

2018-01-25 09:44:15.739 [OCSSD(3717)]CRS-1610: Network communication with node rac122b (2) missing for 90% of timeout interval.  Removal of this node from cluster in 2.830 seconds

2018-01-25 09:44:18.573 [OCSSD(3717)]CRS-1609: This node is unable to communicate with other nodes in the cluster and is going down to preserve cluster integrity; details at (:CSSNM00008:) in /u01/app/grid/diag/crs/rac122a/crs/trace/ocssd.trc.

2018-01-25 09:44:18.573 [OCSSD(3717)]CRS-1656: The CSS daemon is terminating due to a fatal error; Details at (:CSSSC00012:) in /u01/app/grid/diag/crs/rac122a/crs/trace/ocssd.trc

2018-01-25 09:44:18.616 [OCSSD(3717)]CRS-1652: Starting clean up of CRSD resources.

2018-01-25 09:44:20.608 [OCSSD(3717)]CRS-1608: This node was evicted by node 2, rac122b; details at (:CSSNM00005:) in /u01/app/grid/diag/crs/rac122a/crs/trace/ocssd.trc.

節點2集群的alert日志：

2018-01-25 09:44:04.327 [OCSSD(8586)]CRS-1612: Network communication with node rac122a (1) missing for 50% of timeout interval.  Removal of this node from cluster in 14.020 seconds

2018-01-25 09:44:11.609 [OCSSD(8586)]CRS-1611: Network communication with node rac122a (1) missing for 75% of timeout interval.  Removal of this node from cluster in 6.730 seconds

2018-01-25 09:44:15.611 [OCSSD(8586)]CRS-1610: Network communication with node rac122a (1) missing for 90% of timeout interval.  Removal of this node from cluster in 2.730 seconds

2018-01-25 09:44:19.777 [OCSSD(8586)]CRS-1607: Node rac122a is being evicted in cluster incarnation 412336033; details at (:CSSNM00007:) in /u01/app/grid/diag/crs/rac122b/crs/trace/ocssd.trc.

此時，我們從集群的alert日志中看到由于節點1被集群驅逐了。

通過從節點2的ocssd.trc日志我們看到如下信息：

[grid@rac122b trace]$ cat ocssd.trc |egrep -i weight| tail -300

……省略部分信息

2018-01-25 09:44:18.347 :    CSSD:1494243072: clssnmrCheckNodeWeight: node(1) has weight stamp(0) pebbles (0) goldstars (0) flags (0) SpoolVersion (0)

2018-01-25 09:44:18.347 :    CSSD:1494243072: clssnmrCheckNodeWeight: node(2) has weight stamp(412336032) pebbles (0) goldstars (1) flags (3) SpoolVersion (0)

2018-01-25 09:44:18.347 :    CSSD:1494243072: clssnmrCheckNodeWeight: Server pool version not consistent

2018-01-25 09:44:18.347 :    CSSD:1494243072: clssnmrCheckNodeWeight: stamp(412336032), completed(1/2)

2018-01-25 09:44:18.347 :    CSSD:1494243072: clssnmrCheckSplit: Waiting for node weights, stamp(412336032)

2018-01-25 09:44:19.777 :    CSSD:1494243072: clssnmrCheckNodeWeight: node(2) has weight stamp(412336032) pebbles (0) goldstars (1) flags (3) SpoolVersion (0)

2018-01-25 09:44:19.777 :    CSSD:1494243072: clssnmrCheckNodeWeight: Server pool version not consistent

2018-01-25 09:44:19.777 :    CSSD:1494243072: clssnmrCheckNodeWeight: stamp(412336032), completed(1/1)

2018-01-25 09:44:19.777 :    CSSD:1494243072: clssnmCompareNodeWeights: Best map is same as the cohort map of the current node

2018-01-25 09:44:19.777 :    CSSD:1494243072: clssnmFindBestMap: Using base map(2) of node(1) count(0), low(65535), bestcount(0), best_low(65535), cur_weightpebbles (0) goldstars (0) flags (0) SpoolVersion (0)best_weightpebbles (0) goldstars (0) flags (0) SpoolVersion (0)

2018-01-25 09:44:19.777 :    CSSD:1494243072: clssnmCompareNodeWeights: count(1), low(2), bestcount(0), best_low(65535), cur_weight: pebbles(0) goldstars(1) pubnw(1) flexasm(1)best_weight: pebbles(0) goldstars(0)pubnw(0) flexasm(0)

從上述信息來看，CSSD進程會去檢查節點的權重情況(CheckNodeWeight)以及比較節點的權重（CompareNodeWeights）節點2的權重大，此時，節點2這個子群戰勝了節點1子群，因而我們看到的情況是節點1被集群驅逐。

將心跳網絡恢復以及集群狀態恢復正常后，我們將對service級別的權重進行設置。

恢復集群server的權重：

[root@rac122b ~]# crsctl get server css_critical

CRS-5092: Current value of the server attribute CSS_CRITICAL is yes.

[root@rac122b ~]# crsctl set server css_critical no

CRS-4416: Server attribute 'CSS_CRITICAL' successfully changed. Restart Oracle High Availability Services for new value to take effect.

[root@rac122b ~]#  crsctl get server css_critical

CRS-5092: Current value of the server attribute CSS_CRITICAL is no.

[root@rac122b ~]#

設置完成后，提示需要重啟OHAS才能生效。

$ srvctl stop instance -d milodb -i milodb2

# crsctl stop crs

# crsctl start crs

這里我們添加兩個專門用于做權重控制的服務（不對外使用）：

srvctl add service -db milodb -service milodb_wt_srv1 -preferred milodb1

srvctl add service -db milodb -service milodb_wt_srv2 -preferred milodb2

srvctl start service -database milodb -service milodb_wt_srv1

srvctl start service -database milodb -service milodb_wt_srv2

在實例1上的服務不設置權重：

[oracle@rac122b ~]$ crsctl stat res ora.milodb.milodb_wt_srv1.svc -f | grep CSS

CSS_CRITICAL=no

在實例2上的服務設置權重：

[oracle@rac122b ~]$ crsctl stat res ora.milodb.milodb_wt_srv2.svc -f | grep CSS

CSS_CRITICAL=no

[oracle@rac122b ~]$ srvctl modify service -db milodb -service milodb_wt_srv2 -css_critical yes

[oracle@rac122b ~]$ crsctl stat res ora.milodb.milodb_wt_srv2.svc -f | grep CSS

CSS_CRITICAL=yes

[oracle@rac122b ~]$

此時，我們來觀察集群的一些日志情況：

節點1集群的alert日志：

2018-01-25 11:33:30.343 [OCSSD(3652)]CRS-1612: Network communication with node rac122b (2) missing for 50% of timeout interval.  Removal of this node from cluster in 14.910 seconds

2018-01-25 11:33:38.351 [OCSSD(3652)]CRS-1611: Network communication with node rac122b (2) missing for 75% of timeout interval.  Removal of this node from cluster in 6.900 seconds

2018-01-25 11:33:42.576 [OCSSD(3652)]CRS-1610: Network communication with node rac122b (2) missing for 90% of timeout interval.  Removal of this node from cluster in 2.680 seconds

2018-01-25 11:33:45.255 [OCSSD(3652)]CRS-1609: This node is unable to communicate with other nodes in the cluster and is going down to preserve cluster integrity; details at (:CSSNM00008:) in /u01/app/grid/diag/crs/rac122a/crs/trace/ocssd.trc.

2018-01-25 11:33:45.255 [OCSSD(3652)]CRS-1656: The CSS daemon is terminating due to a fatal error; Details at (:CSSSC00012:) in /u01/app/grid/diag/crs/rac122a/crs/trace/ocssd.trc

2018-01-25 11:33:45.269 [OCSSD(3652)]CRS-1652: Starting clean up of CRSD resources.

2018-01-25 11:33:47.289 [OCSSD(3652)]CRS-1608: This node was evicted by node 2, rac122b; details at (:CSSNM00005:) in /u01/app/grid/diag/crs/rac122a/crs/trace/ocssd.trc.

節點2集群的alert日志：

2018-01-25 11:33:29.844 [OCSSD(3983)]CRS-1612: Network communication with node rac122a (1) missing for 50% of timeout interval.  Removal of this node from cluster in 14.900 seconds

2018-01-25 11:33:37.946 [OCSSD(3983)]CRS-1611: Network communication with node rac122a (1) missing for 75% of timeout interval.  Removal of this node from cluster in 6.790 seconds

2018-01-25 11:33:41.947 [OCSSD(3983)]CRS-1610: Network communication with node rac122a (1) missing for 90% of timeout interval.  Removal of this node from cluster in 2.790 seconds

2018-01-25 11:33:46.684 [OCSSD(3983)]CRS-1607: Node rac122a is being evicted in cluster incarnation 412345406; details at (:CSSNM00007:) in /u01/app/grid/diag/crs/rac122b/crs/trace/ocssd.trc.

2018-01-25 11:33:47.755 [ORAAGENT(5634)]CRS-5818: Aborted command 'check' for resource 'ora.SYSTEMDG.dg'. Details at (:CRSAGF00113:) {0:1:11} in /u01/app/grid/diag/crs/rac122b/crs/trace/crsd_oraagent_grid.trc.

2018-01-25 11:33:47.759 [ORAAGENT(5634)]CRS-5818: Aborted command 'check' for resource 'ora.DATADG.dg'. Details at (:CRSAGF00113:) {0:1:11} in /u01/app/grid/diag/crs/rac122b/crs/trace/crsd_oraagent_grid.trc.

2018-01-25 11:33:49.250 [OCSSD(3983)]CRS-1601: CSSD Reconfiguration complete. Active nodes are rac122b .

2018-01-25 11:33:49.617 [CRSD(5349)]CRS-5504: Node down event reported for node 'rac122a'.

從節點2的ocssd日志看到，同樣由于節點2的權重高，因而最終將節點1驅逐除了集群：

后續測試將兩個服務都設置權重時，情況與不設置權重的情況一致，即節點1驅逐節點2。

通過上述場景的測試，我們得知：

• 基于權重的節點驅逐可以在集群出現腦裂時，通過控制減少消除一些影響；

• 配置server方式的權重，需要修改配置后，重啟crs才能生效；

• 通過服務的方式配置節點的權重不需要重啟資源，可以實現動態控制，更靈活。

當然，由于在該特性是12.2引入的，屬于新特性，可能會存在一些bug，建議在使用前應該經過充分測試后使用。

作者介紹：

羅雪原，云和恩墨南區交付技術顧問。

個人有著8年的Oracle技術支持經驗，曾服務過金融、保險、電力、政府、運營商等客戶，有著較豐富的Troubleshooting以及優化經驗。