조회 수 17688 추천 수 0 댓글 0
Atachment
첨부 '1'

[테스트환경]

  Host OS : WIndows XP SP3 32bit / RAM 3GB

  Guest OS : Redhat Enterprise Linux 5 32bit / HDD 각 25GB씩 / RAM 각 1GB씩

  DB : Oracle Database 11gR2 Enterprise Two-Node RAC

 

 

[IP Setting Information - /etc/hosts]

  # Loopback

  127.0.0.1          localhost
  # Public Network - (eth0)
  192.168.52.10    rac1
  192.168.52.20    rac2
  # Private Interconnect - (eth1)
  192.168.121.10   rac1-priv
  192.168.121.20   rac2-priv
  # Public Virtual IP (VIP) addresses for - (eth0)
  192.168.52.110   rac1-vip
  192.168.52.120   rac2-vip
  # SCAN IP
  192.168.52.130   scan-ip

 

 

[ASM Shared Disks Configuration Information]

  파티션명 : /dev/sdb1

  ASM디스크명 : DATA

  용량 : 12GB

 

  파티션명 : /dev/sdc1

  ASM명 : OCR_VOTE

  용량 : 3GB

 

 

[상황]

오라클 설치 매뉴얼에는 11gR2를 설치하기 위한 최소 램이 1.5G 이상이어야 하나 현재 노트북이 3GB의 램이라서 할 수없이 1G씩을 할당해줌.

클러스터웨어 설치 전 사전검사(cluvfy.sh) 실시했을 때 램과 스왑공간 검사 이외에는 모두 성공함.(패키지, 커널파라미터 등)

각 노드끼리의 ping도 서로 잘 전달됨을 확인.

클러스터 설치 100% 후 root.sh 스크립트 실행 시 노드1은 성공적으로 스크립트 수행을 완료하였으나, 노드2에서 실패함.

 

 

[스크린샷]

rac.bmp

 

 

[노드1의 root.sh 전문]

[root@rac1 ~]# /home/oraInventory/orainstRoot.sh
Changing permissions of /home/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /home/oraInventory to dba.
The execution of the script is complete.
[root@rac1 ~]#
[root@rac1 ~]# /home/11.2/grid/root.sh
Running Oracle 11g root.sh script...

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /home/11.2/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2010-12-10 01:34:15: Parsing the host name
2010-12-10 01:34:15: Checking for super user privileges
2010-12-10 01:34:15: User has super user privileges
Using configuration parameter file: /home/11.2/grid/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
  root wallet
  root wallet cert
  root cert export
  peer wallet
  profile reader wallet
  pa wallet
  peer wallet keys
  pa wallet keys
  peer cert request
  pa cert request
  peer cert
  pa cert
  peer root cert TP
  profile reader root cert TP
  pa root cert TP
  peer pa cert TP
  pa peer cert TP
  profile reader pa cert TP
  profile reader peer cert TP
  peer user cert
  pa user cert
Adding daemon to inittab
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded

ASM이 성공적으로 생성되고 시작되었습니다.

디스크 그룹 OCR_VOTE이(가) 성공적으로 생성되었습니다.

clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded
CRS-4256: Updating the profile
Successful addition of voting disk 2786b33cd8f24f4abf2c96b42518255a.
Successfully replaced voting disk group with +OCR_VOTE.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   2786b33cd8f24f4abf2c96b42518255a (ORCL:OCR_VOTE) [OCR_VOTE]
Located 1 voting disk(s).
CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'
CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'
CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'rac1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'
CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'
CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'
CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'
CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac1'
CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'rac1'
CRS-2676: Start of 'ora.evmd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac1'
CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.OCR_VOTE.dg' on 'rac1'
CRS-2676: Start of 'ora.OCR_VOTE.dg' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.registry.acfs' on 'rac1'
CRS-2676: Start of 'ora.registry.acfs' on 'rac1' succeeded

rac1     2010/12/10 01:53:20     /home/11.2/grid/cdata/rac1/backup_20101210_015320.olr
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
Updating inventory properties for clusterware
Oracle Universal Installer 시작 중...

스왑 공간 확인 중: 500MB 이상이어야 합니다..   실제 967MB    성공
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /home/oraInventory
'UpdateNodeList'을(를) 성공했습니다.
[root@rac1 ~]#

 

[노드2의 root.sh 전문]

[root@rac2 ~]# /home/oraInventory/orainstRoot.sh
Changing permissions of /home/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /home/oraInventory to dba.
The execution of the script is complete.
[root@rac2 ~]#
[root@rac2 ~]# /home/11.2/grid/root.sh
Running Oracle 11g root.sh script...

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /home/11.2/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2010-12-10 01:54:37: Parsing the host name
2010-12-10 01:54:37: Checking for super user privileges
2010-12-10 01:54:37: User has super user privileges
Using configuration parameter file: /home/11.2/grid/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Adding daemon to inittab
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node rac1, number 1, and is terminating
An active cluster was found during exclusive startup, restarting to join the cluster
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac2'
CRS-2676: Start of 'ora.mdnsd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'rac2'
CRS-2676: Start of 'ora.gipcd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac2'
CRS-2676: Start of 'ora.gpnpd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac2'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac2'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac2'
CRS-2676: Start of 'ora.diskmon' on 'rac2' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'rac2'
CRS-2676: Start of 'ora.ctssd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'rac2'
CRS-2676: Start of 'ora.drivers.acfs' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac2'
CRS-5011: Check of resource "+ASM" failed: details at "(:CLSN00006:)" in "/home/11.2/grid/log/rac2/agent/ohasd/oraagent_oracle/oraagent_oracle.log"
CRS-2674: Start of 'ora.asm' on 'rac2' failed
CRS-2679: Attempting to clean 'ora.asm' on 'rac2'
CRS-5011: Check of resource "+ASM" failed: details at "(:CLSN00006:)" in "/home/11.2/grid/log/rac2/agent/ohasd/oraagent_oracle/oraagent_oracle.log"
ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
Linux Error: 2: No such file or directory
Process ID: 0
Session ID: 0 Serial number: 0
CRS-5011: Check of resource "+ASM" failed: details at "(:CLSN00006:)" in "/home/11.2/grid/log/rac2/agent/ohasd/oraagent_oracle/oraagent_oracle.log"
CRS-2681: Clean of 'ora.asm' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac2'
CRS-2677: Stop of 'ora.drivers.acfs' on 'rac2' succeeded
CRS-4000: Command Start failed, or completed with errors.
Command return code of 1 (256) from command: /home/11.2/grid/bin/crsctl start resource ora.asm -init
Start of resource "ora.asm -init" failed
Failed to start ASM
Failed to start Oracle Clusterware stack
[root@rac2 ~]#

 

[고찰1]

노드2에서 ASM을 시작하려다가 failed 되었다는 메시지가 난 것 같아 각 노드에서의 ASM인스턴스가 띄워져있는지 조회하였음.

 

 

[노드1의 ASM인스턴스 상황]

[oracle@rac1 bin]$ ps -ef | grep ASM
oracle   13059     1  0 01:45 ?        00:00:02 asm_pmon_+ASM1
oracle   13061     1  0 01:45 ?        00:00:03 asm_vktm_+ASM1
oracle   13065     1  0 01:45 ?        00:00:00 asm_gen0_+ASM1
oracle   13067     1  0 01:45 ?        00:00:02 asm_diag_+ASM1
oracle   13069     1  0 01:45 ?        00:00:01 asm_ping_+ASM1
oracle   13071     1  0 01:45 ?        00:00:00 asm_psp0_+ASM1
oracle   13073     1  0 01:45 ?        00:00:16 asm_dia0_+ASM1
oracle   13075     1  0 01:45 ?        00:00:10 asm_lmon_+ASM1
oracle   13077     1  0 01:45 ?        00:00:07 asm_lmd0_+ASM1
oracle   13080     1  0 01:45 ?        00:00:01 asm_lms0_+ASM1
oracle   13084     1  0 01:45 ?        00:00:00 asm_lmhb_+ASM1
oracle   13086     1  0 01:45 ?        00:00:00 asm_mman_+ASM1
oracle   13088     1  0 01:45 ?        00:00:00 asm_dbw0_+ASM1
oracle   13090     1  0 01:45 ?        00:00:00 asm_lgwr_+ASM1
oracle   13092     1  0 01:45 ?        00:00:00 asm_ckpt_+ASM1
oracle   13094     1  0 01:45 ?        00:00:00 asm_smon_+ASM1
oracle   13096     1  0 01:45 ?        00:00:02 asm_rbal_+ASM1
oracle   13098     1  0 01:45 ?        00:00:02 asm_gmon_+ASM1
oracle   13100     1  0 01:45 ?        00:00:00 asm_mmon_+ASM1
oracle   13102     1  0 01:45 ?        00:00:00 asm_mmnl_+ASM1
oracle   13107     1  0 01:45 ?        00:00:00 asm_lck0_+ASM1
oracle   13156     1  0 01:46 ?        00:00:00 asm_asmb_+ASM1
oracle   13158     1  0 01:46 ?        00:00:00 oracle+ASM1_asmb_+asm1 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
oracle   13183     1  0 01:46 ?        00:00:02 oracle+ASM1_ocr (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
[oracle@rac1 bin]$

 

[노드2의 ASM인스턴스 상황]

[oracle@rac2 ~]$ ps -ef | grep ASM
oracle    6205  6172  0 07:40 pts/1    00:00:00 grep ASM
oracle   15126     1  0 02:20 ?        00:00:02 asm_pmon_+ASM2
oracle   15129     1  0 02:20 ?        00:00:03 asm_vktm_+ASM2
oracle   15133     1  0 02:20 ?        00:00:00 asm_gen0_+ASM2
oracle   15135     1  0 02:20 ?        00:00:01 asm_diag_+ASM2
oracle   15137     1  0 02:20 ?        00:00:01 asm_ping_+ASM2
oracle   15139     1  0 02:20 ?        00:00:00 asm_psp0_+ASM2
oracle   15141     1  0 02:20 ?        00:00:13 asm_dia0_+ASM2
oracle   15143     1  0 02:20 ?        00:00:07 asm_lmon_+ASM2
oracle   15145     1  0 02:20 ?        00:00:05 asm_lmd0_+ASM2
oracle   15147     1  0 02:20 ?        00:00:01 asm_lms0_+ASM2
oracle   15151     1  0 02:20 ?        00:00:00 asm_lmhb_+ASM2
oracle   15153     1  0 02:20 ?        00:00:00 asm_mman_+ASM2
oracle   15155     1  0 02:20 ?        00:00:00 asm_dbw0_+ASM2
oracle   15157     1  0 02:20 ?        00:00:00 asm_lgwr_+ASM2
oracle   15159     1  0 02:20 ?        00:00:00 asm_ckpt_+ASM2
oracle   15161     1  0 02:20 ?        00:00:00 asm_smon_+ASM2
oracle   15163     1  0 02:20 ?        00:00:02 asm_rbal_+ASM2
oracle   15165     1  0 02:20 ?        00:00:00 asm_gmon_+ASM2
oracle   15167     1  0 02:20 ?        00:00:00 asm_mmon_+ASM2
oracle   15169     1  0 02:20 ?        00:00:00 asm_mmnl_+ASM2
oracle   15178     1  0 02:21 ?        00:00:00 asm_lck0_+ASM2
[oracle@rac2 ~]$

 

 

[고찰2]

확실히 노드1쪽과 노드2쪽의 ASM인스턴스가 띄워져있는 상황에서 asmb라는 백그라운드프로세스가 노드2에서는 없는 것을 확인할 수 있음.

 

 

[현재 CRS 상태 조회]

[oracle@rac1 bin]$ ./crs_stat -t
Name           Type                 Target        State        Host
------------------------------------------------------------
ora....N1.lsnr ora....er.type      ONLINE     ONLINE     rac1
ora....VOTE.dg ora....up.type  ONLINE     ONLINE     rac1
ora.asm        ora.asm.type     ONLINE     ONLINE     rac1
ora.eons       ora.eons.type    ONLINE     ONLINE     rac1
ora.gsd        ora.gsd.type      OFFLINE    OFFLINE
ora....network ora....rk.type    ONLINE     ONLINE      rac1
ora.oc4j       ora.oc4j.type      OFFLINE    OFFLINE
ora.ons        ora.ons.type      ONLINE     ONLINE     rac1
ora....SM1.asm application    ONLINE     ONLINE     rac1
ora.rac1.gsd   application      OFFLINE    OFFLINE
ora.rac1.ons   application      ONLINE     ONLINE      rac1
ora.rac1.vip   ora....t1.type     ONLINE     ONLINE     rac1
ora....ry.acfs ora....fs.type     ONLINE     ONLINE     rac1
ora.scan1.vip  ora....ip.type   ONLINE     ONLINE     rac1
[oracle@rac1 bin]$

 

 

[고찰3]

노드2의 root.sh스크립트가 정상적으로 수행되지 못해 아직 노드2의 리소스들이 등록되지 않음을 확인.

관련 에러를 계속 찾아보는 중이고, 네트워크의 MTU값이 동일한지, 각 노드에서 상대 노드로 private 인터페이스 핑이 가는지 확인도 하였고, 시간 동기화를 해주는 NTP 체크도 해보았는데 아직까지 원인을 찾지 못하였음.

 


List of Articles
번호 분류 제목 글쓴이 날짜 조회 수
공지 Q&A Oracle관련 게시물만 Sean 2014.04.09 84839
16 Q&A 오라클 복구 질문 희달이 2015.05.08 1149
15 Q&A import후 simple_size 0일때 고구마 2014.01.13 3194
14 Q&A [답변] 테이블 수정, 삭제 및 LOB에 대해 알려주세요 Tech1 2012.03.26 10658
13 Q&A ORACLE_BASE, ORACLE_HOME 각각의 위치에 대한 설명 부탁드립니다 Tech1 2012.03.20 10900
12 Q&A 파티션 테이블에 대해서 몇가지 궁금합니다. 1 고구마 2011.06.13 11788
11 Q&A Local Prefixed와 Local Non-Prefixed index 차이 ? 1 고구마 2011.06.13 9598
10 Q&A 파티션 테이블 종류가 뭐가 있나요? 1 고구마 2011.06.13 10345
9 Q&A RAC 쿼리 속도 관련 질문입니다. 3 고구마 2011.05.20 8970
8 Q&A ORA--04031 에러에 대해 질문합니다. 4 고구마 2011.05.17 12634
7 Q&A 오라클 프로세스에 대해서 궁금합니다. 4 고구마 2011.05.04 11192
6 Q&A 패키지 BODY 관련 질문 3 고구마 2011.05.03 10182
5 Q&A alert_log는 5 perfstat 2011.04.22 8072
» Q&A RHEL5에 11gR2 RAC 구성시 ASM 관련 에러 file 송기성 2010.12.11 17688
3 Q&A 서버교체에 관해서.. 1 명랑여행 2010.08.20 9641
2 Q&A 오라클 유저 삭제가 안될시? 4 명랑여행 2010.05.27 11548
1 Q&A 업그레이드 이후 안됨 3 유주환 2010.04.23 10385
Board Pagination Prev 1 Next
/ 1