Oracle RAC 11g - very first impressions

After few hours of observation ( in non production environment ) i decided to write down my remarks.

In plus :
#1 - It is possible to make rolling update from CRS 10.2.0.3, great!
#2 - We install CRS 11g without any problems known for earlier versions
#3 - There is no need to manualy reallocate cluster services after node crash or just node maintanance process.

In minus:
#1 - Not working with Quest Spotlight.

Hitting bugs while installing Oracle CRS 10g on Solaris

While installing CRS 10.2.0.2 on Solaris 10 ( and probably other operating systems ) installation process may fail, reporting you a problem with setting up VIP adresses.
Why? Let's take a look..

At the and of CRS installation, OUI will dictate you to run two shell scripts as privileged user. The second one ( root.sh ) calls vipca to make your public addres accessible for database/cluster clients. Before configuring it as an alias for primary interface, vipca checks propriety of all given parameters.
If you decided to configure VIP as set of non-routable IP addreses ( RFC1918 ) from :

10.0.0.0 - 10.255.255.255 (10/8 prefix)
172.16.0.0 - 172.31.255.255 (172.16/12 prefix)
192.168.0.0 - 192.168.255.255 (192.168/16 prefix)

subnets - vipca will fail. Why? Is it a bug or just fussing "feature"? I don't know.. but in 10.2.0.3 Oracle Corporation decided to modify vipca not to check propriety of VIP IPs.

So, how to deal it?
The simpliest way is to install 10.2.0.2 CRS without setting up VIP ( force OK in OUI ), patch CRS software to 10.2.0.3 and finally run vipca by your own.

$CRS_HOME/bin/vipca -silent -nodelist $CRS_NODELIST -nodevips $CRS_NODEVIPS

Unluckily, in CRS 10.2.0.3 for Solaris 64-bit there is a bug in ONS makeing your cluster unstable.
I recommend you to install 5749953 patch asap. You can download it from metalink and follow up README.txt instructions.

Oracle CRS 10g running on Sun Solaris 10

Instead of rewriting cofiguration instructions, i'd like to shed some light on potential problems you may hit during CRS installation process on SUN Solaris 10 64-bit operating system.

Let's start from the begining..

Folowing documentation provided by Oracle Corporation, you set up all preinstallation parameters required to set up Clusterware software ( kernel parameters, environment variables and many other mentioned on manual pages) and you think you're done with operating system preparation? Unfortunately it is barely truth..

Oracle installer will search for SSC/SCP software in hardcoded locations, which are simply not avaliable in standard Sun Solaris 10 installation.
You have to create symlinks:
/usr/local/bin/ssh -> /usr/bin/ssh
/usr/local/bin/scp -> /usr/bin/scp

Than, if your system is 64-bit capable, you have to "tell" Oracle Universal Installer to prepare 64-bit libraries by setting up:

crle -64 -u -l $CRS_HOME/lib

Otherwise you may hit the problem with libclntsh.so. Of course it's corrigible, and it's not much complicated:

export ORACLE_HOME="/oracle/crs/"
cd $ORACLE_HOME/bin
Edit ./genclntsh file and replace:
LD="ld -m -Dfiles -i -G -z text -L${ORACLE_HOME}/${LIB}"
with:
LD="ld -m -Bsymbolic -i -G -z text -L${ORACLE_HOME}/${LIB}"

Run "relink all" again.
./genclntsh

/bin/clscfg -install -nn $CRS_NODE_NAME_LIST -pn $CRS_PRIVATE_NAME_LIST \
$HOSTNAMES_ARGS -o $CH -c $CRS_CLUSTER_NAME -l "$CRS_LANGUAGE_ID" \
-q $CRS_VOTING_DISKS $CLSCFG_MISCNT

Eg.
./clscfg -install -nn node1,1,node2,2,node3,3 -pn node1-priv,1,node2-priv,2,node3-priv,3 -hn node1,1,node2,2,node3,3 -o /oracle/crs/ -c mycluster -l "'AMERICAN_AMERICA.WE8ISO8859P1'" -q /dev/rdsk/c4t600508B400105E660001000003960000d0s6 -force

After above operations, you can check your RAC health.

root@node1:/oracle/crs/bin# ./olsnodes
node1
node2
node3

root@node1:/oracle/crs/bin# ./crsctl check crs
CSS appears healthy
CRS appears healthy
EVM appears healthy

Juppie! ;)

The last step is to configure VIP interfaces:
$CRS_HOME/bin/vipca -silent -nodelist $CRS_NODELIST -nodevips $CRS_NODEVIPS