Thursday, January 26, 2012

Exadata database server Drive Configuration Verification

To verify the database server physical drive configuration, use the following command:

/opt/MegaRAID/MegaCli/MegaCli64 PDList -aALL | grep "Firmware state"

The output should be similar to for the disks:

Firmware state: Online, Spun Up


To verify the database server virtual drive configuration, use the following command:

/opt/MegaRAID/MegaCli/MegaCli64 CfgDsply -aALL | grep "Virtual Drive:";/opt/MegaRAID/MegaCli/MegaCli64 CfgDsply -aALL | grep "Number Of Drives";/opt/MegaRAID/MegaCli/MegaCli64 CfgDsply -aALL | grep "^State"

The output should be similar to:

Virtual Drive: 0 (Target Id: 0)
Number Of Drives    :
State               : Optimal

Monday, January 23, 2012

Exadata [WARNING] This update is not applicable for Not Available BP12

During the application of BP12 on Exadata half rack V2, during the phase of application of db minimal patch, one compute node refused to get patched.

As the ./install.sh script was run, it failed with :

[WARNING] This update is not applicable for Not Available.

Of course it wasn't of much help, so we fished around and then found that because dmidecode command was not returning the system product name at that node, whereas the same command was working at the other nodes, where the patching was successful.

[root@exanode db_minimal_patch]# dmidecode -s system-product-name
Not Available

This was resolved with not that much fanfare. We merely had to reset iLOM from an SSH terminal.Though it took several hours to get there. So instrumentation is the key for debugging.

Friday, January 13, 2012

genclntsh: Could not locate shrept.lst during PSU 11.2.0.2.4

What to do when you are applying PSU 11.2.0.2.4 on your Oracle 11g 11.2.0.2.0 database, and you get following error while running the opatch apply:


Make failed to invoke "/usr/bin/make -f ins_net_client.mk
client_sharedlib ORACLE_HOME=/u01/app/oracle/product/11.2.0/db_1
"....'genclntsh: genclntsh: Could not locate 
/u01/app/oracle/product/11.2.0/db_1/network/admin/shrept.lst
make: *** [client_sharedlib] Error 1
'
Make failed to invoke "/usr/bin/make -f ins_rdbms.mk
client_sharedlib ORACLE_HOME=/u01/app/oracle/product/11.2.0/db_1
"....'genclntsh: genclntsh: Could not locate 
/u01/app/oracle/product/11.2.0/db_1/network/admin/shrept.lst
make: *** [client_sharedlib] Error 1
'

The following make actions have failed :

Re-link fails on target "client_sharedlib".
Re-link fails on target "client_sharedlib".
 
Solution:
Just proceed with the patch installation, it is likely to get completed
successfully with warnings.
After that, go to $ORACLE_HOME/network/admin and see if there is a file shrept.lst?
If there exists a file, make sure that it's permission are set to 644.
If shrept.lst doesn't exist, then create a new file like this:
cd $ORACLE_HOME/network/admin
touch shrept.lst
chmod 644 shrept.lst
Then open the file shrept.lst in text editor, and paste the following lines:
 
network : snaumihi_inithostinfo
network : snaumbg_gmt
network : naedpwd_encrypt
network : naumbsb_bld_singlebyte
network : ztapis
network : nlgh 
Save and exit the file.
 
Now you have to run the relink all command, but before that no oracle process
is running out of that ORACLE_HOME:
 
cd $ORACLE_HOME/bin
./relink all
 
Now you should be good to go.
 
Cheers.
 
Reference: MOS Note ID 340978.1 

Thursday, January 12, 2012

Oracle Connects Big Data to Medium and Small Data

With the announcement of the Oracle Big Data Appliance, Oracle also comes up with some really cool technology stack which is being termed as Oracle Big Data Connectors (OBDC). This piece of software can be used with both Oracle Big Data Appliance and other Apache Hadoop-based systems.

Among other things in OBDC, There is this Oracle Loader for Hadoop which uses MapReduce processing to load data efficiently into Oracle Database 11g and speeds up the loading by using the Oracle’s internal formats. This should be the big incentive and relief and a very solid business rationale for the decision makers to try out No SQL technology with their existing infrastructure.

The main reason of fast adoption of Exadata (Medium data)  in the businesses is it’s ease and speed by which it gets connected to the OLTP (Small Data). Both of them run the same Oracle 11g. Now with these connectors, its’ easy to integrate the NoSQL (Big Data) with these small and medium data sources.

Integration with scalability and interoperability are the keywords here.

Wednesday, January 11, 2012

lsnrctl: symbol lookup error: lsnrctl: undefined symbol: nlgh after PSU 11.2.0.2.4

Applying PSU 11.2.0.2.4 on Oracle 11.2.0.2.0 is often a breezing experience. But sometimes you get a mild jolt kind of thing when at the last step, after a beautifully successful patching session you get following error, while trying to restart the listener:

/u01/app/oracle> lsnrctl start
lsnrctl: symbol lookup error: lsnrctl: undefined symbol: nlgh


If you panic and start getting bad impressions, you are so human.

But don't worry, just relink all the binaries and try again. But before that, you must shutdown all the databases and there should be no Oracle binary running.

After making sure that nothing Oracle is  running, relink:

/u01/app/oracle/product/11.2.0/db_1/bin> ./relink all
writing relink log to: /u01/app/oracle/product/11.2.0/db_1/install/relink.log
/u01/app/oracle/product/11.2.0/db_1/bin> lsnrctl start

And listener should start.

Wednesday, January 4, 2012

Reason 442 : Failed to enable Virtual Adapter in Windows 7 CISCO VPN Client

If all of a sudden, you start receiving the error:

Reason 442 : Failed to enable Virtual Adapter

while you try to access the VPN network through the CISCO VPN client, you might want to disable the Internet sharing for other users. It should work from there.

There are some other things to check if above doesn't work and that is to run the diagnostic in the virtual adapter, or to disable any virtualbox or VMware adapter, or reseting the adapter.

Hope that helps.

Monday, January 2, 2012

Step by Step Implementation of SQL Profiles in Oracle

Original Problematic Query:

SELECT *
FROM TEMP_TAB A2,TEMPOFF A1
WHERE A2.TEMP1=A1.TEMP1
AND (A2.TEMP13=1 OR A2.TEMP13=0 OR A2.TEMP13=0)
AND A2.STEMPDATE>=:B3
AND A2.STEMPDATE<:b2 a2.TEMPTEMP5<='' and=''>000
AND A2.TEMPTEMP5<>000
AND A2.TEMPTEMP5<>0
AND A2.TEMPTEMP5<>0
AND A1.TEMPTEMP6=:B1
AND A1.TEMPTEMP6<>0
GROUP BY A1.TEMPTEMP6,A1.TEMPTEMP7,TRUNC(A2.STEMPDATE,'MONTH')


===================
Plan of above query:

Plan hash value: 2803934278

------------------------------------------------------------------------------------------------------------------
| Id  | Operation                         | Name         | Rows  | Bytes | Cost (%CPU)| Time     | Pstart| Pstop |
------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                  |              |       |       | 18133 (100)|          |       |       |
|   1 |  HASH GROUP BY                    |              |    15 |   720 | 18133  (39)| 00:00:47 |       |       |
|*  2 |   FILTER                          |              |       |       |            |          |       |       |
|*  3 |    HASH JOIN                      |              |   781K|    35M| 17906  (38)| 00:00:47 |       |       |
|   4 |     MAT_VIEW ACCESS BY INDEX ROWID| TEMPOFF        |   104 |  1352 |    54   (0)| 00:00:01 |       |       |
|*  5 |      INDEX RANGE SCAN             | TEMPOFF_TEMPTEMP6    |   104 |       |     1   (0)| 00:00:01 |       |       |
|   6 |     PARTITION RANGE ITERATOR      |              |  9176K|   306M| 17549  (37)| 00:00:46 |   KEY |   KEY |
|*  7 |      TABLE ACCESS FULL            | TEMP_TAB |  9176K|   306M| 17549  (37)| 00:00:46 |   KEY |   KEY |
------------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - filter((:B1<>0 AND :B3<:b2))
   3 - access("A2"."TEMP1"="A1"."TEMP1")
   5 - access("A1"."TEMPTEMP6"=:B1)
       filter("A1"."TEMPTEMP6"<>0)
   7 - filter((INTERNAL_FUNCTION("A2"."TEMP13") AND "A2"."STEMPDATE">=:B3 AND "A2"."TEMPTEMP5"<>0 AND
              "A2"."TEMPTEMP5"<>0 AND "A2"."TEMPTEMP5"<>0 AND "A2"."TEMPTEMP5"<>112 AND "A2"."STEMPDATE"<:b2))


===================

Rationale: As the filter at line 7 in plan is using Full Table Scan (FTS) of partition(s) of TEMPH.TEMP_TAB table, and it is likely that index access would speed up the query.(Also we know that the earlier optimal runs of the query were using index), so we need to enforce index access at line 7.


Table structure:

SQL> desc TEMPH.TEMP_TAB
 Name                                                  Null?    TEMP13
 ----------------------------------------------------- -------- ------------------------------------
 TEMP1                                                   NOT NULL NUMBER
 TEMPTEMP5                                                   NOT NULL NUMBER
 STEMPDATE                                                 NOT NULL DATE
 BTEMPTEMP7                                                  NOT NULL NUMBER
 TEMPCOUNT                                                         NUMBER
 TMPTEMP13                                                  NOT NULL NUMBER
 TMPSUID                                                          NUMBER
 TEMP8_TEMP10_NO                                                 NUMBER


========================

Existing Composite indexes on table TEMPH.TEMP_TAB:

1-      PK_TEMP_TAB(STEMPDATE,TEMP1,TEMP13,BTEMPTEMP7,TEMPTEMP5)
2-      STEMPH_TEMP11(TEMP1,STEMPDATE,TEMP13,TEMPTEMP5)
3-      STEMPH_SSTO (TEMPTEMP5,STEMPDATE,TEMP13,TEMP1)
4-      TEMP_TAB_TEMP8_TEMP10_NO (TEMP8_TEMP10_NO,STEMPDATE) --> not applicable because columns are not in the filter
5-      TEMP_TAB_SHH_TEMP9EMPTEMP5_KEY(SHH_TEMP9EMPTEMP5_KEY,STEMPDATE)    --> not applicable because columns are not in the filter

====================

Filter from the plan on lin 7 using FTS:

filter((INTERNAL_FUNCTION("A2"."TEMP13") AND "A2"."STEMPDATE">=:B3 AND "A2"."TEMPTEMP5"<>0 AND
              "A2"."TEMPTEMP5"<>0 AND "A2"."TEMPTEMP5"<>0 AND "A2"."TEMPTEMP5"<>112 AND "A2"."STEMPDATE"<:b2))

The columns of TEMPH.TEMP_TAB table used in this filter are TEMP13,STEMPDATE,TEMPTEMP5.

Anyway, here we know that TEMPH.STEMPH_TEMP11 index should be used from the TEMPHorical runs and also the test run with bind variable values confirms that.

Now we have to force this query to use the index TEMPH.STEMPH_TEMP11. We can do it with hints, but since we cannot TEMP10 the query, so we are employing the SQL Profile here to enforce the usage of Index.

Now comes implementation of SQL profile:

In order to get string for the SQL profile, first run the query with the hint, and then get the plan with outline information, and then from outline information just get the string, as shown  below:

SQL> set lines 100
SQL> set pages 100
SQL> set timing on
SQL> SELECT /*+index(s STEMPH_TEMP11)*/ *
  2  FROM TEMPH.TEMP_TAB S, TEMPOFF O
  3  WHERE S.TEMP1 = O.TEMP1 AND S.TEMP13 IN (1, 3, 5) AND S.STEMPDATE >= to_date('11/01/11','mm/dd/yy') AND S.STEMPDATE < to_date('12/01/11','mm/dd/yy')
  4  AND S.TEMPTEMP5 NOT IN (0, 112, 0 , 0)
  5  AND (O.TEMPTEMP6 = 0 ) AND O.TEMPTEMP6 NOT IN (0) GROUP BY O.TEMPTEMP6, O.TEMPTEMP7, TRUNC (S.STEMPDATE, 'MONTH');


SQL> select * from table(dbms_xplan.display_cursor(null,null,'ADVANCED +ALLSTATS'));

Query Block Name / Object Alias (identified by operation id):
-------------------------------------------------------------

   1 - SEL$1
   3 - SEL$1 / S@SEL$1
   5 - SEL$1 / O@SEL$1
   6 - SEL$1 / O@SEL$1
   8 - SEL$1 / S@SEL$1

Outline Data
-------------

  /*+
      BEGIN_OUTLINE_DATA
      IGNORE_OPTIM_EMBEDDED_HINTS
      OPTIMIZER_FEATURES_ENABLE('10.2.0.4')
      OPT_PARAM('_optimizer_skip_scan_enabled' 'false')
      OPT_PARAM('optimizer_index_cost_adj' 90)
      ALL_ROWS
      OUTLINE_LEAF(@"SEL$1")
      INDEX_RS_ASC(@"SEL$1" "O"@"SEL$1" ("TEMPOFF"."TEMPTEMP6"))
--->>     INDEX(@"SEL$1" "S"@"SEL$1" ("TEMP_TAB"."TEMP1" "TEMP_TAB"."STEMPDATE" "TEMP_TAB"."TEMP13" "TEMP_TAB"."TEMPTEMP5")) <<------ SQL PROFILE String
      LEADING(@"SEL$1" "O"@"SEL$1" "S"@"SEL$1")
      USE_NL(@"SEL$1" "S"@"SEL$1")
      END_OUTLINE_DATA
  */


==================================


-->> From the outline section above, get the string and create SQL profile for problem query as follows:

begin dbms_sqltune.import_sql_profile (sql_text => 'SELECT * FROM "TEMP_TAB" "A2","TEMPOFF" "A1" WHERE "A2"."TEMP1"="A1"."TEMP1" AND ("A2"."TEMP13"=1 OR "A2"."TEMP13"=3 OR "A2"."TEMP13"=5) AND "A2"."STEMPDATE">=:B3 AND "A2"."STEMPDATE"<:b2 "a2"."TEMPTEMP5"<='' and=''>0 AND "A2"."TEMPTEMP5"<>112 AND "A2"."TEMPTEMP5"<>0 AND "A2"."TEMPTEMP5"<>0 AND "A1"."TEMPTEMP6"=:B1 AND "A1"."TEMPTEMP6"<>0 GROUP BY "A1"."TEMPTEMP6","A1"."TEMPTEMP7",TRUNC("A2"."STEMPDATE",''MONTH'')',
profile => sqlprof_attr('INDEX(@"SEL$1" "A2"@"SEL$1" ("TEMP_TAB"."TEMP1" "TEMP_TAB"."STEMPDATE" "TEMP_TAB"."TEMP13" "TEMP_TAB"."TEMPTEMP5"))'),
name=>'FIX_1',
force_match=>false);
end;
/


Now check the latest plan for the session running the query and verify that index is being used in plan.

Afterwards, the profile can be dropped as:

exec dbms_sqltune.drop_sql_profile('FIX_1');