Thursday, April 24, 2008

Honey, I shrunk the indexes - Part 2

Ver este articulo en Español

How to pick the perfect candidates


Warning: index compression may change execution plans and affect performance, try this on a test database and check if application SQL and PL/SQL code execution improves.

Not every index must be compressed, sometimes compression will give minimal space reductions, that don't compensate the overhead incurred. But how do we know that?

I may suggest two approaches:
1) Size oriented
2) Access frequency oriented

Size Oriented

Let's start with size oriented picking, saying: the bigger the elephant is, better results will get when on diet.

I've used a script like this to get my list of candidates for shrinking:

SELECT
substr(segment_name,1,20) as index_name,
bytes, blocks, extents
FROM dba_segments
WHERE owner = '{write here the owner}'
AND segment_type = 'INDEX'
AND extents > 63 <---this you may change order by bytes desc;
After running above script, you'll get a listing like this:

INDEX NAME BYTES BLOCKS EXTENTS
-------------------- ---------- ---------- ----------
PROD_NAME_IX 524288 64 8
PRD_DESC_PK 327680 40 5
SYS_C009603 131072 16 2
SYS_C009607 65536 8 1
SYS_C009606 65536 8 1
ACTION_TABLE_MEMBERS 65536 8 1
LINEITEM_TABLE_MEMBE 65536 8 1
SYS_C009602 65536 8 1

Now you have to forecast the best compression ratio for your index, and there is a feature very accurate for doing so: ANALYZE the index.

Despite the fact that analyzing tables or objects have deprecated the statistics purpose (one of them), we may use this sentence to test structure. Following command and a quick query to INDEX_STATS will show us if the selected index is a best fit, which compression order to choose and expected size reduction:


SQL> ANALYZE INDEX owner.index_name VALIDATE STRUCTURE OFFLINE;

Index Analyzed

SQL> SELECT name, height, blocks, OPT_CMPR_COUNT, OPT_CMPR_PCTSAVE
FROM index_stats
WHERE name = '{index_name}';

The resulting value OPT_CMPR_COUNT is the value you specify for COMPRESS {n} clause, and OPT_CMPR_PCTSAVE is the "expected" compression ratio for that value. All other values from INDEX_STATS are present figures.

Then your sentences may look like this:

SQL> ALTER INDEX owner.index_name REBUILD COMPRESS {value from OPT_CMPR_COUNT}

or

SQL> CREATE INDEX owner.index_name ON {table_index_clause}
2: TABLESPACE {Tablespace Name}
3: COMPRESS {value from OPT_CMPR_COUNT}
4: {other storage clauses};

Second approach: Access Frequency

For this we're going to need the help of two important views: V$SEGSTAT(9i and up) and ALL_OBJECTS. We need V$SEGSTAT because that dynamic view will show us valuable statistics regarding logical reads/writes or physical reads/writes. Following script is proposed as an aid to find the top used indexes within a schema.

SELECT a.object_name, b.statistic_name, b.value
FROM all_objects a, v$segstat b
WHERE a.object_id = b.obj#
AND a.owner = '{your schema owner here}'
AND a.object_type = 'INDEX'
AND b.statistic_name = 'physical reads' <-- You may change this for physical reads direct ORDER by b.value desc
Above query will give you a list of candidates for compression, now you have to apply the ANALYZE and check if there are good space reductions that 'may' correspond to less IO.

Jump to Part III Honey, I shrunk the indexes - Part 3: Index Compression is good or evil?

View starting post: Honey, I shrunk the indexes - Part 1

Add to Technorati Favorites

Ver este articulo en Español

Subscribe to Oracle Database Disected by Email

Follow IgnacioRuizMX on Twitter

Thursday, April 10, 2008

Honey, I shrunk the indexes

Ver este articulo en Español

Introduction

There is an Oracle feature that may provide savings in space and IO, have you heard of "index key compression"? well, this feature is with us since 8i, but for many, it's obscure or unknown.

Despite the 10g storage management optimizations, always there is gain from index maintenance. If you do index checks regularly you're a good DBA... but if you don't, better take care from now on.

Adding the gains from index rebuild or shrink, you may consider compressing some well-picked indexes, for which the space savings and IO reductions overcome the slight(?) cpu overhead it causes. I wrote a question mark after 'slight' because we will try to estimate that cost in the short term.

I'll propose this starting questions:
* How do you use index key compression?
* What are the first sight results?
* How to pick the best candidates for compression?
* Index compression is good or evil... or both?
* What is the benefit/cost after the shrinking?
* What are the "inside" results or how to analyze the effect on my present queries?

If you have more questions, please feel free to drop a comment and we (all of you and I, because as far as I know I don't have multiple personality disorder) will try to tackle and provide a satisfactory answer.

How do you use index key compression?

There are two ways to accomplish this:
1) drop the index, create it again with COMPRESS
2) rebuild the index with COMPRESS

I will try the second method, with this huge index I've on a test database. These are the starting figures:






TABLE_ROWS TABLE_BLOCKS INDEX_BLOCKS INDEX_BYTES BLEVEL LEAF_BLOCKS
---------- ------------ ------------ ------------- ------ -----------
7,331,706 459,210 155,648 1,275,068,416 3 149,394

Now that we have our baseline, it's time to issue the DDL sentence that will reorganize the index:



SQL> ALTER INDEX idx_big_comp_test REBUILD COMPRESS 2;
Index Rebuild



After that statement our figures are now the following:



TABLE_ROWS   TABLE_BLOCKS   INDEX_BLOCKS   INDEX_BYTES    BLEVEL  LEAF_BLOCKS
---------- ------------ ------------ ------------- ------ -----------
7,331,706 459,210 139,904 1,146,093,568 3 133,682

A quick comparison yields less index blocks and leaf blocks (which is logical and obvious), accounting for 10.5% of space savings.

Let our imagination fly, we're showing our boss the way to extend the out-of-disk due date or justifying a well earned salary rise derived from storage savings. Back to reality... in this life everything has a price, don't rush and compress every index in your databases until we talk about pros and cons, and learn how to choose good candidates for compression.

Jump to Part II Honey, I shrunk the indexes - Part 2: How to pick the perfect candidates


Add to Technorati Favorites

Ver este articulo en Español/Look for this content in spanish

Subscribe to Oracle Database Disected by Email

Thursday, April 3, 2008

Recover that lost space

Do you know what happens when you issue a DELETE? Have you wondered what happen after COMMIT? ... well, I might say that there is life after delete for all those blocks that belonged to the data recently deleted.

I'll present the following test case to show you how this works:

First we are going to create our test environment, borrowing a table from the Examples OE schema, creating it in the SCOTT schema.


SQL> alter session set current_schema=SCOTT;

SQL> create table pd as select * from oe.PRODUCT_DESCRIPTIONS;

Let's see how many blocks our new table PD is using

SQL> column segment_name format a20
2: select segment_name, sum(blocks)
3: from dba_Segments where owner = 'SCOTT' and segment_name = 'PD'
4: group by segment_name
5: order by sum(blocks) desc;

SEGMENT_NAME SUM(BLOCKS)
-------------------- -----------
PD 384

Now we're going to see how it's structured the table:

SQL> select column_name,
2: substr(data_type,1,20) as dt,
3: data_length
4: from all_tab_columns
5: where table_name = 'PD';

COLUMN_NAME DT DATA_LENGTH
------------------------------ -------------------- -----------
PRODUCT_ID NUMBER 22
LANGUAGE_ID VARCHAR2 3
TRANSLATED_NAME NVARCHAR2 100
TRANSLATED_DESCRIPTION NVARCHAR2 4000

That structure yield us a theoretical row length of (22+3+100+4000) = 4125, which if used fully will give us just 1 row per block (see below for DB blocksize). But , rarelly a NVARCHAR2 is fully used.

SQL> show parameters db_block_size

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_block_size integer 8192

We know there is data and how it's structured, but we need something to erase... the following may be politically incorrect but is only for instructional purposes:

SQL> select language_id, count(*) from pd group by language_id;

LAN COUNT(*)
--- ----------
US 288
IW 288
TR 288
...
S 288
SK 288
ZHS 288

30 rows selected.

We're going to delete all the 'S' language registers, I don't know what means 'S' and I'm sorry if I hurt nationalistic feelings, remember this is hypotetical.

SQL> delete from pd where LANGUAGE_ID = 'S';

288 rows deleted.

SQL> commit;

Commit complete.

That was the death sentence for those rows, now will see what happened with the blocks assigned to PD.

SQL> column segment_name format a20
2: select segment_name, sum(blocks)
3: from dba_Segments where owner = 'SCOTT' and segment_name = 'PD'
4: group by segment_name
5: order by sum(blocks) desc;

SEGMENT_NAME SUM(BLOCKS)
-------------------- -----------
PD 384

Wow! the table still has the blocks assigned, and will not return them until a TRUNCATE, DROP ... or ALTER TABLE is issued. Yes! an alter table will return the extents to the tablespace free pool.

SQL> alter table scott.pd deallocate unused;

Table altered.

SQL> column segment_name format a20
2: select segment_name, sum(blocks)
3: from dba_Segments where owner = 'SCOTT' and segment_name = 'PD'
4: group by segment_name
5: order by sum(blocks) desc;

SEGMENT_NAME SUM(BLOCKS)
-------------------- -----------
PD 376

There it is!!! the blocks that contained all the 'S' language registers, now don't belong to the table PD.

At first you may be tempted to include a periodic ALTER TABLE...DEALLOCATE UNUSED in order to recover all that 'wasted' space, you must be carefull when evaluating this, because there are tables that get data erased and never or rarelly get new data, them are perfect candidates for this.

On the other hand, tables with heavy delete activity won't benefit too much because of the overhead related to update the internal tables that keep track of blocks and extents. What is 'heavy' activity, depends on the deleted rows/total rows rate, and you must develop a maintenance criteria to exclude tables.

You're welcome to send your comments... I hope this help you

Please, before you go, don't forget to vote the poll regarding the content of this blog, thank you!
Subscribe to Oracle Database Disected by Email

Friday, March 28, 2008

Sure...your trash will drag you!!!

If you have figured out what happened, you may be wondering why Oracle implemented the free space view in that way, don't you?

In case you are lost in the darkness follow the next real-life case:

Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bit Production
With the Partitioning, OLAP and Data Mining options

SQL> select count(*) from dba_free_space;

COUNT(*)
----------
193822
_
SQL> select count(*) from all_objects where object_name like '%BIN%';

COUNT(*)
----------
175249

SQL> set timing on
SQL> run
1 select count(*) from all_objects where object_name like '%BIN%'
2*

COUNT(*)
----------
175249

Elapsed: 00:00:27.00
SQL> select count(*) from dba_free_space;

COUNT(*)
----------
193821

Elapsed: 00:00:45.71


That is a long waiting time for a free space report, don't you think? ... I'm not a lazy and careless DBA, but you can't take my word for granted, therefore I will check if statistics are more-or-less accurate and later refresh them, anyway.

SQL> select num_rows, blocks from all_Tables where table_name = 'RECYCLEBIN$';

NUM_ROWS BLOCKS
---------- ----------
185640 2308

Elapsed: 00:00:00.05
SQL> exec dbms_stats.gather_table_stats(ownname=> 'SYS',
2: tabname=> 'RECYCLEBIN$', partname=> NULL);

PL/SQL procedure successfully completed.

Elapsed: 00:00:06.70
SQL> select count(*) from dba_free_space;

COUNT(*)
----------
193820

Elapsed: 00:00:46.02
SQL> run
1* select count(*) from dba_free_space

COUNT(*)
----------
193787

Elapsed: 00:00:45.82
SQL> set autotrace on explain statistics
SQL> run
1* select count(*) from dba_free_space

COUNT(*)
----------
193787

Elapsed: 00:00:46.20

Execution Plan
----------------------------------------------------------

------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)|
------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | | 1563 (53)|
| 1 | SORT AGGREGATE | | 1 | | | |
| 2 | VIEW | DBA_FREE_SPACE | 190 | | | 1563 (53)|
| 3 | UNION-ALL | | | | | |
| 4 | NESTED LOOPS | | 1 | 39 | | 4 (0)|
| 5 | NESTED LOOPS | | 1 | 32 | | 3 (0)|
| 6 | TABLE ACCESS FULL | FET$ | 1 | 26 | | 3 (0)|
| 7 | INDEX UNIQUE SCAN | I_FILE2 | 1 | 6 | | 0 (0)|
| 8 | TABLE ACCESS CLUSTER | TS$ | 1 | 7 | | 1 (0)|
| 9 | NESTED LOOPS | | 90 | 4050 | | 12 (9)|
| 10 | NESTED LOOPS | | 90 | 3510 | | 12 (9)|
| 11 | TABLE ACCESS FULL | TS$ | 36 | 468 | | 11 (0)|
| 12 | FIXED TABLE FIXED INDEX | X$KTFBFE (ind:1) | 3 | 78 | | 0 (0)|
| 13 | INDEX UNIQUE SCAN | I_FILE2 | 1 | 6 | | 0 (0)|
| 14 | NESTED LOOPS | | 98 | 6762 | | 1530 (54)|
| 15 | NESTED LOOPS | | 98 | 6174 | | 1530 (54)|
| 16 | HASH JOIN | | 169K| 3983K| 3920K| 840 (16)|
| 17 | TABLE ACCESS FULL | RECYCLEBIN$ | 174K| 1871K| | 585 (14)|
| 18 | TABLE ACCESS FULL | TS$ | 36 | 468 | | 11 (0)|
| 19 | FIXED TABLE FIXED INDEX | X$KTFBUE (ind:1) | 1 | 39 | | 0 (0)|
| 20 | INDEX UNIQUE SCAN | I_FILE2 | 1 | 6 | | 0 (0)|
| 21 | TABLE ACCESS BY INDEX ROWID| RECYCLEBIN$ | 1 | 11 | | 2 (0)|
| 22 | NESTED LOOPS | | 1 | 63 | | 17 (0)|
| 23 | NESTED LOOPS | | 1 | 52 | | 15 (0)|
| 24 | NESTED LOOPS | | 1 | 45 | | 14 (0)|
| 25 | TABLE ACCESS FULL | UET$ | 1 | 39 | | 14 (0)|
| 26 | INDEX UNIQUE SCAN | I_FILE2 | 1 | 6 | | 0 (0)|
| 27 | TABLE ACCESS CLUSTER | TS$ | 1 | 7 | | 1 (0)|
| 28 | INDEX UNIQUE SCAN | I_TS# | 1 | | | 0 (0)|
| 29 | INDEX RANGE SCAN | RECYCLEBIN$_TS | 7961 | | | 2 (0)|
------------------------------------------------------------------------------------------------

Statistics
----------------------------------------------------------
4027917 recursive calls
129 db block gets
891718 consistent gets
175114 physical reads
0 redo size
517 bytes sent via SQL*Net to client
469 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
1 rows processed

SQL> set autotrace off

Let's see what happen if we get rid off all that garbage... be pacient, it'll take 2 or 3 hours to purge the recycle bin.

SQL> purge dbarecycle_bin;

Recyclebin purged

Our timings will improve by the order of thousands, getting results in results in fractions of seconds. That will make easier and faster your space management tasks... say good bye to those long chats with your peers between tablespace resize (if you are using Enterprise Manager is even worse!).

SQL> run
1* select count(*) from all_objects where object_name like '%BIN%'

COUNT(*)
----------
103

Elapsed: 00:00:00.20

SQL> set linesize 255
SQL> set autotrace on explain statistics
SQL> run
1* select count(*) from dba_free_space

COUNT(*)
----------
1140

Elapsed: 00:00:00.06

Execution Plan
----------------------------------------------------------

------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)|
------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | | 1563 (53)|
| 1 | SORT AGGREGATE | | 1 | | | |
| 2 | VIEW | DBA_FREE_SPACE | 190 | | | 1563 (53)|
| 3 | UNION-ALL | | | | | |
| 4 | NESTED LOOPS | | 1 | 39 | | 4 (0)|
| 5 | NESTED LOOPS | | 1 | 32 | | 3 (0)|
| 6 | TABLE ACCESS FULL | FET$ | 1 | 26 | | 3 (0)|
| 7 | INDEX UNIQUE SCAN | I_FILE2 | 1 | 6 | | 0 (0)|
| 8 | TABLE ACCESS CLUSTER | TS$ | 1 | 7 | | 1 (0)|
| 9 | NESTED LOOPS | | 90 | 4050 | | 12 (9)|
| 10 | NESTED LOOPS | | 90 | 3510 | | 12 (9)|
| 11 | TABLE ACCESS FULL | TS$ | 36 | 468 | | 11 (0)|
| 12 | FIXED TABLE FIXED INDEX | X$KTFBFE (ind:1) | 3 | 78 | | 0 (0)|
| 13 | INDEX UNIQUE SCAN | I_FILE2 | 1 | 6 | | 0 (0)|
| 14 | NESTED LOOPS | | 98 | 6762 | | 1530 (54)|
| 15 | NESTED LOOPS | | 98 | 6174 | | 1530 (54)|
| 16 | HASH JOIN | | 169K| 3983K| 3920K| 840 (16)|
| 17 | TABLE ACCESS FULL | RECYCLEBIN$ | 174K| 1871K| | 585 (14)|
| 18 | TABLE ACCESS FULL | TS$ | 36 | 468 | | 11 (0)|
| 19 | FIXED TABLE FIXED INDEX | X$KTFBUE (ind:1) | 1 | 39 | | 0 (0)|
| 20 | INDEX UNIQUE SCAN | I_FILE2 | 1 | 6 | | 0 (0)|
| 21 | TABLE ACCESS BY INDEX ROWID| RECYCLEBIN$ | 1 | 11 | | 2 (0)|
| 22 | NESTED LOOPS | | 1 | 63 | | 17 (0)|
| 23 | NESTED LOOPS | | 1 | 52 | | 15 (0)|
| 24 | NESTED LOOPS | | 1 | 45 | | 14 (0)|
| 25 | TABLE ACCESS FULL | UET$ | 1 | 39 | | 14 (0)|
| 26 | INDEX UNIQUE SCAN | I_FILE2 | 1 | 6 | | 0 (0)|
| 27 | TABLE ACCESS CLUSTER | TS$ | 1 | 7 | | 1 (0)|
| 28 | INDEX UNIQUE SCAN | I_TS# | 1 | | | 0 (0)|
| 29 | INDEX RANGE SCAN | RECYCLEBIN$_TS | 7961 | | | 2 (0)|
------------------------------------------------------------------------------------------------

Statistics
----------------------------------------------------------
295 recursive calls
129 db block gets
12437 consistent gets
0 physical reads
0 redo size
516 bytes sent via SQL*Net to client
469 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
1 rows processed


Please, before you go, don't forget to vote the poll regarding the content of this blog, thank you!

Your trash will drag you

If you think that Oracle recycle bin (feature available since 10g1) has no direct impact on your performance, you better think deep about it... seriously.

Try the following (if possible) : turn on statement timing, then issue a simple SELECT on DBA_FREE_SPACE and keep record of elapsed time; create a couple of hundreds of thousands tables in a 10g database, and then delete it (the DB recycle bin feature must be turned on) ... now you have a full-to-the-top recycle bin, then repeat the SELECT on DBA_FREE_SPACE, you'll see it will take a longer time to complete, and finally will get a huge number of rows. You may think that suddenly the free space has been multiplied... but your response time divided.

I don't know ... I'll let you think about this puzzling situation here and come back later to see how you doing.

Question: Can you figure out what is happening?

Subscribe to Oracle Database Disected by Email

Tuesday, March 25, 2008

2 minute guide for Statspack Installation

Ver este articulo en Español

Statspack is an Oracle database tool that is both powerful and underused, it's the first reporting resource you have (it's free, comes in the box) to see what is happening in your DB at a glance.

Ingredients
1 Oracle sql*plus session
1 Tablespace with at least 200Mb (9i) or 400Mb (10g) of free space*
1 user with DBA power

*Recommend you create a brand new tablespace

Preparation
You need to install Statspack previous to any use, then you have to run just one script... yes, it's that easy.
Login with your powerful user, then at the prompt type:

SQL> @?/rdbms/admin/spcreate

...you've started the Statspack setup, that will create the owner for the Statspack objects, which username is fixed to PERFSTAT. In order to acomplish that, asks for the tablespace where all objects will be stored, the temporary tablespace Statspack will use for sorting data and the user password.

You'll end with a message like this, and a result listing at the place you executed sqlplus.

It's recommended to establish a baseline for your reporting, then you must connect as PERFSTAT and take your first snapshot:
SQL> exec statspack.snap;

It will run with detail level 5, which is the default ... and enough for 90% of the cases. Taking samples 2 to 5 minutes apart is the period recommended, you may take samples with longer intervals but some results will get their formatting exceded and show # marks.
You may be interested on scheduling execution of statspack.snap, please read 2 minute guide for Statspack sampling

Further ... and detailed reading at the Oracle(R) Database Performance Tuning Guide.


Add to Technorati Favorites

Subscribe to Oracle Database Disected by Email

Thursday, March 13, 2008

Your own DB 'Big Brother'

How many times you have been in the unfortunate situation where something catastrophic has happened, caused by an OSI 'Layer-8' error, you've been asked whom did that... and you don't have a clue.

Well, for those cases Oracle provides a very useful auditing feature, that is very simple to enable.

Usually you start enabling auditing on users that have the privilege to do the activity you are interested on, after you may enable auditing on users suspicious of trying to do something they don't have the privilege to.

Let's give a short example, given the case I want to track who drops or modifies any user.


SQL> alter system set audit_trail=DB scope=spfile;

or edit your pfile to reflect above setting
SQL> shutdown
SQL> startup
SQL> audit drop user by admin;


SQL> audit alter user by admin;

SQL> alter user scott identified by tigre;
User Altered
SQL> select count(*) sys.aud$;


How do you check what is fasible to audit or what is beeing audited, you must check these views:
* SYSTEM_PRIVILEGE_MAP
* DBA_PRIV_AUDIT_OPTS
* DBA_AUDIT_OBJECT

If those views aren't available then you must create them running the cataudit.sql script from $ORACLE_HOME/rdbms/admin.

Further explanation you may find in the Oracle Database Administration Guide.

Subscribe to Oracle Database Disected by Email
Custom Search