Tuesday, April 25, 2017

In EBS 12.1.3, Gather Schema Stats fails with Cause: FDPSTP failed due to ORA-29913: error in executing ODCIEXTTABLEOPEN callout

Recently in our environment, Gather Schema Statistics program were failing for custom schema with errors

Cause: FDPSTP failed due to ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-04040: file ***.csv in Custom Directory not found
ORA-06512: at "APPS.FND_STATS",

From the error message, we can easily conclude that the program was trying to find the file in the custom directory. The program failed because the program couldn't find the file in the respective directory

What could be the reason.


The primary cause of this issue is that an OS file for an "external table" existed at some point in time but does not now. However, the database still believes the OS file for the table exists since the dictionary information about the object has not been modified to reflect the change. When DBMS_STATS is run against the table in question, it makes a call out to the external table which fails because the object is not there. The error can also occur during data pump operations.



Solution:

By using the below query, identify the list of external tables in the custom schema

select el.table_name, el.owner, dir.directory_path||'/'||dir.directory_name "path"
from dba_external_locations el
, dba_directories dir
where el.table_name like '%&&table_pattern%'
and el.owner like '%&&owner%'
and el.directory_owner = dir.owner
and el.directory_name = dir.directory_name
order by 1, 2;

This will list down the table name,owner name and path. If the path is incorrect, modify it first.

Now there are two options.

1.Lock the stats collection for the respective external tables under custom schema.

exec DBMS_STATS.LOCK_TABLE_STATS('owner','external table name');

Then run the gather schema statistics.The program will complete nowmal now and the log message will show that the stats have not been gathered for the locked tables.

Once the program completes, it is feasible to unlock the stats gathering

exec DBMS_STATS.UNLOCK_TABLE_STATS('owner','external table name');


2. This is the best option. Run gather table statisctics for the respective tables one by one. For any external tables, if the program throws an error “file not found”, then create a 0 byte file manually and then run it. It should complete normal now
Once gather table stats completes for all the external tables, run gather schema statistics for custom schema

reference:-ORA-20011 ORA-29913 and ORA-29400 with Associated KUP-XXXXX Errors from DBMS_STATS.GATHER_STATS_JOB (Doc ID 1274653.1)

Monday, April 17, 2017

adoacorectl.sh , adformsctl.sh , adoafmctl.sh – exiting with status 204

OC4J:oafm, OC4J:forms not starting and existing with status 204


Processes in Instance: *****************
---------------------------------+--------------------+---------+---------
ias-component                    | process-type       |     pid | status
---------------------------------+--------------------+---------+---------
OC4JGroup:default_group          | OC4J:oafm          |    |      down
OC4JGroup:default_group          | OC4J:oafm          |           down
OC4JGroup:default_group          | OC4J:oafm          |           down
OC4JGroup:default_group          | OC4J:oafm          |           down
OC4JGroup:default_group          | OC4J:oafm          |           down
OC4JGroup:default_group          | OC4J:forms         |           down
OC4JGroup:default_group          | OC4J:oacore        |   29781 | Alive
OC4JGroup:default_group          | OC4J:oacore        |   29780 | Alive
OC4JGroup:default_group          | OC4J:oacore        |   29778 | Alive
OC4JGroup:default_group          | OC4J:oacore        |   29779 | Alive
OC4JGroup:default_group          | OC4J:oacore        |   29777 | Alive
HTTP_Server                      | HTTP_Server        |   29720 | Alive


Symptoms
Unable to start oafm using adstrtal.sh. Running adoafmctl.sh ends with timeout.

You are running adoafmctl.sh version 120.6.12000000.3
Starting OPMN managed OAFM OC4J instance ...
adoafmctl.sh: exiting with status 152
adoafmctl.sh: check the logfile.

Cause
Existing JSP Tag Library Cache content was preventing the TLD Cache for the mapviewer app from initialising correctly.

Solution:

To implement the solution, please execute the following steps:

1)stop all middle tier services

2)Delete/backup all the files under directory:

cd $COMMON_TOP
mv _TldCache _TldCache_bkp

Take persistance backup or remove it.
==================================================

rm -rf $INST_TOP/ora/10.1.3/j2ee/oacore/persistence/*
rm -rf $INST_TOP/ora/10.1.3/j2ee/oafm/persistence/*
rm -rf $INST_TOP/ora/10.1.3/j2ee/forms/persistence/*
rm -f $INST_TOP/ora/10.1.3/opmn/logs/states/.opmndat


3)start all middle tier services

after doing above steps issue got resolved .

You are running adoacorectl.sh version 120.13

Checking status of OPMN managed OACORE OC4J instance ...

Processes in Instance: **************
---------------------------------+--------------------+---------+---------
ias-component                    | process-type       |     pid | status
---------------------------------+--------------------+---------+---------
OC4JGroup:default_group          | OC4J:oafm          |   30149 | Alive
OC4JGroup:default_group          | OC4J:oafm          |   30148 | Alive
OC4JGroup:default_group          | OC4J:oafm          |   30147 | Alive
OC4JGroup:default_group          | OC4J:oafm          |   30146 | Alive
OC4JGroup:default_group          | OC4J:oafm          |   30145 | Alive
OC4JGroup:default_group          | OC4J:forms         |   30075 | Alive
OC4JGroup:default_group          | OC4J:oacore        |   29781 | Alive
OC4JGroup:default_group          | OC4J:oacore        |   29780 | Alive
OC4JGroup:default_group          | OC4J:oacore        |   29778 | Alive
OC4JGroup:default_group          | OC4J:oacore        |   29779 | Alive
OC4JGroup:default_group          | OC4J:oacore        |   29777 | Alive
HTTP_Server                      | HTTP_Server        |   29720 | Alive


Reference
http://onlineappsdba.com/index.php/2009/07/23/adoacorectlsh-adformsctlsh-adoafmctlsh-exiting-with-status-204/
OAFM OC4J is Not Starting. Error: "time out while waiting for a managed process to start" [ID 952583.1]