Today we had an issue where just after recent refresh, users started complaining that they are not able to see the concurrent program log and out files. The error was "The Applications File Server could not open the file "path" for read.
How i did the troubleshooting.
1. We all know that RRA(FNDFS executable) is the default text viewer to view the out and log files. I verified that the profile option "RRA:Enabled is set to yes or not. It was set to yes in my case. If not, then change it to yes.
2. Then i verified whether the apps listener was running in all the nodes(it was shared appl top with 2 apps tier and 2 cm tier). It was running in all the nodes.
3. Then i started checking the files inside $TNS_ADMIN because RRA uses configuration file tnsnames.ora and listener.ora. As it was shared appl top so the FNDFS and FNDSM entry should have the details for all the nodes in tnsnames.ora file. In my case, there were few entries missing and i had to add them manually.
tnsnames.ora file.
example-FNDFS_nodename=(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=NODENAME)(PORT=1234))
(CONNECT_DATA=(SID=FNDFS)))
FNDFS_nodename_domainname=
FNDFS_SID_nodename=
Repeat the same and add all the nodes like above.
In the same way, add the FNDSM entries.
And then create a ifile and add all the entries.
ifile entry
FNDFS_APPLTOP_nodename1=(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=NODENAME1)(PORT=1234))
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=NODENAME2)(PORT=1234))
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=NODENAME3)(PORT=1234))
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=NODENAME4)(PORT=1234))
)
(CONNECT_DATA=(SID=FNDFS)))
Note: The FNDFS_APPL_TOP_nodename1 should have CM1 as first entry in app1 and app2 server.
Repeated the same in all the nodes and bounced the ebs services. Asked the users to validate the same and they confirmed that it is working fine.
Sample TNSNAMES.ORA(END Portion)
FNDFS_APPLTOP_nodename1=
(DESCRIPTION=
(ADDRESS_LIST=
(ADDRESS=(PROTOCOL=tcp)(HOST=CM node hostname)(PORT=port no))
(ADDRESS=(PROTOCOL=tcp)(HOST=app2 node)(PORT=port no))
(ADDRESS=(PROTOCOL=tcp)(HOST=app1 node)(PORT=port no))
(ADDRESS=(PROTOCOL=tcp)(HOST=cm 2 node)(PORT=port no))
(ADDRESS=(PROTOCOL=tcp)(HOST=dmz node)(PORT=port no))
(ADDRESS=(PROTOCOL=tcp)(HOST=dmz node)(PORT=port no)))
(CONNECT_DATA=
(SID=FNDFS)
)
)
How i did the troubleshooting.
1. We all know that RRA(FNDFS executable) is the default text viewer to view the out and log files. I verified that the profile option "RRA:Enabled is set to yes or not. It was set to yes in my case. If not, then change it to yes.
2. Then i verified whether the apps listener was running in all the nodes(it was shared appl top with 2 apps tier and 2 cm tier). It was running in all the nodes.
3. Then i started checking the files inside $TNS_ADMIN because RRA uses configuration file tnsnames.ora and listener.ora. As it was shared appl top so the FNDFS and FNDSM entry should have the details for all the nodes in tnsnames.ora file. In my case, there were few entries missing and i had to add them manually.
tnsnames.ora file.
example-FNDFS_nodename=(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=NODENAME)(PORT=1234))
(CONNECT_DATA=(SID=FNDFS)))
FNDFS_nodename_domainname=
FNDFS_SID_nodename=
Repeat the same and add all the nodes like above.
In the same way, add the FNDSM entries.
And then create a ifile and add all the entries.
ifile entry
FNDFS_APPLTOP_nodename1=(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=NODENAME1)(PORT=1234))
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=NODENAME2)(PORT=1234))
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=NODENAME3)(PORT=1234))
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=NODENAME4)(PORT=1234))
)
(CONNECT_DATA=(SID=FNDFS)))
Note: The FNDFS_APPL_TOP_nodename1 should have CM1 as first entry in app1 and app2 server.
Repeated the same in all the nodes and bounced the ebs services. Asked the users to validate the same and they confirmed that it is working fine.
Sample TNSNAMES.ORA(END Portion)
FNDFS_APPLTOP_nodename1=
(DESCRIPTION=
(ADDRESS_LIST=
(ADDRESS=(PROTOCOL=tcp)(HOST=CM node hostname)(PORT=port no))
(ADDRESS=(PROTOCOL=tcp)(HOST=app2 node)(PORT=port no))
(ADDRESS=(PROTOCOL=tcp)(HOST=app1 node)(PORT=port no))
(ADDRESS=(PROTOCOL=tcp)(HOST=cm 2 node)(PORT=port no))
(ADDRESS=(PROTOCOL=tcp)(HOST=dmz node)(PORT=port no))
(ADDRESS=(PROTOCOL=tcp)(HOST=dmz node)(PORT=port no)))
(CONNECT_DATA=
(SID=FNDFS)
)
)
No comments:
Post a Comment