Below is the document containing the most common database waits seen during the day to day activity of an Oracle DBA. It contains the possible causes and different ways to alleviate the wait.
Below is the ready reckoner for the location of various log files in Oracle apps 11i. This is very much helpful when troubleshooting any issue
- Database Tier
- Alert Log File – $ORACLE_HOME/admin/$CONTEXT_NAME/bdump/alert_$SID.log
- Trace file – $ORACLE_HOME/admin/SID_Hostname/udump
- Autoconfig Log file – $ORACLE_HOME/appsutil/log/SID_Hostname/DDMMTime/adconfig.log
- Application Tier
- Autoconfig Log file – $APPL_TOP/admin/SID_Hostname/log/DDMMTime/adconfig.log
- Start/Stop script log files – $COMMON_TOP/admin/log/CONTEXT_NAME/<mmddhhmi.log>
- Concurrent log file – $APPLCSF/$APPLLOG
- Patch Log file – $APPL_TOP/admin/<SID>/log
- Worker Log file – $APPL_TOP/admin/<SID>/log
- Apache Log file – $IAS_ORACLE_HOME/Apache/Apache/logs/
- Jserv log File – $IAS_ORACLE_HOME/Apache/Jserv/logs/
- JVM Log File – $IAS_ORACLE_HOME/Apache/Jserv/logs/jvm
- ICM log file – $APPLCSF/$APPLLOG/
- Workflow Mailer log file – $APPLCSF/$APPLLOG/FNDCPGSC*.txt
Support team reported an issue where they were not able to see some of the new users created in AD (Active Directory) flowing to OID (Oracle Internet Directory)
I started looking into the issue. I checked ODI synchronization services (odisrv process)
Logged into the infra tier and ran the command
$ $ORACLE_HOME/ldap/bin/ldapcheck Checking Oracle Internet Directory Processes ...ALL Process oidmon is Alive as PID 30596 Process oidldapd is Alive as PID 30603 Process oidldapd is Alive as PID 30609 Not Running ---- Process oidrepld Process odisrv is Alive as PID 17381
then checked the ldap synchronization log ($ORACLE_HOME/ldap/oid/log/_.aud
and this showed last successful synchronization happened on 23rd Feb 2015
No Errors were seen in the ldap trace file, oidsrv log or oidmon log.
I looked into the highest COMMITTED USN in the AD
$ ldapsearch -h -b "" -s base "objectclass=*" highestcommittedusn highestCommittedUSN=464557938
And searched for highest committed USN from the DIP profile
$ ldapsearch -h -p -D "cn=orcladmin" -w -b "cn=subscriber profile,cn=changelog subscriber,cn=oracle internet directory" -s sub "objectclass=*" > /tmp/PROD_DIP_Profile.txt $ grep -i "orclodipcondirlastappliedchgnum" /tmp/PROD_DIP_Profile1.txt $ orclodipcondirlastappliedchgnum=0 orclodipcondirlastappliedchgnum=455736457 orclodipcondirlastappliedchgnum=0 orclodipcondirlastappliedchgnum=0 orclodipcondirlastappliedchgnum=0 orclodipcondirlastappliedchgnum=0 orclodipcondirlastappliedchgnum=0 orclodipcondirlastappliedchgnum=0
So there was huge difference between the highestCommittedUSN between AD and that in the DIPprofile.
This is the reason Synchronization got stopped.
I followed below steps to make this synchronization work again.
1) Stop the odisrv process
$ oidctl connect= server=odisrv instance=1 stop
2) Disable the DIP profile
– Set the DISPLAY
– Open a VNC sessions and run the below command
Login to oid console with user as “cn=orcladmin”
Go to Integration Server
Select the profile which should be something like <OID_SID>_PortalImport
Edit the same and make it “Disable”
You can check this by running the below command on Infra Node.
ldapsearch –h [oid_hostname with fully qualified domain] –p 389 –D “cn=orcladmin” –w –b “orclodipAgentName=xxxx_PortalImport,cn=subscriber profile ,cn=changelog subscriber,cn=oracle internet directory” –s base “objectclass=*” orclodipagentcontrol
3) update the Highest committed USN in the DIP profile with that obtained from AD
— Now again edit the same profile and go into ‘Status’ Tab
— Update the value against “Last Applied Change Number” with the value obtained above from AD
4) Enable the profile again
— Edit the Profile and make it enable
5) Move the old and existing odisrv log, odimon log, synchronization log ($ORACLE_INFRA_HOME/ldap/oid/log/*.aud and *.trc)
6) Start the odisrv process
$oidctl connect==[tns_alias] server=odisrv instance=1 configset=1 flags=”host=[oid_hostname with fully qualified domain] port=389″ start
7) Run “ldapcheck” command to see the odisrv process.
$ORACLE_HOME/ldap/bin/ldapcheck Process oidmon is Alive as PID 28339 Process oidldapd is Alive as PID 28347 Process oidldapd is Alive as PID 28361 Not Running ---- Process oidrepld Process odisrv is Alive as PID 28348
Now again search for Highest Committed USN in AD and in DIP profile and it matches.
This resolved the issue and synchronization resumed between Microsoft AD and OID.