Monday, November 16, 2015

Can't start listener after server reboot. TNS-12531: TNS:cannot allocate memory

Had an interesting problem after rebooting a test Oracle DB server. The listener won't come back properly post a reboot. It's simply hung and we found following errors in logfile eventually.

TNS-12531: TNS:cannot allocate memory

However this error is very misleading and confusing because there's plenty of memory on the host. As matter of fact the Oracle instance itself is not started yet. 
Found some hint from Oracle support doc. 

Oracle 11g Listener Services & Databases fails to start after server reboot

Apparently, the /etc/resolv.conf miss a domain in search string. We undertook some domain changes internally. Since all our DB servers are not managed by puppet, this host was overlooked.

Problem solved by adding proper domain names into /etc/resolv.conf
However still need to dig deeper as to why Oracle behave so strangely and spit out misleading errors.