We suddenly have an issue that the Planning forms will not open. If we restart the services they open fine, but within an hour or two it starts again and you can pick the form off the list but it just hangs and never opens. Same in Smartview and Web. on version 9.3.1

Hi, Darin.

Have you looked at the (many) Planning logs? They are difficult (to me at least) to read and make sense of, but you may get a hint out of a Java stack dump as to what is going on, or at least have something to go to support with.

Also, have you changed any of your forms? Made any of them really big and caused Planning to lose its marbles? I wonder if it Planning is running out of memory on another series of forms and it that problem manifests itself by all forms not working.

Regards,

Cameron Lackpour

Hey Cameron,

Yes, we are trying to dig through them, but have not turned up anything that looks useful. I was thinking maybe memory, because it works great when the service first comes up but then hangs after some time of use. I will monitor memory and see how it looks over time.

Thanks for the leads. And really got a lot out of your presentations last week, thanks

Darin

>>Thanks for the leads. And really got a lot out of your presentations last week, thanks

It's gratifying to hear that you found the presentations useful. Kaleidoscope is an awesome conference.

Regards,

Cameron Lackpour

As Cameron mentioned a large form could be causing this behavior.

What's bad is that when a user logs out it will tend to re-open the same form they were on and if you have a form which has a lot of cells it can wreak havoc.

Check out the Planning Service in task manager and see if it's using a lot of memory and/or CPU when this is happening.

Also if you have users seeing the freeze you can look at the memory consumption of their web browser for really large forms you will see IE/Firefox use up to 500MB of ram on the client side.

On the planning server also review the memory settings for Planning and bump them up if they are defaults. I've also seen people suggest changing the stack space to 4mb for really large forms -- you may want to be a little more conservative and use 2m for your -Xss and 1024m for your -Xmx settings.

The Xss and Xmx settings are under the key:
HKEY_LOCAL_MACHINE\SOFTWARE\Hyperion Solutions\HyperionPlanning\HyS9PlanningI

Regards,

John

Don't have any very large forme, and none have changes for months. But the issue suddenly started Monday when we restarted the services and has been a persistant issue since.

>>What's bad is that when a user logs out it will tend to re-open the same form they were on and if you have a form which has a lot of cells it can wreak havoc.

I totally agree -- this can be breathtakingly annoying.

At least with SmartView you have to choose the form before it opens. If Task Lists were there in 9.3.1 I think almost everyone would be on SV already.

Per the proposed Planning roadmap, in a short time the clients will be at parity. That will make the web vs. SmartView selection process interesting.

Regards,

Cameron Lackpour

> Per the proposed Planning roadmap, in a short time the clients will be at parity. That will make the web vs. SmartView selection process interesting.

No! You have to stay on the web and use nifty JavaScript customizations!

-- Joe

.

Regarding the stack spaces I noticed it was pretty high for workspace and we were seeing weird issues with logins -- after making the stack space on Weblogic Workspace more like defaults (128k) it seemed to clear things right up. So I'm not sure that changing stack size is a good idea in general -- the heap size (mx) is definitely something to change like you found out.

Regards,

John A. Booth
http://www.metavero.com

Here is the entry in the weblogic log when the problem occurs. I am not a strong weblogic person, but since we lost the perosn in the group that was doing that, we will have to bone up on it.

So know its stuck and then times out, but not sure if its in Planning Form or waiting on Essbase response. Even if I try the smallest form it happens, but only after Planning has been running a while. So memory issues seem likely.


####<Jun 30, 2009 8:46:03 PM CDT> <Error> <WebLogicServer> <hsf-corphyp08> <HyperionPlanning> <[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1246412763247> <BEA-000337> <[STUCK] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "707" seconds working on the request "Http Request: /HyperionPlanning/SmartView", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
Thread-17 "[STUCK] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, waiting, priority=1, DAEMON> {
-- Waiting for notification on: com.sun.jndi.ldap.LdapRequest@16fb5b6[fat lock]
jrockit.vm.Threads.waitForSignalWithTimeout(Native Method)
java.lang.Object.wait(Native Method)
com.sun.jndi.ldap.Connection.readReply(Connection.java:418)
^-- Lock released while waiting: com.sun.jndi.ldap.LdapRequest@16fb5b6[fat lock]
com.sun.jndi.ldap.LdapClient.getSearchReply(LdapClient.java:611)
com.sun.jndi.ldap.LdapClient.search(LdapClient.java:534)
com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:1958)
com.sun.jndi.ldap.LdapCtx.doSearchOnce(LdapCtx.java:1893)
com.sun.jndi.ldap.LdapCtx.c_lookup(LdapCtx.java:987)
com.sun.jndi.toolkit.ctx.ComponentContext.p_lookup(ComponentContext.java:526)
com.sun.jndi.toolkit.ctx.PartialCompositeContext.lookup(PartialCompositeContext.java:159)
com.sun.jndi.toolkit.ctx.PartialCompositeContext.lookup(PartialCompositeContext.java:148)
javax.naming.InitialContext.lookup(InitialContext.java:351)
com.hyperion.css.spi.impl.nv.JNDIHelper.isContextActive(Unknown Source)
^-- Holding lock: javax.naming.directory.InitialDirContext@186765[thin lock]
com.hyperion.css.spi.impl.nv.JNDIHelper.isContextActive(Unknown Source)
com.hyperion.css.spi.impl.nv.JNDIHelper.getDirectoryContextForRead(Unknown Source)
com.hyperion.css.spi.impl.nv.JNDIHelper.getDirectoryContext(Unknown Source)
com.hyperion.css.spi.impl.nv.JNDIHelper.getDirectoryContext(Unknown Source)
com.hyperion.css.spi.impl.nv.NativeProviderBase.getBaseContext(Unknown Source)
com.hyperion.css.spi.impl.nv.NativeProvider.getIndirectRolesListForEntries(Unknown Source)
com.hyperion.css.spi.impl.nv.NativeProvider.getRolesListForEntries(Unknown Source)
com.hyperion.css.spi.impl.nv.NativeProvider.getRolesListForEntry(Unknown Source)
com.hyperion.hbr.security.HbrSecurityAPIHelper.getMergedCSSentriesArray(Unknown Source)
com.hyperion.hbr.security.HbrSecurityAPIHelper.updateAllHashes(Unknown Source)
com.hyperion.hbr.security.HbrSecurityAPI.getUserOrGroupRoles(Unknown Source)
com.hyperion.hbr.db.DBUsers.getRolesForUser(Unknown Source)
com.hyperion.hbr.core.UserManagerServer.getRolesForUser(Unknown Source)
com.hyperion.hbr.core.AccessControlManagerServer.getOnlyUserPrivileges(Unknown Source)
com.hyperion.hbr.core.AccessControlManagerServer.getUserPrivileges(Unknown Source)
com.hyperion.hbr.core.UserManagerServer.authenticateUser(Unknown Source)
com.hyperion.hbr.api.HBRServerLocal.init(Unknown Source)
com.hyperion.hbr.api.HBRServerLocal.<init>(Unknown Source)
jrockit.vm.RNI.c2java(Native Method)
jrockit.reflect.NativeMethodInvoker.invoke0(Native Method)
jrockit.reflect.NativeConstructorInvoker.newInstance(Unknown Source)
java.lang.reflect.Constructor.newInstance(Unknown Source)
com.hyperion.hbr.api.thin.HBR.getHBRServerImpl(Unknown Source)
com.hyperion.hbr.api.thin.HBR.getHBRServer(Unknown Source)
com.hyperion.hbr.api.thin.HBR.getRTPs(Unknown Source)
com.hyperion.planning.db.HspFMDBImpl.getRTPs(Unknown Source)
com.hyperion.planning.db.HspFMDBImpl.getRTPs(Unknown Source)
com.hyperion.planning.db.HspFMDBImpl.getHBRRunOnSaveRTP(Unknown Source)
com.hyperion.planning.HspSmartViewProvider.bHasRTPonsave(Unknown Source)
com.hyperion.planning.HspSmartViewProvider.getFormGrid(Unknown Source)
com.hyperion.planning.HspSmartViewProvider.getFormSlice(Unknown Source)
jrockit.vm.RNI.c2java(Native Method)
jrockit.reflect.NativeMethodInvoker.invoke0(Native Method)
jrockit.reflect.NativeMethodInvoker.invoke(Unknown Source)
jrockit.reflect.VirtualNativeMethodInvoker.invoke(Unknown Source)
java.lang.reflect.Method.invoke(Unknown Source)
org.apache.commons.beanutils.MethodUtils.invokeMethod(MethodUtils.java:252)
org.apache.commons.digester.CallMethodRule.end(CallMethodRule.java:585)
org.apache.commons.digester.Rule.end(Rule.java:228)
org.apache.commons.digester.Digester.endElement(Digester.java:1067)
com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:633)
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1241)
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(XMLDocumentFragmentScannerImpl.java:1685)
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:368)
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:834)
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:764)
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:148)
com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1242)
weblogic.xml.jaxp.WebLogicXMLReader.parse(WebLogicXMLReader.java:133)
weblogic.xml.jaxp.RegistryXMLReader.parse(RegistryXMLReader.java:153)
org.apache.commons.digester.Digester.parse(Digester.java:1591)
HspSmartViewServlet.Handle(Unknown Source)
HspSmartViewServlet.Handle(Unknown Source)
HspSmartViewServlet.doPost(Unknown Source)
javax.servlet.http.HttpServlet.service(HttpServlet.java:763)
javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:225)
weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:127)
weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:272)
weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
HspValidationFilter.doFilter(Unknown Source)
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3151)
weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:1973)
weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:1880)
weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1310)
weblogic.work.ExecuteThread.execute(ExecuteThread.java:207)
weblogic.work.ExecuteThread.run(ExecuteThread.java:179)
jrockit.vm.RNI.c2java(Native Method)
}

One of the earlier things I see in that web logic dump is related to LDAP -- com.sun.jndi.ldap.LdapRequest

Suggest you bounce Shared Services, review it's log files make sure it seems ok then restart Planning.

GL,

John

in the docs it refers to shared services responsiveness:
When Shared Services is configured with WebLogic 9.2.1 and Oracle 11g, Shared Services
provisioning, Oracle Hyperion Enterprise Performance Management System Lifecycle
Management, or EPM System auditing may stop responding. To resolve this issue, use the
WebLogic Console to change the Stuck Thread Max Time value to 60000 and the Accept Backlog
value to 50000 for Shared Services.

Your error references a stuck thread timeout, perhaps this may be related?

We increased the heap size and have not seen the issue again. Also the speed has returned to normal. thanks all

Try looking at the values of your heap sizes for EAS and Planning, recommended setting of 1024 min and max no greater than 1280.