arjuna: TransactionReaper::check timeout for TX

Please help me solve one problem. Our project uses Java 11, WildFly 20.0.1, and MS-SQL. The problem is that when users try to open some windows for editing information, for example, Clients or State in the web application, the error `java.sql.SQLTimeoutException: The query has timed out.` appears. This happens when the audit.jsp file is loaded, which is used in four similar windows as an iframe, but the error appears in only two of them and this is confusing because, in these windows, the functionality that calls audit.jsp is the same. Stacktrace here, due to the post char limit.

Also part of the code from the audit.jsp where the error comes from:

<script type="text/javascript" <cms:nonce/>>                toolbarDisabledImages = "../images/disabled/";                toolbarEnabledImages = "../images/enabled/";                var Bar = new ToolBar("Bar","tbFrameToolBar", document.getElementById("ToolBarId"), true);                Bar.addItem(new Handle());                Bar.addItem(new Button("Prev","Previous Page","16x16/page_prev.gif", null, false, page_prev, true));                Bar.addItem(new Button("Next","Next Page","16x16/page_next.gif", null, false, page_next, true));                Bar.addControl('<P class="textCommon textBlackB">Table:</P>',true);                Bar.addControl("<SELECT id='tableId' class='dlgInput dlgText'>"+                               "<OPTION>All</OPTION>"+                        <ejbexec:bodycall model="model" classname="CmsCaseEJB" methodname="getTables"                            paramtypes="<%=new Class[]{Long.class, Integer.class}%>" paramvalues="<%=new Object[]{objectId, type}%>">                               "<OPTION value=\"<%=model.getObjectId()%>\" <%=model.getObjectId().equals(tableId) ? "selected" : ""%>><%=Utils.formatScriptHTML(model.getObjectValue())%></OPTION>"+                        </ejbexec:bodycall>                               "</SELECT>",true);

But according to the log file, the root of the problem is in two places, namely here:

public static ArrayList load(Connection conn, String query, Class modelClass, PagingContext pc) throws SQLException {  /*45 line*/ try (Statement stmt = conn.createStatement(); ResultSet rset = stmt.executeQuery(query.toUpperCase())) {            return load(rset, modelClass, pc);        }    }

And here:

public void initObjectModelList() throws Exception {        // Get EJB remote interface        Class cls = null;        // Invoke remote EJB method        Object result = null;        try {            Object ejb = Utils.getInstanceByClassName(getClassname());            cls = ejb.getClass(); /*115 line*/ result = cls.getMethod(getMethodname(), getParamtypes()).invoke(ejb, getParamvalues());        } catch (NoSuchMethodException e) {            // we want to include searched method into exception message            throw new NoSuchMethodException(                        "EJBExec, found class: " + Utils.EJB_PACKAGE + "." + getClassname() +                        "\nMethod: " + getMethodSignature() + " not found.");        } catch (Exception ex) {            System.out.println("Class name: " + (cls != null ? cls.getName() : "Unknown!!!") + ", declares " + (cls != null ? cls.getDeclaredMethods().length : "???") + " methods.");            System.out.println("Exception while calling: " + getMethodSignature());            throw ex;        }

If clients try to execute an SQL query, that does not work, manually, through MS SQL Server, it works fine.
But what is worse is that there is no such problem on the localhost as well as on our UAT, when the client has it both on Prod and UAT.

We tried the solution from this question, namely the first post, but it didn't help.

If you need, I can provide more logs or some additional information.

Thank you!

EDIT:

Add a link to the log file at OP.

Welcome to CodeRanch, Bebra.

I'm sorry, I disabled the link to your stack trace. We do not want our users to download files from unknown origins, especially not when the origins are non-English.

Please share your stack trace by posting it (or if it's too big, only the relevant part) on our website directly, or use a site like Paste Bin.

It would also be helpful if you posted the EJB class you're calling, and your transaction configuration.

Thank you! I can do it like this through PasteBin?

Welcome to the Ranch (again)

Please don't quote the whole of the preceding post; that adds nothing new and is liable to removal. I could see a stack trace in your pastebin link. Some of the entries seem to be only WARN, however. I have copied it for the convenience of readers; I am presuming there is nothing confidential in it.

2024-01-25 13:55:28,088  WARN  [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffffc0a864c9:30ba8c4b:65afc50d:52dd2c8 in state  RUN

2024-01-25 13:55:28,088  WARN  [com.arjuna.ats.arjuna] (Transaction Reaper Worker 1) ARJUNA012095: Abort of action id 0:ffffc0a864c9:30ba8c4b:65afc50d:52dd2c8 invoked while multiple threads active within it.

2024-01-25 13:55:28,088  WARN  [com.arjuna.ats.arjuna] (Transaction Reaper Worker 1) ARJUNA012381: Action id 0:ffffc0a864c9:30ba8c4b:65afc50d:52dd2c8 completed with multiple threads - thread default task-10662 was in progress with java.base@11.0.2/java.net.SocketInputStream.socketRead0(Native Method)
java.base@11.0.2/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
java.base@11.0.2/java.net.SocketInputStream.read(SocketInputStream.java:168)
java.base@11.0.2/java.net.SocketInputStream.read(SocketInputStream.java:140)
com.microsoft.sqlserver//com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:2026)
com.microsoft.sqlserver//com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:6446)
com.microsoft.sqlserver//com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7610)
com.microsoft.sqlserver//com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteCursored(SQLServerStatement.java:2028)
com.microsoft.sqlserver//com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:846)
com.microsoft.sqlserver//com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:767)
com.microsoft.sqlserver//com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7225)
com.microsoft.sqlserver//com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:3053)
com.microsoft.sqlserver//com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:247)
com.microsoft.sqlserver//com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:222)
com.microsoft.sqlserver//com.microsoft.sqlserver.jdbc.SQLServerStatement.executeQuery(SQLServerStatement.java:692)
org.jboss.ironjacamar.jdbcadapters@1.4.22.Final//org.jboss.jca.adapters.jdbc.WrappedStatement.executeQuery(WrappedStatement.java:397)
deployment.CMS.ear.cmsEJB.jar//com.bebra.util.ModelLoader.load(ModelLoader.java:45)
deployment.CMS.ear.cmsEJB.jar//com.bebra.util.ModelLoader.loadList(ModelLoader.java:37)
deployment.CMS.ear.cmsEJB.jar//com.bebra.cms.ejb.CmsCaseEJB$DAO.getTables(CmsCaseEJB.java:3121)
deployment.CMS.ear.cmsEJB.jar//com.bebra.cms.ejb.CmsCaseEJB.getTables(CmsCaseEJB.java:158)
jdk.internal.reflect.GeneratedMethodAccessor1309.invoke(Unknown Source)
java.base@11.0.2/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.base@11.0.2/java.lang.reflect.Method.invoke(Method.java:566)
org.jboss.as.ee@20.0.1.Final//org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
org.jboss.as.weld.common@20.0.1.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:79)
org.jboss.as.weld.common@20.0.1.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:89)
org.jboss.as.weld.common@20.0.1.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:102)
org.jboss.as.ee@20.0.1.Final//org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.as.ejb3@20.0.1.Final//org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.as.jpa@20.0.1.Final//org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.as.ee@20.0.1.Final//org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:40)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53)
org.jboss.as.ee@20.0.1.Final//org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.as.ejb3@20.0.1.Final//org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.as.ejb3@20.0.1.Final//org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.as.ejb3@20.0.1.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252)
org.jboss.as.ejb3@20.0.1.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:388)
org.jboss.as.ejb3@20.0.1.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:158)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
org.jboss.weld.core@3.1.4.Final//org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:72)
org.jboss.as.weld.common@20.0.1.Final//org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.as.ejb3@20.0.1.Final//org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.as.ejb3@20.0.1.Final//org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.as.ejb3@20.0.1.Final//org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.as.ejb3@20.0.1.Final//org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.as.ejb3@20.0.1.Final//org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.as.ejb3@20.0.1.Final//org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.as.ee@20.0.1.Final//org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438)
org.wildfly.security.elytron-private@1.12.1.Final//org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:627)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
org.jboss.invocation@1.5.2.Final//org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53)
org.jboss.as.ee@20.0.1.Final//org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198)
org.jboss.as.ee@20.0.1.Final//org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:191)
org.jboss.as.ee@20.0.1.Final//org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81)
deployment.CMS.ear.cmsEJB.jar//com.bebra.cms.ejb.CmsCaseEJB$$$view29.getTables(Unknown Source)
jdk.internal.reflect.GeneratedMethodAccessor1308.invoke(Unknown Source)
java.base@11.0.2/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.base@11.0.2/java.lang.reflect.Method.invoke(Method.java:566)
deployment.CMS.ear.cmsWEB.war//com.bebra.taglib.ejbexec.EJBExecTag.initObjectModelList(EJBExecTag.java:115)

Thank you (again) and sorry, make a post here the first time

Think nothing of it.

Can you show us CmsCaseEJB.getTables(), DAO.getTables() and ModelLoader.loadList()?

CmsCaseEJB.getTables():

/**     * get list of all system tables which has at least one record in history     *     * @return list of all tables which has at least one record in history     * @ejb:interface-method     * @ejb:transaction type="Supports"     */    public List<BasicModel> getTables(Long objectId, Integer type) throws ApplicationException {        try (DAO caseDao = dao) {            log.debug("getTables(" + objectId + ", " + type + ")");            caseDao.openConnection();            return caseDao.getTables(objectId, type);        } catch (Exception e) {            log.error("getTables", e);            throw new ApplicationException(e);        }    }

DAO.getTables():

public List<BasicModel> getTables(Long objectId, Integer type) throws Exception {            String where;            switch (type) {                case Consts.OWNER_TYPE_OFFICE:                    where = "OFFICE_ID=" + objectId;                    break;                case Consts.OWNER_TYPE_CLIENT:                    where = "CLIENT_ID=" + objectId;                    break;                case Consts.OWNER_TYPE_COURT:                    where = "COURT_ID=" + objectId;                    break;                case Consts.OWNER_TYPE_WIZARD:                    where = "WIZARD_ID=" + objectId;                    break;                case Consts.OWNER_TYPE_CASEID:                    where = "CASE_FILE_ID=(select CASE_FILE_ID from CMS_CASE where CASE_ID=" + objectId + ") and (CASE_ID is null or CASE_ID=" + objectId + ")";                    break;                default:                    where = "1 = 1";                    break;            }            String query = "select distinct t.TABLE_ID objectId, TABLE_SCR_NAME objectValue from CMS_TABLE t inner join CMS_HISTORY h on (t.TABLE_ID=h.TABLE_ID) " +                    "where " + where + " order by TABLE_SCR_NAME";            return ModelLoader.loadList(connection, query, BasicModel.class);        }

ModelLoader.loadList():

public static <T> List<T> loadList(Connection conn, String query, Class modelClass) throws SQLException {        return load(conn, query, modelClass, null);    }

How is dao initialized in CmsCaseEJB?

How is connection initialized in DAO?

Also, are there any annotations present on these classes and fields?

The critical information here is that it's a JDBC Timeout Exception. That means that the JDBC driver made a request to the MS SQL Server and did not receive a response within the alloted timeout interval. I believe that arjuna is one of the available SQL server drivers these days.

That cam happen if you ask the server to do too much processing within a single JDBC request. "Too much processing" meaning that you're doing something that, for example, will return 1 lakh rows while doing multiple joins without suitable indexes.

It's possible that the SQL Server DBA can provide you with diagnostic and tuning information that will improve the efficiency of your SQL request.

The other alternative is to tune the Connection Pool parameters to permit a longer iimeout interval.

I must note, however, that since this is WildFly that webapps should generally not make excessively-ambitious requests as you can bog down WildFly's ability to service other user requests. An HTTP request/response must always strive to take as little time as possible. Plus, the client itself may timeout if an HTTP response doesn't come back before its own timeout limit expires.

As a standard side note, I am also obliged to note that you should be using a Connection Pool in webapps that use JDBC and not Driver getConnection and NEVER store a Connection between requests. Always obtain your Connections from the server Connection pool, use them and close them as soon as possible before returning from your request handler.