一次oom引发的无法提供服务

/ 后端 / 没有评论 / 430浏览

起因:

大年初一早上,安卓端又突然部分用户登录不上了,之前是我刚接到该项目时候,出现一次,然后修改了一些导致内存无法回收的业务代码; 赶紧看了下,发现负载均衡的其中一台服务器安卓端口异常断开无法连接; 登入服务器,发现服务还在,但是接口服务已经无法正常提供服务,处于假死状态,于是重启服务,暂时恢复正常; 在文件夹中发现该服务oom后产生的dump日志文件,下载到本地分析问题;

1.使用神器mat分析内存日志,查看内存占用饼图及提示的可能出现问题的原因,如图所示:

alt 其中一块内存占用高达89.86%,关键词:com.mysql.jdbc.MysqlIO.readSingleRowSet;

查看引用的对象,按照内存占用降序查看,找到大量的com.mysql.jdbc.ByteArrayRow对象,继续往里面看,是订单信息.到此很明显了,就是有大量查数据库操作导致的oom; alt 查看具体信息: alt

2.查看线程栈信息:

catalina-exec-23
  at com.mysql.jdbc.MysqlIO.nextRowFast([Lcom/mysql/jdbc/Field;IZIZZZ)Lcom/mysql/jdbc/ResultSetRow; (MysqlIO.java:2157)
  at com.mysql.jdbc.MysqlIO.nextRow([Lcom/mysql/jdbc/Field;IZIZZZLcom/mysql/jdbc/Buffer;)Lcom/mysql/jdbc/ResultSetRow; (MysqlIO.java:1964)
  at com.mysql.jdbc.MysqlIO.readSingleRowSet(JIIZ[Lcom/mysql/jdbc/Field;)Lcom/mysql/jdbc/RowData; (MysqlIO.java:3316)
  at com.mysql.jdbc.MysqlIO.getResultSet(Lcom/mysql/jdbc/StatementImpl;JIIIZLjava/lang/String;Z[Lcom/mysql/jdbc/Field;)Lcom/mysql/jdbc/ResultSetImpl; (MysqlIO.java:463)
  at com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(Lcom/mysql/jdbc/StatementImpl;IIIZLjava/lang/String;Lcom/mysql/jdbc/Buffer;ZJ[Lcom/mysql/jdbc/Field;)Lcom/mysql/jdbc/ResultSetImpl; (MysqlIO.java:3040)
  at com.mysql.jdbc.MysqlIO.readAllResults(Lcom/mysql/jdbc/StatementImpl;IIIZLjava/lang/String;Lcom/mysql/jdbc/Buffer;ZJ[Lcom/mysql/jdbc/Field;)Lcom/mysql/jdbc/ResultSetImpl; (MysqlIO.java:2288)
  at com.mysql.jdbc.MysqlIO.sqlQueryDirect(Lcom/mysql/jdbc/StatementImpl;Ljava/lang/String;Ljava/lang/String;Lcom/mysql/jdbc/Buffer;IIIZLjava/lang/String;[Lcom/mysql/jdbc/Field;)Lcom/mysql/jdbc/ResultSetInternalMethods; (MysqlIO.java:2681)
  at com.mysql.jdbc.ConnectionImpl.execSQL(Lcom/mysql/jdbc/StatementImpl;Ljava/lang/String;ILcom/mysql/jdbc/Buffer;IIZLjava/lang/String;[Lcom/mysql/jdbc/Field;Z)Lcom/mysql/jdbc/ResultSetInternalMethods; (ConnectionImpl.java:2551)
  at com.mysql.jdbc.PreparedStatement.executeInternal(ILcom/mysql/jdbc/Buffer;ZZ[Lcom/mysql/jdbc/Field;Z)Lcom/mysql/jdbc/ResultSetInternalMethods; (PreparedStatement.java:1861)
  at com.mysql.jdbc.PreparedStatement.execute()Z (PreparedStatement.java:1192)
  at com.zaxxer.hikari.pool.ProxyPreparedStatement.execute()Z (ProxyPreparedStatement.java:44)
  at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute()Z (Unknown Source)
  at org.apache.ibatis.executor.statement.PreparedStatementHandler.query(Ljava/sql/Statement;Lorg/apache/ibatis/session/ResultHandler;)Ljava/util/List; (PreparedStatementHandler.java:63)
  at org.apache.ibatis.executor.statement.RoutingStatementHandler.query(Ljava/sql/Statement;Lorg/apache/ibatis/session/ResultHandler;)Ljava/util/List; (RoutingStatementHandler.java:79)
  at org.apache.ibatis.executor.SimpleExecutor.doQuery(Lorg/apache/ibatis/mapping/MappedStatement;Ljava/lang/Object;Lorg/apache/ibatis/session/RowBounds;Lorg/apache/ibatis/session/ResultHandler;Lorg/apache/ibatis/mapping/BoundSql;)Ljava/util/List; (SimpleExecutor.java:63)
  at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(Lorg/apache/ibatis/mapping/MappedStatement;Ljava/lang/Object;Lorg/apache/ibatis/session/RowBounds;Lorg/apache/ibatis/session/ResultHandler;Lorg/apache/ibatis/cache/CacheKey;Lorg/apache/ibatis/mapping/BoundSql;)Ljava/util/List; (BaseExecutor.java:324)
  at org.apache.ibatis.executor.BaseExecutor.query(Lorg/apache/ibatis/mapping/MappedStatement;Ljava/lang/Object;Lorg/apache/ibatis/session/RowBounds;Lorg/apache/ibatis/session/ResultHandler;Lorg/apache/ibatis/cache/CacheKey;Lorg/apache/ibatis/mapping/BoundSql;)Ljava/util/List; (BaseExecutor.java:156)
  at org.apache.ibatis.executor.CachingExecutor.query(Lorg/apache/ibatis/mapping/MappedStatement;Ljava/lang/Object;Lorg/apache/ibatis/session/RowBounds;Lorg/apache/ibatis/session/ResultHandler;Lorg/apache/ibatis/cache/CacheKey;Lorg/apache/ibatis/mapping/BoundSql;)Ljava/util/List; (CachingExecutor.java:109)
  at org.apache.ibatis.executor.CachingExecutor.query(Lorg/apache/ibatis/mapping/MappedStatement;Ljava/lang/Object;Lorg/apache/ibatis/session/RowBounds;Lorg/apache/ibatis/session/ResultHandler;)Ljava/util/List; (CachingExecutor.java:83)
  at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(Ljava/lang/String;Ljava/lang/Object;Lorg/apache/ibatis/session/RowBounds;)Ljava/util/List; (DefaultSqlSession.java:148)
  at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(Ljava/lang/String;Ljava/lang/Object;)Ljava/util/List; (DefaultSqlSession.java:141)
  at sun.reflect.GeneratedMethodAccessor148.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (Unknown Source)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (Method.java:498)
  at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object; (SqlSessionTemplate.java:434)
  at com.sun.proxy.$Proxy113.selectList(Ljava/lang/String;Ljava/lang/Object;)Ljava/util/List; (Unknown Source)
  at org.mybatis.spring.SqlSessionTemplate.selectList(Ljava/lang/String;Ljava/lang/Object;)Ljava/util/List; (SqlSessionTemplate.java:231)
  at org.apache.ibatis.binding.MapperMethod.executeForMany(Lorg/apache/ibatis/session/SqlSession;[Ljava/lang/Object;)Ljava/lang/Object; (MapperMethod.java:137)
  at org.apache.ibatis.binding.MapperMethod.execute(Lorg/apache/ibatis/session/SqlSession;[Ljava/lang/Object;)Ljava/lang/Object; (MapperMethod.java:75)
  at org.apache.ibatis.binding.MapperProxy.invoke(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object; (MapperProxy.java:53)
  at com.sun.proxy.$Proxy139.queryList(Ljava/util/Map;)Ljava/util/List; (Unknown Source)
  at com.wfcm.service.impl.WfGoodsOrderServiceImpl.queryList(Ljava/util/Map;)Ljava/util/List; (WfGoodsOrderServiceImpl.java:82)
  at com.wfcm.service.impl.WfGoodsOrderServiceImpl$$FastClassBySpringCGLIB$$fc197e01.invoke(ILjava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (Unknown Source)
  at org.springframework.cglib.proxy.MethodProxy.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (MethodProxy.java:218)
  at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint()Ljava/lang/Object; (CglibAopProxy.java:779)
  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()Ljava/lang/Object; (ReflectiveMethodInvocation.java:163)
  at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed()Ljava/lang/Object; (CglibAopProxy.java:750)
  at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation()Ljava/lang/Object; (TransactionInterceptor.java:123)
  at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(Ljava/lang/reflect/Method;Ljava/lang/Class;Lorg/springframework/transaction/interceptor/TransactionAspectSupport$InvocationCallback;)Ljava/lang/Object; (TransactionAspectSupport.java:388)
  at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(Lorg/aopalliance/intercept/MethodInvocation;)Ljava/lang/Object; (TransactionInterceptor.java:119)
  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()Ljava/lang/Object; (ReflectiveMethodInvocation.java:186)
  at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed()Ljava/lang/Object; (CglibAopProxy.java:750)
  at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(Lorg/aopalliance/intercept/MethodInvocation;)Ljava/lang/Object; (ExposeInvocationInterceptor.java:97)
  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()Ljava/lang/Object; (ReflectiveMethodInvocation.java:186)
  at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed()Ljava/lang/Object; (CglibAopProxy.java:750)
  at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;Lorg/springframework/cglib/proxy/MethodProxy;)Ljava/lang/Object; (CglibAopProxy.java:692)
  at com.wfcm.service.impl.WfGoodsOrderServiceImpl$$EnhancerBySpringCGLIB$$70508291.queryList(Ljava/util/Map;)Ljava/util/List; (Unknown Source)
  at com.wfcm.controller.unclassified.OrderController.mine(Ljavax/servlet/http/HttpServletRequest;)Lcom/wfcm/utils/R; (OrderController.java:472)
  at com.wfcm.controller.unclassified.OrderController$$FastClassBySpringCGLIB$$81da07d2.invoke(ILjava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (Unknown Source)
  at org.springframework.cglib.proxy.MethodProxy.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (MethodProxy.java:218)
  at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint()Ljava/lang/Object; (CglibAopProxy.java:779)
  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()Ljava/lang/Object; (ReflectiveMethodInvocation.java:163)
  at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed()Ljava/lang/Object; (CglibAopProxy.java:750)
  at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(Lorg/aopalliance/intercept/MethodInvocation;)Ljava/lang/Object; (ExposeInvocationInterceptor.java:97)
  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()Ljava/lang/Object; (ReflectiveMethodInvocation.java:186)
  at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed()Ljava/lang/Object; (CglibAopProxy.java:750)
  at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;Lorg/springframework/cglib/proxy/MethodProxy;)Ljava/lang/Object; (CglibAopProxy.java:692)
  at com.wfcm.controller.unclassified.OrderController$$EnhancerBySpringCGLIB$$f7cebb39.mine(Ljavax/servlet/http/HttpServletRequest;)Lcom/wfcm/utils/R; (Unknown Source)
  at sun.reflect.GeneratedMethodAccessor987.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (Unknown Source)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (Method.java:498)
  at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke([Ljava/lang/Object;)Ljava/lang/Object; (InvocableHandlerMethod.java:197)
  at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(Lorg/springframework/web/context/request/NativeWebRequest;Lorg/springframework/web/method/support/ModelAndViewContainer;[Ljava/lang/Object;)Ljava/lang/Object; (InvocableHandlerMethod.java:141)
  at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(Lorg/springframework/web/context/request/ServletWebRequest;Lorg/springframework/web/method/support/ModelAndViewContainer;[Ljava/lang/Object;)V (ServletInvocableHandlerMethod.java:106)
  at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Lorg/springframework/web/method/HandlerMethod;)Lorg/springframework/web/servlet/ModelAndView; (RequestMappingHandlerAdapter.java:894)
  at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Lorg/springframework/web/method/HandlerMethod;)Lorg/springframework/web/servlet/ModelAndView; (RequestMappingHandlerAdapter.java:808)
  at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljava/lang/Object;)Lorg/springframework/web/servlet/ModelAndView; (AbstractHandlerMethodAdapter.java:87)
  at org.springframework.web.servlet.DispatcherServlet.doDispatch(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V (DispatcherServlet.java:1060)
  at org.springframework.web.servlet.DispatcherServlet.doService(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V (DispatcherServlet.java:962)
  at org.springframework.web.servlet.FrameworkServlet.processRequest(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V (FrameworkServlet.java:1006)
  at org.springframework.web.servlet.FrameworkServlet.doPost(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V (FrameworkServlet.java:909)
  at javax.servlet.http.HttpServlet.service(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V (HttpServlet.java:648)
  at org.springframework.web.servlet.FrameworkServlet.service(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V (FrameworkServlet.java:883)
  at javax.servlet.http.HttpServlet.service(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (HttpServlet.java:729)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (ApplicationFilterChain.java:292)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (ApplicationFilterChain.java:207)
  at org.apache.tomcat.websocket.server.WsFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V (WsFilter.java:52)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (ApplicationFilterChain.java:240)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (ApplicationFilterChain.java:207)
  at com.wfcm.config.filter.CrosFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V (CrosFilter.java:36)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (ApplicationFilterChain.java:240)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (ApplicationFilterChain.java:207)
  at org.springframework.web.filter.RequestContextFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V (RequestContextFilter.java:100)
  at org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V (OncePerRequestFilter.java:119)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (ApplicationFilterChain.java:240)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (ApplicationFilterChain.java:207)
  at org.springframework.web.filter.FormContentFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V (FormContentFilter.java:93)
  at org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V (OncePerRequestFilter.java:119)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (ApplicationFilterChain.java:240)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (ApplicationFilterChain.java:207)
  at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V (WebMvcMetricsFilter.java:93)
  at org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V (OncePerRequestFilter.java:119)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (ApplicationFilterChain.java:240)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (ApplicationFilterChain.java:207)
  at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V (ErrorPageFilter.java:126)
  at org.springframework.boot.web.servlet.support.ErrorPageFilter.access$000(Lorg/springframework/boot/web/servlet/support/ErrorPageFilter;Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V (ErrorPageFilter.java:64)
  at org.springframework.boot.web.servlet.support.ErrorPageFilter$1.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V (ErrorPageFilter.java:101)
  at org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V (OncePerRequestFilter.java:119)
  at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V (ErrorPageFilter.java:119)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (ApplicationFilterChain.java:240)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (ApplicationFilterChain.java:207)
  at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V (CharacterEncodingFilter.java:201)
  at org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V (OncePerRequestFilter.java:119)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (ApplicationFilterChain.java:240)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (ApplicationFilterChain.java:207)
  at org.apache.catalina.core.StandardWrapperValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V (StandardWrapperValve.java:212)
  at org.apache.catalina.core.StandardContextValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V (StandardContextValve.java:94)
  at org.apache.catalina.authenticator.AuthenticatorBase.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V (AuthenticatorBase.java:492)
  at org.apache.catalina.core.StandardHostValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V (StandardHostValve.java:141)
  at org.apache.catalina.valves.ErrorReportValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V (ErrorReportValve.java:80)
  at org.apache.catalina.valves.AbstractAccessLogValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V (AbstractAccessLogValve.java:620)
  at org.apache.catalina.core.StandardEngineValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V (StandardEngineValve.java:88)
  at org.apache.catalina.connector.CoyoteAdapter.service(Lorg/apache/coyote/Request;Lorg/apache/coyote/Response;)V (CoyoteAdapter.java:502)
  at org.apache.coyote.http11.AbstractHttp11Processor.process(Lorg/apache/tomcat/util/net/SocketWrapper;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState; (AbstractHttp11Processor.java:1152)
  at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(Lorg/apache/tomcat/util/net/SocketWrapper;Lorg/apache/tomcat/util/net/SocketStatus;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState; (AbstractProtocol.java:684)
  at org.apache.tomcat.util.net.Nio2Endpoint$SocketProcessor.doRun()V (Nio2Endpoint.java:1097)
  at org.apache.tomcat.util.net.Nio2Endpoint$SocketProcessor.run()V (Nio2Endpoint.java:1056)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V (ThreadPoolExecutor.java:1149)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run()V (ThreadPoolExecutor.java:624)
  at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run()V (TaskThread.java:61)
  at java.lang.Thread.run()V (Thread.java:748)

看到了其中的业务代码,OrderController下的mine方法;

com.wfcm.controller.unclassified.OrderController.mine

3.定位到业务代码:

Map<String, Object> map = new HashMap<String, Object>();
        Integer uid = ISsoLoginHelper.confirm(req);
        //获取待付款个数
        map.put("uid", uid);
        //获取已完成个数
        int alreadyPayTotal = goodsOrderService.queryAlreadyPay(map);
        int allTotal = goodsOrderService.queryTotal(map);
        //获取优惠券个数
        int couponsTotal = memberCouponsService.queryTotal(map);

        map.put("type", 1);//商品类型(1.课程 2.章节3.vip卡4商品)
        int courseCollect = collectionService.queryTotal(map);
        map.put("type", 2);
        int chapterCollect = collectionService.queryTotal(map);
        map.put("type", 3);
        int vipCollect = collectionService.queryTotal(map);

        *//*recordService.selectByCreateTime();*//*

        //获取已购课程个数
        Map<String, Object> map1 = new HashMap<String, Object>();
        map1.put("uid", uid);
        map1.put("orderStatus", 4);//订单状态(1.待付款2.待收货3.待评价4.已完成5.取消订单)
        List<WfGoodsOrderEntity> goodsOrderList = goodsOrderService.queryList(map1);
        List<WfGoodsOrderRelationEntity> goodsCourseList = new ArrayList<WfGoodsOrderRelationEntity>();//获取所有已完成订单
        List<WfGoodsOrderRelationEntity> goodsVipList = new ArrayList<WfGoodsOrderRelationEntity>();//获取所有已完成订单
        for (int i = 0; i < goodsOrderList.size(); i++) {
            WfGoodsOrderEntity goodsOrderEntity = goodsOrderList.get(i);
            map.put("orderCode", goodsOrderEntity.getOrdercode());
            List<WfGoodsOrderRelationEntity> courseList = goodsOrderRelationService.queryList(map);//根据订单号获取其中所有课程
            for (int j = 0; j < courseList.size(); j++) {
                int m = courseList.get(j).getGoodssort();
                if (m == 1 || m == 2) {
                    goodsCourseList.add(courseList.get(j));
                } else if (m == 3) {
                    goodsVipList.add(courseList.get(j));
                }
            }
        }

        Map<String, Object> resultMap = new HashMap<String, Object>();
        resultMap.put("allTotal", allTotal);
        resultMap.put("alreadyPayTotal", alreadyPayTotal);
        resultMap.put("couponsTotal", couponsTotal);
        resultMap.put("courseCollect", courseCollect);
        resultMap.put("chapterCollect", chapterCollect);
        resultMap.put("allCollect", chapterCollect + courseCollect + vipCollect);
        resultMap.put("vipCollect", vipCollect);
        resultMap.put("courseTotal", goodsCourseList.size());
        resultMap.put("vipTotal", goodsVipList.size());
        return R.ok(resultMap);

(1)令人窒息的循环查询操作;不过就算单个用户查询全量订单,数据量也不算台打,也不至于oom; (2)通过深入查看内存日志中占用较大的数据库返回数据,可以看到有不同用户的订单数据,那么就是筛选用户id没有生效;所以问题出现在这里.查询了全量订单数据; (3)问题出在Integer uid = ISsoLoginHelper.confirm(req);具体为uid获取后为null,然后下面的查询方法,如果为null则不作为筛选条件查询;

可以看到如下图,用于筛选的条件map中,uid为null:

alt

4.解决问题

(1)查看日志,其他端未发生oom,并且其他端也没有,只有安卓端;于是查看请求日志发现只有这一端再用;结合前端业务可知,该业务方法其实已经不需要,只是安卓没有删除,依然在使用; (2)直接跟安卓端沟通后,后端禁用该接口;安卓下个版本删除该逻辑;

5.分析

(1)该接口是需要认证登录的,所以如果进入到该接口,正常来说uid不可能为null; (2)查看ISsoLoginHelper.confirm(req)方法,是根据token取出用户信息;这一步方法在其他地方很少使用了,因为是个重复获取redis信息的操作,其他地方在拦截器验证登录时,使用了本地变量里保存用户信息了; (3)想到安卓最近一段时间,更新某个版本后,偶尔会出现用户异常退出登录的问题,原因是token失效;目前用户的登录有限制,会挤掉以前的登录信息; (4)于是分析可知,在以下条件下可能产生问题: 1.该接口在安卓端是个异步请求,有其他接口同时进行请求;此时正常认证用户进入该接口方法; 2.安卓端此时再次出现了将自己登录信息挤掉的操作,由于ISsoLoginHelper.confirm(req)方法获取用户信息需要与redis交互,并且此时已经可能会出现跟拦截器验证用户时的登录信息'数据不一致';所以获取uid为null; (5)禁用该无用接口后,其他涉及ISsoLoginHelper.confirm(req)获取用户信息的方法,修改为从本地变量获取; (6)查找屎山代码中,可能出现全量查询大量数据的操作,进行修改;