我们是nginx做域名转发到workerman的, php start.php status查看的连接数,每次这个连接数上到25000,那么BusinessWorker进程全部busy,导致nginx的work进程的CPU飙升到100%,每次的处理方案是执行重启workerman就好,是我的业务逻辑复杂还是有些连接已关闭但是没有释放呀,这样的情况持续一个月
看看是不是没优化linux内核,安装event扩展
谢答,已优化linux内核,event 也安装啦;下面我把strace分析与lsof分析的结果贴一下
谢答,已优化linux内核,event 也安装啦 通过strace -ttp 分析
09:21:20.539487 sendto(17, "1\r\n$4\r\nPING\r\n", 14, MSG_DONTWAIT, NULL, 0) = 14 09:21:20.539829 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:20.539946 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:20.543752 recvfrom(17, "+PONG\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 7 09:21:20.544123 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:20.544246 sendto(17, "3\r\n$4\r\nHGET\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64 09:21:20.544402 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:20.544523 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:20.546624 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5 09:21:20.547232 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:20.547793 sendto(17, "3\r\n$4\r\nHGET\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44 09:21:20.547981 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:20.548117 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:20.550221 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5 09:21:20.550522 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:20.550817 sendto(17, "3\r\n$4\r\nHDEL\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44 09:21:20.551003 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:20.551142 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:20.553339 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4 09:21:20.554654 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:20.554776 sendto(17, "3\r\n$4\r\nHDEL\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64 09:21:20.554941 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:20.555065 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:20.557071 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4 09:21:20.557262 alarm(0) = 30 09:21:20.557816 gettimeofday({tv_sec=1620609680, tv_usec=557870}, NULL) = 0 09:21:20.557949 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 16 09:21:20.558068 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR) 09:21:20.558305 fcntl(16, F_SETFL, O_RDWR|O_NONBLOCK) = 0 09:21:20.558418 connect(16, {sa_family=AF_INET, sin_port=htons(2303), sin_addr=inet_addr("10.168.91.75")}, 16) = -1 EINPROGRESS (操作现在正在进行) 09:21:20.558882 poll([{fd=16, events=POLLIN|POLLOUT|POLLERR|POLLHUP}], 1, 3000) = 1 ([{fd=16, revents=POLLOUT}]) 09:21:20.559012 getsockopt(16, SOL_SOCKET, SO_ERROR, [0], [4]) = 0 09:21:20.559122 fcntl(16, F_SETFL, O_RDWR) = 0 09:21:20.559244 sendto(16, "\0\0\0\34\313\0\0\0\0\0\0\0\0\0\0\0\0\0\00011\1\0\0\0\0\0\0", 28, 0, NULL, 0) = 28 09:21:20.559409 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR) 09:21:20.559522 fcntl(16, F_SETFL, O_RDWR) = 0 09:21:20.559822 gettimeofday({tv_sec=1620609680, tv_usec=559868}, NULL) = 0 09:21:20.559951 recvfrom(16, "\0\0\0\2N;", 655350, 0, NULL, NULL) = 6 09:21:20.922527 close(16) = 0 09:21:20.923284 alarm(30) = 0 09:21:20.924116 sendto(10, "\0\0\0+\5\0\0\0\0\0\0\0\0\0\0\0\0\0\00011\1\0\0\0\0\0\0{\"ty"..., 43, 0, NULL, 0) = 43 09:21:20.925805 alarm(0) = 30 09:21:20.926070 alarm(30) = 0 09:21:20.926686 alarm(0) = 30 09:21:20.927195 alarm(30) = 0 09:21:20.927641 write(18, "[api-host-2][NOTICE][2021-05-10 "..., 118) = 118 09:21:20.927817 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:20.927948 sendto(17, "1\r\n$4\r\nPING\r\n", 14, MSG_DONTWAIT, NULL, 0) = 14 09:21:20.928096 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:20.928196 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:20.930932 recvfrom(17, "+PONG\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 7 09:21:20.931125 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:20.931614 sendto(17, "3\r\n$4\r\nHGET\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64 09:21:20.931823 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:20.932100 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:20.933818 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5 09:21:20.934038 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:20.934150 sendto(17, "3\r\n$4\r\nHGET\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44 09:21:20.934437 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:20.934615 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:20.936742 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5 09:21:20.936970 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:20.937099 sendto(17, "3\r\n$4\r\nHDEL\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44 09:21:20.937334 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:20.937503 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:20.939613 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4 09:21:20.939802 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:20.939923 sendto(17, "3\r\n$4\r\nHDEL\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64 09:21:20.940093 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:20.940352 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:20.942155 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4 09:21:20.942362 alarm(0) = 30 09:21:20.943212 gettimeofday({tv_sec=1620609680, tv_usec=943268}, NULL) = 0 09:21:20.943346 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 16 09:21:20.944675 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR) 09:21:20.944804 fcntl(16, F_SETFL, O_RDWR|O_NONBLOCK) = 0 09:21:20.944921 connect(16, {sa_family=AF_INET, sin_port=htons(2303), sin_addr=inet_addr("10.168.91.75")}, 16) = -1 EINPROGRESS (操作现在正在进行) 09:21:20.945135 poll([{fd=16, events=POLLIN|POLLOUT|POLLERR|POLLHUP}], 1, 3000) = 1 ([{fd=16, revents=POLLOUT}]) 09:21:20.945267 getsockopt(16, SOL_SOCKET, SO_ERROR, [0], [4]) = 0 09:21:20.945533 fcntl(16, F_SETFL, O_RDWR) = 0 09:21:20.945702 sendto(16, "\0\0\0\34\313\0\0\0\0\0\0\0\0\0\0\0\0\0\0+\f\1\0\0\0\0\0\0", 28, 0, NULL, 0) = 28 09:21:20.945886 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR) 09:21:20.946003 fcntl(16, F_SETFL, O_RDWR) = 0 09:21:20.946127 gettimeofday({tv_sec=1620609680, tv_usec=946174}, NULL) = 0 09:21:20.946257 recvfrom(16, "\0\0\0\2N;", 655350, 0, NULL, NULL) = 6 09:21:21.271293 close(16) = 0 09:21:21.271483 alarm(30) = 0 09:21:21.271763 sendto(10, "\0\0\0+\5\0\0\0\0\0\0\0\0\0\0\0\0\0\0+\f\1\0\0\0\0\0\0{\"ty"..., 43, 0, NULL, 0) = 43 09:21:21.271963 alarm(0) = 30 09:21:21.272336 gettimeofday({tv_sec=1620609681, tv_usec=272389}, NULL) = 0 09:21:21.272465 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 16 09:21:21.272614 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR) 09:21:21.272718 fcntl(16, F_SETFL, O_RDWR|O_NONBLOCK) = 0 09:21:21.272815 connect(16, {sa_family=AF_INET, sin_port=htons(2303), sin_addr=inet_addr("10.168.91.75")}, 16) = -1 EINPROGRESS (操作现在正在进行) 09:21:21.273112 poll([{fd=16, events=POLLIN|POLLOUT|POLLERR|POLLHUP}], 1, 3000) = 1 ([{fd=16, revents=POLLOUT}]) 09:21:21.273240 getsockopt(16, SOL_SOCKET, SO_ERROR, [0], [4]) = 0 09:21:21.273359 fcntl(16, F_SETFL, O_RDWR) = 0 09:21:21.273474 sendto(16, "\0\0\0\34\313\0\0\0\0\0\0\0\0\0\0\0\0\0\00011\1\0\0\0\0\0\0", 28, 0, NULL, 0) = 28 09:21:21.273670 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR) 09:21:21.273779 fcntl(16, F_SETFL, O_RDWR) = 0 09:21:21.273885 gettimeofday({tv_sec=1620609681, tv_usec=273931}, NULL) = 0 09:21:21.274086 recvfrom(16, "\0\0\0\2N;", 655350, 0, NULL, NULL) = 6 09:21:21.805334 close(16) = 0 09:21:21.805627 alarm(30) = 0 09:21:21.806368 sendto(10, "\0\0\0+\5\0\0\0\0\0\0\0\0\0\0\0\0\0\00011\1\0\0\0\0\0\0{\"ty"..., 43, 0, NULL, 0) = 43 09:21:21.806634 alarm(0) = 30 09:21:21.806985 alarm(30) = 0 09:21:21.807132 alarm(0) = 30 09:21:21.807693 alarm(30) = 0 09:21:21.808011 write(18, "[api-host-2][NOTICE][2021-05-10 "..., 118) = 118 09:21:21.808298 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.808588 sendto(17, "1\r\n$4\r\nPING\r\n", 14, MSG_DONTWAIT, NULL, 0) = 14 09:21:21.808847 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.809021 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:21.825032 recvfrom(17, "+PONG\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 7 09:21:21.825296 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.825420 sendto(17, "3\r\n$4\r\nHGET\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64 09:21:21.825623 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.825766 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:21.828824 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5 09:21:21.829031 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.829162 sendto(17, "3\r\n$4\r\nHGET\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44 09:21:21.829313 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.829447 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:21.834519 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5 09:21:21.834758 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.834881 sendto(17, "3\r\n$4\r\nHDEL\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44 09:21:21.835016 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.835133 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:21.853262 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4 09:21:21.853484 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.853653 sendto(17, "3\r\n$4\r\nHDEL\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64 09:21:21.853815 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.853934 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:21.857867 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4 09:21:21.860674 alarm(0) = 30 09:21:21.861217 alarm(30) = 0 09:21:21.861526 alarm(0) = 30 09:21:21.862322 alarm(30) = 0 09:21:21.862726 write(18, "[api-host-2][NOTICE][2021-05-10 "..., 118) = 118 09:21:21.863315 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.863601 sendto(17, "1\r\n$4\r\nPING\r\n", 14, MSG_DONTWAIT, NULL, 0) = 14 09:21:21.864835 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.865095 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:21.870268 recvfrom(17, "+PONG\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 7 09:21:21.870788 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.871260 sendto(17, "3\r\n$4\r\nHGET\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64 09:21:21.871787 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.872482 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:21.873753 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5 09:21:21.874128 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.874498 sendto(17, "3\r\n$4\r\nHGET\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44 09:21:21.874896 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.875239 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:21.877850 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5 09:21:21.878196 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.878515 sendto(17, "3\r\n$4\r\nHDEL\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44 09:21:21.878901 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.879211 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:21.880923 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4 09:21:21.881275 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.881634 sendto(17, "3\r\n$4\r\nHDEL\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64 09:21:21.881985 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.882298 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:21.884083 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4 09:21:21.884480 alarm(0) = 30 09:21:21.885182 alarm(30) = 0 09:21:21.885493 alarm(0) = 30 09:21:21.885893 alarm(30) = 0 09:21:21.886252 write(18, "[api-host-2][NOTICE][2021-05-10 "..., 118) = 118 09:21:21.886650 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.886951 sendto(17, "1\r\n$4\r\nPING\r\n", 14, MSG_DONTWAIT, NULL, 0) = 14 09:21:21.887287 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.887624 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:21.889965 recvfrom(17, "+PONG\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 7 09:21:21.892213 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.892419 sendto(17, "3\r\n$4\r\nHGET\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64 09:21:21.893001 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.893121 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:21.899878 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5 09:21:21.900781 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.900910 sendto(17, "3\r\n$4\r\nHGET\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44 09:21:21.901069 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.901185 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:21.903927 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5 09:21:21.904103 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.904210 sendto(17, "3\r\n$4\r\nHDEL\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44 09:21:21.904341 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.904450 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:21.906366 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4 09:21:21.906541 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.906687 sendto(17, "3\r\n$4\r\nHDEL\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64 09:21:21.906820 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:21.906928 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:21.909143 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4 09:21:21.909326 alarm(0) = 30 09:21:21.909700 gettimeofday({tv_sec=1620609681, tv_usec=909745}, NULL) = 0 09:21:21.909812 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 16 09:21:21.909923 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR) 09:21:21.910021 fcntl(16, F_SETFL, O_RDWR|O_NONBLOCK) = 0 09:21:21.910126 connect(16, {sa_family=AF_INET, sin_port=htons(2303), sin_addr=inet_addr("10.168.91.75")}, 16) = -1 EINPROGRESS (操作现在正在进行) 09:21:21.910312 poll([{fd=16, events=POLLIN|POLLOUT|POLLERR|POLLHUP}], 1, 3000) = 1 ([{fd=16, revents=POLLOUT}]) 09:21:21.910438 getsockopt(16, SOL_SOCKET, SO_ERROR, [0], [4]) = 0 09:21:21.910586 fcntl(16, F_SETFL, O_RDWR) = 0 09:21:21.910694 sendto(16, "\0\0\0\34\313\0\0\0\0\0\0\0\0\0\0\0\0\0\0+\f\1\0\0\0\0\0\0", 28, 0, NULL, 0) = 28 09:21:21.910839 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR) 09:21:21.910932 fcntl(16, F_SETFL, O_RDWR) = 0 09:21:21.911038 gettimeofday({tv_sec=1620609681, tv_usec=911079}, NULL) = 0 09:21:21.911160 recvfrom(16, "\0\0\0\2N;", 655350, 0, NULL, NULL) = 6 09:21:22.451501 close(16) = 0 09:21:22.451702 alarm(30) = 0 09:21:22.451903 sendto(10, "\0\0\0+\5\0\0\0\0\0\0\0\0\0\0\0\0\0\0+\f\1\0\0\0\0\0\0{\"ty"..., 43, 0, NULL, 0) = 43 09:21:22.452075 alarm(0) = 30 09:21:22.452270 alarm(30) = 0 09:21:22.452375 alarm(0) = 30 09:21:22.452553 alarm(30) = 0 09:21:22.452769 write(18, "[api-host-2][NOTICE][2021-05-10 "..., 118) = 118 09:21:22.452938 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:22.453036 sendto(17, "1\r\n$4\r\nPING\r\n", 14, MSG_DONTWAIT, NULL, 0) = 14 09:21:22.453165 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:22.453259 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:22.455444 recvfrom(17, "+PONG\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 7 09:21:22.455641 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:22.455745 sendto(17, "3\r\n$4\r\nHGET\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64 09:21:22.455887 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:22.455992 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:22.459278 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5 09:21:22.459533 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:22.459672 sendto(17, "3\r\n$4\r\nHGET\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44 09:21:22.459813 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:22.459917 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:22.464095 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5 09:21:22.464285 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:22.464391 sendto(17, "3\r\n$4\r\nHDEL\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44 09:21:22.464530 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:22.464678 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:22.466924 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4 09:21:22.467119 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:22.467230 sendto(17, "3\r\n$4\r\nHDEL\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64 09:21:22.467377 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:22.467494 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:22.473443 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4 09:21:22.475078 alarm(0) = 30 09:21:22.475390 gettimeofday({tv_sec=1620609682, tv_usec=475433}, NULL) = 0 09:21:22.475505 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 16 09:21:22.475980 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR) 09:21:22.476083 fcntl(16, F_SETFL, O_RDWR|O_NONBLOCK) = 0 09:21:22.476178 connect(16, {sa_family=AF_INET, sin_port=htons(2303), sin_addr=inet_addr("10.168.91.75")}, 16) = -1 EINPROGRESS (操作现在正在进行) 09:21:22.476373 poll([{fd=16, events=POLLIN|POLLOUT|POLLERR|POLLHUP}], 1, 3000) = 1 ([{fd=16, revents=POLLOUT}]) 09:21:22.476500 getsockopt(16, SOL_SOCKET, SO_ERROR, [0], [4]) = 0 09:21:22.477472 fcntl(16, F_SETFL, O_RDWR) = 0 09:21:22.478328 sendto(16, "\0\0\0\34\313\0\0\0\0\0\0\0\0\0\0\0\0\0\00011\1\0\0\0\0\0\0", 28, 0, NULL, 0) = 28 09:21:22.478504 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR) 09:21:22.478966 fcntl(16, F_SETFL, O_RDWR) = 0 09:21:22.479071 gettimeofday({tv_sec=1620609682, tv_usec=479108}, NULL) = 0 09:21:22.479183 recvfrom(16, "\0\0\0\2N;", 655350, 0, NULL, NULL) = 6 09:21:23.136261 close(16) = 0 09:21:23.136497 alarm(30) = 0 09:21:23.136749 sendto(10, "\0\0\0+\5\0\0\0\0\0\0\0\0\0\0\0\0\0\00011\1\0\0\0\0\0\0{\"ty"..., 43, 0, NULL, 0) = 43 09:21:23.147287 alarm(0) = 30 09:21:23.147537 alarm(30) = 0 09:21:23.147791 alarm(0) = 30 09:21:23.147960 alarm(30) = 0 09:21:23.148126 write(18, "[api-host-2][NOTICE][2021-05-10 "..., 118) = 118 09:21:23.148296 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.148391 sendto(17, "1\r\n$4\r\nPING\r\n", 14, MSG_DONTWAIT, NULL, 0) = 14 09:21:23.148505 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.148628 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:23.179974 recvfrom(17, "+PONG\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 7 09:21:23.180221 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.180329 sendto(17, "3\r\n$4\r\nHGET\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64 09:21:23.180981 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.181093 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:23.183216 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5 09:21:23.186886 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.188151 sendto(17, "3\r\n$4\r\nHGET\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44 09:21:23.188338 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.188459 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:23.202631 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5 09:21:23.202879 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.203008 sendto(17, "3\r\n$4\r\nHDEL\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44 09:21:23.203167 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.203292 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:23.205155 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4 09:21:23.205329 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.205451 sendto(17, "3\r\n$4\r\nHDEL\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64 09:21:23.205638 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.205745 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:23.208125 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4 09:21:23.208291 alarm(0) = 30 09:21:23.208518 alarm(30) = 0 09:21:23.208674 alarm(0) = 30 09:21:23.208851 alarm(30) = 0 09:21:23.209017 write(18, "[api-host-2][NOTICE][2021-05-10 "..., 118) = 118 09:21:23.209171 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.209282 sendto(17, "1\r\n$4\r\nPING\r\n", 14, MSG_DONTWAIT, NULL, 0) = 14 09:21:23.209444 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.209551 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:23.211959 recvfrom(17, "+PONG\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 7 09:21:23.212136 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.212250 sendto(17, "3\r\n$4\r\nHGET\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64 09:21:23.212399 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.212518 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:23.223226 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5 09:21:23.223501 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.223653 sendto(17, "3\r\n$4\r\nHGET\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44 09:21:23.223797 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.223920 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:23.226627 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5 09:21:23.226839 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.226967 sendto(17, "3\r\n$4\r\nHDEL\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44 09:21:23.227947 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.228085 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:23.232497 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4 09:21:23.232717 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.232841 sendto(17, "3\r\n$4\r\nHDEL\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64 09:21:23.232990 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) 09:21:23.233100 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}]) 09:21:23.244036 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4 09:21:23.245634 alarm(0) = 30 09:21:23.246193 gettimeofday({tv_sec=1620609683, tv_usec=246252}, NULL) = 0 09:21:23.246330 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 16 09:21:23.246472 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR) 09:21:23.246627 fcntl(16, F_SETFL, O_RDWR|O_NONBLOCK) = 0 09:21:23.246756 connect(16, {sa_family=AF_INET, sin_port=htons(2303), sin_addr=inet_addr("10.168.91.75")}, 16) = -1 EINPROGRESS (操作现在正在进行) 09:21:23.247047 poll([{fd=16, events=POLLIN|POLLOUT|POLLERR|POLLHUP}], 1, 3000) = 1 ([{fd=16, revents=POLLOUT}]) 09:21:23.247214 getsockopt(16, SOL_SOCKET, SO_ERROR, [0], [4]) = 0 09:21:23.247353 fcntl(16, F_SETFL, O_RDWR) = 0 09:21:23.247488 sendto(16, "\0\0\0\34\313\0\0\0\0\0\0\0\0\0\0\0\0\0\0+\f\1\0\0\0\0\0\0", 28, 0, NULL, 0) = 28 09:21:23.247704 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR) 09:21:23.249515 fcntl(16, F_SETFL, O_RDWR) = 0 09:21:23.249758 gettimeofday({tv_sec=1620609683, tv_usec=249807}, NULL) = 0 09:21:23.249928 recvfrom(16, "\0\0\0\2N;", 655350, 0, NULL, NULL) = 6 09:21:23.637876 close(16) = 0 09:21:23.638187 alarm(30) = 0 09:21:23.638630 sendto(10, "\0\0\0+\5\0\0\0\0\0\0\0\0\0\0\0\0\0\0+\f\1\0\0\0\0\0\0{\"ty"..., 43, 0, NULL, 0) = 43 09:21:23.639026 alarm(0) = 30 09:21:23.641024 gettimeofday({tv_sec=1620609683, tv_usec=641103}, NULL) = 0 09:21:23.641818 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 16 09:21:23.642103 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR) 09:21:23.642341 fcntl(16, F_SETFL, O_RDWR|O_NONBLOCK) = 0 09:21:23.643719 connect(16, {sa_family=AF_INET, sin_port=htons(2303), sin_addr=inet_addr("10.168.91.75")}, 16) = -1 EINPROGRESS (操作现在正在进行) 09:21:23.644015 poll([{fd=16, events=POLLIN|POLLOUT|POLLERR|POLLHUP}], 1, 3000) = 1 ([{fd=16, revents=POLLOUT}]) 09:21:23.644194 getsockopt(16, SOL_SOCKET, SO_ERROR, [0], [4]) = 0 09:21:23.644368 fcntl(16, F_SETFL, O_RDWR) = 0 09:21:23.644535 sendto(16, "\0\0\0\34\313\0\0\0\0\0\0\0\0\0\0\0\0\0\00011\1\0\0\0\0\0\0", 28, 0, NULL, 0) = 28 09:21:23.644887 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR) 09:21:23.645031 fcntl(16, F_SETFL, O_RDWR) = 0 09:21:23.645687 gettimeofday({tv_sec=1620609683, tv_usec=645740}, NULL) = 0 09:21:23.645869 recvfrom(16, ^Cstrace: Process 129499 detached
lsof -nPp 分析后
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME php 129499 root cwd DIR 202,1 4096 1199944 /home/www/chat50 php 129499 root rtd DIR 202,1 4096 2 / php 129499 root txt REG 202,1 31908647 1575462 /usr/local/php56/bin/php php 129499 root mem REG 202,1 217032 401548 /var/db/nscd/hosts php 129499 root mem REG 202,1 61560 162406 /usr/lib64/libnss_files-2.17.so php 129499 root DEL REG 202,1 262841 /tmp/xcache php 129499 root mem REG 202,1 708960 1453372 /usr/local/php56/lib/php/extensions/no-debug-non-zts-20131226/xcache.so php 129499 root mem REG 202,1 1362871 1453336 /usr/local/php56/lib/php/extensions/no-debug-non-zts-20131226/soap.so php 129499 root mem REG 202,1 2500552 1445058 /usr/local/php56/lib/php/extensions/no-debug-non-zts-20131226/redis.so php 129499 root mem REG 202,1 678263 1461496 /usr/local/php56/lib/php/extensions/no-debug-non-zts-20131226/mysqli.so php 129499 root mem REG 202,1 662449 158254 /usr/local/lib/libmcrypt.so.4.4.8 php 129499 root mem REG 202,1 169570 1454154 /usr/local/php56/lib/php/extensions/no-debug-non-zts-20131226/mcrypt.so php 129499 root mem REG 202,1 855115 1454297 /usr/local/php56/lib/php/extensions/no-debug-non-zts-20131226/zip.so php 129499 root mem REG 202,1 179800 163519 /usr/lib64/libevent_core-2.0.so.5.1.9 php 129499 root mem REG 202,1 133864 163521 /usr/lib64/libevent_extra-2.0.so.5.1.9 php 129499 root mem REG 202,1 24456 163523 /usr/lib64/libevent_openssl-2.0.so.5.1.9 php 129499 root mem REG 202,1 803448 1442226 /usr/local/php56/lib/php/extensions/no-debug-non-zts-20131226/event.so php 129499 root mem REG 202,1 106172832 166453 /usr/lib/locale/locale-archive php 129499 root mem REG 202,1 402384 134594 /usr/lib64/libpcre.so.1.2.0 php 129499 root mem REG 202,1 155744 132878 /usr/lib64/libselinux.so.1 php 129499 root mem REG 202,1 121320 133626 /usr/lib64/libsasl2.so.3.0.0 php 129499 root mem REG 202,1 15688 134676 /usr/lib64/libkeyutils.so.1.5 php 129499 root mem REG 202,1 67104 161754 /usr/lib64/libkrb5support.so.0.1 php 129499 root mem REG 202,1 68192 133155 /usr/lib64/libbz2.so.1.0.6 php 129499 root mem REG 202,1 157424 151357 /usr/lib64/liblzma.so.5.2.2 php 129499 root mem REG 202,1 352608 151584 /usr/lib64/libldap-2.4.so.2.10.7 php 129499 root mem REG 202,1 61952 141025 /usr/lib64/liblber-2.4.so.2.10.7 php 129499 root mem REG 202,1 15856 137501 /usr/lib64/libcom_err.so.2.1 php 129499 root mem REG 202,1 210784 151492 /usr/lib64/libk5crypto.so.3.1 php 129499 root mem REG 202,1 967760 137059 /usr/lib64/libkrb5.so.3.3 php 129499 root mem REG 202,1 320720 146456 /usr/lib64/libgssapi_krb5.so.2.2 php 129499 root mem REG 202,1 142144 132848 /usr/lib64/libpthread-2.17.so php 129499 root mem REG 202,1 251792 162750 /usr/lib64/libnspr4.so php 129499 root mem REG 202,1 20040 166470 /usr/lib64/libplc4.so php 129499 root mem REG 202,1 15744 166471 /usr/lib64/libplds4.so php 129499 root mem REG 202,1 198968 166472 /usr/lib64/libnssutil3.so php 129499 root mem REG 202,1 1257728 166614 /usr/lib64/libnss3.so php 129499 root mem REG 202,1 168336 166615 /usr/lib64/libsmime3.so php 129499 root mem REG 202,1 370584 166616 /usr/lib64/libssl3.so php 129499 root mem REG 202,1 186680 142759 /usr/lib64/libssh2.so.1.0.1 php 129499 root mem REG 202,1 208920 133506 /usr/lib64/libidn.so.11.6.11 php 129499 root mem REG 202,1 11392 157241 /usr/lib64/libfreebl3.so php 129499 root mem REG 202,1 2156240 132822 /usr/lib64/libc-2.17.so php 129499 root mem REG 202,1 795592 161017 /usr/lib64/libfreetype.so.6.14.0 php 129499 root mem REG 202,1 2521144 132882 /usr/lib64/libcrypto.so.1.0.2k php 129499 root mem REG 202,1 470376 160594 /usr/lib64/libssl.so.1.0.2k php 129499 root mem REG 202,1 1509600 143280 /usr/lib64/libxml2.so.2.9.1 php 129499 root mem REG 202,1 115816 162404 /usr/lib64/libnsl-2.17.so php 129499 root mem REG 202,1 19248 162402 /usr/lib64/libdl-2.17.so php 129499 root mem REG 202,1 1136944 162403 /usr/lib64/libm-2.17.so php 129499 root mem REG 202,1 43712 162409 /usr/lib64/librt-2.17.so php 129499 root mem REG 202,1 439280 151366 /usr/lib64/libcurl.so.4.3.0 php 129499 root mem REG 202,1 285328 133423 /usr/lib64/libjpeg.so.62.1.0 php 129499 root mem REG 202,1 90248 132818 /usr/lib64/libz.so.1.2.7 php 129499 root mem REG 202,1 179296 133405 /usr/lib64/libpng15.so.15.13.0 php 129499 root mem REG 202,1 109976 162408 /usr/lib64/libresolv-2.17.so php 129499 root mem REG 202,1 40600 151333 /usr/lib64/libcrypt-2.17.so php 129499 root mem REG 202,1 163312 166456 /usr/lib64/ld-2.17.so php 129499 root mem REG 202,1 26970 408331 /usr/lib64/gconv/gconv-modules.cache php 129499 root 0u CHR 136,1 0t0 4 /dev/pts/1 (deleted) php 129499 root 1w CHR 1,3 0t0 1028 /dev/null php 129499 root 2w CHR 1,3 0t0 1028 /dev/null php 129499 root 3u REG 202,1 0 262854 /tmp/.xcache.0.129265.1.mutex (deleted) php 129499 root 4u a_inode 0,10 0 9546 [eventpoll] php 129499 root 5u unix 0xffff880c64b99dc0 0t0 759626038 socket php 129499 root 6u unix 0xffff880c64b9e1c0 0t0 759626039 socket php 129499 root 7u IPv4 759626041 0t0 TCP 10.168.91.75:40904->10.168.91.75:1236 (ESTABLISHED) php 129499 root 8u IPv4 759626049 0t0 TCP 10.168.91.75:55834->10.168.91.75:2300 (ESTABLISHED) php 129499 root 9u IPv4 759626052 0t0 TCP 10.168.91.75:50054->10.168.91.75:2301 (ESTABLISHED) php 129499 root 10u IPv4 759626056 0t0 TCP 10.168.91.75:59092->10.168.91.75:2303 (ESTABLISHED) php 129499 root 11u IPv4 759626059 0t0 TCP 10.168.91.75:51924->10.168.91.75:2306 (ESTABLISHED) php 129499 root 12u IPv4 759626061 0t0 TCP 10.168.91.75:58294->10.168.91.75:2307 (ESTABLISHED) php 129499 root 13u IPv4 759626077 0t0 TCP 10.168.91.75:47806->10.168.91.75:2304 (ESTABLISHED) php 129499 root 14u IPv4 759626083 0t0 TCP 10.168.91.75:42994->10.168.91.75:2305 (ESTABLISHED) php 129499 root 15u IPv4 759626089 0t0 TCP 10.168.91.75:32772->10.168.91.75:2302 (ESTABLISHED) php 129499 root 17u IPv4 759621951 0t0 TCP 10.168.91.75:39288->10.146.29.193:6379 (ESTABLISHED) php 129499 root 18w REG 202,1 67425954 405332 /var/log/chatWorker/2021-05-10.log php 129499 root 19u IPv4 764258273 0t0 TCP 127.0.0.1:43048->127.0.0.1:6688 (CLOSE_WAIT) php 129499 root 20u IPv4 764269827 0t0 TCP 127.0.0.1:44020->127.0.0.1:6688 (CLOSE_WAIT) php 129499 root 21u IPv4 764293538 0t0 TCP 127.0.0.1:39318->127.0.0.1:6688 (CLOSE_WAIT) php 129499 root 22u IPv4 764298809 0t0 TCP 127.0.0.1:42902->127.0.0.1:6688 (CLOSE_WAIT) php 129499 root 23u IPv4 764321399 0t0 TCP 127.0.0.1:35588->127.0.0.1:6688 (CLOSE_WAIT) php 129499 root 24u IPv4 764339709 0t0 TCP 127.0.0.1:47380->127.0.0.1:6688 (CLOSE_WAIT) php 129499 root 25u IPv4 764843705 0t0 TCP 127.0.0.1:50452->127.0.0.1:6688 (CLOSE_WAIT) php 129499 root 26u IPv4 764843923 0t0 TCP 127.0.0.1:53740->127.0.0.1:6688 (CLOSE_WAIT) php 129499 root 27u IPv4 764862631 0t0 TCP 127.0.0.1:57462->127.0.0.1:6688 (CLOSE_WAIT) php 129499 root 28u IPv4 765089929 0t0 TCP 127.0.0.1:45636->127.0.0.1:6688 (CLOSE_WAIT) php 129499 root 29u IPv4 765080227 0t0 TCP 127.0.0.1:47420->127.0.0.1:6688 (CLOSE_WAIT) php 129499 root 30u IPv4 765087559 0t0 TCP 127.0.0.1:52486->127.0.0.1:6688 (CLOSE_WAIT) php 129499 root 31u IPv4 765613071 0t0 TCP 127.0.0.1:50078->127.0.0.1:6688 (CLOSE_WAIT)
咋感觉你代码死循环了,一直读写redis
@1393:php 我们是有读写Redis;从哪看出死循环读写Redis呀
php 129499 root 31u IPv4 765613071 0t0 TCP 127.0.0.1:50078->127.0.0.1:6688 (CLOSE_WAIT) 关闭异常的,应该也是那个问题导致的吧?
我的这个应用的逻辑应该有点复杂,发一条消息,需要做到接收这不同客户端都要有收到消息,自己的其他客户端也要有这条消息显示,导致了一个用户在一个端发出一条消息,最终可能需要发送的有多次,得根据接受者与自己的客户端在线数而定
strace 下gateway进程看下吧,估计系统资源用完了
是的,资源被占用光啦
解决了?啥资源用光了?难道是打开的文件数超过了系统内核配置?
@1393:没有呢,这样的情况我重启服务器又正常啦,有没有办法可以记录到这种情况的日志的呀
看看是不是没优化linux内核,安装event扩展
谢答,已优化linux内核,event 也安装啦;下面我把strace分析与lsof分析的结果贴一下
谢答,已优化linux内核,event 也安装啦
通过strace -ttp 分析
09:21:20.539487 sendto(17, "1\r\n$4\r\nPING\r\n", 14, MSG_DONTWAIT, NULL, 0) = 14
09:21:20.539829 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:20.539946 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:20.543752 recvfrom(17, "+PONG\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 7
09:21:20.544123 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:20.544246 sendto(17, "3\r\n$4\r\nHGET\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64
09:21:20.544402 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:20.544523 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:20.546624 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5
09:21:20.547232 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:20.547793 sendto(17, "3\r\n$4\r\nHGET\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44
09:21:20.547981 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:20.548117 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:20.550221 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5
09:21:20.550522 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:20.550817 sendto(17, "3\r\n$4\r\nHDEL\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44
09:21:20.551003 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:20.551142 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:20.553339 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4
09:21:20.554654 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:20.554776 sendto(17, "3\r\n$4\r\nHDEL\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64
09:21:20.554941 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:20.555065 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:20.557071 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4
09:21:20.557262 alarm(0) = 30
09:21:20.557816 gettimeofday({tv_sec=1620609680, tv_usec=557870}, NULL) = 0
09:21:20.557949 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 16
09:21:20.558068 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR)
09:21:20.558305 fcntl(16, F_SETFL, O_RDWR|O_NONBLOCK) = 0
09:21:20.558418 connect(16, {sa_family=AF_INET, sin_port=htons(2303), sin_addr=inet_addr("10.168.91.75")}, 16) = -1 EINPROGRESS (操作现在正在进行)
09:21:20.558882 poll([{fd=16, events=POLLIN|POLLOUT|POLLERR|POLLHUP}], 1, 3000) = 1 ([{fd=16, revents=POLLOUT}])
09:21:20.559012 getsockopt(16, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
09:21:20.559122 fcntl(16, F_SETFL, O_RDWR) = 0
09:21:20.559244 sendto(16, "\0\0\0\34\313\0\0\0\0\0\0\0\0\0\0\0\0\0\00011\1\0\0\0\0\0\0", 28, 0, NULL, 0) = 28
09:21:20.559409 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR)
09:21:20.559522 fcntl(16, F_SETFL, O_RDWR) = 0
09:21:20.559822 gettimeofday({tv_sec=1620609680, tv_usec=559868}, NULL) = 0
09:21:20.559951 recvfrom(16, "\0\0\0\2N;", 655350, 0, NULL, NULL) = 6
09:21:20.922527 close(16) = 0
09:21:20.923284 alarm(30) = 0
09:21:20.924116 sendto(10, "\0\0\0+\5\0\0\0\0\0\0\0\0\0\0\0\0\0\00011\1\0\0\0\0\0\0{\"ty"..., 43, 0, NULL, 0) = 43
09:21:20.925805 alarm(0) = 30
09:21:20.926070 alarm(30) = 0
09:21:20.926686 alarm(0) = 30
09:21:20.927195 alarm(30) = 0
09:21:20.927641 write(18, "[api-host-2][NOTICE][2021-05-10 "..., 118) = 118
09:21:20.927817 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:20.927948 sendto(17, "1\r\n$4\r\nPING\r\n", 14, MSG_DONTWAIT, NULL, 0) = 14
09:21:20.928096 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:20.928196 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:20.930932 recvfrom(17, "+PONG\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 7
09:21:20.931125 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:20.931614 sendto(17, "3\r\n$4\r\nHGET\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64
09:21:20.931823 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:20.932100 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:20.933818 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5
09:21:20.934038 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:20.934150 sendto(17, "3\r\n$4\r\nHGET\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44
09:21:20.934437 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:20.934615 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:20.936742 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5
09:21:20.936970 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:20.937099 sendto(17, "3\r\n$4\r\nHDEL\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44
09:21:20.937334 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:20.937503 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:20.939613 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4
09:21:20.939802 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:20.939923 sendto(17, "3\r\n$4\r\nHDEL\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64
09:21:20.940093 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:20.940352 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:20.942155 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4
09:21:20.942362 alarm(0) = 30
09:21:20.943212 gettimeofday({tv_sec=1620609680, tv_usec=943268}, NULL) = 0
09:21:20.943346 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 16
09:21:20.944675 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR)
09:21:20.944804 fcntl(16, F_SETFL, O_RDWR|O_NONBLOCK) = 0
09:21:20.944921 connect(16, {sa_family=AF_INET, sin_port=htons(2303), sin_addr=inet_addr("10.168.91.75")}, 16) = -1 EINPROGRESS (操作现在正在进行)
09:21:20.945135 poll([{fd=16, events=POLLIN|POLLOUT|POLLERR|POLLHUP}], 1, 3000) = 1 ([{fd=16, revents=POLLOUT}])
09:21:20.945267 getsockopt(16, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
09:21:20.945533 fcntl(16, F_SETFL, O_RDWR) = 0
09:21:20.945702 sendto(16, "\0\0\0\34\313\0\0\0\0\0\0\0\0\0\0\0\0\0\0+\f\1\0\0\0\0\0\0", 28, 0, NULL, 0) = 28
09:21:20.945886 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR)
09:21:20.946003 fcntl(16, F_SETFL, O_RDWR) = 0
09:21:20.946127 gettimeofday({tv_sec=1620609680, tv_usec=946174}, NULL) = 0
09:21:20.946257 recvfrom(16, "\0\0\0\2N;", 655350, 0, NULL, NULL) = 6
09:21:21.271293 close(16) = 0
09:21:21.271483 alarm(30) = 0
09:21:21.271763 sendto(10, "\0\0\0+\5\0\0\0\0\0\0\0\0\0\0\0\0\0\0+\f\1\0\0\0\0\0\0{\"ty"..., 43, 0, NULL, 0) = 43
09:21:21.271963 alarm(0) = 30
09:21:21.272336 gettimeofday({tv_sec=1620609681, tv_usec=272389}, NULL) = 0
09:21:21.272465 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 16
09:21:21.272614 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR)
09:21:21.272718 fcntl(16, F_SETFL, O_RDWR|O_NONBLOCK) = 0
09:21:21.272815 connect(16, {sa_family=AF_INET, sin_port=htons(2303), sin_addr=inet_addr("10.168.91.75")}, 16) = -1 EINPROGRESS (操作现在正在进行)
09:21:21.273112 poll([{fd=16, events=POLLIN|POLLOUT|POLLERR|POLLHUP}], 1, 3000) = 1 ([{fd=16, revents=POLLOUT}])
09:21:21.273240 getsockopt(16, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
09:21:21.273359 fcntl(16, F_SETFL, O_RDWR) = 0
09:21:21.273474 sendto(16, "\0\0\0\34\313\0\0\0\0\0\0\0\0\0\0\0\0\0\00011\1\0\0\0\0\0\0", 28, 0, NULL, 0) = 28
09:21:21.273670 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR)
09:21:21.273779 fcntl(16, F_SETFL, O_RDWR) = 0
09:21:21.273885 gettimeofday({tv_sec=1620609681, tv_usec=273931}, NULL) = 0
09:21:21.274086 recvfrom(16, "\0\0\0\2N;", 655350, 0, NULL, NULL) = 6
09:21:21.805334 close(16) = 0
09:21:21.805627 alarm(30) = 0
09:21:21.806368 sendto(10, "\0\0\0+\5\0\0\0\0\0\0\0\0\0\0\0\0\0\00011\1\0\0\0\0\0\0{\"ty"..., 43, 0, NULL, 0) = 43
09:21:21.806634 alarm(0) = 30
09:21:21.806985 alarm(30) = 0
09:21:21.807132 alarm(0) = 30
09:21:21.807693 alarm(30) = 0
09:21:21.808011 write(18, "[api-host-2][NOTICE][2021-05-10 "..., 118) = 118
09:21:21.808298 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.808588 sendto(17, "1\r\n$4\r\nPING\r\n", 14, MSG_DONTWAIT, NULL, 0) = 14
09:21:21.808847 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.809021 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:21.825032 recvfrom(17, "+PONG\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 7
09:21:21.825296 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.825420 sendto(17, "3\r\n$4\r\nHGET\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64
09:21:21.825623 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.825766 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:21.828824 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5
09:21:21.829031 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.829162 sendto(17, "3\r\n$4\r\nHGET\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44
09:21:21.829313 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.829447 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:21.834519 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5
09:21:21.834758 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.834881 sendto(17, "3\r\n$4\r\nHDEL\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44
09:21:21.835016 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.835133 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:21.853262 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4
09:21:21.853484 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.853653 sendto(17, "3\r\n$4\r\nHDEL\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64
09:21:21.853815 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.853934 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:21.857867 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4
09:21:21.860674 alarm(0) = 30
09:21:21.861217 alarm(30) = 0
09:21:21.861526 alarm(0) = 30
09:21:21.862322 alarm(30) = 0
09:21:21.862726 write(18, "[api-host-2][NOTICE][2021-05-10 "..., 118) = 118
09:21:21.863315 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.863601 sendto(17, "1\r\n$4\r\nPING\r\n", 14, MSG_DONTWAIT, NULL, 0) = 14
09:21:21.864835 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.865095 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:21.870268 recvfrom(17, "+PONG\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 7
09:21:21.870788 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.871260 sendto(17, "3\r\n$4\r\nHGET\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64
09:21:21.871787 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.872482 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:21.873753 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5
09:21:21.874128 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.874498 sendto(17, "3\r\n$4\r\nHGET\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44
09:21:21.874896 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.875239 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:21.877850 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5
09:21:21.878196 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.878515 sendto(17, "3\r\n$4\r\nHDEL\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44
09:21:21.878901 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.879211 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:21.880923 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4
09:21:21.881275 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.881634 sendto(17, "3\r\n$4\r\nHDEL\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64
09:21:21.881985 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.882298 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:21.884083 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4
09:21:21.884480 alarm(0) = 30
09:21:21.885182 alarm(30) = 0
09:21:21.885493 alarm(0) = 30
09:21:21.885893 alarm(30) = 0
09:21:21.886252 write(18, "[api-host-2][NOTICE][2021-05-10 "..., 118) = 118
09:21:21.886650 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.886951 sendto(17, "1\r\n$4\r\nPING\r\n", 14, MSG_DONTWAIT, NULL, 0) = 14
09:21:21.887287 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.887624 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:21.889965 recvfrom(17, "+PONG\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 7
09:21:21.892213 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.892419 sendto(17, "3\r\n$4\r\nHGET\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64
09:21:21.893001 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.893121 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:21.899878 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5
09:21:21.900781 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.900910 sendto(17, "3\r\n$4\r\nHGET\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44
09:21:21.901069 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.901185 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:21.903927 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5
09:21:21.904103 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.904210 sendto(17, "3\r\n$4\r\nHDEL\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44
09:21:21.904341 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.904450 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:21.906366 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4
09:21:21.906541 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.906687 sendto(17, "3\r\n$4\r\nHDEL\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64
09:21:21.906820 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:21.906928 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:21.909143 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4
09:21:21.909326 alarm(0) = 30
09:21:21.909700 gettimeofday({tv_sec=1620609681, tv_usec=909745}, NULL) = 0
09:21:21.909812 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 16
09:21:21.909923 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR)
09:21:21.910021 fcntl(16, F_SETFL, O_RDWR|O_NONBLOCK) = 0
09:21:21.910126 connect(16, {sa_family=AF_INET, sin_port=htons(2303), sin_addr=inet_addr("10.168.91.75")}, 16) = -1 EINPROGRESS (操作现在正在进行)
09:21:21.910312 poll([{fd=16, events=POLLIN|POLLOUT|POLLERR|POLLHUP}], 1, 3000) = 1 ([{fd=16, revents=POLLOUT}])
09:21:21.910438 getsockopt(16, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
09:21:21.910586 fcntl(16, F_SETFL, O_RDWR) = 0
09:21:21.910694 sendto(16, "\0\0\0\34\313\0\0\0\0\0\0\0\0\0\0\0\0\0\0+\f\1\0\0\0\0\0\0", 28, 0, NULL, 0) = 28
09:21:21.910839 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR)
09:21:21.910932 fcntl(16, F_SETFL, O_RDWR) = 0
09:21:21.911038 gettimeofday({tv_sec=1620609681, tv_usec=911079}, NULL) = 0
09:21:21.911160 recvfrom(16, "\0\0\0\2N;", 655350, 0, NULL, NULL) = 6
09:21:22.451501 close(16) = 0
09:21:22.451702 alarm(30) = 0
09:21:22.451903 sendto(10, "\0\0\0+\5\0\0\0\0\0\0\0\0\0\0\0\0\0\0+\f\1\0\0\0\0\0\0{\"ty"..., 43, 0, NULL, 0) = 43
09:21:22.452075 alarm(0) = 30
09:21:22.452270 alarm(30) = 0
09:21:22.452375 alarm(0) = 30
09:21:22.452553 alarm(30) = 0
09:21:22.452769 write(18, "[api-host-2][NOTICE][2021-05-10 "..., 118) = 118
09:21:22.452938 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:22.453036 sendto(17, "1\r\n$4\r\nPING\r\n", 14, MSG_DONTWAIT, NULL, 0) = 14
09:21:22.453165 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:22.453259 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:22.455444 recvfrom(17, "+PONG\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 7
09:21:22.455641 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:22.455745 sendto(17, "3\r\n$4\r\nHGET\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64
09:21:22.455887 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:22.455992 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:22.459278 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5
09:21:22.459533 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:22.459672 sendto(17, "3\r\n$4\r\nHGET\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44
09:21:22.459813 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:22.459917 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:22.464095 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5
09:21:22.464285 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:22.464391 sendto(17, "3\r\n$4\r\nHDEL\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44
09:21:22.464530 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:22.464678 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:22.466924 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4
09:21:22.467119 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:22.467230 sendto(17, "3\r\n$4\r\nHDEL\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64
09:21:22.467377 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:22.467494 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:22.473443 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4
09:21:22.475078 alarm(0) = 30
09:21:22.475390 gettimeofday({tv_sec=1620609682, tv_usec=475433}, NULL) = 0
09:21:22.475505 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 16
09:21:22.475980 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR)
09:21:22.476083 fcntl(16, F_SETFL, O_RDWR|O_NONBLOCK) = 0
09:21:22.476178 connect(16, {sa_family=AF_INET, sin_port=htons(2303), sin_addr=inet_addr("10.168.91.75")}, 16) = -1 EINPROGRESS (操作现在正在进行)
09:21:22.476373 poll([{fd=16, events=POLLIN|POLLOUT|POLLERR|POLLHUP}], 1, 3000) = 1 ([{fd=16, revents=POLLOUT}])
09:21:22.476500 getsockopt(16, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
09:21:22.477472 fcntl(16, F_SETFL, O_RDWR) = 0
09:21:22.478328 sendto(16, "\0\0\0\34\313\0\0\0\0\0\0\0\0\0\0\0\0\0\00011\1\0\0\0\0\0\0", 28, 0, NULL, 0) = 28
09:21:22.478504 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR)
09:21:22.478966 fcntl(16, F_SETFL, O_RDWR) = 0
09:21:22.479071 gettimeofday({tv_sec=1620609682, tv_usec=479108}, NULL) = 0
09:21:22.479183 recvfrom(16, "\0\0\0\2N;", 655350, 0, NULL, NULL) = 6
09:21:23.136261 close(16) = 0
09:21:23.136497 alarm(30) = 0
09:21:23.136749 sendto(10, "\0\0\0+\5\0\0\0\0\0\0\0\0\0\0\0\0\0\00011\1\0\0\0\0\0\0{\"ty"..., 43, 0, NULL, 0) = 43
09:21:23.147287 alarm(0) = 30
09:21:23.147537 alarm(30) = 0
09:21:23.147791 alarm(0) = 30
09:21:23.147960 alarm(30) = 0
09:21:23.148126 write(18, "[api-host-2][NOTICE][2021-05-10 "..., 118) = 118
09:21:23.148296 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.148391 sendto(17, "1\r\n$4\r\nPING\r\n", 14, MSG_DONTWAIT, NULL, 0) = 14
09:21:23.148505 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.148628 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:23.179974 recvfrom(17, "+PONG\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 7
09:21:23.180221 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.180329 sendto(17, "3\r\n$4\r\nHGET\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64
09:21:23.180981 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.181093 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:23.183216 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5
09:21:23.186886 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.188151 sendto(17, "3\r\n$4\r\nHGET\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44
09:21:23.188338 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.188459 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:23.202631 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5
09:21:23.202879 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.203008 sendto(17, "3\r\n$4\r\nHDEL\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44
09:21:23.203167 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.203292 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:23.205155 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4
09:21:23.205329 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.205451 sendto(17, "3\r\n$4\r\nHDEL\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64
09:21:23.205638 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.205745 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:23.208125 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4
09:21:23.208291 alarm(0) = 30
09:21:23.208518 alarm(30) = 0
09:21:23.208674 alarm(0) = 30
09:21:23.208851 alarm(30) = 0
09:21:23.209017 write(18, "[api-host-2][NOTICE][2021-05-10 "..., 118) = 118
09:21:23.209171 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.209282 sendto(17, "1\r\n$4\r\nPING\r\n", 14, MSG_DONTWAIT, NULL, 0) = 14
09:21:23.209444 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.209551 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:23.211959 recvfrom(17, "+PONG\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 7
09:21:23.212136 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.212250 sendto(17, "3\r\n$4\r\nHGET\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64
09:21:23.212399 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.212518 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:23.223226 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5
09:21:23.223501 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.223653 sendto(17, "3\r\n$4\r\nHGET\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44
09:21:23.223797 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.223920 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:23.226627 recvfrom(17, "$-1\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5
09:21:23.226839 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.226967 sendto(17, "3\r\n$4\r\nHDEL\r\n$17\r\nQXB_MOB2PLAT_"..., 44, MSG_DONTWAIT, NULL, 0) = 44
09:21:23.227947 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.228085 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:23.232497 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4
09:21:23.232717 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.232841 sendto(17, "3\r\n$4\r\nHDEL\r\n$16\r\nQXB_CID2MOB_n"..., 64, MSG_DONTWAIT, NULL, 0) = 64
09:21:23.232990 poll([{fd=17, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
09:21:23.233100 poll([{fd=17, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=17, revents=POLLIN}])
09:21:23.244036 recvfrom(17, ":0\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 4
09:21:23.245634 alarm(0) = 30
09:21:23.246193 gettimeofday({tv_sec=1620609683, tv_usec=246252}, NULL) = 0
09:21:23.246330 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 16
09:21:23.246472 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR)
09:21:23.246627 fcntl(16, F_SETFL, O_RDWR|O_NONBLOCK) = 0
09:21:23.246756 connect(16, {sa_family=AF_INET, sin_port=htons(2303), sin_addr=inet_addr("10.168.91.75")}, 16) = -1 EINPROGRESS (操作现在正在进行)
09:21:23.247047 poll([{fd=16, events=POLLIN|POLLOUT|POLLERR|POLLHUP}], 1, 3000) = 1 ([{fd=16, revents=POLLOUT}])
09:21:23.247214 getsockopt(16, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
09:21:23.247353 fcntl(16, F_SETFL, O_RDWR) = 0
09:21:23.247488 sendto(16, "\0\0\0\34\313\0\0\0\0\0\0\0\0\0\0\0\0\0\0+\f\1\0\0\0\0\0\0", 28, 0, NULL, 0) = 28
09:21:23.247704 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR)
09:21:23.249515 fcntl(16, F_SETFL, O_RDWR) = 0
09:21:23.249758 gettimeofday({tv_sec=1620609683, tv_usec=249807}, NULL) = 0
09:21:23.249928 recvfrom(16, "\0\0\0\2N;", 655350, 0, NULL, NULL) = 6
09:21:23.637876 close(16) = 0
09:21:23.638187 alarm(30) = 0
09:21:23.638630 sendto(10, "\0\0\0+\5\0\0\0\0\0\0\0\0\0\0\0\0\0\0+\f\1\0\0\0\0\0\0{\"ty"..., 43, 0, NULL, 0) = 43
09:21:23.639026 alarm(0) = 30
09:21:23.641024 gettimeofday({tv_sec=1620609683, tv_usec=641103}, NULL) = 0
09:21:23.641818 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 16
09:21:23.642103 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR)
09:21:23.642341 fcntl(16, F_SETFL, O_RDWR|O_NONBLOCK) = 0
09:21:23.643719 connect(16, {sa_family=AF_INET, sin_port=htons(2303), sin_addr=inet_addr("10.168.91.75")}, 16) = -1 EINPROGRESS (操作现在正在进行)
09:21:23.644015 poll([{fd=16, events=POLLIN|POLLOUT|POLLERR|POLLHUP}], 1, 3000) = 1 ([{fd=16, revents=POLLOUT}])
09:21:23.644194 getsockopt(16, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
09:21:23.644368 fcntl(16, F_SETFL, O_RDWR) = 0
09:21:23.644535 sendto(16, "\0\0\0\34\313\0\0\0\0\0\0\0\0\0\0\0\0\0\00011\1\0\0\0\0\0\0", 28, 0, NULL, 0) = 28
09:21:23.644887 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR)
09:21:23.645031 fcntl(16, F_SETFL, O_RDWR) = 0
09:21:23.645687 gettimeofday({tv_sec=1620609683, tv_usec=645740}, NULL) = 0
09:21:23.645869 recvfrom(16, ^Cstrace: Process 129499 detached
lsof -nPp 分析后
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
php 129499 root cwd DIR 202,1 4096 1199944 /home/www/chat50
php 129499 root rtd DIR 202,1 4096 2 /
php 129499 root txt REG 202,1 31908647 1575462 /usr/local/php56/bin/php
php 129499 root mem REG 202,1 217032 401548 /var/db/nscd/hosts
php 129499 root mem REG 202,1 61560 162406 /usr/lib64/libnss_files-2.17.so
php 129499 root DEL REG 202,1 262841 /tmp/xcache
php 129499 root mem REG 202,1 708960 1453372 /usr/local/php56/lib/php/extensions/no-debug-non-zts-20131226/xcache.so
php 129499 root mem REG 202,1 1362871 1453336 /usr/local/php56/lib/php/extensions/no-debug-non-zts-20131226/soap.so
php 129499 root mem REG 202,1 2500552 1445058 /usr/local/php56/lib/php/extensions/no-debug-non-zts-20131226/redis.so
php 129499 root mem REG 202,1 678263 1461496 /usr/local/php56/lib/php/extensions/no-debug-non-zts-20131226/mysqli.so
php 129499 root mem REG 202,1 662449 158254 /usr/local/lib/libmcrypt.so.4.4.8
php 129499 root mem REG 202,1 169570 1454154 /usr/local/php56/lib/php/extensions/no-debug-non-zts-20131226/mcrypt.so
php 129499 root mem REG 202,1 855115 1454297 /usr/local/php56/lib/php/extensions/no-debug-non-zts-20131226/zip.so
php 129499 root mem REG 202,1 179800 163519 /usr/lib64/libevent_core-2.0.so.5.1.9
php 129499 root mem REG 202,1 133864 163521 /usr/lib64/libevent_extra-2.0.so.5.1.9
php 129499 root mem REG 202,1 24456 163523 /usr/lib64/libevent_openssl-2.0.so.5.1.9
php 129499 root mem REG 202,1 803448 1442226 /usr/local/php56/lib/php/extensions/no-debug-non-zts-20131226/event.so
php 129499 root mem REG 202,1 106172832 166453 /usr/lib/locale/locale-archive
php 129499 root mem REG 202,1 402384 134594 /usr/lib64/libpcre.so.1.2.0
php 129499 root mem REG 202,1 155744 132878 /usr/lib64/libselinux.so.1
php 129499 root mem REG 202,1 121320 133626 /usr/lib64/libsasl2.so.3.0.0
php 129499 root mem REG 202,1 15688 134676 /usr/lib64/libkeyutils.so.1.5
php 129499 root mem REG 202,1 67104 161754 /usr/lib64/libkrb5support.so.0.1
php 129499 root mem REG 202,1 68192 133155 /usr/lib64/libbz2.so.1.0.6
php 129499 root mem REG 202,1 157424 151357 /usr/lib64/liblzma.so.5.2.2
php 129499 root mem REG 202,1 352608 151584 /usr/lib64/libldap-2.4.so.2.10.7
php 129499 root mem REG 202,1 61952 141025 /usr/lib64/liblber-2.4.so.2.10.7
php 129499 root mem REG 202,1 15856 137501 /usr/lib64/libcom_err.so.2.1
php 129499 root mem REG 202,1 210784 151492 /usr/lib64/libk5crypto.so.3.1
php 129499 root mem REG 202,1 967760 137059 /usr/lib64/libkrb5.so.3.3
php 129499 root mem REG 202,1 320720 146456 /usr/lib64/libgssapi_krb5.so.2.2
php 129499 root mem REG 202,1 142144 132848 /usr/lib64/libpthread-2.17.so
php 129499 root mem REG 202,1 251792 162750 /usr/lib64/libnspr4.so
php 129499 root mem REG 202,1 20040 166470 /usr/lib64/libplc4.so
php 129499 root mem REG 202,1 15744 166471 /usr/lib64/libplds4.so
php 129499 root mem REG 202,1 198968 166472 /usr/lib64/libnssutil3.so
php 129499 root mem REG 202,1 1257728 166614 /usr/lib64/libnss3.so
php 129499 root mem REG 202,1 168336 166615 /usr/lib64/libsmime3.so
php 129499 root mem REG 202,1 370584 166616 /usr/lib64/libssl3.so
php 129499 root mem REG 202,1 186680 142759 /usr/lib64/libssh2.so.1.0.1
php 129499 root mem REG 202,1 208920 133506 /usr/lib64/libidn.so.11.6.11
php 129499 root mem REG 202,1 11392 157241 /usr/lib64/libfreebl3.so
php 129499 root mem REG 202,1 2156240 132822 /usr/lib64/libc-2.17.so
php 129499 root mem REG 202,1 795592 161017 /usr/lib64/libfreetype.so.6.14.0
php 129499 root mem REG 202,1 2521144 132882 /usr/lib64/libcrypto.so.1.0.2k
php 129499 root mem REG 202,1 470376 160594 /usr/lib64/libssl.so.1.0.2k
php 129499 root mem REG 202,1 1509600 143280 /usr/lib64/libxml2.so.2.9.1
php 129499 root mem REG 202,1 115816 162404 /usr/lib64/libnsl-2.17.so
php 129499 root mem REG 202,1 19248 162402 /usr/lib64/libdl-2.17.so
php 129499 root mem REG 202,1 1136944 162403 /usr/lib64/libm-2.17.so
php 129499 root mem REG 202,1 43712 162409 /usr/lib64/librt-2.17.so
php 129499 root mem REG 202,1 439280 151366 /usr/lib64/libcurl.so.4.3.0
php 129499 root mem REG 202,1 285328 133423 /usr/lib64/libjpeg.so.62.1.0
php 129499 root mem REG 202,1 90248 132818 /usr/lib64/libz.so.1.2.7
php 129499 root mem REG 202,1 179296 133405 /usr/lib64/libpng15.so.15.13.0
php 129499 root mem REG 202,1 109976 162408 /usr/lib64/libresolv-2.17.so
php 129499 root mem REG 202,1 40600 151333 /usr/lib64/libcrypt-2.17.so
php 129499 root mem REG 202,1 163312 166456 /usr/lib64/ld-2.17.so
php 129499 root mem REG 202,1 26970 408331 /usr/lib64/gconv/gconv-modules.cache
php 129499 root 0u CHR 136,1 0t0 4 /dev/pts/1 (deleted)
php 129499 root 1w CHR 1,3 0t0 1028 /dev/null
php 129499 root 2w CHR 1,3 0t0 1028 /dev/null
php 129499 root 3u REG 202,1 0 262854 /tmp/.xcache.0.129265.1.mutex (deleted)
php 129499 root 4u a_inode 0,10 0 9546 [eventpoll]
php 129499 root 5u unix 0xffff880c64b99dc0 0t0 759626038 socket
php 129499 root 6u unix 0xffff880c64b9e1c0 0t0 759626039 socket
php 129499 root 7u IPv4 759626041 0t0 TCP 10.168.91.75:40904->10.168.91.75:1236 (ESTABLISHED)
php 129499 root 8u IPv4 759626049 0t0 TCP 10.168.91.75:55834->10.168.91.75:2300 (ESTABLISHED)
php 129499 root 9u IPv4 759626052 0t0 TCP 10.168.91.75:50054->10.168.91.75:2301 (ESTABLISHED)
php 129499 root 10u IPv4 759626056 0t0 TCP 10.168.91.75:59092->10.168.91.75:2303 (ESTABLISHED)
php 129499 root 11u IPv4 759626059 0t0 TCP 10.168.91.75:51924->10.168.91.75:2306 (ESTABLISHED)
php 129499 root 12u IPv4 759626061 0t0 TCP 10.168.91.75:58294->10.168.91.75:2307 (ESTABLISHED)
php 129499 root 13u IPv4 759626077 0t0 TCP 10.168.91.75:47806->10.168.91.75:2304 (ESTABLISHED)
php 129499 root 14u IPv4 759626083 0t0 TCP 10.168.91.75:42994->10.168.91.75:2305 (ESTABLISHED)
php 129499 root 15u IPv4 759626089 0t0 TCP 10.168.91.75:32772->10.168.91.75:2302 (ESTABLISHED)
php 129499 root 17u IPv4 759621951 0t0 TCP 10.168.91.75:39288->10.146.29.193:6379 (ESTABLISHED)
php 129499 root 18w REG 202,1 67425954 405332 /var/log/chatWorker/2021-05-10.log
php 129499 root 19u IPv4 764258273 0t0 TCP 127.0.0.1:43048->127.0.0.1:6688 (CLOSE_WAIT)
php 129499 root 20u IPv4 764269827 0t0 TCP 127.0.0.1:44020->127.0.0.1:6688 (CLOSE_WAIT)
php 129499 root 21u IPv4 764293538 0t0 TCP 127.0.0.1:39318->127.0.0.1:6688 (CLOSE_WAIT)
php 129499 root 22u IPv4 764298809 0t0 TCP 127.0.0.1:42902->127.0.0.1:6688 (CLOSE_WAIT)
php 129499 root 23u IPv4 764321399 0t0 TCP 127.0.0.1:35588->127.0.0.1:6688 (CLOSE_WAIT)
php 129499 root 24u IPv4 764339709 0t0 TCP 127.0.0.1:47380->127.0.0.1:6688 (CLOSE_WAIT)
php 129499 root 25u IPv4 764843705 0t0 TCP 127.0.0.1:50452->127.0.0.1:6688 (CLOSE_WAIT)
php 129499 root 26u IPv4 764843923 0t0 TCP 127.0.0.1:53740->127.0.0.1:6688 (CLOSE_WAIT)
php 129499 root 27u IPv4 764862631 0t0 TCP 127.0.0.1:57462->127.0.0.1:6688 (CLOSE_WAIT)
php 129499 root 28u IPv4 765089929 0t0 TCP 127.0.0.1:45636->127.0.0.1:6688 (CLOSE_WAIT)
php 129499 root 29u IPv4 765080227 0t0 TCP 127.0.0.1:47420->127.0.0.1:6688 (CLOSE_WAIT)
php 129499 root 30u IPv4 765087559 0t0 TCP 127.0.0.1:52486->127.0.0.1:6688 (CLOSE_WAIT)
php 129499 root 31u IPv4 765613071 0t0 TCP 127.0.0.1:50078->127.0.0.1:6688 (CLOSE_WAIT)
咋感觉你代码死循环了,一直读写redis
@1393:php 我们是有读写Redis;从哪看出死循环读写Redis呀
php 129499 root 31u IPv4 765613071 0t0 TCP 127.0.0.1:50078->127.0.0.1:6688 (CLOSE_WAIT) 关闭异常的,应该也是那个问题导致的吧?
我的这个应用的逻辑应该有点复杂,发一条消息,需要做到接收这不同客户端都要有收到消息,自己的其他客户端也要有这条消息显示,导致了一个用户在一个端发出一条消息,最终可能需要发送的有多次,得根据接受者与自己的客户端在线数而定
strace 下gateway进程看下吧,估计系统资源用完了
是的,资源被占用光啦
解决了?啥资源用光了?难道是打开的文件数超过了系统内核配置?
@1393:没有呢,这样的情况我重启服务器又正常啦,有没有办法可以记录到这种情况的日志的呀