Opened 4 years ago

Last modified 4 years ago

#1456 accepted

SCS mxtce daemon memory leak

Reported by: xyang@maxgigapop.net Owned by: xyang@maxgigapop.net
Priority: major Milestone:
Component: STITCHING Version: SPIRAL7
Keywords: Cc: ckotil@grnoc.iu.edu
Dependencies:

Description

As reported by Chad Kotil:

There appears to be a memory leak in the SCS service. Today we received a low memory alarm for the host running SCS.

[ckotil@geni-scs ~]$ ps aux | grep mxt ckotil 5901 0.0 0.0 103248 840 pts/2 S+ 12:55 0:00 grep mxt root 8873 19.2 85.6 4375164 3351352 ? Ssl Mar18 477:43 /usr/local/geni-scssrc/main/mxtce -d

The host only has 4gb of memory, and the SCS was using about 85% of that.

After restart:

[ckotil@geni-scs ~]$ ps aux | grep mxt root 5971 33.0 3.5 563308 137288 ? Ssl 12:56 0:00 /usr/local/geni-scs/src/main/mxtce -d ckotil 5978 0.0 0.0 103248 840 pts/2 S+ 12:56 0:00 grep mxt

10 minutes later:

[ckotil@geni-scs ~]$ ps aux | grep mxt root 5971 19.1 6.8 760044 268248 ? Ssl 12:56 1:36 /usr/local/geni-scs/src/main/mxtce -d ckotil 6486 0.0 0.0 103248 840 pts/2 S+ 13:04 0:00 grep mxt

Change History (1)

comment:1 Changed 4 years ago by xyang@maxgigapop.net

Status: newaccepted

Fixes have been made into an experiment branch 'mem-leak-fix' out of the 2.5 branch.

They do not resolve all the leaks but should have got those that had caused a real problem.

The code is not deployed on Test SCS.

I expected a merge back to the 2.5 release branch once it looks stable on the Test SCS. Then we can patch the Production SCS.

Note: See TracTickets for help on using tickets.