Up to top level
AO15   AO16   AO17   AO18   AO19   Backgrounds   Calibration   Conference   Data   Docs   EPICMOS   EPICpn   Feedback   Gallery   Misc   OM   Pending   PhD_Theses   Publications   RGS   RadMonitor   SAS_Hardware   SAS_WS   SASv16.0   SASv16.0_Installation   SASv16.1   SASv16.1_Installation   SASv17.0   SASv17.0_Installation   SASv18.0   SASv18.0_Installation   SciSim   Simulators_other   Suggestions   Trash   Visibility   XMM-bouncing   XMM-news   XRPS   XSA   esas   incoming  

Logged in as guest

Viewing SASv17.0_Installation/84877
Full headers

From: rmj@ast.cam.ac.uk
Subject: Memory allocation errors in hkgtigen in Fedora 28 and 30 beta
Compose reply
Download message
Move To:
2 replies: 1 2
0 followups:

Private message: yes  no

Notes:

Notification:


Date: Wed, 17 Apr 2019 11:43:35 +0200
From: rmj@ast.cam.ac.uk
To: xmmhelp@sciops.esa.int
Subject: Memory allocation errors in hkgtigen in Fedora 28 and 30 beta
Full_Name: Roderick Johnstone
Submission from: (NULL) (193.147.152.102)


Hi

This is a follow up to #84519.

Running rgsproc on recent versions of Fedora linux causes hkgtigen to fail with
the message:

** rgsproc::hkgtigen: fatal error (OutOfMemory), An out-of-memory condition has
occurred: A component of the task `rgsproc' has tried to allocate more dynamic
memory than what is currently available in the system. A likely cause of this is
an attempt to read and/or write a very large data set with the DAL in its
high-memory-mode. In this case, please choose the low-memory-model (see the DAL
documentation for details) and try again. If this does not help it might be that
too many users are using the system or the swap space is insufficient.

I don't think the system is really out of memory because the task runs correctly
on the same hardware with an earlier glibc. See details below...

I've seen this on Fedora 28 and now on Fedora 30 beta which I am evaluating with
both sas 16.1.0 and 17.0.0.

The task works normally under Fedora 28 with glibc-2.27-35 or previous packages,
but fails with the more recent updates, eg glibc-2.27-38.

My Fedora 30 testing system has glibc-2.29-9 on which it also fails.

It is possible to work around the error by setting the environment variable
MALLOC_CHECK_ to the value 1 or 2. Setting it to 0 still causes the error.

When setting MALLOC_CHECK_ to 1 or 2 the task rgsproc appears to run to
completion without any errors and the output files appear to be reasonable.
There are no warning printed from glibc which is what might be expected from if
there was a coding problem.

Could someone look at this please?

Thanks.

Roderick Johnstone

This is with:
SAS release: xmmsas_20180620_1732-17.0.0
Compiled on: Wed Jun 20 21:06:03 CEST 2018
Compiled by: sasbuild@sasbld03n
Platform   : Ubuntu16.04 64


Reply 1

Resend
From: Pedro Rodriguez <xmmhelp@sciops.esa.int>
To: rmj@ast.cam.ac.uk
Subject:   Re:    Memory   allocation errors in hkgtigen in Fedora 28 and 30 beta (PR#84877)
Date: Wed Apr 17 13:11:23 2019
Hello,

your question has been forwarded to SAS developers for detailed analysis.

Please, note that due to Easter holidays the reply will not be available before
next week. 

Pedro

> Full_Name: Roderick Johnstone
> Submission from: (NULL) (193.147.152.102)
> 
> 
> Hi
> 
> This is a follow up to #84519.
> 
> Running rgsproc on recent versions of Fedora linux causes hkgtigen to fail
with
> the message:
> 
> ** rgsproc::hkgtigen: fatal error (OutOfMemory), An out-of-memory condition
has
> occurred: A component of the task `rgsproc' has tried to allocate more
dynamic
> memory than what is currently available in the system. A likely cause of
this
is
> an attempt to read and/or write a very large data set with the DAL in its
> high-memory-mode. In this case, please choose the low-memory-model (see the
DAL
> documentation for details) and try again. If this does not help it might be
that
> too many users are using the system or the swap space is insufficient.
> 
> I don't think the system is really out of memory because the task runs
correctly
> on the same hardware with an earlier glibc. See details below...
> 
> I've seen this on Fedora 28 and now on Fedora 30 beta which I am evaluating
with
> both sas 16.1.0 and 17.0.0.
> 
> The task works normally under Fedora 28 with glibc-2.27-35 or previous
packages,
> but fails with the more recent updates, eg glibc-2.27-38.
> 
> My Fedora 30 testing system has glibc-2.29-9 on which it also fails.
> 
> It is possible to work around the error by setting the environment
variable
> MALLOC_CHECK_ to the value 1 or 2. Setting it to 0 still causes the error.
> 
> When setting MALLOC_CHECK_ to 1 or 2 the task rgsproc appears to run to
> completion without any errors and the output files appear to be
reasonable.
> There are no warning printed from glibc which is what might be expected
from
if
> there was a coding problem.
> 
> Could someone look at this please?
> 
> Thanks.
> 
> Roderick Johnstone
> 
> This is with:
> SAS release: xmmsas_20180620_1732-17.0.0
> Compiled on: Wed Jun 20 21:06:03 CEST 2018
> Compiled by: sasbuild@sasbld03n
> Platform   : Ubuntu16.04 64
> 
> 


Reply 2

Resend
From: Pedro Rodriguez <xmmhelp@sciops.esa.int>
To: rmj@ast.cam.ac.uk
Subject: Re:    Memory   allocation errors in hkgtigen in Fedora 28 and 30 beta (PR#84877)
Date: Mon Apr 29 16:38:32 2019
Hi,

thank you very much for the detailed analysis made on the fatal error 
issued by rgsproc.

SAS analysts have been able to identify the task hkgtigen as the one responsible

for the fatal error. The error is issued also when processing EPIC pn and MOS
data and not only associated to RGS. Furthermore, the problem is persistent in
the
forthcoming SAS18.0 release.

As mentioned in your email, the error occurs for most recent versions of 
glibc library but apparently there is nothing in their release notes that 
justify it. The SAS analysts are debugging the code as well as re-checking
the library release notes and SAS compilation options to understand properly
the
origin of the problem and, eventually, fix it.

Meanwhile, for future questions in our helpdesk about the same or similar issue
we will refer to the workaround in your email to set MALLOC_CHECK to 1 or 2.

If by the time of SAS18.0 is released we have not found a solution to the 
problem we are considering to add a specific entry in our SAS Watchout page.

Thanks again,

Pedro

 

> Full_Name: Roderick Johnstone
> Submission from: (NULL) (193.147.152.102)
> 
> 
> Hi
> 
> This is a follow up to #84519.
> 
> Running rgsproc on recent versions of Fedora linux causes hkgtigen to fail
with
> the message:
> 
> ** rgsproc::hkgtigen: fatal error (OutOfMemory), An out-of-memory condition
has
> occurred: A component of the task `rgsproc' has tried to allocate more
dynamic
> memory than what is currently available in the system. A likely cause of
this
is
> an attempt to read and/or write a very large data set with the DAL in its
> high-memory-mode. In this case, please choose the low-memory-model (see the
DAL
> documentation for details) and try again. If this does not help it might be
that
> too many users are using the system or the swap space is insufficient.
> 
> I don't think the system is really out of memory because the task runs
correctly
> on the same hardware with an earlier glibc. See details below...
> 
> I've seen this on Fedora 28 and now on Fedora 30 beta which I am evaluating
with
> both sas 16.1.0 and 17.0.0.
> 
> The task works normally under Fedora 28 with glibc-2.27-35 or previous
packages,
> but fails with the more recent updates, eg glibc-2.27-38.
> 
> My Fedora 30 testing system has glibc-2.29-9 on which it also fails.
> 
> It is possible to work around the error by setting the environment
variable
> MALLOC_CHECK_ to the value 1 or 2. Setting it to 0 still causes the error.
> 
> When setting MALLOC_CHECK_ to 1 or 2 the task rgsproc appears to run to
> completion without any errors and the output files appear to be
reasonable.
> There are no warning printed from glibc which is what might be expected
from
if
> there was a coding problem.
> 
> Could someone look at this please?
> 
> Thanks.
> 
> Roderick Johnstone
> 
> This is with:
> SAS release: xmmsas_20180620_1732-17.0.0
> Compiled on: Wed Jun 20 21:06:03 CEST 2018
> Compiled by: sasbuild@sasbld03n
> Platform   : Ubuntu16.04 64
> 
> 

Up to top level
AO15   AO16   AO17   AO18   AO19   Backgrounds   Calibration   Conference   Data   Docs   EPICMOS   EPICpn   Feedback   Gallery   Misc   OM   Pending   PhD_Theses   Publications   RGS   RadMonitor   SAS_Hardware   SAS_WS   SASv16.0   SASv16.0_Installation   SASv16.1   SASv16.1_Installation   SASv17.0   SASv17.0_Installation   SASv18.0   SASv18.0_Installation   SciSim   Simulators_other   Suggestions   Trash   Visibility   XMM-bouncing   XMM-news   XRPS   XSA   esas   incoming  

Logged in as guest


Please make your (short) question the subject of your request!


Web interface using JitterBug ... back to the XMM home page