Up to top level
AO15   AO16   AO17   AO18   AO19   Backgrounds   Calibration   Conference   Data   Docs   EPICMOS   EPICpn   Feedback   Gallery   Misc   OM   Pending   PhD_Theses   Publications   RGS   RadMonitor   SAS_Hardware   SAS_WS   SASv16.0   SASv16.0_Installation   SASv16.1   SASv16.1_Installation   SASv17.0   SASv17.0_Installation   SASv18.0   SASv18.0_Installation   SciSim   Simulators_other   Suggestions   Trash   Visibility   XMM-bouncing   XMM-news   XRPS   XSA   esas   incoming  

Logged in as guest

Viewing EPICpn/37424
Full headers

From: "Pawel Lachowicz" <tslpl@nus.edu.sg>
Subject: pn/data reduction problem
Compose reply
Download message
Move To:
1 replies: 1
1 followups: 1

Private message: yes  no

Notes:

Notification:


From: "Pawel Lachowicz" <tslpl@nus.edu.sg>
To: <xmmhelp@sciops.esa.int>
Subject: pn/data reduction problem
Date: Thu, 31 Jul 2008 17:43:57 +0800
This is a multi-part message in MIME format.

------=_NextPart_000_01C1_01C8F335.01F1CFE0
Content-Type: text/plain;
	format=flowed;
	charset="Windows-1252";
	reply-type=original
Content-Transfer-Encoding: 7bit

Hello,

Recently I discovered a strange problem with 'epchain' reduction
of ODF files for pn-detector. Namely, after almost one year of
reduction of the same data set of 0109141401 (corresponding to
NGC 4051, rev. 263), now the resulting event file for pn seems to
be too small in its size:

-rw-r--r-- 1 pawel 172800 2008-07-29 14:38 P0109141401PNU002PIEVLI0000.FIT

comapring to pn-event file which I obtained a year ago following
exactly the same steps of ODF reduction. I found that at the
beginning of 'epchain' running, the following errors occur:

epchain..
EXPOSURE = 1
** odfexpand: error (NoSuchSet), The ODF in directory 
`/home/pawel/xray/xmm/data/ngc4051/rev263/0109141401/odf' does
not contain a file corresponding to the request: 
instrument=EPN/dataMode=Imaging/exposureNr=1/ccdNr=1
** -: warning (notInODF), requested CCD file seems to be non-existent in 
ODF, skip processing for this CCD
** odfexpand: error (NoSuchSet), The ODF in directory 
`/home/pawel/xray/xmm/data/ngc4051/rev263/0109141401/odf' does
not contain a file corresponding to the request: 
instrument=EPN/dataMode=Imaging/exposureNr=1/ccdNr=2
** -: warning (notInODF), requested CCD file seems to be non-existent in 
ODF, skip processing for this CCD
** odfexpand: error (NoSuchSet), The ODF in directory 
`/home/pawel/xray/xmm/data/ngc4051/rev263/0109141401/odf' does
not contain a file corresponding to the request: 
instrument=EPN/dataMode=Imaging/exposureNr=1/ccdNr=3
** -: warning (notInODF), requested CCD file seems to be non-existent in 
ODF, skip processing for this CCD
/home/pawel/xray/xmm/data/ngc4051/rev263/0109141401/odf/0263_0109141401_PNU00204IME.FIT
** epframes: warning (Extrapolation), Evaluating OBT-MET fit polynomial at 
x=27704990 which is outside the time
correlation data abscissa range [27627670.91, 27703420.08] - result of 
extrapolation might be afflicted with large
numerical uncertainty.
** epframes: warning (Extrapolation), Evaluating OBT-MET fit polynomial at 
x=27737390 which is outside the time
correlation data abscissa range [27627670.91, 27703420.08] - result of 
extrapolation might be afflicted with large
numerical uncertainty.
** epframes: warning (Extrapolation), Evaluating OBT-MET fit polynomial at 
x=27703520.08 which is outside the time
correlation data abscissa range [27627670.91, 27703420.08] - result of 
extrapolation might be afflicted with large
numerical uncertainty
(...)

what would mean that some data files are missing! Strange!
I would like to ask you kindly to look closely on a possible
cause of that problem.

I run my pn reduction script (enclosed as an attachment to
this email). My working directory looks like:

perth:~/xray/xmm/data/ngc4051/rev263/0109141401> ls -lotr
total 40
-rwxr-xr-x 1 pawel  1080 2008-07-29 14:29 reduction-pn.csh
drwxr-xr-x 2 pawel  4096 2008-07-29 14:38 pn
drwxr-xr-x 2 pawel 20480 2008-07-31 17:13 odf
drwxr-xr-x 2 pawel  4096 2008-07-31 17:14 evt

The same problem I have for second observation of NGC 4051
namely, rev. 541.

Thank you, regards,

Pawel


--
National University of Singapore, Centre for Wavelets,
Approximation and Information Processing
Dr. Pawel Lachowicz
http://www.temasek-lab.nus.edu.sg/~tslpl 

------=_NextPart_000_01C1_01C8F335.01F1CFE0
Content-Type: application/octet-stream;
	name="reduction-pn.csh"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
	filename="reduction-pn.csh"

#!/bin/csh=0A=
=0A=
#############################################=0A=
#   XMM-Newton Reduction of ODF files       #=0A=
#          ver. 17.7.2006                   #=0A=
#         by Pawel Lachowicz                #=0A=
#               --ALL--                     #=0A=
#############################################=0A=
=0A=
echo ""=0A=
echo "Reduction of the XMM-pn-mos-rgs/odf data"=0A=
echo ""=0A=
=0A=
echo ""=0A=
echo "cleaning /odf directory from .SAS and .cif files"=0A=
=0A=
cd odf=0A=
rm *SUM.SAS=0A=
rm ccf.cif=0A=
cd ..=0A=
=0A=
set DIR=3D`pwd`=0A=
=0A=
setenv SAS_ODF $DIR/odf=0A=
setenv SAS_CCFPATH /home/pawel/xray/xmm/xmmsas_20080701_1801/ccf=0A=
setenv SAS_CCF $DIR/evt=0A=
=0A=
echo ""=0A=
echo "making /env directory"=0A=
=0A=
if( -d ./evt) then=0A=
  rm -rf ./evt/=0A=
endif=0A=
=0A=
mkdir evt=0A=
cd evt=0A=
=0A=
echo ""=0A=
echo "cifbuild.."=0A=
cifbuild withccfpath=3Dno analysisdate=3Dnow category=3DXMMCCF =
fullpath=3Dyes=0A=
=0A=
echo ""=0A=
echo "odfingest.."=0A=
odfingest odfdir=3D$SAS_ODF outdir=3D$SAS_ODF=0A=
=0A=
echo ""=0A=
echo "epchain.."=0A=
epchain keepintermediate=3Dnone=0A=
cp *PN*PIEVLI* pn.evt=0A=
=0A=
echo ""=0A=
echo "The processing appears to be successfull. Check these event files:"=0A=
echo ""=0A=
ls -ltr *.evt=0A=
ls -ltr *.lst=0A=
=0A=
mkdir ../pn ; cp pn.evt ../pn -f=0A=

------=_NextPart_000_01C1_01C8F335.01F1CFE

Message of length 5005 truncated

Reply 1

Resend
From: Nora Loiseau <xmmhelp@sciops.esa.int>
To: tslpl@nus.edu.sg
Subject: Re: pn/data reduction problem (PR#37424)
Date: Thu Jul 31 18:25:20 2008
Dear Dr. Pawel Lachowicz

please find here the analysis of a SAS developer:

1) you should always give us details about machine / OS / SAS version used,

2) the files corresponding to CCD 1, 2, 3, are not found just due to this being
a Small Window observation, therefore only CCD4 contains data. Why are these
warnings then given? This is due to the use of epchain instead of the officially
supported "epproc". Although they are basically doing the same, we do recommend
the use of epproc for data reduction, (however, these warnings should not affect
at all the data processing),

3) the large number of warnings like

** epframes: warning (Extrapolation), Evaluating OBT-MET fit polynomial at
x=27704990 which is outside the time correlation data abscissa range
[27627670.91, 27703420.08] - result of extrapolation might be afflicted with
large numerical uncertainty.

are due to a defective TCX (Time Correlation file), in this case. This file is
derived from the original Time Correlation file (TCS) taking into account very
precisely diverse shifts affecting the information on the photons' time
arrivals. For general purposes it is good enough to use the TCS file instead of
the TCX. This is done by defining the environment variable SAS_TIMECORR to TCS
(eg. > setenv SAS_TIMECORR TCS in a c-shell). This should be done in this
case
before launching "epproc".

4) despite all these problems there is no indication in the log extract we got
that the event file should be affected. It is though definitively. I have
reduced the data and the event list, output of epproc (independently of using
TCS or TCX) is larger than 400 MB. So, we recommend the user to run epproc with
verbosity 5 after setting the SAS_TIMECORR environment variable, and send us the
whole log file together with the info asked in (1) if the problems persist.

-----

Best regards,

Nora

----
Dr. Nora Loiseau
XMM-Newton User Support Group


Followup 1

Compose reply
Download message
From: "Pawel Lachowicz" <tslpl@nus.edu.sg>
To: "Nora Loiseau" <xmmhelp@sciops.esa.int>
Subject: Re: pn/data reduction problem (PR#37424)
Date: Sun, 3 Aug 2008 23:54:01 +0800
Dear Nora,

thank you very much for your comments on my problem. They helped
me a lot, though the problem persisted even after minor changes I did
after your advice. I found it was solved as I downloaded newest
version of CCF files. Now, I can reduce ODF files without any problems.
Again, thanks!

Best of life,
Pawel


Up to top level
AO15   AO16   AO17   AO18   AO19   Backgrounds   Calibration   Conference   Data   Docs   EPICMOS   EPICpn   Feedback   Gallery   Misc   OM   Pending   PhD_Theses   Publications   RGS   RadMonitor   SAS_Hardware   SAS_WS   SASv16.0   SASv16.0_Installation   SASv16.1   SASv16.1_Installation   SASv17.0   SASv17.0_Installation   SASv18.0   SASv18.0_Installation   SciSim   Simulators_other   Suggestions   Trash   Visibility   XMM-bouncing   XMM-news   XRPS   XSA   esas   incoming  

Logged in as guest


Please make your (short) question the subject of your request!


Web interface using JitterBug ... back to the XMM home page