Up to top level
AO15   AO16   AO17   AO18   Backgrounds   Calibration   Conference   Data   Docs   EPICMOS   EPICpn   Feedback   Gallery   Misc   OM   Pending   PhD_Theses   Publications   RGS   RadMonitor   SAS_Hardware   SAS_WS   SASv16.0   SASv16.0_Installation   SASv16.1   SASv16.1_Installation   SASv17.0   SASv17.0_Installation   SciSim   Simulators_other   Suggestions   Trash   Visibility   XMM-bouncing   XMM-news   XRPS   XSA   esas   incoming  

Logged in as guest

Viewing EPICpn/8560
Full headers

From: cocj@roe.ac.uk
Subject: EPIC pn FLAG filtering
Compose reply
Download message
Move To:
2 replies: 1 2
2 followups: 1 2

Private message: yes  no

Notes:

Notification:


Date: Sun, 18 May 2003 19:15:34 GMT
From: cocj@roe.ac.uk
To: xmmhelp@xmm.vilspa.esa.es
CC: cocj@roe.ac.uk
Subject: EPIC pn FLAG filtering
Full_Name: Olivia Johnson
Submission from: (NULL) (195.194.120.192)


I'm finding no change in the number of events when I filter a pn 
exposure with the #XMMEA_EP flag set.  (I've tried to submit this through the
SAS bug form interface, but my Netscape stalls before displaying the form.)

I am using data from exposures 0094310101 and 0094310201 which I've merged
using
merge.  Filtering the MOS exposures with #XMMEA_EM removes ~40% of events, but 
filtering PN with #XMMEA_EP removes no events. 

The command I am using is:

evselect table=pn.fits:EVENTS withfilteredset=yes expression='(#XMMEA_EP)'
filteredset=pn_flag_filt.fits filtertype=expression keepfilteroutput=yes
updateexposure=yes filterexposure=yes

Using the more conservative FLAG==0 filter, I loose ~40% of my pn counts.
Could you explain this behavior?

Thanks very much, 
-Olivia 

PS.  I'm also curious if there's a SAS-approved recommendation for filtering out

low energy events.  I've seen values of 150, 200, and 300 eV in the literature,
and
wonder if there's been any study of which value is best in terms of general
signal
to noise.


Reply 1

Resend
From: Matthias Ehle <xmmhelp@xmm.vilspa.esa.es>
To: cocj@roe.ac.uk
Subject: Re: EPIC pn FLAG filtering (PR#8560)
Date: Mon May 19 08:19:05 2003
Dear Olivia 

you wrote
> I'm finding no change in the number of events when I filter a pn 
> exposure with the #XMMEA_EP flag set.  (I've tried to submit this through
the
> SAS bug form interface, but my Netscape stalls before displaying the
form.)
> 
> I am using data from exposures 0094310101 and 0094310201 which I've merged
> using
> merge.  Filtering the MOS exposures with #XMMEA_EM removes ~40% of events,
but

> filtering PN with #XMMEA_EP removes no events. 
> 
> The command I am using is:
> 
> evselect table=pn.fits:EVENTS withfilteredset=yes expression='(#XMMEA_EP)'
> filteredset=pn_flag_filt.fits filtertype=expression keepfilteroutput=yes
> updateexposure=yes filterexposure=yes
> 
> Using the more conservative FLAG==0 filter, I loose ~40% of my pn counts.
> Could you explain this behavior?

First question: how did you generate the pn event list? If you used epproc, 
several (default) event selections already have been applied (see task
description
of epicproc, pn and MOS sections). If you generated the pn calibrated event
list
with the task 'epchain', by default #XMMEA_EP is applied in the screening
step (see epchain task description). 

So, could it be that the pn event list (pn.fits) already had been filtered
with #XMMEA_EP? An easy way to test this is to open the file with
xmmselect table=pn.fits and click on "yes" in the pop-up window
"The table contains data subspace information..." to display the
table selection history. Can you, please, confirm that #XMMEA_EP
was not applied yet. 

> PS.  I'm also curious if there's a SAS-approved recommendation for
filtering
out
> low energy events.  I've seen values of 150, 200, and 300 eV in the
literature,
> and wonder if there's been any study of which value is best in terms of
general
> signal to noise.

By default epproc & epchain remove all events with an energy below 150 eV.
This reduces the lengths of the event list significantly and was choosen
by the pn team as appropriate lower limit. Please, also have a look at the
EPIC status of calibration document (available at
http://xmm.vilspa.esa.es/external/xmm_sw_cal/calib/index.shtml)
where it is stated that for MOS imaging mode you can go down to 0.2 keV
(BUT care must be taken below 0.3 keV) and for pn imaging mode down to 0.15
keV.
Sometimes you see in EPIC in the very soft energy band bright columns or
pixels which go away if you restrict your analysis to somewhat higher energies.
Of course the energy selection also depends on the source type you are
looking at: e.g. for diffuse extended emission you certainly want to keep
the soft events in your image and spectrum.

Cheers,
Matthias.

Matthias Ehle 
XMM-Newton SOC 
User Support Group


Followup 1

Compose reply
Download message
Date: Mon, 19 May 2003 14:11:22 +0100 (BST)
From: Olivia Johnson <cocj@roe.ac.uk>
To: Matthias Ehle <xmmhelp@xmm.vilspa.esa.es>
Subject: Re: EPIC pn FLAG filtering (PR#8560)
Hi again, and thanks for your quick response.

> First question: how did you generate the pn event list?

I'm using an eventlist made from merging the pipeline processed eventlists.  
Since there were no changes in the calibration which would effect imaging 
data since the pipeline processing, this should be ok?  I have data from 
two different orbits of the same field, and the first orbit data is split 
into scheduled and unscheduled chunks.  I've merged these, and then merged 
the result with the second orbit data, and this is what pn.fits is.

> So, could it be that the pn event list (pn.fits) already had been filtered
> with #XMMEA_EP? An easy way to test this is to open the file with
> xmmselect table=pn.fits and click on "yes" in the pop-up window
> "The table contains data subspace information..." to display the
> table selection history. Can you, please, confirm that #XMMEA_EP
> was not applied yet. 

I don't think this is the case.  Here's what I see in the xmmselect 
window:

(( CCDNR in [1] && gti(pn.fits:STDGTI01,TIME) ) || ( CCDNR in [2]
&& 
gti(pn.fits:STDGTI02,TIME) ) || ( CCDNR in [3] && 
gti(pn.fits:STDGTI03,TIME) ) || ( CCDNR in [4] && 
gti(pn.fits:STDGTI04,TIME) ) || ( CCDNR in [5] && 
gti(pn.fits:STDGTI05,TIME) ) || ( CCDNR in [6] && 
gti(pn.fits:STDGTI06,TIME) ) || ( CCDNR in [7] && 
gti(pn.fits:STDGTI07,TIME) ) || ( CCDNR in [8] && 
gti(pn.fits:STDGTI08,TIME) ) || ( CCDNR in [9] && 
gti(pn.fits:STDGTI09,TIME) ) || ( CCDNR in [10] && 
gti(pn.fits:STDGTI10,TIME) ) || ( CCDNR in [11] && 
gti(pn.fits:STDGTI11,TIME) ) || ( CCDNR in [12] && 
gti(pn.fits:STDGTI12,TIME) ))

Any suggestions?

Thanks again,

Olivia 




Reply 2

Resend
From: Matthias Ehle <xmmhelp@xmm.vilspa.esa.es>
To: cocj@roe.ac.uk
Subject: Re: EPIC pn FLAG filtering (PR#8560)
Date: Mon May 19 13:59:14 2003
Dear Olivia,

I think your reply clarifies the situation:

If you base your analysis on SSC pipeline processed calibrated event lists,
you can check the configuration of the pipeline reading the
P009431010XOBX000SCRLOG0000.ASZ (gziped ASCII) log file.
There you will see that evselect was executed at a certain stage
on the pn intermediate products with expression='#XMMEA_EP && (PI>150
||
PI<-150)'.
So if you use pipeline processed pn data this selection indeed already
has been applied and filtering again with #XMMEA_EP will have no effect.

> I don't think this is the case.  Here's what I see in the xmmselect 
> window: ...

OK as the final lists were created with a consecutive call to evselect
you only see this last filter selection expression.

Another way to check the applied filter is to open the event list with
the ftool fv and look at the header of the EVENTS extension:
there you find a list of XMMEA_?? keywords and also the XMMEA_EP flag
which have been applied to your data.

Cheers,
  Matthias.


Followup 2

Compose reply
Download message
Date: Mon, 19 May 2003 15:19:36 +0100 (BST)
From: Olivia Johnson <cocj@roe.ac.uk>
To: Matthias Ehle <xmmhelp@xmm.vilspa.esa.es>
Subject: Re: EPIC pn FLAG filtering (PR#8560)
Yes that explains it.  I've looked at the ASCII file and see where the 
#XMMEA_EP events were destructively filtered, but can find no 
corresponding expression for the MOS data, which explains why #XMMEA_EM 
still has an effect.

Cheers for your help, 
-olivia


On Mon, 19 May 2003, Matthias Ehle wrote:

> Dear Olivia,
> 
> I think your reply clarifies the situation:
> 
> If you base your analysis on SSC pipeline processed calibrated event lists,
> you can check the configuration of the pipeline reading the
> P009431010XOBX000SCRLOG0000.ASZ (gziped ASCII) log file.
> There you will see that evselect was executed at a certain stage
> on the pn intermediate products with expression='#XMMEA_EP &&
(PI>150 ||
> PI<-150)'.
> So if you use pipeline processed pn data this selection indeed already
> has been applied and filtering again with #XMMEA_EP will have no effect.
> 
> > I don't think this is the case.  Here's what I see in the xmmselect 
> > window: ...
> 
> OK as the final lists were created with a consecutive call to evselect
> you only see this last filter selection expression.
> 
> Another way to check the applied filter is to open the event list with
> the ftool fv and look at the header of the EVENTS extension:
> there you find a list of XMMEA_?? keywords and also the XMMEA_EP flag
> which have been applied to your data.
> 
> Cheers,
>   Matthias.
> 



Up to top level
AO15   AO16   AO17   AO18   Backgrounds   Calibration   Conference   Data   Docs   EPICMOS   EPICpn   Feedback   Gallery   Misc   OM   Pending   PhD_Theses   Publications   RGS   RadMonitor   SAS_Hardware   SAS_WS   SASv16.0   SASv16.0_Installation   SASv16.1   SASv16.1_Installation   SASv17.0   SASv17.0_Installation   SciSim   Simulators_other   Suggestions   Trash   Visibility   XMM-bouncing   XMM-news   XRPS   XSA   esas   incoming  

Logged in as guest


Please make your (short) question the subject of your request!


Web interface using JitterBug ... back to the XMM home page