Up to top level
AO15   AO16   AO17   AO18   AO19   Backgrounds   Calibration   Conference   Data   Docs   EPICMOS   EPICpn   Feedback   Gallery   Misc   OM   Pending   PhD_Theses   Publications   RGS   RadMonitor   SAS_Hardware   SAS_WS   SASv16.0   SASv16.0_Installation   SASv16.1   SASv16.1_Installation   SASv17.0   SASv17.0_Installation   SASv18.0   SASv18.0_Installation   SciSim   Simulators_other   Suggestions   Trash   Visibility   XMM-bouncing   XMM-news   XRPS   XSA   esas   incoming  

Logged in as guest

Viewing SASv16.0/81138
Full headers

From: wangjingyi14@fudan.edu.cn
Subject: Filtering the flaring background
Compose reply
Download message
Move To:
5 replies: 1 2 3 4 5
4 followups: 1 2 3 4

Private message: yes  no

Notes:

Notification:


Date: Mon, 13 Mar 2017 02:02:25 GMT
From: wangjingyi14@fudan.edu.cn
To: xmmhelp@sciops.esa.int
CC: wangjingyi14@fudan.edu.cn
Subject: Filtering the flaring background
Full_Name: Jingyi Wang
Submission from: (NULL) (58.246.118.133)


The observation ID is 0760646601 and I'm reducing the EPIC-pn data. In the
quality report shown in the archive, it says Percentage of flaring time:
P0760646601PNS003FBKTSR0000.FIT: 16.0%, BKGDSCRN = T

So I began filtering the flaring background. After "epproc", I used the command
below to generate the light curve:
evselect table=2889_0760646601_EPN_S003_ImagingEvts.ds withrateset=Y
rateset=PN_rate.fits maketimecolumn=Y timebinsize=100 makeratecolumn=Y
expression=' #XMMEA_EP && (PI>10000&&PI<12000) &&
(PATTERN==0)'

But then when I look at the PN_rate.fits, I found almost all the rate is below
0.4 (this value is the recommended one to generate the gti file, right?). I
don't understand this because 16% is obviously a very large number and this
doesn't match the light-curve I saw. So do I still need to filter the
background? 

I'd appreciate it if you can help. 


Reply 1

Resend
From: Jan-Uwe Ness <xmmhelp@sciops.esa.int>
To: wangjingyi14@fudan.edu.cn
Subject: Re: Filtering the flaring background (PR#81138)
Date: Mon Mar 13 09:24:35 2017
Dear Jingyi Wang,

 the decision which count rate threshold to chose depends on what you want to
do. The 16% in the quality report results from a threshold that aims for maximum
S/N to detect point sources in the image. If you have a very bright source, you
may want chose a lower threshold. For timing analysis, you may even want to
apply no filtering at all.


If you open the file quoted before the value of 16% in the quality report,
  P0760646601PNS003FBKTSR0000.FTZ
you will find a light curve in the first extension that goes much above 0.4
counts per second. This file is a pipeline product which you find in the PPS
products. The quality report refers to observations made during pipeline
production.

The reason this light curve differs from the one you have derived is that the
pipeline is using a more optimized approach to apply flare filtering than simply
looking at the full-field high-energy light curve. The rationale and algorithm
for the method applied in the pipeline are described in Section 3.2.3 in Rosen
et al. (2016):
http://cdsads.u-strasbg.fr/abs/2016A%26A...590A...1R

The commands used for creating the light curve (in Rosen et al. referred to as
'in-band' light curve) can be found in the log file
   P0760646601OBX000SCRLOG0000.ASC
where you will find an evselect and an epiclccorr command (search for FBKTSR).
The optimization algorithm derives a threshold count rate of 0.637. This value
can be found in the header of the RATE extension (header keyword FLCUTTHR).

If you want to generate a gti file applying this threshold, you can use the
in-band light curve from the PPS products, thus: 
tabgtigen table=P0760646601PNS003FBKTSR0000.FTZ expression='RATE<=0.637'
gtiset=EPICgti.fits

Best wishes,
Jan-Uwe Ness



> The observation ID is 0760646601 and I'm reducing the EPIC-pn data. In the
> quality report shown in the archive, it says Percentage of flaring time:
> P0760646601PNS003FBKTSR0000.FIT: 16.0%, BKGDSCRN = T
> 
> So I began filtering the flaring background. After "epproc", I used the
command
> below to generate the light curve:
> evselect table=2889_0760646601_EPN_S003_ImagingEvts.ds withrateset=Y
> rateset=PN_rate.fits maketimecolumn=Y timebinsize=100 makeratecolumn=Y
> expression=' #XMMEA_EP && (PI>10000&&PI<12000)
&& (PATTERN==0)'
> 
> But then when I look at the PN_rate.fits, I found almost all the rate is
below
> 0.4 (this value is the recommended one to generate the gti file, right?).
I
> don't understand this because 16% is obviously a very large number and
this
> doesn't match the light-curve I saw. So do I still need to filter the
> background? 
> 
> I'd appreciate it if you can help. 
> 
> 


Followup 1

Compose reply
Download message
From: Jingyi Wang <jingyi_wang@berkeley.edu>
Date: Tue, 14 Mar 2017 10:33:15 +0800
Subject: Re: Re: Filtering the flaring background (PR#81138)
To: Jan-Uwe Ness <xmmhelp@sciops.esa.int>
--94eb2c0d0e3478b921054aa7a679
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Dear Jan-Uwe,

The other email server seems to have some issue sending emails, so I change
into this one, hope you wouldn't mind. I'm really new in the reduction
process of XMM-Newton data, so I just want to check a few things with you.

1) About the gti file generated with the pipeline product
P0760646601PNS003FBKTSR0000.FTZ, can I then use it on the ImagingEvts file
from epproc like following and get the clean event list?
evselect table=3D2889_0760646601_EPN_S003_ImagingEvts.ds withfilteredset=3D=
Y
filteredset=3DPN_clean.fits destruct=3DY keepfilteroutput=3DT expression=3D=
'#XMMEA_EP
&& gti(PN_gti_pps_threshold.fits,TIME) && (PI>150)'

Or, I should use it on some other file from the pps directory?

2) I couldn't find the XXXXPNS003FBKTSR0000.FTZ file for PN instrument in
observations like 0760646201 (Obs.ID) which was in the timing mode, but
there are same kind of files for MOS1/2. So, does this mean I can only
adopt the method for imaging data?

For PN data in timing mode, how can I choose threshold then? From threads
provided by XMM-Newton website, it seems it has to be a little higher than
the low & steady region. But it's quite vague....Would this choice affect
the spectrum a lot, or it mostly a matter of S/N?


Best regards,
Jingyi

> -----=E5=8E=9F=E5=A7=8B=E9=82=AE=E4=BB=B6-----
> =E5=8F=91=E4=BB=B6=E4=BA=BA: "Jan-Uwe Ness" <xmmhelp@sciops.esa.int>
> =E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4: 2017-03-13 17:24:34 (=E6=98=9F=E6=
=9C=9F=E4=B8=80)
> =E6=94=B6=E4=BB=B6=E4=BA=BA: wangjingyi14@fudan.edu.cn
> =E6=8A=84=E9=80=81:
> =E4=B8=BB=E9=A2=98: Re: Filtering the flaring background (PR#81138)
>
> Dear Jingyi Wang,
>
>  the decision which count rate threshold to chose depends on what you
want to
> do. The 16% in the quality report results from a threshold that aims for
maximum
> S/N to detect point sources in the image. If you have a very bright
source, you
> may want chose a lower threshold. For timing analysis, you may even want
to
> apply no filtering at all.
>
>
> If you open the file quoted before the value of 16% in the quality report=
,
>   P0760646601PNS003FBKTSR0000.FTZ
> you will find a light curve in the first extension that goes much above
0.4
> counts per second. This file is a pipeline product which you find in the
PPS
> products. The quality report refers to observations made during pipeline
> production.
>
> The reason this light curve differs from the one you have derived is that
the
> pipeline is using a more optimized approach to apply flare filtering than
simply
> looking at the full-field high-energy light curve. The rationale and
algorithm
> for the method applied in the pipeline are described in Section 3.2.3 in
Rosen
> et al. (2016):
> http://cdsads.u-strasbg.fr/abs/2016A%26A...590A...1R
>
> The commands used for creating the light curve (in Rosen et al. referred
to as
> 'in-band' light curve) can be found in the log file
>    P0760646601OBX000SCRLOG0000.ASC
> where you will find an evselect and an epiclccorr command (search for
FBKTSR).
> The optimization algorithm derives a threshold count rate of 0.637. This
value
> can be found in the header of the RATE extension (header keyword
FLCUTTHR).
>
> If you want to generate a gti file applying this threshold, you can use
the
> in-band light curve from the PPS products, thus:
> tabgtigen table=3DP0760646601PNS003FBKTSR0000.FTZ
expression=3D'RATE<=3D0=
.637'
> gtiset=3DEPICgti.fits
>
> Best wishes,
> Jan-Uwe Ness
>
>
>
> > The observation ID is 0760646601 and I'm reducing the EPIC-pn data. In
the
> > quality report shown in the archive, it says Percentage of flaring
time=
:
> > P0760646601PNS003FBKTSR0000.FIT: 16.0%, BKGDSCRN =3D T
> >
> > So I began filtering the flaring background. After "epproc", I used
the
> command
> > below to generate the light curve:
> > evselect table=3D2889_0760646601_EPN_S003_ImagingEvts.ds
withrateset=3D=
Y
> > rateset=3DPN_rate.fits maketimecolumn=3DY timebinsize=3D100
makeratecol=
umn=3DY
> > expression=3D' #XMMEA_EP && (PI>10000&&PI<12000)
&& (PATTERN=3D=3D0)'
> >
> > But then when I look at the PN_rate.fits, I found almost all the rate
i=
s
> below
> > 0.4 (this value is the recommended one to generate the gti file,
right?). I
> > don't understand this because 16% is obviously a very large number and
this
> > doesn't match the light-curve I saw. So do I still need to filter the
> > background?
> >
> > I'd appreciate it if you can help.
> >
> >
>
> This message and any attachments are intended for the use of the
addressee or addressees only.
> The unauthorised discl

Message of length 17879 truncated


Reply 2

Resend
From: Jan-Uwe Ness <xmmhelp@sciops.esa.int>
To: jingyi_wang@berkeley.edu
Subject: Re: Filtering the flaring background (PR#81138)
Date: Tue Mar 14 08:17:09 2017
Dear Jingyi,

 no problem using the other email address.

1) The gti file is simply a list of start/end times of intervals that shall be
used for extraction. You can use on the events file produced with epproc or the
events file from the pipeline because the time stamps are in the same units.

2) The pipeline currently only runs on imaging data, but even if, the objective
of source detection doesn't apply to timing mode data, so another threshold is
needed. Since the timing mode is usually chosen for very bright sources, you
probably do not need very strict flare filtering. I thus suggest to go the
simple way and filter out really strong flares. The filtering is only about S/N,
if you do not do any flare filtering, you would have a higher background, but if
the source is bright this matters less. There is not really a strict recipe,
that is why the formulations are a bit vague in the documentation, you will need
to play around a bit to see yourself which threshold allows you to derive most
robust conclusions.

 Best wishes,
Jan-Uwe

> 
> Dear Jan-Uwe,
> 
> The other email server seems to have some issue sending emails, so I
change
> into this one, hope you wouldn't mind. I'm really new in the reduction
> process of XMM-Newton data, so I just want to check a few things with you.
> 
> 1) About the gti file generated with the pipeline product
> P0760646601PNS003FBKTSR0000.FTZ, can I then use it on the ImagingEvts file
> from epproc like following and get the clean event list?
> evselect table=3D2889_0760646601_EPN_S003_ImagingEvts.ds
withfilteredset=3D=
> Y
> filteredset=3DPN_clean.fits destruct=3DY keepfilteroutput=3DT
expression=3D=
> '#XMMEA_EP
> && gti(PN_gti_pps_threshold.fits,TIME) && (PI>150)'
> 
> Or, I should use it on some other file from the pps directory?
> 
> 2) I couldn't find the XXXXPNS003FBKTSR0000.FTZ file for PN instrument in
> observations like 0760646201 (Obs.ID) which was in the timing mode, but
> there are same kind of files for MOS1/2. So, does this mean I can only
> adopt the method for imaging data?
> 
> For PN data in timing mode, how can I choose threshold then? From threads
> provided by XMM-Newton website, it seems it has to be a little higher than
> the low & steady region. But it's quite vague....Would this choice
affect
> the spectrum a lot, or it mostly a matter of S/N?
> 
> 
> Best regards,
> Jingyi
> 


Followup 2

Compose reply
Download message
From: Jingyi Wang <jingyi_wang@berkeley.edu>
Date: Thu, 16 Mar 2017 10:17:25 +0800
Subject: Re: Filtering the flaring background (PR#81138)
To: Jan-Uwe Ness <xmmhelp@sciops.esa.int>
--001a114b6cdc7db5d8054acfa957
Content-Type: text/plain; charset=UTF-8

Dear Jan-Uwe,

Thank you so much for your answers, I've gained a lot.

1) Another question, which may be stupid, is that should we always use the
same gti file to generate the source and background spectrum? To conclude,
the better way to treat MOS1/2 timing mode data I can think of now is to
obtain the gti file using the imaging data's pipeline product, and apply
this to both TimingEvts and ImagingEvts generated by emproc and get the
source and background spectrum respectively after that. Is this the right
way?

2) Is the pipeline products in the pps directory better than the
epproc/emproc products to use?

Best regards,
Jingyi


On Tue, Mar 14, 2017 at 4:17 PM, Jan-Uwe Ness <xmmhelp@sciops.esa.int>
wrote:

> Dear Jingyi,
>
>  no problem using the other email address.
>
> 1) The gti file is simply a list of start/end times of intervals that
> shall be
> used for extraction. You can use on the events file produced with epproc
> or the
> events file from the pipeline because the time stamps are in the same
> units.
>
> 2) The pipeline currently only runs on imaging data, but even if, the
> objective
> of source detection doesn't apply to timing mode data, so another
> threshold is
> needed. Since the timing mode is usually chosen for very bright sources,
> you
> probably do not need very strict flare filtering. I thus suggest to go the
> simple way and filter out really strong flares. The filtering is only
> about S/N,
> if you do not do any flare filtering, you would have a higher background,
> but if
> the source is bright this matters less. There is not really a strict
> recipe,
> that is why the formulations are a bit vague in the documentation, you
> will need
> to play around a bit to see yourself which threshold allows you to derive
> most
> robust conclusions.
>
>  Best wishes,
> Jan-Uwe
>
> >
> > Dear Jan-Uwe,
> >
> > The other email server seems to have some issue sending emails, so I
> change
> > into this one, hope you wouldn't mind. I'm really new in the reduction
> > process of XMM-Newton data, so I just want to check a few things with
> you.
> >
> > 1) About the gti file generated with the pipeline product
> > P0760646601PNS003FBKTSR0000.FTZ, can I then use it on the ImagingEvts
> file
> > from epproc like following and get the clean event list?
> > evselect table=3D2889_0760646601_EPN_S003_ImagingEvts.ds
> withfilteredset=3D=
> > Y
> > filteredset=3DPN_clean.fits destruct=3DY keepfilteroutput=3DT
> expression=3D=
> > '#XMMEA_EP
> > && gti(PN_gti_pps_threshold.fits,TIME) && (PI>150)'
> >
> > Or, I should use it on some other file from the pps directory?
> >
> > 2) I couldn't find the XXXXPNS003FBKTSR0000.FTZ file for PN instrument
in
> > observations like 0760646201 (Obs.ID) which was in the timing mode,
but
> > there are same kind of files for MOS1/2. So, does this mean I can only
> > adopt the method for imaging data?
> >
> > For PN data in timing mode, how can I choose threshold then? From
threads
> > provided by XMM-Newton website, it seems it has to be a little higher
> than
> > the low & steady region. But it's quite vague....Would this choice
affect
> > the spectrum a lot, or it mostly a matter of S/N?
> >
> >
> > Best regards,
> > Jingyi
> >
>
> This message and any attachments are intended for the use of the addressee
> or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in
> whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete
> it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the
> sender.
>
> Please consider the environment before printing this email.
>
>
>


-- 
Jingyi Wang
Exchange at UCB

Department of Physics, Fudan University, Shanghai

Tel
(510)345-9320
(+86)151-2108-3826
Email
jingyi_wang@berkeley.edu
wangjingyi14@fudan.edu.cn
jingyi_wang@126.com

--001a114b6cdc7db5d8054acfa957
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Dear
Jan-Uwe,<div><br></div><div>Thank you so much for you=
r answers, I&#39;ve gained a
lot.=C2=A0</div><div><br></div><div>1) Another=
 question, which may be stupid, is that should we always use the same gti f=
ile to generate the source and background spectrum? To conclude, the better=
 way to treat MOS1/2 timing mode data I can think of now is to obtain the g=
ti file using the imaging data&#39;s pipeline product, and apply this to b

Message of length 12264 truncated


Reply 3

Resend
From: Jan-Uwe Ness <xmmhelp@sciops.esa.int>
To: jingyi_wang@berkeley.edu
Subject: Re: Filtering the flaring background (PR#81138)
Date: Thu Mar 16 08:53:06 2017
Dear Jinyi,

 glad to hear you've gained a lot!

1)
  a) For source and background, you have to apply the same gti file, otherwise
you are assuming too much or too little background.
  b) If you have an imaging and a timing exposure of the same instrument, they
were not taken at the same time and you need to generate different gti files
because the flaring background will be different in the exposures.
  Although MOS1 and MOS2 exposures overlap to a high degree in time, you also
need to produce different gti files for each exposure to account for the
different behaviour of the different modes and also the detectors.

2) Your second question is essentially answered with the last paragraph in the
thread about creation of events files:
https://www.cosmos.esa.int/web/xmm-newton/sas-thread-epic-reprocessing

 Best wishes,
Jan-Uwe


> Dear Jan-Uwe,
> 
> Thank you so much for your answers, I've gained a lot.
> 
> 1) Another question, which may be stupid, is that should we always use the
> same gti file to generate the source and background spectrum? To conclude,
> the better way to treat MOS1/2 timing mode data I can think of now is to
> obtain the gti file using the imaging data's pipeline product, and apply
> this to both TimingEvts and ImagingEvts generated by emproc and get the
> source and background spectrum respectively after that. Is this the right
> way?
> 
> 2) Is the pipeline products in the pps directory better than the
> epproc/emproc products to use?
> 
> Best regards,
> Jingyi
> 


Followup 3

Compose reply
Download message
From: Jingyi Wang <jingyi_wang@berkeley.edu>
Date: Thu, 16 Mar 2017 17:06:39 +0800
Subject: Re: Filtering the flaring background (PR#81138)
To: Jan-Uwe Ness <xmmhelp@sciops.esa.int>
--94eb2c1847ac0e3757054ad561d4
Content-Type: text/plain; charset=UTF-8

Dear Jan-Uwe,

Thanks for your reply.

Actually I thought in the timing mode, MOS1 or MOS2 both have the inner CCD
recording timing data and some outer CCDs taking imaging data, and that's
where the source and background spectrum can be generated from
respectively, according to the threads provided on the SAS website (
https://www.cosmos.esa.int/web/xmm-newton/sas-thread-mos-spectrum-timing).
So I thought the timing & imaging data should be taken at the same time,
thus, can share the same gti file generated from the the light-curve from
pipeline product.

Am I right, or I misunderstood something...?


Jingyi

On Thu, Mar 16, 2017 at 4:53 PM, Jan-Uwe Ness <xmmhelp@sciops.esa.int>
wrote:

> Dear Jinyi,
>
>  glad to hear you've gained a lot!
>
> 1)
>   a) For source and background, you have to apply the same gti file,
> otherwise
> you are assuming too much or too little background.
>   b) If you have an imaging and a timing exposure of the same instrument,
> they
> were not taken at the same time and you need to generate different gti
> files
> because the flaring background will be different in the exposures.
>   Although MOS1 and MOS2 exposures overlap to a high degree in time, you
> also
> need to produce different gti files for each exposure to account for the
> different behaviour of the different modes and also the detectors.
>
> 2) Your second question is essentially answered with the last paragraph in
> the
> thread about creation of events files:
> https://www.cosmos.esa.int/web/xmm-newton/sas-thread-epic-reprocessing
>
>  Best wishes,
> Jan-Uwe
>
>
> > Dear Jan-Uwe,
> >
> > Thank you so much for your answers, I've gained a lot.
> >
> > 1) Another question, which may be stupid, is that should we always use
> the
> > same gti file to generate the source and background spectrum? To
> conclude,
> > the better way to treat MOS1/2 timing mode data I can think of now is
to
> > obtain the gti file using the imaging data's pipeline product, and
apply
> > this to both TimingEvts and ImagingEvts generated by emproc and get
the
> > source and background spectrum respectively after that. Is this the
right
> > way?
> >
> > 2) Is the pipeline products in the pps directory better than the
> > epproc/emproc products to use?
> >
> > Best regards,
> > Jingyi
> >
>
> This message and any attachments are intended for the use of the addressee
> or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in
> whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete
> it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the
> sender.
>
> Please consider the environment before printing this email.
>
>
>


-- 
Jingyi Wang
Exchange at UCB

Department of Physics, Fudan University, Shanghai

Tel
(510)345-9320
(+86)151-2108-3826
Email
jingyi_wang@berkeley.edu
wangjingyi14@fudan.edu.cn
jingyi_wang@126.com

--94eb2c1847ac0e3757054ad561d4
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Dear
Jan-Uwe,<div><br></div><div>Thanks for your
reply.</d=
iv><div><br></div><div>Actually I thought in the
timing mode, MOS1 or MOS2 =
both have the inner CCD recording timing data and some outer CCDs taking im=
aging data, and that&#39;s where the source and background spectrum can be =
generated from respectively, according to the threads provided on the SAS w=
ebsite (<a href=3D"https://www.cosmos.esa.int/web/xmm-newton/sas-thread-mos=
-spectrum-timing">https://www.cosmos.esa.int/web/xmm-newton/sas-thread-mos-=
spectrum-timing</a>). So I thought the timing &amp; imaging data
should be =
taken at the same time, thus, can share the same gti file generated from th=
e the light-curve from pipeline
product.=C2=A0</div><div><br></div><div>Am =
I right, or I misunderstood
something...?</div><div><br></div><div><br></di=
v><div>Jingyi</div></div><div
class=3D"gmail_extra"><br><div class=3D"gmail=
_quote">On Thu, Mar 16, 2017 at 4:53 PM, Jan-Uwe Ness <span
dir=3D"ltr">&lt=
;<a href=3D"mailto:xmmhelp@sciops.esa.int"
target=3D"_blank">xmmhelp@sciops=
.esa.int</a>&gt;</span> wrote:<br><blockquote
class=3D"gmail_quote" style=
=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Dear Jin=
yi,<br>
<br>
=C2=A0glad to hear you&#39;ve gained a lot!<br>
<br>
1)<br>
=C2=A0 a) For source 

Message of length 10196 truncated


Reply 4

Resend
From: Jan-Uwe Ness <xmmhelp@sciops.esa.int>
To: jingyi_wang@berkeley.edu
Subject: Re: Filtering the flaring background (PR#81138)
Date: Thu Mar 16 11:19:25 2017
Dear Jingyi,

 ok now I understand, yes the image part and the timing part are of course
strictly simultaneous so the same gti file can be used.

Note, however, that the timing mode is usually used only for bright sources, and
the background may be marginal against the source even during times of flaring.
If you filter out a lot of time bins, you may end up filtering out more source
than background. In extreme cases you may even increase the signal to noise by
reducing the effective exposure time than reducing it by having less noise from
the background. So this is an individual decision, and you have to study which
way forward is more efficient at the end of the day.

 Regards,
Jan-Uwe

 
> Dear Jan-Uwe,
> 
> Thanks for your reply.
> 
> Actually I thought in the timing mode, MOS1 or MOS2 both have the inner
CCD
> recording timing data and some outer CCDs taking imaging data, and that's
> where the source and background spectrum can be generated from
> respectively, according to the threads provided on the SAS website (
> https://www.cosmos.esa.int/web/xmm-newton/sas-thread-mos-spectrum-timing).
> So I thought the timing & imaging data should be taken at the same
time,
> thus, can share the same gti file generated from the the light-curve from
> pipeline product.
> 
> Am I right, or I misunderstood something...?
> 
> 
> Jingyi
> 
> On Thu, Mar 16, 2017 at 4:53 PM, Jan-Uwe Ness
<xmmhelp@sciops.esa.int>
> wrote:
> 
>> Dear Jinyi,
>>
>>  glad to hear you've gained a lot!
>>
>> 1)
>>   a) For source and background, you have to apply the same gti file,
>> otherwise
>> you are assuming too much or too little background.
>>   b) If you have an imaging and a timing exposure of the same
instrument,
>> they
>> were not taken at the same time and you need to generate different gti
>> files
>> because the flaring background will be different in the exposures.
>>   Although MOS1 and MOS2 exposures overlap to a high degree in time,
you
>> also
>> need to produce different gti files for each exposure to account for
the
>> different behaviour of the different modes and also the detectors.
>>
>> 2) Your second question is essentially answered with the last paragraph
in
>> the
>> thread about creation of events files:
>> https://www.cosmos.esa.int/web/xmm-newton/sas-thread-epic-reprocessing
>>
>>  Best wishes,
>> Jan-Uwe
>>
>>
>> > Dear Jan-Uwe,
>> >
>> > Thank you so much for your answers, I've gained a lot.
>> >
>> > 1) Another question, which may be stupid, is that should we always
use
>> the
>> > same gti file to generate the source and background spectrum? To
>> conclude,
>> > the better way to treat MOS1/2 timing mode data I can think of now
is to
>> > obtain the gti file using the imaging data's pipeline product, and
apply
>> > this to both TimingEvts and ImagingEvts generated by emproc and
get the
>> > source and background spectrum respectively after that. Is this
the right
>> > way?
>> >
>> > 2) Is the pipeline products in the pps directory better than the
>> > epproc/emproc products to use?
>> >
>> > Best regards,
>> > Jingyi


Followup 4

Compose reply
Download message
From: Jingyi Wang <jingyi_wang@berkeley.edu>
Date: Thu, 16 Mar 2017 19:30:32 +0800
Subject: Re: Filtering the flaring background (PR#81138)
To: Jan-Uwe Ness <xmmhelp@sciops.esa.int>
--001a114a511a9b2bc7054ad763a4
Content-Type: text/plain; charset=UTF-8

Dear Jan-Uwe,

Sorry about the confusion earlier. May I ask then where and how can I see
the data's signal-to-noise ratio?

Jingyi

On Thu, Mar 16, 2017 at 7:19 PM, Jan-Uwe Ness <xmmhelp@sciops.esa.int>
wrote:

> Dear Jingyi,
>
>  ok now I understand, yes the image part and the timing part are of course
> strictly simultaneous so the same gti file can be used.
>
> Note, however, that the timing mode is usually used only for bright
> sources, and
> the background may be marginal against the source even during times of
> flaring.
> If you filter out a lot of time bins, you may end up filtering out more
> source
> than background. In extreme cases you may even increase the signal to
> noise by
> reducing the effective exposure time than reducing it by having less noise
> from
> the background. So this is an individual decision, and you have to study
> which
> way forward is more efficient at the end of the day.
>
>  Regards,
> Jan-Uwe
>
>
> > Dear Jan-Uwe,
> >
> > Thanks for your reply.
> >
> > Actually I thought in the timing mode, MOS1 or MOS2 both have the
inner
> CCD
> > recording timing data and some outer CCDs taking imaging data, and
that's
> > where the source and background spectrum can be generated from
> > respectively, according to the threads provided on the SAS website (
> > https://www.cosmos.esa.int/web/xmm-newton/sas-thread-mos-spectrum-timing
> ).
> > So I thought the timing & imaging data should be taken at the same
time,
> > thus, can share the same gti file generated from the the light-curve
from
> > pipeline product.
> >
> > Am I right, or I misunderstood something...?
> >
> >
> > Jingyi
> >
> > On Thu, Mar 16, 2017 at 4:53 PM, Jan-Uwe Ness
<xmmhelp@sciops.esa.int>
> > wrote:
> >
> >> Dear Jinyi,
> >>
> >>  glad to hear you've gained a lot!
> >>
> >> 1)
> >>   a) For source and background, you have to apply the same gti
file,
> >> otherwise
> >> you are assuming too much or too little background.
> >>   b) If you have an imaging and a timing exposure of the same
> instrument,
> >> they
> >> were not taken at the same time and you need to generate different
gti
> >> files
> >> because the flaring background will be different in the exposures.
> >>   Although MOS1 and MOS2 exposures overlap to a high degree in
time, you
> >> also
> >> need to produce different gti files for each exposure to account
for the
> >> different behaviour of the different modes and also the detectors.
> >>
> >> 2) Your second question is essentially answered with the last
paragraph
> in
> >> the
> >> thread about creation of events files:
> >> https://www.cosmos.esa.int/web/xmm-newton/sas-thread-epic-reprocessing
> >>
> >>  Best wishes,
> >> Jan-Uwe
> >>
> >>
> >> > Dear Jan-Uwe,
> >> >
> >> > Thank you so much for your answers, I've gained a lot.
> >> >
> >> > 1) Another question, which may be stupid, is that should we
always use
> >> the
> >> > same gti file to generate the source and background spectrum?
To
> >> conclude,
> >> > the better way to treat MOS1/2 timing mode data I can think
of now is
> to
> >> > obtain the gti file using the imaging data's pipeline
product, and
> apply
> >> > this to both TimingEvts and ImagingEvts generated by emproc
and get
> the
> >> > source and background spectrum respectively after that. Is
this the
> right
> >> > way?
> >> >
> >> > 2) Is the pipeline products in the pps directory better than
the
> >> > epproc/emproc products to use?
> >> >
> >> > Best regards,
> >> > Jingyi
>
> This message and any attachments are intended for the use of the addressee
> or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in
> whole or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete
> it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the
> sender.
>
> Please consider the environment before printing this email.
>
>
>


-- 
Jingyi Wang
Exchange at UCB

Department of Physics, Fudan University, Shanghai

Tel
(510)345-9320
(+86)151-2108-3826
Email
jingyi_wang@berkeley.edu
wangjingyi14@fudan.edu.cn
jingyi_wang@126.com

--001a114a511a9b2bc7054ad763a4
C

Message of length 14114 truncated


Reply 5

Resend
From: Jan-Uwe Ness <xmmhelp@sciops.esa.int>
To: jingyi_wang@berkeley.edu
Subject: Re: Filtering the flaring background (PR#81138)
Date: Thu Mar 16 12:31:08 2017
Dear Jingyi,

 signal-to-noise is the ratio of meaningful information (source) to unwanted
signal (background):
https://en.wikipedia.org/wiki/Signal-to-noise_ratio

Regards,
Jan-Uwe

 
> Dear Jan-Uwe,
> 
> Sorry about the confusion earlier. May I ask then where and how can I see
> the data's signal-to-noise ratio?
> 
> Jingyi
> 
> On Thu, Mar 16, 2017 at 7:19 PM, Jan-Uwe Ness
<xmmhelp@sciops.esa.int>
> wrote:
> 
>> Dear Jingyi,
>>
>>  ok now I understand, yes the image part and the timing part are of
course
>> strictly simultaneous so the same gti file can be used.
>>
>> Note, however, that the timing mode is usually used only for bright
>> sources, and
>> the background may be marginal against the source even during times of
>> flaring.
>> If you filter out a lot of time bins, you may end up filtering out
more
>> source
>> than background. In extreme cases you may even increase the signal to
>> noise by
>> reducing the effective exposure time than reducing it by having less
noise
>> from
>> the background. So this is an individual decision, and you have to
study
>> which
>> way forward is more efficient at the end of the day.
>>
>>  Regards,
>> Jan-Uwe

Up to top level
AO15   AO16   AO17   AO18   AO19   Backgrounds   Calibration   Conference   Data   Docs   EPICMOS   EPICpn   Feedback   Gallery   Misc   OM   Pending   PhD_Theses   Publications   RGS   RadMonitor   SAS_Hardware   SAS_WS   SASv16.0   SASv16.0_Installation   SASv16.1   SASv16.1_Installation   SASv17.0   SASv17.0_Installation   SASv18.0   SASv18.0_Installation   SciSim   Simulators_other   Suggestions   Trash   Visibility   XMM-bouncing   XMM-news   XRPS   XSA   esas   incoming  

Logged in as guest


Please make your (short) question the subject of your request!


Web interface using JitterBug ... back to the XMM home page