Up to top level
AO15   AO16   AO17   AO18   AO19   Backgrounds   Calibration   Conference   Data   Docs   EPICMOS   EPICpn   Feedback   Gallery   Misc   OM   Pending   PhD_Theses   Publications   RGS   RadMonitor   SAS_Hardware   SAS_WS   SASv16.0   SASv16.0_Installation   SASv16.1   SASv16.1_Installation   SASv17.0   SASv17.0_Installation   SASv18.0   SASv18.0_Installation   SciSim   Simulators_other   Suggestions   Trash   Visibility   XMM-bouncing   XMM-news   XRPS   XSA   esas   incoming  

Logged in as guest

Viewing EPICpn/9382
Full headers

From: Domitilla de Martino <demartin@na.astro.it>
Subject: Background treatment and subtraction
Compose reply
Download message
Move To:
3 replies: 1 2 3
3 followups: 1 2 3

Private message: yes  no

Notes:

Notification:


Date: Mon, 27 Oct 2003 13:25:15 +0100 (MET)
From: Domitilla de Martino <demartin@na.astro.it>
To: xmmhelp@xmm.vilspa.esa.es
Subject: Background treatment and subtraction
Dear Helpdesk

I would like to ask you about the treatment of  background
for variable sources, that is for sources which have intrinsic variations
on timescales ranging from minutes to hours and for which a
background subtracted light curve has to be extracted before performing
any timing analysis (fourier transforms etc....).

The manual provides information on how to check for high background periods
during the observation by extracting a rate curve at energies above 10keV
including filtering with specific PATTERN and FLAGS.
I did so and indeed the background has a rate which is not constant with
time. This is so both extracting the background curve in the whole FOV
as well as in a predefined region which  I chose free of source
contamination.
Then I selected some periods during the observation where the background is
lower than a predifined (eye estimated) threshold and defined with
tabgengti a further filtering selection to extract the event file withouth
contamination of high background periods. However, the background still has
a behaviour which is not constant. Therefore, my question is:

The light curve of the source (filtered and extracted between gti
defined above) has to be subtracted with this background? Is this the
correct procedure?
Indeed in the  manual there is no specific reference to background
subtraction but only to exclusion of high background periods from the data.


Thanking you in advance for your help
Sincerely yours

Domitilla de Martino

=================================================================
Domitilla de Martino                                            |
                                                                |
INAF Osservatorio Astronomico di Capodimonte Napoli             |
Via Moiariello 16 80131 Naples Italy                            |
                                                                |
TEL: 39-081-5575580                                             |
FAX: 39-081-456710                                              |
e-mail: demartino@na.astro.it or demartin@cerere.na.astro.it    |
=================================================================




Reply 1

Resend
From: Norbert Schartel <xmmhelp@xmm.vilspa.esa.es>
To: demartin@na.astro.it
Subject: Re: Background treatment and subtraction (PR#9382)
Date: Mon Oct 27 16:14:07 2003
Dear Domitilla,

the procedures described for the background subtractions 
are not suited for variability studies. For such studies 
I would recommend the following procedure:
 - continue to select for PATTERN and FLAGS
 - make a light curve of a background region
   (same CCD, same or larger area than the source), 
   but define the start time and the end time of
   the light curve (+ binning) 
   + determine the backscale 
 - make a light curve for the source region using
   the same start and end time (+ binning) as used for the
   background
   + determine the backscale
 - make the light curve of the source 
    source counts = source region counts - F * background region counts
    F = backscale source region / backscale background region
 - calculate the errors for the light curve of the
   source (Gaussian or Poisson depending on the number of
   counts)

This light curve gives the exact and correct values. 
Periods with a high background will have a larger error. 
However, this light curve can be analyzed without any constraints 
if the error of the counts is taken into account for the analysis
(fourier transforms etc....).

With best regards
Norbert


Followup 1

Compose reply
Download message
Date: Tue, 28 Oct 2003 11:55:54 +0100 (MET)
From: Domitilla de Martino <demartin@na.astro.it>
To: Norbert Schartel <xmmhelp@xmm.vilspa.esa.es>
Subject: Re: Background treatment and subtraction (PR#9382)
 Dear Norbert

Thank you very much for your suggestion.  I assume that for extraction
of spectra along a specified range of phases for a periodic variation.
(Phase-resolved spectra), I have to create  background spectra
relative to the same phase bins and compute the backscale accordingly.
Is it correct?

Thanking you again
Best wishes
Domitilla

On Mon, 27 Oct 2003, Norbert Schartel wrote:

> Dear Domitilla,
>
> the procedures described for the background subtractions
> are not suited for variability studies. For such studies
> I would recommend the following procedure:
>  - continue to select for PATTERN and FLAGS
>  - make a light curve of a background region
>    (same CCD, same or larger area than the source),
>    but define the start time and the end time of
>    the light curve (+ binning)
>    + determine the backscale
>  - make a light curve for the source region using
>    the same start and end time (+ binning) as used for the
>    background
>    + determine the backscale
>  - make the light curve of the source
>     source counts = source region counts - F * background region counts
>     F = backscale source region / backscale background region
>  - calculate the errors for the light curve of the
>    source (Gaussian or Poisson depending on the number of
>    counts)
>
> This light curve gives the exact and correct values.
> Periods with a high background will have a larger error.
> However, this light curve can be analyzed without any constraints
> if the error of the counts is taken into account for the analysis
> (fourier transforms etc....).
>
> With best regards
> Norbert
>
>

=================================================================
Domitilla de Martino                                            |
                                                                |
INAF Osservatorio Astronomico di Capodimonte Napoli             |
Via Moiariello 16 80131 Naples Italy                            |
                                                                |
TEL: 39-081-5575580                                             |
FAX: 39-081-456710                                              |
e-mail: demartino@na.astro.it or demartin@cerere.na.astro.it    |
=================================================================



Reply 2

Resend
From: Norbert Schartel <xmmhelp@xmm.vilspa.esa.es>
To: demartin@na.astro.it
Subject: Re: Background treatment and subtraction (PR#9382)
Date: Tue Oct 28 17:26:48 2003
Dear Domitilla,

yes, this is correct. Most probably you have
to select the spectra along the specific 
phases for the source spectra and for the 
background spectra.
In addition you should screen the background
and exclude periods of flares.

With best regards
Norbert


Followup 2

Compose reply
Download message
Date: Wed, 29 Oct 2003 11:37:23 +0100 (MET)
From: Domitilla de Martino <demartin@na.astro.it>
To: Norbert Schartel <xmmhelp@xmm.vilspa.esa.es>
Subject: Re: Background treatment and subtraction (PR#9382)
Dear Norbert,

> yes, this is correct. Most probably you have
> to select the spectra along the specific
> phases for the source spectra and for the
> background spectra.

 Indeed is what I was planning to do for the
 phase-resolved spectra.

> In addition you should screen the background
> and exclude periods of flares.

 What do you mean with "screening the background"?
 Does it mean that I have to follow the
 procedure suggested in the manual to select the high energy
 (E>10keV) photons and create a light curve to exclude
 the high background periods?. This would mean that
 this step has to be done before extracting any spectrum
 of the source and background. Also this should be done
 before creating a light curve for timing analysis purposes.
 This is indeed what I did before. Please let me know
  if it is correct.

 Hoping to hearing from you
 Best wishes
 Domitilla

=================================================================
Domitilla de Martino                                            |
                                                                |
INAF Osservatorio Astronomico di Capodimonte Napoli             |
Via Moiariello 16 80131 Naples Italy                            |
                                                                |
TEL: 39-081-5575580                                             |
FAX: 39-081-456710                                              |
e-mail: demartino@na.astro.it or demartin@cerere.na.astro.it    |
=================================================================



Reply 3

Resend
From: Norbert Schartel <xmmhelp@xmm.vilspa.esa.es>
To: demartin@na.astro.it
Subject: Re: Background treatment and subtraction (PR#9382)
Date: Wed Oct 29 11:43:24 2003
Dear Domitilla,

our recommendations for light curves (i.e. to be used
for fourier analysis etc. ) and for spectra are different.

1) Light-curves: the high background effects only the
error of the affected bins. In the analysis
such bins get a very low weight. Therefore, the results
are not impacted by the background flares.

2) Spectra: here the situation is different as the high background
periods effects all accumulated photons. Therefore, you should
exclude high background periods.

From my own experiences I have the strong impression that
the background rejection criteria are too conservative.
They work very well for very faint sources, but if a source
has some flux then they are too conservative. With out
knowing the details of your study I would suggest the
following:
1) extract the source counts for the periods as required by
   the source. If they are very long then split them in
   smaller periods.
2) extract the background for the same time intervals
3) calculate signal to noise for every time interval
4) define a signal to noise ration for which you consider
   counts for the final composed spectra

Point four need a little bit playing as it depends really on
the source flux in comparison to the background flux. But I
have had cases where I could use say 80% of the exposure time
in comparison to 20% if I would have followed the >10keV criteria.

Maybe we should simply call and speak,

With best regards
Norbert


Followup 3

Compose reply
Download message
Date: Wed, 29 Oct 2003 13:11:22 +0100 (MET)
From: Domitilla de Martino <demartin@na.astro.it>
To: Norbert Schartel <xmmhelp@xmm.vilspa.esa.es>
Subject: Re: Background treatment and subtraction (PR#9382)
  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.
  Send mail to mime@docserver.cac.washington.edu for more info.

--0-1369300554-1067429482=:17008
Content-Type: TEXT/PLAIN; charset=US-ASCII


Dear Norbert,

I do not have your phone number.
However I agree with your suggestion. However it is worth to
let you know that the source rate in the 2-15keV range is on average
about 1.5counts/sec and the background on average is 0.2counts/sec in
the same energy range (you might wish to look at the attachedd PS figures
of background and  source).

 So you can advice me on what to do.

 In any case I do not know how to compute the errors of the
 light curve with SAS or command line.

 Cheers
 Domitilla

=================================================================
Domitilla de Martino                                            |
                                                                |
INAF Osservatorio Astronomico di Capodimonte Napoli             |
Via Moiariello 16 80131 Naples Italy                            |
                                                                |
TEL: 39-081-5575580                                             |
FAX: 39-081-456710                                              |
e-mail: demartino@na.astro.it or demartin@cerere.na.astro.it    |
=================================================================

--0-1369300554-1067429482=:17008
Content-Type: APPLICATION/PostScript; name="pn-back.ps"
Content-Transfer-Encoding: BASE64
Content-ID: <Pine.OSF.4.44.0310291311220.17008@cerere.na.astro.it>
Content-Description: 
Content-Disposition: attachment; filename="pn-back.ps"

JSFQUy1BZG9iZS0zLjAgRVBTRi0zLjAKJSVGb3I6IGRlbWFydGluCiUlVGl0
bGU6IFBHUExPVCBQb3N0U2NyaXB0IHBsb3QKJSVDcmVhdG9yOiBQR1BMT1QK
JSVDcmVhdGlvbkRhdGU6IDI5LU9jdC0yMDAzIDEzOjA2CiUlQm91bmRpbmdC
b3g6IChhdGVuZCkKJSVEb2N1bWVudEZvbnRzOiAoYXRlbmQpCiUlTGFuZ3Vh
Z2VMZXZlbDogMQolJU9yaWVudGF0aW9uOiBMYW5kc2NhcGUKJSVQYWdlczog
KGF0ZW5kKQolJUVuZENvbW1lbnRzCiUlQmVnaW5Qcm9sb2cKL0wge21vdmV0
byBybGluZXRvIGN1cnJlbnRwb2ludCBzdHJva2UgbW92ZXRvfSBiaW5kIGRl
ZgovQyB7cmxpbmV0byBjdXJyZW50cG9pbnQgc3Ryb2tlIG1vdmV0b30gYmlu
ZCBkZWYKL0Qge21vdmV0byAwIDAgcmxpbmV0byBjdXJyZW50cG9pbnQgc3Ry
b2tlIG1vdmV0b30gYmluZCBkZWYKL1NMVyB7NSBtdWwgc2V0bGluZXdpZHRo
fSBiaW5kIGRlZgovU0NGIC9wb3AgbG9hZCBkZWYKL0JQIHtuZXdwYXRoIG1v
dmV0b30gYmluZCBkZWYKL0xQIC9ybGluZXRvIGxvYWQgZGVmCi9FUCB7cmxp
bmV0byBjbG9zZXBhdGggZW9maWxsfSBiaW5kIGRlZgovTUIge2dzYXZlIHRy
YW5zbGF0ZSBNRkFDIGR1cCBzY2FsZSAxIHNldGxpbmV3aWR0aCAyIHNldGxp
bmVjYXAgMCBzZXRsaW5lam9pbiBuZXdwYXRofSBiaW5kIGRlZgovTUUgL2dy
ZXN0b3JlIGxvYWQgZGVmCi9DQyB7MCAzNjAgYXJjIHN0cm9rZX0gYmluZCBk
ZWYKL0ZDIHswIDM2MCBhcmMgZmlsbH0gYmluZCBkZWYKJSVFbmRQcm9sb2cK
IAolJVBhZ2U6IDEgMQolJUJlZ2luUGFnZVNldHVwCi9QR1BMT1Qgc2F2ZSBk
ZWYKMC4wNzIgMC4wNzIgc2NhbGUKODE0OSAyNTAgdHJhbnNsYXRlIDkwIHJv
dGF0ZQoxIHNldGxpbmVqb2luIDEgc2V0bGluZWNhcCAxIFNMVyAxIFNDRgol
JUVuZFBhZ2VTZXR1cAolJVBhZ2VCb3VuZGluZ0JveDogKGF0ZW5kKQowLjAw
MCBzZXRncmF5IDEgU0xXIDgzOTkgMCAxMDQ5IDc4MCBMIDAgNjIzOSBDIC04
Mzk5IDAgQyAwIC02MjM5IEMgMCA5MCAxMzg2IDc4MCBMIDAgNDUgMTc5OCA3
ODAgTCAwIDQ1IDIyMTEgNzgwIEwgMCA0NSAyNjIzIDc4MCBMCjAgNDUgMzAz
NiA3ODAgTCAwIDkwIDM0NDggNzgwIEwgMCA0NSAzODYxIDc4MCBMIDAgNDUg
NDI3MyA3ODAgTCAwIDQ1IDQ2ODYgNzgwIEwgMCA0NSA1MDk4IDc4MCBMIDAg
OTAgNTUxMSA3ODAgTCAwIDQ1IDU5MjQgNzgwIEwKMCA0NSA2MzM2IDc4MCBM
IDAgNDUgNjc0OSA3ODAgTCAwIDQ1IDcxNjEgNzgwIEwgMCA5MCA3NTc0IDc4
MCBMIDAgNDUgNzk4NiA3ODAgTCAwIDQ1IDgzOTkgNzgwIEwgMCA0NSA4ODEx
IDc4MCBMIDAgNDUgOTIyNCA3ODAgTAowIDkwIDEzODYgNjkyOSBMIDAgNDUg
MTc5OCA2OTc0IEwgMCA0NSAyMjExIDY5NzQgTCAwIDQ1IDI2MjMgNjk3NCBM
IDAgNDUgMzAzNiA2OTc0IEwgMCA5MCAzNDQ4IDY5MjkgTCAwIDQ1IDM4NjEg
Njk3NCBMCjAgNDUgNDI3MyA2OTc0IEwgMCA0NSA0Njg2IDY5NzQgTCAwIDQ1
IDUwOTggNjk3NCBMIDAgOTAgNTUxMSA2OTI5IEwgMCA0NSA1OTI0IDY5NzQg
TCAwIDQ1IDYzMzYgNjk3NCBMIDAgNDUgNjc0OSA2OTc0IEwKMCA0NSA3MTYx
IDY5NzQgTCAwIDkwIDc1NzQgNjkyOSBMIDAgNDUgNzk4NiA2OTc0IEwgMCA0
NSA4Mzk5IDY5NzQgTCAwIDQ1IDg4MTEgNjk3NCBMIDAgNDUgOTIyNCA2OTc0
IEwgMTIgNiA5MzEgNjQ4IEwgMTggMTggQwowIC0xMjYgQyAtNiAtNiAxMDQ1
IDU1OCBMIDYgLTYgQyA2IDYgQyAtNiA2IEMgLTYgMTIgMTE3MSA2NTQgTCAt
MTggNiBDIC0xMiAwIEMgLTE4IC02IEMgLTEyIC0xOCBDIC02IC0zMCBDIDAg
LTMwIEMgNiAtMjQgQyAxMiAtMTIgQwoxOCAtNiBDIDYgMCBDIDE4IDYgQyAx
MiAxMiBDIDYgMTggQyAwIDYgQyAtNiAxOCBDIC0xMiAxMiBDIC0xOCA2IEMg
LTYgMCBDIC0xOCAtNiBDIC0xMiAtMTIgQyAtNiAtMTggQyAtNjAgLTg0IDEy
NzMgNjcyIEwgOTAgMCBDCjAgLTEyNiAxMjczIDY3MiBMIC02IC0xOCAxNDEx
IDYzMCBMIC0xMiAtMTIgQyAtMTggLTYgQyAtNiAwIEMgLTE4IDYgQyAtMTIg
MTIgQyAtNiAxOCBDIDAgNiBDIDYgMTggQyAxMiAxMiBDIDE4IDYgQyA2IDAg
QyAxOCAtNiBDCjEyIC0xMiBDIDYgLTI0IEMgMCAtMzAgQyAtNiAtMzAgQyAt
MTIgLTE4IEMgLTE4IC02IEMgLTEyIDAgQyAtMTggNiBDIC02IDEyIEMgODQg
LTg0IDE0NTkgNjQyIEwgLTg0IC04NCAxNTQzIDY0MiBMIDExIDYgMTYwMyA2
NDggTAoxOCAxOCBDIDAgLTEyNiBDIC0xOCAtNiAxNzQwIDY3MiBMIC0xMiAt
MTggQyAtNiAtMzAgQyAwIC0xOCBDIDYgLTMwIEMgMTIgLTE4IEMgMTggLTYg
QyAxMiAwIEMgMTggNiBDIDEyIDE4IEMgNiAzMCBDIDAgMTggQyAtNiAzMCBD
Ci0xMiAxOCBDIC0xOCA2IEMgLTEyIDAgQyAtMTMgLTQgMTg0MiA3MzYgTCAt
NSAtOSBDIDAgLTkgQyA1IC05IEMgO

Message of length 117703 truncated

Up to top level
AO15   AO16   AO17   AO18   AO19   Backgrounds   Calibration   Conference   Data   Docs   EPICMOS   EPICpn   Feedback   Gallery   Misc   OM   Pending   PhD_Theses   Publications   RGS   RadMonitor   SAS_Hardware   SAS_WS   SASv16.0   SASv16.0_Installation   SASv16.1   SASv16.1_Installation   SASv17.0   SASv17.0_Installation   SASv18.0   SASv18.0_Installation   SciSim   Simulators_other   Suggestions   Trash   Visibility   XMM-bouncing   XMM-news   XRPS   XSA   esas   incoming  

Logged in as guest


Please make your (short) question the subject of your request!


Web interface using JitterBug ... back to the XMM home page