Up to top level
AO15   AO16   AO17   AO18   Backgrounds   Calibration   Conference   Data   Docs   EPICMOS   EPICpn   Feedback   Gallery   Misc   OM   Pending   PhD_Theses   Publications   RGS   RadMonitor   SAS_Hardware   SAS_WS   SASv16.0   SASv16.0_Installation   SASv16.1   SASv16.1_Installation   SASv17.0   SASv17.0_Installation   SciSim   Simulators_other   Suggestions   Trash   Visibility   XMM-bouncing   XMM-news   XRPS   XSA   esas   incoming  

Logged in as guest

Viewing SASv17.0/83866
Full headers

From: Teng Liu <liu@mpe.mpg.de>
Subject: Re: which CCF to use when running emldetect on multiple pointings
Compose reply
Download message
Move To:
2 replies: 1 2
0 followups:

Private message: yes  no

Notes:

Notification:


From: Teng Liu <liu@mpe.mpg.de>
Subject: Re: which CCF to use when running emldetect on multiple pointings
Date: Tue, 21 Aug 2018 11:50:49 +0200
To: Nora Loiseau <xmmhelp@sciops.esa.int>
Dear Nora,

I have checked both the source code and output (using verbose=10) of emosaicproc
and I found two facts:

1. When analyzing the data of each individual obsid, emosaicproc uses the
corresponding cif correctly. But then, before running the final step of
emldetect, emosaicproc does not reset SAS_CCF. Therefore, the last cif in the
loop is used. Could you please ask the author of emosaicproc whether the last
cif in the loop is selected on purpose, or the cif needed by emldetect is not
set by emosaicproc with any special intention?

2. As shown by the output of emldetect ("emldetect:- CCF constituents accessed
by the calibration server:...), the CCF files used by emldetect are always the
same between the cif of differrent obsid. It means at least in the cases I
checked, the input cif does not affect emldetect results. Could you please ask
the author of emldetect, is it true that the emldetect results are independent
on cif?

Best regards,
Teng

> On Aug 17, 2018, at 2:17 PM, Nora Loiseau <xmmhelp@sciops.esa.int>
wrote:
> 
> Dear Teng,
> 
> I consulted the SAS developer and he said that the CCF used by emosaic prep
and
> proc should be the correct ones. If you want to check he suggests to run
the
> tasks with higher verbosity, i.e. 6 or 7, and you should be able to see in
the
> logs which CCFs were used.
> 
> Best regards,
> 
> Nora
> 
> 
>> 
>> Dear Nora,
>> 
>> Thanks for your reply. But my question is not about mosaic
observations. I
>> know that mosaic observations have the same cif. My question is about
>> non-mosaic, different-epoc observations. Take the example of the 3
>> observations:
>> 0677600133
>> 0677600134
>> 0404964701
>> The first two have the same cif. But the third observation has a
different
>> one. Which cif should be used when running emldetect? A cif is
requested by
>> emldetect, but there is no instruction about which one to used.
>> 
>> Best regards,
>> Teng
>> 
>> 
>> On Thu, Aug 16, 2018 at 4:24 PM Nora Loiseau
<xmmhelp@sciops.esa.int> wrote:
>> 
>>> Dear Teng,
>>> 
>>> For the mosaic observation the ccf to be used will be the same, as
the
>>> observations were performed together. For the mosaic the best is to
treat
>>> the
>>> different sub-observations as independent observations.
>>> 
>>> For instance for the source at RA=2h 29m 18s Dec=-5d 38' 16" you
will find
>>> 3 observations:
>>> 0677600133
>>> 0677600134
>>> 0404964701
>>> 
>>> The first 2 belong to the same mosaic and will have the same ccf.
>>> 
>>> Best regards,
>>> 
>>> Nora
>>> 
>>>> 
>>>> As you suggested, I run emosaic_prep and emosaicproc for one
mosaic
>>>> observation 0677600101. In the output of emosaicproc, the CCF
used by
>>>> emldetect are listed as:
>>>> emldetect:- CCF constituents accessed by the calibration
server:
>>>> CifEntry{EMOS1, LINCOORD, 18,
/xmm/ccf/public/EMOS1_LINCOORD_0018.CCF,
>>>> 1998-01-01T00:00:00.000}
>>>> CifEntry{EMOS2, LINCOORD, 18,
/xmm/ccf/public/EMOS2_LINCOORD_0018.CCF,
>>>> 1998-01-01T00:00:00.000}
>>>> ......
>>>> 
>>>> However, through these outputs, we can not tell which cif was
used by
>>>> emldetect. In the case of running emldetect for non-mosaic
different
>>>> observations, there are more than one different cif. It is
never
>>> mentioned
>>>> anywhere which one should be used. Does it mean that emldetect
does not
>>>> depend on cif and any one can be used?
>>>> 
>>>> 
>>>> Cheers,
>>>> Teng
>>>> 
>>>> On Mon, Aug 13, 2018 at 12:02 PM, Nora Loiseau
<xmmhelp@sciops.esa.int>
>>>> wrote:
>>>> 
>>>>> Dear Teng,
>>>>> 
>>>>> It is not clear looking at the code, but please check the
log of the
>>> tasks
>>>>> performed and you will see that the proper ccf are applied.
>>>>> 
>>>>> Best regards,
>>>>> 
>>>>> Nora
>>>>> 
>>>>> 
>>>>>> Thank you for your reply. I checked the PERL source
code of
>>> emosaicproc,
>>>>>> which you suggested to use. I am not good at the PERL
language. But
>>> seems
>>>>>> the emosaicproc is not setting any cif before running
emldetect, and
>>>>>> therefore likely the cif of the last pointing in the
loop is used by
>>>>>> emldetect. Is it true?
>&g

Message of length 73962 truncated

Reply 1

Resend
From: Nora Loiseau <xmmhelp@sciops.esa.int>
To: liu@mpe.mpg.de
Subject: Re: which CCF to use when running emldetect on multiple pointings (PR#83866)
Date: Tue Aug 21 10:05:09 2018
Dear Teng,

I am forwarding your comments to the SAS developer. I will come back to you
as soon as I have his inputs.

Best regards,

Nora


> 
> I have checked both the source code and output (using verbose=10) of
emosaicproc and I found two facts:
> 
> 1. When analyzing the data of each individual obsid, emosaicproc uses the
> corresponding cif correctly. But then, before running the final step of
> emldetect, emosaicproc does not reset SAS_CCF. Therefore, the last cif in
the
> loop is used. Could you please ask the author of emosaicproc whether the
last
> cif in the loop is selected on purpose, or the cif needed by emldetect is
not
> set by emosaicproc with any special intention?
> 
> 2. As shown by the output of emldetect ("emldetect:- CCF constituents
accessed
> by the calibration server:...), the CCF files used by emldetect are always
the
> same between the cif of differrent obsid. It means at least in the cases I
> checked, the input cif does not affect emldetect results. Could you please
ask
> the author of emldetect, is it true that the emldetect results are
independent
> on cif?
> 
> Best regards,
> Teng
> 

----
Dr. Nora Loiseau
XMM-Newton User Support Group


Reply 2

Resend
From: Nora Loiseau <xmmhelp@sciops.esa.int>
To: liu@mpe.mpg.de
Subject: Re: which CCF to use when running emldetect on multiple pointings (PR#83866)
Date: Mon Sep  3 10:56:36 2018
Dear Teng,

Sorry for the delay in coming back to you. The SAS developer confirms
that you were right that the individual ccf are not used by emldetect. 
The proper calibration files are used by emosaicproc, each ODF using their 
corresponding ccf.cif.
The task emldetect has as input parameters all the images, background 
maps, exposure maps and detection lists of all the ODF processed. It is 
performed only once, using the ccf.cif of the last ODF processed by 
emosaicproc, but the ccf parameters used for this task have not changed 
since many years, so this is not critical. It would only be recommended 
to process the ODF by chronological order so the ccf.cif used will be 
modern enough.
The SAS developer is thankful for your comments and he will modify 
the task so emldetect uses automatically the most modern ccf.cif of 
the set of ODF processed, this functionality will be ready for next SAS 
release.

There is a new task doing the same, edetect_stack, but it is not yet 
completely documented, we hope to have soon a thread and explanations 
for it. This task also uses a single common ccf.cif and it was tested for 
stacking very old and very new ODF without problems.

Best regards,

Nora

 

> 
> I have checked both the source code and output (using verbose=10) of
emosaicproc
> and I found two facts:
> 
> 1. When analyzing the data of each individual obsid, emosaicproc uses the
> corresponding cif correctly. But then, before running the final step of
> emldetect, emosaicproc does not reset SAS_CCF. Therefore, the last cif in
the
> loop is used. Could you please ask the author of emosaicproc whether the
last
> cif in the loop is selected on purpose, or the cif needed by emldetect is
not
> set by emosaicproc with any special intention?
> 
> 2. As shown by the output of emldetect ("emldetect:- CCF constituents
accessed
> by the calibration server:...), the CCF files used by emldetect are always
the
> same between the cif of differrent obsid. It means at least in the cases I
> checked, the input cif does not affect emldetect results. Could you please
ask
> the author of emldetect, is it true that the emldetect results are
independent
> on cif?
> 
> Best regards,
> Teng
> 
>> On Aug 17, 2018, at 2:17 PM, Nora Loiseau
<xmmhelp@sciops.esa.int> wrote:
>> 
>> Dear Teng,
>> 
>> I consulted the SAS developer and he said that the CCF used by emosaic
prep
> and
>> proc should be the correct ones. If you want to check he suggests to
run the
>> tasks with higher verbosity, i.e. 6 or 7, and you should be able to see
in
the
>> logs which CCFs were used.
>> 
>> Best regards,
>> 
>> Nora
>> 
>> 
>>> 
>>> Dear Nora,
>>> 
>>> Thanks for your reply. But my question is not about mosaic
observations. I
>>> know that mosaic observations have the same cif. My question is
about
>>> non-mosaic, different-epoc observations. Take the example of the 3
>>> observations:
>>> 0677600133
>>> 0677600134
>>> 0404964701
>>> The first two have the same cif. But the third observation has a
different
>>> one. Which cif should be used when running emldetect? A cif is
requested by
>>> emldetect, but there is no instruction about which one to used.
>>> 
>>> Best regards,
>>> Teng
>>> 
>>> 
>>> On Thu, Aug 16, 2018 at 4:24 PM Nora Loiseau
<xmmhelp@sciops.esa.int>
wrote:
>>> 
>>>> Dear Teng,
>>>> 
>>>> For the mosaic observation the ccf to be used will be the same,
as the
>>>> observations were performed together. For the mosaic the best
is to treat
>>>> the
>>>> different sub-observations as independent observations.
>>>> 
>>>> For instance for the source at RA=2h 29m 18s Dec=-5d 38' 16"
you will find
>>>> 3 observations:
>>>> 0677600133
>>>> 0677600134
>>>> 0404964701
>>>> 
>>>> The first 2 belong to the same mosaic and will have the same
ccf.
>>>> 
>>>> Best regards,
>>>> 
>>>> Nora
>>>> 
>>>>> 
>>>>> As you suggested, I run emosaic_prep and emosaicproc for
one mosaic
>>>>> observation 0677600101. In the output of emosaicproc, the
CCF used by
>>>>> emldetect are listed as:
>>>>> emldetect:- CCF constituents accessed by the calibration
server:
>>>>> CifEntry{EMOS1, LINCOORD, 18,
/xmm/ccf/public/EMOS1_LINCOORD_0018.CCF,
>>>>> 1998-01-01T00:00:00.000}
>>>>> CifEntry

Message of length 81993 truncated

Up to top level
AO15   AO16   AO17   AO18   Backgrounds   Calibration   Conference   Data   Docs   EPICMOS   EPICpn   Feedback   Gallery   Misc   OM   Pending   PhD_Theses   Publications   RGS   RadMonitor   SAS_Hardware   SAS_WS   SASv16.0   SASv16.0_Installation   SASv16.1   SASv16.1_Installation   SASv17.0   SASv17.0_Installation   SciSim   Simulators_other   Suggestions   Trash   Visibility   XMM-bouncing   XMM-news   XRPS   XSA   esas   incoming  

Logged in as guest


Please make your (short) question the subject of your request!


Web interface using JitterBug ... back to the XMM home page