!SBGTaiLOMWKgoOYewU:matrix.org

ctlearn

19 Members
Deep Learning for IACT Event Reconstruction4 Servers

Load older messages


SenderMessageTime
24 Mar 2020
@_slack_ctlearn_UCTDX00UB:matrix.orgTjark (edited) ... start training the ... => ... start train the ... 10:32:54
@shlydv:openastronomy.orgshlydvRedacted or Malformed Event10:50:15
25 Mar 2020
@shlydv:openastronomy.orgshlydv Hey Tjark sorry for the late reply. I was able to get the hdf5 output file. I'm getting familiar with the image mapper and uproot. Will share squared images soon! 14:00:32
@daria.ge:matrix.orgdaria.ge joined the room.14:55:46
26 Mar 2020
@ayushvpaliwal:matrix.orgayushvpaliwal Dear Tjark,
I have a doubt. Till now, I got familiar with DL1DH. And reading the stuff regarding uproot package. I understood what we can do with both packages, but I want to know what step by step process we will follow to make ROOT enable. Can you please explain rough map for this ??
17:35:30
@_slack_ctlearn_UCTDX00UB:matrix.orgTjark Great @shlydv! 18:28:35
@_slack_ctlearn_UCTDX00UB:matrix.orgTjark Since most of you got familiar with the DL1 data handler, we can dig deeper and in a more technical way into the project. A step by step would be: 1. Implement a DL1DataWriter (or modifiy the current one) for the root files of MAGIC and VERITAS. For MAGIC you "only" need to change the event source with the MAGIC event source, which is already in place (https://github.com/cta-observatory/ctapipe_io_magic). For the VERITAS file you would need to open the file with the uproot packages to retrieve the events "by hand". 2. Produce hdf5 files for MAGIC and VERITAS, which can be read directly from the DL1Reader (without any changes). 3. Test the hdf5 files with the existing models in CTLearn. Implement new models in CTLearn, which are especially design for the array layout of MAGIC (2 IACTs) and VERITAS (4 IACTs) 4. (If time allows) Compare to existing analysis chains like Random Forest or Boosted Decision Trees. 5. Update code, documentation and plots in Github Hope that helps. Don't hesitate to ask further questions in case something is unclear! 18:57:05
@shlydv:openastronomy.orgshlydvThanks for the info! Will start working on it. Also, do we have to submit a PR before the application deadline i.e. 30 March or just the proposal?19:13:22
@ganeshtero:matrix.orgganeshteroproposal deadline is 31st March right?19:14:07
@ayushvpaliwal:matrix.orgayushvpaliwalWhat we will do for FACT, HESS, CTA?19:31:59
@ayushvpaliwal:matrix.orgayushvpaliwal Our first step was to run and verify existing DL model and for preprocessing we are using DL1DH. Right? Second was to make it ROOT enable using uproot. How we will do this ? I stuck here.
And at last we will make it multi telescopes enable. Right?
Please guide me on this
19:43:38
@_slack_ctlearn_UCTDX00UB:matrix.orgTjark The proposal deadline is next Tuesday (31.03). Yes you should submit a PR (including your progress over the last weeks; for example the squared images for MAGIC) and a proposal before Tuesday. 19:56:33
@_slack_ctlearn_UCTDX00UB:matrix.orgTjark CTA is already in place! For FACT and HESS we don't have access to data. 19:59:08
@_slack_ctlearn_UCTDX00UB:matrix.orgTjark You need to implement a class similar to the DL1DataWriter class in writer.py. Instead of reading the data from a simtel file, you need to read the data from the root file. For MAGIC you only need to change the event source (from the default CTA to the MAGIC one), since a plugin is already available. 20:07:31
@_slack_ctlearn_UCTDX00UB:matrix.orgTjark I think I'm not getting your question. What do you mean with multi telescopes? 20:08:31
@ayushvpaliwal:matrix.orgayushvpaliwalWhat mentioned in idea page was initially we will make it to enable data in ROOT format from specific IACT. And afterwords from given collection of IACTs20:12:00
@_slack_ctlearn_UCTDX00UB:matrix.orgTjark Okay, I see! MAGIC is a collection of two IACTs and VERITAS is a collection of four IACTs. However, the data structure is the same for the files of each IACT in their array (collection). Shouldn’t cause any troubles scaling up for „multi telescopes“. 20:41:46
27 Mar 2020
@shlydv:openastronomy.orgshlydvI read the Magic root file using uproot and used bilinear interpolation in the image_mapper. The output image doesn't seem right though12:48:37
@shlydv:openastronomy.orgshlydvindex1.png
Download index1.png
12:48:44
@shlydv:openastronomy.orgshlydvBut each event in 'Events.MCerPhotEvt.fPixels.fPhot' had a shape of (1183,1) while MagicCam images have a size of (1039,1), right? So I used the first 1039 pixel values for this image.12:51:05
@_slack_ctlearn_UCTDX00UB:matrix.orgTjark Awesome progress! Yes, that's was right. The pixel values for indices >1039 are not relevant. 16:51:16
@_slack_ctlearn_UCTDX00UB:matrix.orgTjark One side note: The image look like the default oversampling and not bilinear interpolation. Are you sure you passing "mapping_method={'MAGICCam': 'bilinear_interpolation'}" in the ImageMapper? 16:54:17
@shlydv:openastronomy.orgshlydvOh yes I forgot that. Does this work?17:09:00
@shlydv:openastronomy.orgshlydvfinal.png
Download final.png
17:09:08
@_slack_ctlearn_UCTDX00UB:matrix.orgTjark That looks good! 17:11:47
28 Mar 2020
@shlydv:openastronomy.orgshlydvFor the MAGIC files, we have to change this part of the code to the ctapipe_io_magic's MAGICEventSource, right? And then use the hdf5 files in CTLearn using the dl1reader.11:54:51
@shlydv:openastronomy.orgshlydvInkedScreenshot from 2020-03-28 17-19-40_LI.jpg
Download InkedScreenshot from 2020-03-28 17-19-40_LI.jpg
11:55:51
@_slack_ctlearn_UCTDX00UB:matrix.orgTjark Yes, instead of using the EventSource from ctapipe, we should use the EventSource from ctapipe_io_magic, which were build following the convention and syntax from ctapipe. 11:57:38
@shlydv:openastronomy.orgshlydvGreat!11:58:11
@_slack_ctlearn_UCTDX00UB:matrix.orgTjark Did you tried to pass in the eventsource class plus settings from ctapipe_io_magic into the event_source_class & event_source_settings from the DL1DataWriter? Then you should execute the if case in your image. 12:02:13

Show newer messages


Back to Room List