| 24 Mar 2020 |
 Tjark | (edited) ... start training the ... => ... start train the ... | 10:32:54 |
 shlydv | Redacted or Malformed Event | 10:50:15 |
| 25 Mar 2020 |
| shlydv | Hey Tjark sorry for the late reply. I was able to get the hdf5 output file. I'm getting familiar with the image mapper and uproot. Will share squared images soon! | 14:00:32 |
|  daria.ge joined the room. | 14:55:46 |
| 26 Mar 2020 |
 ayushvpaliwal | Dear Tjark,
I have a doubt. Till now, I got familiar with DL1DH. And reading the stuff regarding uproot package. I understood what we can do with both packages, but I want to know what step by step process we will follow to make ROOT enable. Can you please explain rough map for this ?? | 17:35:30 |
| Tjark | Great @shlydv! | 18:28:35 |
| Tjark | Since most of you got familiar with the DL1 data handler, we can dig deeper and in a more technical way into the project. A step by step would be:
1. Implement a DL1DataWriter (or modifiy the current one) for the root files of MAGIC and VERITAS. For MAGIC you "only" need to change the event source with the MAGIC event source, which is already in place (https://github.com/cta-observatory/ctapipe_io_magic). For the VERITAS file you would need to open the file with the uproot packages to retrieve the events "by hand".
2. Produce hdf5 files for MAGIC and VERITAS, which can be read directly from the DL1Reader (without any changes).
3. Test the hdf5 files with the existing models in CTLearn. Implement new models in CTLearn, which are especially design for the array layout of MAGIC (2 IACTs) and VERITAS (4 IACTs)
4. (If time allows) Compare to existing analysis chains like Random Forest or Boosted Decision Trees.
5. Update code, documentation and plots in Github
Hope that helps. Don't hesitate to ask further questions in case something is unclear! | 18:57:05 |
| shlydv | Thanks for the info! Will start working on it. Also, do we have to submit a PR before the application deadline i.e. 30 March or just the proposal? | 19:13:22 |
 ganeshtero | proposal deadline is 31st March right? | 19:14:07 |
| ayushvpaliwal | What we will do for FACT, HESS, CTA? | 19:31:59 |
| ayushvpaliwal | Our first step was to run and verify existing DL model and for preprocessing we are using DL1DH. Right? Second was to make it ROOT enable using uproot. How we will do this ? I stuck here.
And at last we will make it multi telescopes enable. Right?
Please guide me on this | 19:43:38 |
| Tjark | The proposal deadline is next Tuesday (31.03). Yes you should submit a PR (including your progress over the last weeks; for example the squared images for MAGIC) and a proposal before Tuesday. | 19:56:33 |
| Tjark | CTA is already in place! For FACT and HESS we don't have access to data. | 19:59:08 |
| Tjark | You need to implement a class similar to the DL1DataWriter class in writer.py. Instead of reading the data from a simtel file, you need to read the data from the root file. For MAGIC you only need to change the event source (from the default CTA to the MAGIC one), since a plugin is already available. | 20:07:31 |
| Tjark | I think I'm not getting your question. What do you mean with multi telescopes? | 20:08:31 |
| ayushvpaliwal | What mentioned in idea page was initially we will make it to enable data in ROOT format from specific IACT. And afterwords from given collection of IACTs | 20:12:00 |
| Tjark | Okay, I see! MAGIC is a collection of two IACTs and VERITAS is a collection of four IACTs. However, the data structure is the same for the files of each IACT in their array (collection). Shouldn’t cause any troubles scaling up for „multi telescopes“. | 20:41:46 |
| 27 Mar 2020 |
| shlydv | I read the Magic root file using uproot and used bilinear interpolation in the image_mapper. The output image doesn't seem right though | 12:48:37 |
| shlydv | 
Download index1.png | 12:48:44 |
| shlydv | But each event in 'Events.MCerPhotEvt.fPixels.fPhot' had a shape of (1183,1) while MagicCam images have a size of (1039,1), right? So I used the first 1039 pixel values for this image. | 12:51:05 |
| Tjark | Awesome progress! Yes, that's was right. The pixel values for indices >1039 are not relevant. | 16:51:16 |
| Tjark | One side note: The image look like the default oversampling and not bilinear interpolation. Are you sure you passing "mapping_method={'MAGICCam': 'bilinear_interpolation'}" in the ImageMapper? | 16:54:17 |
| shlydv | Oh yes I forgot that. Does this work? | 17:09:00 |
| shlydv | 
Download final.png | 17:09:08 |
| Tjark | That looks good! | 17:11:47 |
| 28 Mar 2020 |
| shlydv | For the MAGIC files, we have to change this part of the code to the ctapipe_io_magic's MAGICEventSource, right? And then use the hdf5 files in CTLearn using the dl1reader. | 11:54:51 |
| shlydv | 
Download InkedScreenshot from 2020-03-28 17-19-40_LI.jpg | 11:55:51 |
| Tjark | Yes, instead of using the EventSource from ctapipe, we should use the EventSource from ctapipe_io_magic, which were build following the convention and syntax from ctapipe. | 11:57:38 |
| shlydv | Great! | 11:58:11 |
| Tjark | Did you tried to pass in the eventsource class plus settings from ctapipe_io_magic into the event_source_class & event_source_settings from the DL1DataWriter? Then you should execute the if case in your image. | 12:02:13 |
