RIT CIS Summer Internship

The paired setting of the model I tested yesterday was not good at all. I decided not to test any more and instead focused on my presentation. I fixed an image that I realized didn’t match, got some more questions answered, and reread some of the papers from earlier. I also went over each of my slides and practiced presenting.  In the afternoon I discovered that using PSNR for the unpaired settings of the models doesn’t make a lot of sense (I had suspicions earlier but got them confirmed today)- it doesn’t affect the image results, but the graphs I created for those settings are now misleading/incorrect. Oh well, something to add to the reflection. Overall today wasn’t too eventful since it was all preparation. Hopefully I’m ready for tomorrow!

I tested the model from yesterday and it looks surprisingly similar to the best one- the color of the sky is a little split between pink and blue but is blended in most of the images, and overall the objects look a little sharper. The graphs for the loss don’t make that much sense though, and the PSNR for the RGB images fluctuates a lot. This morning we ran through our presentations in the auditorium. We started at 9 am and didn’t finish until 11:50, but we had a lot of time in between each one for feedback. Mine was second and went 16 minutes (better time than I thought) and I got a lot of good advice too. Sometimes I forget that other people haven’t seen my presentation/topic before so I have to remember to explain/define everything. At lunch Jocelyn, Hannah, and I went out to eat and then to Abbott's. We also saw a beaver by the RIT entrance!  In the afternoon I worked on my presentation some more- I cut out some of the graphs as Joe suggested and also made the diagrams clear

I tested some more models today- no significant improvements. But I got to meet with both of my advisors in the morning for at least an hour to go over my presentation so I got a lot of good feedback and also some explanations for why the results haven’t been that great. It turns out that the U-Net structure is ok for RGB-to-thermal translation, which is reflected in the graphs and output images, but not so great for thermal to RGB because that translation ends up with increased complexity/texture. Translating from thermal to RGB accurately is difficult on its own, but with a model and/or loss function not suited to that translation specifically, it could mislead the training. That could also explain why the results using the paired dataset are worse- it’s not making the right connections between the two modalities and when it only has the exact image to compare to rather than something more varied it actually makes it worse. I also got suggestions to improve the image quality like add

We started having morning meetings again. I started the day by testing the three models waiting for me. Two were pretty bad, one was ok. I’m running two more right now, but I’m not sure what I should be doing to make them improve. Right now I'm changing how often the networks are optimized to hopefully balance them more. I spent most of the day working on my presentation by organizing the graphs and images I want to show. They’re not great, but if they’re the best I can do then so be it. In the afternoon I had another meeting where Joe went over David's presentation and then mine to give feedback and see if there was any overlap in background information. I hope I can meet with my advisors soon to go over my presentation as well.

The three models were done training by this morning, and I tested all of them- they were ok, but the paired dataset ones still have splotches on them. Some of the generator losses were actually increasing after a certain number of epochs (around 11-12) so I decided to train another one with the paired dataset for 12 epochs. I’m also retraining the best unpaired one I have so far and also the paired version for the original model because I still don’t have the RGB to thermal images. However I might end up having to retrain them because I might have messed up the saved files again. There are so many of them! I also think that some of the issues might be due to overfitting so I added more dropouts to the model and starting running that too. I can train four right now because I managed to get one of them onto Grissom. Eventually I'm going to change more of the hyperparameters, and I'm also thinking of optimizing the generator more often. I can always run more at home and get the

I had some minor computer issues today so I didn’t get to test that much. Yesterday I put the desktop into suspension (which I now know I’m not supposed to do) and couldn’t get it back on, so I tried using a screen at home and detaching it but then I wasn’t able to reconnect at RIT today (probably because of the Matplotlib graphs). In any case I had to restart the training this morning. Early in the afternoon, I put the computer into lock because that was supposed to be the proper way to do it but it didn’t work and in bringing the computer back to life my training was lost again. Basically it still hasn't finished, but I'll get results tomorrow! Hopefully they look better… testing right now it doesn't look too promising- the ones using the paired dataset are still looking strange and the ones using the unpaired dataset pretty much only get the sky the right color... but it was only after a few epochs so it might still improve. I also met with Joe in the afternoon to get so