Day 22


When I came in this morning all three of the models were done training. After testing each of them and seeing the results, I had to abandon the simplified model since it seems to think the sky is red (among other problems).

(Ahhhhhh)       

The original model fared better. Here’s a result from using the unpaired dataset.
The colors are pretty dull and there are weird outlines between the sky and the trees/buildings but it looks better at least.

And here's a RGB to thermal translation.
I realize I should probably include the ground truth for all of these. There are thousands of testing images, but I might cut it down so it’s easier for me to match up images for the presentation.

As for using the paired dataset, the outputted images somehow look worse and the graphs from the training were also strange. Most of the images have white dots over them, and while there are warmer colors it lost the blue… I’m wondering if I need to change the model more for it to properly use the two groups of images...
I spent the rest of the day working on my presentation and testing different changes to the model/training code, ultimately ending up with two new models I want to try. They're going to be trained for 50 epochs with the unpaired dataset.
  • The first one is saving the model every 5 epochs instead of just after high PSNRs, and uses LeakyReLU in the generator instead of RELU [session 2, saved under 6]
  • The second one is saving the model every 5 epochs only (not affected by PSNR at all), and uses LeakyRELU in the generator [session 1, saved under 7]
  • I'm also retraining the original model with the paired dataset because I messed up the file holding the saved parameters before I could save the RGB to thermal images [session 4, saved under 3]
I might also change the model structure and/or hyperparameters more.

Hopefully I'll see some improvements in the outputted images when I test tomorrow.



Comments

Post a Comment

Popular posts from this blog

Day 30 (Last Day!!)

It's hard to believe it's over. This morning we had our presentations in the auditorium, and everyone did really well. I remembered pretty much everything I wanted to say, and I think I explained my topic thoroughly. The presentations ended at almost exactly 11, and after closing remarks, photographs, and conversations, I stayed behind at RIT for a while and sent my files with the final code to my advisors. At noon my lab (which also includes two other graduate students) went out to lunch to celebrate, which was really nice because we ended up talking a lot (and not about debugging!) and I got to know them better. It's a little bittersweet because now my involvement is over, but it was a really rewarding experience and I learned how to do something new (many things actually). Overall, although it was stressful at times, I'm glad I ended up having so much autonomy with my project because it really taught me a lot about myself and about making mistakes. I'm going
Read more

Problem Statement

Amy Ruan Problem Statement           Image-to-image translation is a widely applicable area of computer vision; with it, street scenes can be generated from label maps, color pictures from grayscale ones, full objects from outlines, and photographic images from paintings. In short, image-to-image translation is the transformation of one type of scene representation into another. The development of generative adversarial networks (GANs) has facilitated this process, since its structure composes of two networks- the generator and the discriminator- that are trained to compete against each other until the generator has learned to mimic the data it was trained on, which results in clear and realistic output images in image-to-image translation.           Image-to-image translation and GANs have the potential to solve many problems. Autonomous vehicles, for example, must be able to perceive their surroundings even in the dark. While methods exist that translate nighttime photographs into RG
Read more

Day 28

I tested the model from yesterday and it looks surprisingly similar to the best one- the color of the sky is a little split between pink and blue but is blended in most of the images, and overall the objects look a little sharper. The graphs for the loss don’t make that much sense though, and the PSNR for the RGB images fluctuates a lot. This morning we ran through our presentations in the auditorium. We started at 9 am and didn’t finish until 11:50, but we had a lot of time in between each one for feedback. Mine was second and went 16 minutes (better time than I thought) and I got a lot of good advice too. Sometimes I forget that other people haven’t seen my presentation/topic before so I have to remember to explain/define everything. At lunch Jocelyn, Hannah, and I went out to eat and then to Abbott's. We also saw a beaver by the RIT entrance!  In the afternoon I worked on my presentation some more- I cut out some of the graphs as Joe suggested and also made the diagrams clear
Read more