Problem Statement
Amy Ruan
Problem Statement
Image-to-image translation is a widely applicable area of computer vision; with it, street scenes can be generated from label maps, color pictures from grayscale ones, full objects from outlines, and photographic images from paintings. In short, image-to-image translation is the transformation of one type of scene representation into another. The development of generative adversarial networks (GANs) has facilitated this process, since its structure composes of two networks- the generator and the discriminator- that are trained to compete against each other until the generator has learned to mimic the data it was trained on, which results in clear and realistic output images in image-to-image translation.
Image-to-image translation and GANs have the potential to solve many problems. Autonomous vehicles, for example, must be able to perceive their surroundings even in the dark. While methods exist that translate nighttime photographs into RGB (normal color) photographs, as well as analyze them (e.g. for depth estimation), it is an area of image-to-image translation that has not been extensively explored. This translation would be applicable in other fields as well, such as mobile navigation, augmented reality, and robotics, so that even when a thermal camera or depth sensor isn’t available, that information could still be obtained from a regular image.
The objective of this project is to modify preexisting algorithms for image-to-image translation to improve their accuracy. This will involve learning to use generative adversarial networks and image-to-image translation techniques to translate thermal images to RGB and also vice versa. In addition, if progress allows, these RGB photographs can be used to perform depth and camera pose estimation, which would map out the depths and orientations of various objects in the image. Combined together, a 3D reconstruction of the scene could eventually be created as well. In all, this project will involve learning to code these networks, run tests, and analyze the results.
Problem Statement
Image-to-image translation is a widely applicable area of computer vision; with it, street scenes can be generated from label maps, color pictures from grayscale ones, full objects from outlines, and photographic images from paintings. In short, image-to-image translation is the transformation of one type of scene representation into another. The development of generative adversarial networks (GANs) has facilitated this process, since its structure composes of two networks- the generator and the discriminator- that are trained to compete against each other until the generator has learned to mimic the data it was trained on, which results in clear and realistic output images in image-to-image translation.
Image-to-image translation and GANs have the potential to solve many problems. Autonomous vehicles, for example, must be able to perceive their surroundings even in the dark. While methods exist that translate nighttime photographs into RGB (normal color) photographs, as well as analyze them (e.g. for depth estimation), it is an area of image-to-image translation that has not been extensively explored. This translation would be applicable in other fields as well, such as mobile navigation, augmented reality, and robotics, so that even when a thermal camera or depth sensor isn’t available, that information could still be obtained from a regular image.
The objective of this project is to modify preexisting algorithms for image-to-image translation to improve their accuracy. This will involve learning to use generative adversarial networks and image-to-image translation techniques to translate thermal images to RGB and also vice versa. In addition, if progress allows, these RGB photographs can be used to perform depth and camera pose estimation, which would map out the depths and orientations of various objects in the image. Combined together, a 3D reconstruction of the scene could eventually be created as well. In all, this project will involve learning to code these networks, run tests, and analyze the results.
Comments
Post a Comment