Advancements in Synthetic Video Generation for Autonomous Driving
CONTENTS 1 Overview ...................................................................................................................... 108
2 Motivation .................................................................................................................... 109 2.1 Survey ............................................................................................................................. 109
3 Proposals and Contributions .......................................................................................... 117
4 Methodology ................................................................................................................ 118 |
4.1 |
Generative Adversarial Networks ..................................................................................... 118 |
4.2 |
Training of Generative Adversarial Networks ................................................................... 118 |
4.3 |
Objective Functions in Commonly used GANs ................................................................... 118 |
4.4 |
Generator Architecture .................................................................................................... 119 |
4.5 |
Discriminator Architecture ............................................................................................... 121 |
4.6 |
Objective Functions Used in Proposed GAN ...................................................................... 122 |
4.7 |
Training and Validation .................................................................................................... 123 |
5 Results and Conclusions ................................................................................................ 124 |
5.1 |
Sample Frame Output ...................................................................................................... 124 |
5.2 |
KPI ( Key Performance Indicators ) Evaluation .................................................................... 125 |
5.3 |
Conclusion ....................................................................................................................... 127 |
6 References .................................................................................................................... 127 7 Acknowledgements ....................................................................................................... 129
FIGURES
Figure 2-1 : Different environmental conditions clockwise from top-left daylight , rain scene , sunlight , evening . [ 15 ] ..................................................................................................................................... 110
Figure 2-2 : Depth and segmentation output from CARLA simulator . [ 15 ]................................................ 110 Figure 2-3 : Architecture for low resolution outputs . [ 5 ] ........................................................................... 111
Figure 2-4 : Architecture for high resolution outputs G1 corresponds to lower architecture network and G2 corresponds to higher resolution network . [ 5 ] ........................................................................... 111
Figure 2-5 : Architecture of label / flow embedding , image and segmentation used in world-consistent video-to-video synthesis . [ 4 ] ............................................................................................................. 112
Figure 2-6 : Architecture of generator used in world-consistent video-to-video synthesis . [ 4 ] ................ 113
Figure 2-7 : Architecture of multi SPADE residual block used in world-consistent video-to-video synthesis . [ 4 ] ..................................................................................................................................... 113
Figure 2-8 : Instance as well as label used for encoding . [ 12 ] .................................................................... 114 Figure 2-9 : SPADE normalization . [ 13 ] ....................................................................................................... 115
106 March 2024