Query-EPI
Learn Disparity Space from Light Field via a Query Mechanism
As one representation of the scene depth information in light field(LF), disparity space is a 2.5-dimensional(2.5D) data composed of a disparity point set $(x, y, d)$, a feature that is often overlooked in deep learning era yet. Current advanced learning-based methods usually adopt an end-to-end approach for LF disparity estimation: input 2D LF images and output the target 2D disparity map. These 2D to 2D methods tend to ignore the additional 0.5D disparity information, named disparity continuity in this paper, and fails to learn and estimate disparity better. Although traditional methods with spatial division could hold the continuity, slow speed and low precision limit its application. To solve the above problem, we propose a query mechanism to learn 2.5D disparity space directly from light field images, rather than just a 2D disparity map. Firstly, we take samples directly from disparity space for learning, which complements the missing 0.5D information compared to those only use ground truth. Secondly, encoder network is designed for LF encoding, where we propose three encoders for disparity estimation task with different light field representations, to prove the generality of our query mechanism. Thirdly, we make the query process conditioned on disparity value $d$, then design a conditional decoder to perform disparity query by combining sampled disparity point $p$ with LF features extracted. At last, to extract the target disparity map from the 2.5D space learned, three query-based generation algorithms are proposed and compared in terms of efficiency and precision, including space query, point query and region query. This is for EPI encoder network(Query-EPI)
2-view direction
python
Ubuntu 18.04.6 LTS with one NVIDIA GeForce RTX 3090 Ti
