Underwater object tracking attracts a growing interest because of its importance for marine engineering and aquatic robotics. Despite numerous tracking algorithms have been proposed over the last decade, they are mainly focused on general open air datasets. In contrast, objects in underwater videos are small and densely distributed, and have more complex moving patterns in the unique water environment. To develop tracking algorithms that can deal with these challenges posed by underwater videos, we construct the first real-world underwater tracking benchmark called UW360, which contains 360 videos with over 180K frames that cover a diverse range of underwater scenes. Our UW360 dataset exhibits several challenges including (1) Missing discriminative information. (2) Dense scenarios. (3) Complex motion trajectories and appearance variations. We experimentally evaluate the state-of-the-art trackers on UW360 and present a comprehensive analysis. Furthermore, we conduct some method design explorations to shed light on the potential future directions towards this new scenario beyond open-air. [Paper] [Benchmark] [Dataset]