Swarm Robotics has been on a raise and one of the domains which is actively been research is Swarm Robotics using Unmanned Aerial Vehicles. The ability of Drones to be able to maintain a fixed inter-agent geometry is vital for this to work effectively. The most effective systems now use a Motion Capture system which is not scalable at a production level. As a result of this scalability issue, this thesis proposes a robust framework that would help in drone bearing formation by building drone detection, tracking and bearing estimator using a Convolutional Neural Network. Due to the high dynamics of drones a fish-eye camera is the visual sensor of choice.