Files and links (2)
Published (Version of record)CC BY V4.0, Open Access
Published (Version of record)CC BY V4.0, Open
Journal article
Assessing the ability of deep learning techniques to perform real-time identification of shark species in live streaming video from drones
Frontiers in Marine Science, Vol.9, pp.1-26
20/10/2022
Metrics
1 File views/ downloads
18 Record Views
UN Sustainable Development Goals (SDGs)
This output has contributed to the advancement of the following goals:
Source: InCites
Abstract
Over the last five years remotely piloted drones have become the tool of choice to spot potentially dangerous sharks in New South Wales, Australia. They have proven to be a more effective, accessible and cheaper solution compared to crewed aircraft. However, the ability to reliably detect and identify marine fauna is closely tied to pilot skill, experience and level of fatigue. Modern computer vision technology offers the possibility of improving detection reliability and even automating the surveillance process in the future. In this work we investigate the ability of commodity deep learning algorithms to detect marine objects in video footage from drones, with a focus on distinguishing between shark species. This study was enabled by the large archive of video footage gathered during the NSW Department of Primary Industries Drone Trials since 2016. We used this data to train two neural networks, based on the ResNet-50 and MobileNet V1 architectures, to detect and identify ten classes of marine object in 1080p resolution video footage. Both networks are capable of reliably detecting dangerous sharks: 80% accuracy for RetinaNet-50 and 78% for MobileNet V1 when tested on a challenging external dataset, which compares well to human observers. The object detection models correctly detect and localise most objects, produce few false-positive detections and can successfully distinguish between species of marine fauna in good conditions. We find that shallower network architectures, like MobileNet V1, tend to perform slightly worse on smaller objects, so care is needed when selecting a network to match deployment needs. We show that inherent biases in the training set have the largest effect on reliability. Some of these biases can be mitigated by pre-processing the data prior to training, however, this requires a large store of high resolution images that supports augmentation. A key finding is that models need to be carefully tuned for new locations and water conditions. Finally, we built an Android mobile application to run inference on real-time streaming video and demonstrated a working prototype during fields trials run in partnership with Surf Life Saving NSW.
Details
- Title
- Assessing the ability of deep learning techniques to perform real-time identification of shark species in live streaming video from drones
- Creators
- Cormac R. Purcell - Macquarie UniversityAndrew J. Walsh - Macquarie UniversityAndrew P Colefax - Sci-EyePaul Allan Butcher - NSW Department of Primary Industries
- Publication Details
- Frontiers in Marine Science, Vol.9, pp.1-26
- Publisher
- Frontiers Research Foundation
- Grant note
- CP and AW were supported by a Macquarie University New Staff Grant. Drone footage was collected under NSW DPI and Office of Environment and Heritage scientific permits (Ref. P01/0059; MWL000102746) and animal ethics (16/09 – SIMS). Project funding and support was provided by NSW DPI and associated NSW Shark Management Strategy, and NSW DPI and Macquarie University grant: ‘Improved drone detection using artificial intelligence and portable devices’, CONT19/673 for CRP, AW and PB.
- Identifiers
- 991013055412402368
- Copyright
- © 2022 Purcell, Walsh, Colefax and Butcher. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
- Academic Unit
- Faculty of Science and Engineering
- Language
- English
- Resource Type
- Journal article