Validating a Tethered Balloon System and Optical Technologies for Marine Wildlife Detection and Tracking

Abstract

The interactions between marine wildlife and marine energy devices are not well understood, leading to regulatory delays for device deployments and testing. Technologies that enable marine wildlife observations can help to fill data gaps and reduce uncertainties about animal–device interactions. A validation test conducted in Galveston Bay near La Porte, Texas, in December 2022 used a technology package consisting of a tethered balloon system and three independent sensor systems, including three-band visible, eight-band multispectral, and single-band thermal to detect three marine-mammal-shaped surrogates. The field campaign aimed to provide an initial step to evaluating the use of the TBS and the effectiveness of the sensor suite for marine wildlife observations and detection. From 2 December to 7 December 2022, 6 flights were conducted under varying altitudes and environmental conditions resulting in the collection of 5454 images. A subset of the images was classified and analyzed with two collection criteria including Beaufort wind force scale and TBS altitude to assess a range of observations of a surrogate from near-shore to offshore based on pixel count. The results of this validation test demonstrate the potential for using TBSs and imaging sensors for marine wildlife observations and offer valuable information for further development and application of this technology for marine energy and other blue economy sectors.

1. Introduction

Improving high-frequency data collection technologies and observation platforms to monitor marine wildlife behavior is a priority for emerging renewable energy industries such as marine energy (ME). Marine energy (ME), which includes wave, tidal, ocean current, free-flowing river, and ocean thermal energy conversion, is an emerging renewable energy source [1]. Current marine wildlife observational technologies, such as human observations [2,3], hydrophones [4,5,6,7], underwater cameras [8,9], and small uncrewed aerial systems (UASs) [10,11,12,13], have limitations when observing wildlife behavior. The advancement in small UAS technologies has provided increased opportunities to observe marine wildlife behavior in the ocean from an overhead perspective using multispectral sensors capable of collection in the 395–950 nm range to increase visibility at the surface and at shallow depths [14]. However, there are several limitations when using small UASs for marine wildlife observations [15]. One limitation of the small UAS is the short battery life—an average flight time of around 20 min—which may not be sufficient to capture the behavioral patterns of marine wildlife, such as large cetaceans, that may surface for only a few breaths and then dive for extended periods of time [16]. Another limitation is the weight of the sensor payload on the small UAS because regulations limit the total weight of the UAS and sensor payload to 24.95 kg or less [17,18,19]. This limits the type and number of sensors that can be used for wildlife observation on a small UAS. Additionally, the power consumption of the sensors for data collection further reduces the battery lifespan of the UAS. Sun glare on water surfaces can hinder aerial technologies, making it more challenging for remote pilots to efficiently locate surfacing wildlife during observations. Consequently, this may restrict data collection when using a battery-operated, small UAS [18,20,21]. In general, the recommendations, such as those detailed in [22], include conducting flights of sufficient duration to capture wildlife behavioral patterns adequately and operating larger sensor payload packages.

Human observation methods used for marine mammal survey sightings can typically require a research vessel that creates an underwater noise disturbance and increases the researchers’ exposure to risks associated with open ocean conditions. Human observations from the research vessel provide an opaque view, which does not reveal whale behavior in the same way as aerial technology [7,23].

To overcome these limitations, a helium-filled TBS can be used for marine wildlife detection, tracking, and observation while carrying a significantly heavier payload with multiple sensors. TBS technology has been used for more than 40 years to collect atmospheric measurements, including biogenic compounds, chemical species, turbulence, radiation, cloud microphysics, and meteorological parameters, and has demonstrated long-term observation capabilities and robustness in harsh environments [24]. Previous marine mammal research illustrates how a video camera suspended from a tethered airship was an effective way to observe manatees remotely [25].

Advancing marine environment observational platforms, allows researchers to study marine wildlife behavior, such as displacement, from a unique overhead perspective, providing insights into their behavior and ecology that might not be possible using other observation methods. Displacement occurs when an animal in its natural habitat changes its behavior and location or can no longer use its habitat because of human activities [26]. TBSs with advanced optical sensors, can provide an overhead perspective that enables the capture of behavioral changes in marine wildlife, especially around an array of ME devices. Using a TBS for observing marine wildlife behavior can provide valuable data.

The present study seeks to address existing uncertainties pertaining to the ecological ramifications of ME devices on marine wildlife, a critical factor impeding the regulatory process and leading to testing and deployment delays [1,27]. This investigation endeavors to provide preliminary validation to furnish essential baseline data regarding the presence or absence of marine wildlife surrogates that would later be utilized in marine areas zoned for ME extraction with live targets. Furthermore, a pivotal objective of this research is to unravel the applicability of the TBS and sensor systems to ascertain the optimal sensor coverage range, thereby informing the required number of TBS units and imagers necessary for comprehensive monitoring of an entire ME facility. These insights hold the potential to guide management decisions aimed to minimize or mitigate impacts on the ecosystems and wildlife, all while laying the groundwork for the integration of ME installations in the future.

2. Materials and Methods

The initial test of the TBS technology involved stationing the TBS near the shoreline, while towing three two-dimensional surrogates shaped like large and small whales, as well as a three-dimensional pinniped model, behind a boat for detection by the sensors mounted on the TBS. Here we discuss the field location, TBS, sensors, boat operations, and surrogates used during the field trial. Additionally, we discuss how images were processed for detection and the range of detection of surrogates using the various sensors at different altitudes.

2.1. TBS and Sensor Descriptions

The TBS was a 104 m³ helium-filled aerostat deployed from an electric winch using a 5 HP DC Permanent Magnet Motor powered by a reversible regenerative-driven variable-speed controller connected to a 7 kW gas-powered generator. The balloon-level and surface wind speeds were recorded throughout the flights using NRG 40C calibrated anemometers. The balloon-level wind direction was measured using two Tallysman HC872 helical antennae coupled with a Vega 28 GNSS compass board. iMet-4 RSB research radiosondes were deployed on the balloon and boat to record meteorological parameters such as pressure, relative humidity, and temperature; 3D Global Positioning System (GPS) locations; and barometric altitude. A Vaisala CL-51 ceilometer provided real-time cloud base altitude. The four imagers tested include a Sony UMC-R10C visible imager (Sony RGB) [28], MAIA-WV multispectral imager (MAIA) [29], Infrared Cameras Inc. (ICI) Mirage 640P (Mid-Wave Infrared) MWIR imager (ICI Mirage) [30], and ICI 8640 (Long-Wave Infrared) LWIR imager (ICI 8640) [31]; all imagers are shown in Figure 1 and specifications are detailed in

Table 1. Specifications of the visible, multispectral, and thermal imagers.

Sensor	Resolution	Spectral Range (nm)	Weight (g)	Input Power (W)
Sony RGB UMC-R10C	5456 × 3632 pixels	RGB	158	2.2
MAIA-WV	1280 × 960 pixels	395–950	420	~7.5
ICI Mirage 640P MWIR	640 × 512 pixels (14-bit) @ 30 Hz	3000–5000	765	7
ICI 8640 LWIR	640 × 512 pixels (14-bit) @ 30 Hz	7000–14,000	136	<1.2

.

2.2. TBS Deployment and Field Campaign Locations

The deployment of the TBS was conducted by Sandia National Laboratories. The TBS was stationed and deployed from land at Bayshore Park, located in La Porte, Texas, as shown in Figure 2. All surrogate deployments were conducted from the vessel offshore from Bayshore Park Pier, as indicated in Figure 3. The total flight time for the TBS was 14 h and 50 min, averaging 2 h and 6 min per flight. A flight was typically concluded to collect and review camera images during a change in the type of surrogate being deployed, rather than ending due to a limitation of the TBS. TBS flight approvals for this campaign required ground visibility of at least three statute miles and for the balloon to maintain 500’ of separation below the base of any cloud. The TBS operational criteria for this campaign limited flights to when wind speeds were below 11 m/s at the surface or when lightning was not within a 16 km radius. Flights occurred during daylight and collected images at 50, 150, and 250 m above ground level for durations of 1 to 4 h in clear sky, broken to overcast clouds, temperatures from 16 °C to 30 °C, relative humidities from 54% to 100%, and wind speeds from 0 to 7.4 m/s at the surface and 0 to 9.6 m/s aloft. Six flights were conducted from 2 December to 7 December 2022, as detailed in

Table 2. Details of six flights conducted in La Porte, Texas, during December 2022.

Date	Start Time (UTC)	End Time (UTC)	Flight Time (hh:mm)	Cameras	Notes
12/2/2022	18:19	19:14	0:55	MAIA, 8640	MAIA images were under- or over-exposed.
12/3/2022	15:50	18:52	3:02	MAIA, ICI Mirage	Flights to 175 m above ground level (agl).
12/3/2022	19:55	21:39	1:44	MAIA, ICI Mirage	Flights to 300 m agl.
12/4/2022	17:52	21:45	3:53	MAIA, ICI 8640, ICI Mirage	Initial surrogate tow tests.
12/6/2022	14:48	18:10	3:22	MAIA, ICI 8640, ICI Mirage	Tow tests with surrogates from side arm in higher Beaufort scale.
12/7/2022	17:20	18:45	1:25	MAIA, ICI 8640	Ascent to 250 m before precipitation.

.

2.3. Boat Operations and Deployment of Surrogates

Three different sized surrogates were tested. The small whale surrogate (Figure 4) was the first to be deployed, with a tow bridle that was 17 m in length. Once the towing method was established with the small whale surrogate, the large whale surrogate (Figure 5b) was towed to perform the direct distance observational test with the objective of evaluating the platform to surrogate detection range of each sensor. While the large whale surrogate was being towed, the smaller whale and pinniped-shaped (Figure 5a) surrogates were anchored near the shore for continual monitoring while the TBS was in flight, acquiring data. As an additional validation test to obtain visually distinguishable depictions and pixel space comparisons of the surrogates at different altitudes the three surrogates were anchored to a single point and linked in series by surface lines. To establish the line, the small whale surrogate was connected to the anchor point, followed by the pinniped surrogate, and terminating with the large whale surrogate.

2.4. Image Classification and Analysis

A subset of imagery from the four sensors was classified and analyzed according to two collection criteria: (1) the Beaufort wind force scale and (2) the TBS flying altitude, where low Beaufort may provide the farthest detection under the best conditions. These two specifications were chosen because we consider them to be salient and relevant collection parameters that could impact detection in any similar data acquisition campaign. Since the boat was not equipped with an anemometer, corrections for boat motion would have to be applied to derive wind speed and the resulting Beaufort scale value. Thus, the Beaufort scale values were determined by anemometer readings at a land-based station (Figure 1) in the deployment area, and each image was assigned a value of 0 (below 0.5 m/s), 1 (0.5–1.5 m/s), or greater than 1 (above 1.5 m/s). For this study we wanted to focus on the lower end of the scale when operating conditions are favorable and when ocean surface roughness is not extreme, in order to first test detection feasibility under calmer but still representative scenarios before possibly testing more extreme scenarios in future work. Images were paired with the temporally closest land-based anemometer reading using the respective GPS time stamps of the anemometer and imaging sensors.

Images were classified by three flying altitude levels: 50 m, 150 m, or 250 m. The altitude at the time of image acquisition was determined using the iMet-4 GPS of the TBS and image time stamp tags. The collection criteria for the Beaufort scale and flying altitude created a set of nine unique conditions across four sensors and three surrogate sizes, yielding a matrix with 108 cells. Where possible, representative images for each condition were identified and analyzed. The use of a representative conditional matrix in this study aligns with the broader aim of exploring the feasibility of the TBS-imager system for marine animal detection rather than conducting an exhaustive statistical analysis of the collected data.

Analysis consisted of a calculation of the approximate distance from the sensor to the surrogate, which was derived using GPS data from the two iMet-4s onboard the TBS and towing boat, respectively. This calculation represents the three-dimensional straight-line distance between imager and towboat. The surrogates were not equipped with geolocation units, meaning the exact distance from sensor to surrogate depends on the tow distance and its orientation relative to the imager. In addition, manual visual inspection was performed using a Geographic Information System (GIS) to detect surrogates of the three sizes. Basic image enhancement techniques (e.g., stretching) were applied to facilitate object recognition. Manual digitization was then performed on each identified surrogate to delineate it into a vector polygon. Finally, the zonal statistics GIS method derived pixel counts for each surrogate/vector.

3. Results

The capacities of all four sensors to capture visually identifiable representations of the surrogates spanning the three tested sizes and most of the collection conditions were tested. The only condition for which no matching imagery was obtained was for the 250 m level at Beaufort 0. This resulted from the scarcity of Beaufort 0 conditions at that altitude during the flying periods together with occasional GPS signal drops rendering some images unusable.

Table 3. Maximum distance for analyzed images.

Sensor	Maximum Distance (m)	Flying Altitude (m)	Beaufort Category	Large Surrogate Pixels	Medium Surrogate Pixels	Small Surrogate Pixels
ICI Mirage	724	50	>1	212	52	7
ICI 8640	629	250	1	60	22	0
Sony RGB	452	250	0	6768	1847	190
MAIA	623	50	>1	934	219	57

shows the maximum distance for the representative images analyzed in the condition matrix, and the companion images are shown in Figure 3. Note that the table is not an exhaustive accounting of all captured images, but a subset was used to populate the image matrix. The maximum distance for image analysis for each of the sensors establishes a range that can be used to determine the detection and tracking capabilities in an area. For example, the PacWave South Marine Energy Test Site, located 7 miles off the shore of Newport, Oregon, has a 3219 m² operational area [32], using maximum distance measurements, but with the highest accuracy of identification, we can estimate how many TBS and sensor packages would be required to monitor the entire operational area. The greatest distance in Table 3 corresponds to the ICI Mirage thermal imager capture at a flying altitude of 50 m under Beaufort > 1 conditions, with a distance to the vessel of 724 m. In this case, the large and medium surrogates show a clear signal, while the small surrogate is detectable but less clearly distinguished from surface wavetops (Figure 6a). An image matrix in its entirety is provided in the Supplemental Material. The large surrogate is partially submerged, and the bulk of its body is not clearly visible. The shape of the tail, however, is still discernible, making positive identification viable. The signal from the medium surrogate is strong but less distinguishable in shape, while the small surrogate is hardly detectable with a resolution of a mere seven pixels, highlighting the limited potential for detectability in such cases.

The ICI 8640 image (Figure 6b) at a distance of 629 m yielded a distinguishable large surrogate detection at 60 pixels. The medium surrogate occupies only 22 pixels and exhibits a lower shape definition but still produces a noticeable signal. The small surrogate, located between the other two, was not visually detectable—one of only two such cases in the image matrix.

The MAIA image with a distance to surrogates of 623 m showed a well-defined shape and strong signal for the large and medium surrogates, and the small surrogate was still easily identifiable (Figure 6d). The band shown in Figure 3d is near-infrared (825–950 nm) and reveals good potential for detection, even under the Beaufort > 1 conditions. While distinctly present in the image, the waves are easily distinguished from the surrogate signals. Other bands in the multispectral array also yielded positive surrogate identifications, indicating the potential to increase their use in the diversity of the signals together with the distinguishability of the surrogates the multispectral sensor has to offer.

The Sony RGB image in Figure 6c stands out because of its unmatched detail. As Table 2 reveals, the pixel counts for the Sony RGB far exceed those of the other sensors, allowing for sharp shape delineations. The difference in color between the large, medium, and small surrogates visible in the image is also notable because it points to the use of possible spectral signatures in species determination, even with only three bands.

Note that three Beaufort classes are represented in the maximum distances of Table 3. Qualitatively, the effect of Beaufort class on object visibility/identification was less consistently apparent than that of distance and flying altitude, although without a more quantitative analysis such observations remain tentative. With said qualification, we speculate that this discrepancy of influence could partially result from the fact that the surrogates were floating at the surface level; thus, the effects of any resultant increased turbidity and other possible below-surface obscuring cannot be assessed from the imagery. Nevertheless, influences were observed, as in the ICI Mirage image (Figure 6a), in which the large surrogate was partially covered by water and harder to delineate, and it was more challenging to distinguish the small surrogate from the whitecaps created by higher wind conditions.

Conditions where no successful detection was achieved involved the previously mentioned ICI 8640 Beaufort 1 small surrogate at 250 m, and the ICI Mirage Beaufort 1 small at 50 m (not shown). It is noteworthy that in all cases where any matching imagery existed for a condition, the large surrogate was detectable. Cases in which detection was possible but distinguishability was low were most notable with increased distance and flying altitude, although this was not a rule across the board. In the case of thermal imagers, low pixel counts and consequent amorphous object shapes presented fewer hindrances to marine mammal detection than non-thermal imagers, because other objects/debris are less likely to carry a heat signature than a surfacing mammal. Nevertheless, species identification functionality still needs to be refined due to the limited meaningful shape of the object, as shown in Figure 7a,b. Without such refinement, it remains unclear whether successful whale identification would stand under similar conditions.

The greatest distance of all images taken (not just those used to satisfy the condition matrix) was 1243 m with the ICI Mirage and Sony RGB sensor under the 250 m altitude Beaufort 1 condition (Figure 7). In this case, the boat was towing only the large surrogate, which is easily distinguishable in both images. The detail is seen in the Sony RGB image (Figure 7a), albeit markedly lower than in closer captures, it still renders enough clarity for possible species identification. As revealed by the pixel counts, the deployed Sony RGB sensor pixel density per unit area significantly exceeds its thermal and multispectral counterparts. The sharpness of the Sony RGB images provides a unique advantage of its potential utility in distinguishing between species because it exhibits the greatest detail. The Sony RGB includes the benefits of being the least expensive, most widely available sensor of the tested suite. Limitations of the sensors lie in their spectral range and resolution, rendering them less suitable to be used in isolation for marine mammal detection where highest accuracy of identification is desired.

4. Discussion

4.1. Image Data Plays a Crucial Role in Advancing ME Testing and Deployment

Imaging data from a TBS can be instrumental in advancing ME testing and deployment, benefiting the ME regulatory community in several ways. The high-resolution aerial images captured by the TBS can assist in site characterization and resource assessment [33], enabling developers to identify suitable locations for marine renewable energy projects and regulators to approve the use of those sites. This information can help assess the feasibility and potential environmental impacts of ME projects, allowing regulatory agencies to make informed decisions about permitting and compliance.

The imaging data from the TBS can be used for environmental monitoring, operational monitoring, and safety assessment. For example, ecological models may use data to understand population effects over time [34]. Collectively, these data may inform mitigations to reduce localized impacts and promote ecological resilience [35]. Future development of real-time or near-real-time imaging data can provide valuable insights into the performance and safety of ME devices during operation, enabling regulatory agencies to monitor and ensure compliance with operational and safety regulations. Additionally, the images can be used to assess potential environmental impacts on species when paired with other recommended ME environmental monitoring technologies and methods, such as those that measure environmental effects of underwater noise [36], electromagnetic fields [37], collision risk [38], anthropogenic light [39], and habitat changes [40], and aid in compliance with environmental regulations. The imaging dataset, in addition to its primary application, holds the potential for diverse utility encompassing public outreach, stakeholder engagement, and research and development purposes, thereby facilitating effective communication, transparency, and progressive evolution within the ME industry [41]. A notable instance is the use of aerial images of southern resident killer whales captured by a UAS operated by the National Oceanographic and Atmospheric Administration (NOAA), which has prominently contributed to regulatory management and augmented public awareness through distinct platforms, exemplified by NOAA in a web-based news article [42] and an ArcGIS StoryMaps integration [43].

Visualizations or images have been increasingly used to influence environmental policy and planning over the past 20 years [44]. To advance emerging clean-energy resources such as ME, science communication must be tailored to specific end-users [41]. The images collected from the validation test were used to create a story blog [45] published in May 2023 on the Triton Initiative (Triton) project website with 134 views as of 31 July 2023, and a YouTube video [46] published in August 2023 with 131 views after being published for nine days. These media seek to increase ME stakeholder engagement and regulatory understanding about the capability of the TBS and sensor technologies for marine wildlife monitoring. These media campaigns support the framework for effective science communication and outreach strategies and dissemination of research findings for ME projects developed by the Triton [41]. Triton is a marine energy project funded by the U.S. Department of Energy Water Power Technologies Office to advance the understanding of technologies and methodologies for monitoring the environment around marine energy devices. Additionally, these campaigns align with the energy equity framing and environmental justice practices we seek to incorporate into our research and inspire collaboration with all stakeholders involved in ME [47,48].

4.2. RGB Images for Species Detection and Pairing Them with Thermal and Multispectral Sensors for Machine-Learning-Based Image Detection

The clarity of RGB imaging is a crucial factor in informing species identification and can be applied to enhance the analysis of thermal and multispectral camera data. RGB imaging provides high-resolution color images that are easily interpretable by human observers, allowing for visual identification of species based on their physical characteristics. The fine details and nuances captured by RGB imaging, such as color patterns, markings, and shapes, enable researchers and experts to accurately identify species, especially those with distinct visual features. This makes RGB imaging a valuable tool for field applications requiring real-time species identification. RGB imaging can be analyzed with thermal and multispectral cameras. Thermal cameras capture heat signatures, providing insights into species’ thermoregulatory behaviors, such as a large whale resting or logging at the water surface. Multispectral sensors are commonly able to capture spectra in the coastal-blue, VNIR, red-edge, and short-wave infrared ranges, revealing information about environmental factors that can influence species distribution under the water and at the surface. By comparing RGB images with thermal or multispectral images, researchers can gain a more comprehensive understanding of species distribution, behavior, and ecological interactions. With the ability to image marine wildlife from extended distances, this technology provides a promising solution for monitoring these sensitive species in offshore environments. RGB imaging can also significantly contribute to artificial intelligence (AI)-based image detection and species identification. High-resolution RGB images can serve as valuable training data for machine-learning algorithms, with further capability development allowing them to recognize species based on visual features and patterns, and to even recognize individuals within a population.

Compared to thermal and multispectral cameras, RGB cameras are generally more affordable, widely available, and easy to operate, making them accessible to researchers and conservation practitioners. This makes RGB imaging a practical choice for many biodiversity monitoring and species identification applications, especially in resource-constrained settings. The cost-effectiveness and accessibility of RGB imaging make it a practical choice for collecting large amounts of training data. The availability of affordable and widely accessible RGB cameras allows for the collection of extensive image datasets from various locations, habitats, and species, which can greatly contribute to developing robust and accurate ML models for species identification.

4.3. Comparing Two Thermal Cameras’ Capabilities and Costs

The ICI Mirage and the ICI 8640 thermal imagers have distinct features and capabilities. The ICI Mirage is a MWIR (Mid-Wave Infrared) imager that captures temperature and radiative emissivity. In contrast, ICI 8640 LWIR (Long-Wave Infrared) imager captures thermal emissions of objects in the field of view.

The ICI Mirage is known for its high-quality MWIR spectrum imaging capabilities; it can capture temperature via radiative emissivity with high accuracy and thermal sensitivity. MWIR imagers are generally known to provide higher-quality images than LWIR imagers, and they effectively capture “warmer” objects in the landscape. However, the ICI Mirage may also come with a higher cost than the ICI 8640 LWIR imager, making it less suitable for budget-constrained scenarios.

Sweeping differences in detection capacity between the ICI Mirage and ICI 8640 imagers were not apparent during this field campaign, suggesting the far more inexpensive ICI 8640 imager could be sufficient. However, it remains to be seen whether greater differences exist when imaging heat sources just below the water surface, when there is a reduced temperature offset between the detection target and its surroundings, or when imaging targets at colder temperatures. The comparison in Figure 3a,b could illustrate potential advantages of the ICI Mirage over the ICI 8640. The ICI 8640 capture fails to resolve the small surrogate, even though the distance is closer and the sensor array dimensions between the two imagers are equivalent (640 × 512), while the ICI Mirage can see all three surrogates, including the partially submerged large one.

Thermal imaging using the ICI 8640 LWIR imager can be a cost-effective approach for marine wildlife detection, providing valuable insights into temperature distribution and the thermal emissions of objects in the landscape. The results of our field campaigns demonstrate the reliability and accuracy of the ICI 8640 LWIR imager for capturing thermal information in diverse environmental conditions. This imager can be a suitable option for budget-constrained scenarios that require thermal imaging capabilities for environmental monitoring applications. Further research and application of the ICI 8640 LWIR imager in different environmental settings are warranted to understand its potential for marine wildlife detection and tracking.

4.4. Human Observation Methodologies to Compare with TBS-Collected Images

Boat-based surveys involving human observations are essential for assessing the abundance and distribution of marine mammal species. However, these surveys have limitations as they primarily capture surface interactions, providing only partial behavioral information. Understanding the underwater behavior and the driving factors behind it requires techniques such as tagging or the utilization of overhead aerial technologies that offer a vertical perspective into the water. In this particular study, human observations of the surrogates were not included. However, it is highly recommended that future research incorporates traditional methods of collecting biological images and conducting human observations to facilitate comparisons with the images obtained from the TBS. The coastal-blue band (395–450 nm) of the MAIA sensor has the potential to see through the water column to some extent. Such sensors are used for coastal bathymetry. Thus, it remains the work of a future study to examine whether such a sensor could be used to see sufficiently through the water column for near-surface but submerged animal detection. Figure 8 depicts a large whale surrogate floating on the water’s surface, albeit lacking the three-dimensional body shape of a living animal. This representation serves to highlight one of the visual challenges associated with human observation methodologies from a research vessel.

4.5. Inclusion of 3-Dimensional Targets

The absence of three-dimensional characteristics in both the large and small whale targets in our study presents a noteworthy limitation, potentially impacting detection probability. It is important to highlight that our investigation was primarily oriented towards surface detection. Consequently, the decision to employ two-dimensional targets was made in line with this focus. Research literature shows that photographs taken of the dorsal side of the whales as they surfaced to breathe [49] followed a protocol in [10], where each photograph was quality graded based on camera focus, body posture (degree of rolling and arching), and body shape visibility (ability to see the rostrum, fluke notch, and the body contour), and only photographs of adequate quality [10] were kept for the analyses. Furthermore, we advocate for future experiments employing these sensors and TBSs to involve live targets and for analysis to use the methods described in [10], as these adjustments may enhance overall detection probability.

4.6. Marine Mammal Skin Temperature and Detection Rates

In this study, we did not directly investigate the skin temperature of marine mammals. Although the research team deliberated incorporating heating technologies at specific locations on the targets, it was unclear whether the temperature differential between the water in Galveston Bay and the body temperature of marine mammals frequenting areas slated for marine energy test sites was substantial enough to yield a notable improvement in detectability via thermal cameras. Furthermore, since testing the viability of successful detection using our sensor package was the primary research objective, any positive thermal detection without heating elements present substantiates such viability, given that the presence of a heat source on a target should only serve to increase contrast and therefore detectability in the thermal spectral range, all else being equal. We therefore made the informed decision not to include heated targets at this stage. However, with surface detection viability established, future research aimed towards quantifying overall detection probabilities would benefit from the inclusion of heated targets. Additionally, thermal imaging did reveal the presence of a cooler water plume in the wake of the boat while towing the surrogate target, which warrants further exploration.

4.7. Tow Patterns Recommendations

In our study, maintaining the buoyancy of the surrogates necessitated the research vessel to operate at a reduced speed. Unfavorable wind and current conditions further complicated our ability to maintain straight tow patterns departing from the shore. As the day progressed, the intensification of wind and current exacerbated these challenges, hindering our efforts to maintain consistent straight tracks. It is evident from Figure 3 that constructing more robust targets capable of withstanding higher vessel speeds would enhance our capacity to tow these targets consistently across the study area. While the tow pattern employed in this study may limit generalization, the results obtained serve as a detection viability baseline that can be built upon in future work employing more controlled towing patterns.

4.8. Limitations of Farthest Detection Distance and Other Indicators

In order to address the limitations associated with the farthest detection distance and other relevant indicators, it is strongly recommended that future studies incorporate live targets. Working with live targets introduces dynamic factors, such as the variability in target locations, which will provide a more accurate assessment of detectability at different distances. In future research involving live targets, it is advisable to implement controls and detectability measures to establish a more dependable detection rate and to effectively differentiate marine wildlife from other sources such as marine debris or flotsam. Additionally, it is crucial to consider the impact of survey altitude on detectability and explore how various altitudes of the TBS may influence the detection process when working with live targets. Future work would benefit from an experimental design incorporating these factors to produce results amenable to statistical determinations of successful detection. This holds true for statistically quantifying the effect of environmental conditions such as altitude on successful detection as well.

5. Conclusions

5.1. Future Applications of the TBS and Sensors for Detecting and Tracking Wildlife and Biodiversity Monitoring

The successful field campaign of the TBS and imaging sensors in La Porte, Texas, has opened up promising opportunities for future tests and discussions about novel applications in marine environments and biodiversity monitoring. We recommend TBSs and sensors to

• Be further tested and used in ME sectors, such as offshore wind, wave energy, and tidal energy systems;
• Be further evaluated for submerged and underwater marine wildlife detection and tracking;
• Have applications in wildlife conservation and research efforts, such as monitoring endangered marine mammal populations, studying their behavior and habitat use, and evaluating the effectiveness of conservation measures, as it can provide valuable data for informing conservation strategies and decision-making;
• Have applications in marine biodiversity monitoring including tracking migratory patterns of marine species, monitoring breeding and nesting grounds, and identifying potential changes in marine ecosystems.

5.2. Blue Economy Sector Considerations for the Use of TBSs

The Blue Economy encompasses various economic activities and sectors that are connected to the world’s oceans and coastal areas. These include aquaculture and desalination. For the preservation and conservation of marine resources, blue economy sectors are integrating environmental monitoring technologies, and adaptations of technologies such as those investigated for marine energy, similar to the TBS, can play a significant role in performing environmental monitoring, aiding ocean and coastal development, and preserving marine resources. More specifically TBS technologies may

• Be employed in aquaculture to monitor interactions between farmed species and wild marine mammals, birds, and other wildlife, as well as monitoring crop health and biomass density, and ideal harvest time. This information can inform best practices for sustainable and responsible aquaculture operations.
• Be used to monitor the effects of desalination facilities on marine wildlife, particularly marine mammals and birds. As desalination becomes an increasingly important solution for addressing freshwater scarcity in coastal regions, using TBSs and imaging sensors can aid in assessing the environmental impacts of these facilities and developing appropriate mitigation measures.
• Be employed for pollution monitoring to detect pollutants, oil spills, and other contaminants in the water.

5.3. Limitations to Consider for Future Work

In this research article, it is crucial to acknowledge several limitations before conclusively endorsing the use of TBSs and sensors for marine wildlife detection and identification. Our findings suggest that further investigation is essential to ascertain the accuracy of species identification and to evaluate the trade-offs associated with each sensor.

While this study has demonstrated the capability of sensors to detect surrogates at long distances when positioned behind a vessel, it is important to note that the results do not provide a clear structural outline of the surrogate’s body at far distances. Therefore, the accuracy of each sensor in identifying actual marine species needs to be comprehensively explored in subsequent studies, which should ideally involve live targets, such as large whales, studied at different surfacing intervals and locations within a study area. Collecting such data is imperative for making informed decisions regarding ME environmental monitoring and the potential use of TBS and sensors for species identification within an ME deployment area.

To enhance the credibility of a live target study, a thorough comparison of the sensors and an assessment of potential trade-offs are necessary. Unfortunately, due to the absence of marine wildlife in this particular study, a direct comparison of the sensors and the identification of trade-offs were not possible. Therefore, it is recommended that future research takes place in environments that closely mimic the environmental conditions of an ME site of interest. Validating the TBS and sensors in these environments will offer crucial insights into their potential use, trade-offs, and accuracy for species identification, thus assessing their suitability for advancing environmental monitoring and reducing barriers to ME testing and deployment.

5.4. Relative Cost Considerations for TBSs and Sensors in Environmental Monitoring

While this research delves into the intricacies of the TBS and its applications for environmental monitoring, it is important to note that the assessment of relative cost between a TBS and conventional technologies and methodologies was not within the scope of this research. Without undertaking this comprehensive research endeavor utilizing the TBS and sensors, there exists inadequate data to conduct a thorough cost analysis for a meaningful comparison. However, a comprehensive analysis of costs, juxtaposed with alternative environmental monitoring technologies and methodologies, particularly in a context linked to marine energy, warrants further evaluation for a more holistic understanding of the comparative merits including human resources, opportunity for autonomy, and above all safety to deploy, implement, and operate in at-sea conditions. Future investigations should aim to expound these economic aspects to guide informed decisions and promote efficient utilization of resources within the realms of environmental monitoring, observing marine wildlife, and aiding the transition to ocean-generated renewable energy.

In conclusion, imaging data from a TBS can be a valuable tool for the ME community, providing critical information for marine mammal presence and site characterization, environmental monitoring technology applications, operational monitoring when testing devices with marine mammal presence, public outreach, and research and development. By using such imaging data, regulatory agencies can make informed decisions, promote sustainable ME development, and ensure compliance with regulatory requirements.