Next Article in Journal
Unresolved CFD and DEM Coupled Simulations on Scour around a Subsea Pipeline
Next Article in Special Issue
A Summary of Environmental Monitoring Recommendations for Marine Energy Development That Considers Life Cycle Sustainability
Previous Article in Journal
Multi-Scale Influence of Flexible Submerged Aquatic Vegetation (SAV) on Estuarine Hydrodynamics
Previous Article in Special Issue
Capabilities of an Acoustic Camera to Inform Fish Collision Risk with Current Energy Converter Turbines
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Use of a 360-Degree Underwater Camera to Characterize Artificial Reef and Fish Aggregating Effects around Marine Energy Devices

1
Coastal Sciences Division, Pacific Northwest National Laboratory, Sequim, WA 98382, USA
2
Visual Communications, Pacific Northwest National Laboratory, Richland, WA 99354, USA
*
Author to whom correspondence should be addressed.
J. Mar. Sci. Eng. 2022, 10(5), 555; https://doi.org/10.3390/jmse10050555
Submission received: 7 March 2022 / Revised: 15 April 2022 / Accepted: 17 April 2022 / Published: 19 April 2022

Abstract

:
Marine energy devices must be attached to the seafloor by their foundations, pilings, or anchors, and will have other parts in the water column like the devices themselves, mooring lines, and power export cables running along the seafloor. The installation and presence of these artificial structures will create physical changes that can disrupt or create new habitats, and potentially alter the behavior of mobile organisms such as fish around a device by attracting them to these new artificial reefs and fish aggregating devices. In this study, we tested a new approach for monitoring fish activity around a marine energy device anchor: a 360-degree underwater camera to keep the target (a wave energy converter’s anchor) in the field of view of the camera. The camera was deployed in three configurations (hand-held, tripod, video lander) at sites with different hydrodynamics and underwater visibilities. The video lander was the best configuration: very stable, versatile, and easy to handle. The 360-degree field of view enabled observing and counting fishes, which were more abundant at dusk than dawn or noon, around the anchor. Despite remaining challenges, 360-degree cameras are useful tools for monitoring animal interactions with marine energy devices.

1. Introduction

Marine energy devices, such as wave energy converters (WECs) and instream tidal turbines, are artificial structures that are placed in the natural marine environment. As such, they are colonized by mobile and sessile organisms because of the hard substrate, vertical relief, and habitat complexity that they provide, and subsequently, they become artificial reefs (ARs) and fish aggregating devices (FADs) [1,2,3,4]. In addition to the devices themselves, various components of marine energy systems offer new structures on the seafloor or in the water column, such as anchors, mooring lines, unburied cables, and cable protections. Many demersal and pelagic fish species are particularly attracted to artificial structures at sea, due to a common phenomenon called a thigmotactic response, in which fish tend to “move toward structured rather than bare, featureless habitat” [5]. While true ARs and FADs exploit this behavior for conservation and fishing purposes, respectively, marine energy devices are not specifically designed to enhance fish habitat and the effect is only secondary [6]. Recent monitoring of offshore wind turbines along the east coast of the United States (U.S.) showed a myriad of fish swimming around the monopods just a few months after completion of their installation [7]. These new artificial structures may enhance the regional production of fish and invertebrates by providing shelter and food [2]. However, the extent to and pace at which marine energy devices attract fish and mobile invertebrates and potentially displace them away from nearby natural and already-established ARs remains uncertain and a concern for regulators and stakeholders.
The AR and FAD effects of marine energy devices are usually assessed visually during monitoring surveys, using technologies such as remotely operated vehicles (ROVs), drop cameras, video landers, or scuba divers equipped (or not) with underwater cameras [8]. These technologies are adaptable, and each has pros and cons for monitoring AR and FAD effects. For instance, ROVs, drop cameras, and video landers can be deployed over depth ranges spanning hundreds to thousands of meters for several hours at a time, while scuba divers are usually limited to the first 30 m of the water column and about an hour of dive time [8,9,10,11]. Divers, because of their maneuverability, are better equipped for navigating in areas where the risk of tether or line entanglement exists, such as in kelp fields or close to/underneath artificial structures [8,12,13]. Yet, diver and ROV motion, especially if bearing lights, may affect animals’ behavior, either scaring them away or, sometimes, attracting them [8,14]. In that regard, drop cameras or video landers left on the seafloor for some amount of time may have less of an effect on animals’ behavior and may be more suitable for observing mobile invertebrates and fish aggregating around marine energy devices [15].
A challenge of many drop cameras and video landers for surveying the AR and FAD effects of artificial structures such as marine energy devices is the total field of view. Often, the frame only bears one camera facing one direction (e.g., [16]), or two video cameras facing opposite directions, and all or parts of the targeted structure will be missed if not in the field of view [8,10,17]. One way to alleviate this issue would be to use 360-degree underwater video footage, which would then cover all directions. One could use three individual video cameras, mounted at 120° from each other on a frame for deployment as a drop camera or video lander (e.g., [10,18]), or use actual commercial off-the-shelf (COTS) 360-degree video cameras (e.g., [15,19]). Recording underwater footage over a 360-degree field of view for surveying AR and FAD effects assures that the targeted structure will be visible, assuming that water visibility and the distance from the target are considered. In addition, while using three individual video cameras may be cheaper than a proper 360-degree video camera because of the relative novelty of the latter, processing and analyzing the video footage from three non-stitched individual sources is much more time-consuming and challenging than from a single source.
As identified by a recent literature review [8], ROVs, scuba divers, drop cameras, and video landers are common monitoring technologies at marine energy sites, but the use of 360-degree video cameras is much more infrequent (e.g., [19]). The literature review aforementioned [8] did not identify any successful static passive technology (i.e., different from moving ROVs or scuba divers) for monitoring the AR and FAD effects of artificial structures, especially in high-energy environments such as tidal estuaries or offshore wave fields. The goal of the study reported here was to assess the use of a COTS 360-degree video camera in different deployment configurations for monitoring the AR and FAD effects at marine energy sites, particularly around the mooring anchors of a WEC. Special attention was given to the applicability of the technology to high-energy environments with deployment challenges such as strong current, mooring lines, and other entanglement risks.

2. Materials and Methods

2.1. Technology

Because high-quality custom 360-degree camera setups could present a high cost, we focused here on COTS cameras. To select the right camera, we compared the following technical considerations of six COTS 360-degree cameras that could fit the purpose of this study (Figure 1): depth rating compatible with all known marine energy deployments to date, runtime of several hours, the possibility for time-lapse frame capture, possibility to attach external lights, and possibility to mount on a tripod or custom-made frame for deployment as a video lander on the seafloor. Depth ratings range from 4.8 m (without dedicated underwater housing) to 999 m (with pressure-proof housing); the runtime varies from 3 min to 70 min using internal storage and to 140 min or 280 min using memory cards (with 64 Gb and 128 Gb capacities, respectively). All these 360-degree cameras offer time-lapse options and various possible settings. The size and shape of the cameras’ housing matter when it comes to attaching external lights and mounting the camera on a tripod or custom-made frame. Smaller builds such as the GoPro Max 360, Insta360 One R, Kandao QooCam, or Ricoh Theta cannot handle many external accessories and are better hand-held or mounted on a monopod. The 360RIZE 360Abyss is a cube housing with a GoPro and lens dome on each of the six faces, which makes it challenging to attach lights and mount them on a frame. In addition, a previous study reported that the GoPro Max 360 tends to overheat after an hour, both on land and underwater, and the Insta360 One R tends to move in the housing if not safely secured [19]. Overall, the Boxfish 360 camera met all our criteria (Figure 1). In addition, it comprises a sensor package that records environmental data (depth, water temperature, and camera orientation) every second.
Three possible methods of deployment were identified for the Boxfish 360 (Figure 2): hand-held from a line attached to the top of the housing by a ¼-inch eyebolt, for lowering from the side of a boat; mounted on a tripod for deployment from shore or by scuba divers, with lead soft pouch weights attached to the tripod legs to prevent tipping in faster currents; and mounted on a lander with a wide stability base to be deployed by boat at greater depth. The lander was designed in-house and comprised a three-face aluminum frame with a lifting eyehole at the top and attached to a triangular-shaped fiberglass grate ballasted with lead soft pouch weights. The lander was 94 cm tall, had a footprint of 0.37 m2, and weighed 24 kg (including 6 kg of neutrally buoyant camera) in air, to which up to 12 kg of ballast weight were added. The amount of necessary ballast weight was determined by analyzing the expected sum of forces and moments on the structure. To remain stable, the friction force between the lander base and seafloor must be able to resist drag from the fluid flow. Drag and friction coefficients were conservatively estimated from the range of expected values based on the lander design and individual deployment locations. Drag forces were calculated from expected flow velocities, which were calculated from empirical current data at Sequim Bay inlet and wave data at La Jolla Pier, accessed from the Scripps Institute of Oceanography. The adequate weight to stabilize the lander with a large factor of safety for both locations was determined to be 12 kg of rebar or a material of equal or greater relative density to seawater. External lighting was provided by a rig of six Light Motion SOLA Video Pro LED 3800-lumen lights and, after drowning two of them, by three SOLA and six FoxFury Rugo 620-lumens lights, screwed in the ¼-inch thread on top of the camera housing.

2.2. Field Approach

The camera was deployed at three locations (Figure 3) using the following configurations: the tripod and lander in Sequim Bay, WA; hand-held and tripod near Astoria, OR; and the lander in La Jolla, CA.
In Sequim Bay, the tripod was deployed and recovered by a scuba diver at slack tide at 4 m depth on a cobbled section of seabed in the energetic tidal channel leading into Sequim Bay and left to record for 15 min. The lander was then lowered three times to the seafloor with a davit from a small boat to a sandy bottom at 12 m depth in Sequim Bay protected from tidal currents for 5 min, then for a duration of an hour without a specific target to look at. Next, the lander was lowered to the same sandy location three times for 10 min each, at about 5 m away from a target made of a line with suspended toggle floats. The deployments in Sequim Bay were used to become familiar with the camera and troubleshoot the two configurations. From these tests, it appeared that the lander’s bare aluminum frame was reflecting the light from the external lighting rig and required a matte black coat of paint. Additionally, sun glare on the water surface affected the video quality at the shallower location.
The camera was then taken to two tidal marsh channels in the Grays River Estuary within the Columbia River Estuary near Astoria, to test its ability to see targets in low-visibility settings. The camera was lowered by hand alongside a small boat to about 1 m depth next to partially submerged logs in Secret River as the tide receded, and Seal Slough close to slack high tide and held to record for 20 min. Additionally, the tripod was deployed at slack low tide in Secret River about 0.5 m above the river bottom. The tripod was placed approximately 1 m upstream from a submerged stump to protect the camera from debris carried by the incoming tide. The tripod was flanked by two smaller pieces of wooden debris serving as targets, and the camera was left to record for 60 min as the tide rose to 2 m in depth. In parallel with recording the video footage, a Secchi disc was lowered into the water to assess the visibility. At each site, the visibility was approximately 1 m, in brown-yellow water.
Lastly, the video lander was deployed off the coast of La Jolla, CA from a small boat using a davit on a sandy bottom in 20 m of water. The location was approximately 1 km from shore and near a gravity mooring anchor of a WEC (CalWave Power Technologies’ xWave WEC) near the Scripps Institution of Oceanography’s Ellen Browning Scripps Memorial pier. The lander was deployed for about 1 h durations, three times a day, for three consecutive days at dawn, noon, and dusk (Table 1). Dawn and dusk setups required the lights to be turned on. For each deployment, a cluster of toggle floats was attached to the surface line about 3 m above the lander to ensure that the line floated above the frame during operation and out of the field of view. To help with the deployments and getting close enough to the anchors (≈5 m away), a live-stream Aquaview Scout XL camera was attached to the top of the lander with a quick-release mechanism that enabled pulling the live-stream camera back into the boat once the lander was in position. Because GPS locations were recorded each time, the locations of the first three deployments were used for the subsequent deployments and the live-stream camera was retired. Water turbidity was assessed before and after each drop by lowering a Secchi disc from the side of the boat.
The camera’s parameters were set according to the manual recommendations for each deployment at every site: f/2.8 aperture, −1 exposure compensation, 29.97 frames per second, auto white balance, auto ISO, and auto shutter angle.

2.3. Data Processing and Analysis

After each deployment, the video footage and environmental data were recovered from the three memory cards and the Boxfish internal memory, respectively, and the cards and memory were erased in preparation for the next deployment.
The Boxfish 360 uses three micro-four-thirds camera sensors, each with a lens that provides a 185-degree field of view. The three cameras are positioned to provide overlapping information that can be stitched together in MistikaVR software to create a 360-degree field of view. To get started, Boxfish provides a stitching template that is designed to roughly align each camera. To create an almost seamless stitch line and unnoticeable zenith and nadir, advanced parameters needed to be adjusted in MistikaVR. These adjustments included the horizon line, convergence locations of the stitch lines, stitch feathering, optical flow, masking, and edge points. Determining edge points was one of the most critical tasks. When objects are too close or too far away from different lenses, a parallax effect develops causing a disruption in the stitch line. Changing an edge point’s location, size, shape, and feathering will prioritize a specific camera’s view at a certain point along the stitch line, which removes the unwanted parallax. Once the clips were properly stitched, each camera angle needed color and exposure adjustments for optimal viewing. Finally, each file was rendered in an equirectangular MP4 video file format to be recognized and viewed as a spherical image in any video reader.
Video footage from Sequim Bay and Astoria were only reviewed qualitatively, using QuickTime Player, to assess image quality, water visibility, target visibility, and identify any necessary troubleshooting needs. A review of La Jolla footage for the visibility of the target, and the identification, counting, and tracking of fish was done at 5-min intervals throughout the stitched videos using a VLC media player. The stitching method of the 360-degree videos resulted in three viewing frames that were classified as left, middle, and right, where the middle frame contained the majority of the WEC anchor when it was visible, and each viewing frame was differentiated by the lander margins visible between frames (Figure 4). Each viewing frame was reviewed independently to isolate the organisms observed in the vicinity of the anchor from those observed away from it, and the stitched footage allowed tracking animals to move from one viewing frame to another. Data were collected for each viewing frame per 5-min time stamp to record MaxN, the maximum number of individuals present at one time for each taxon. Observations were conducted during 30-s intervals around each 5-min mark to ensure that slow-moving and cryptic organisms were more likely to be noticed. Videos were first reviewed at real-time framerate for each 5-min mark, then sped up to double the framerate for a second viewing to detect slow-moving or cryptic organisms such as flatfish. When many organisms were present, all frames for that time mark were reviewed to capture the total count of organisms present. Visible organisms were identified to the lowest taxonomical level possible and enumerated but no statistical analyses were conducted on MaxN due to the limited sampling size.

3. Results

3.1. Field Considerations

Three different deployment configurations of the Boxfish camera were tested to identify the most suitable one for high-energy sites (i.e., providing stable images and safety for the camera). The most stable configuration was the lander because of its wide base, which allowed for the attachment of weights as ballast providing a low and adjustable center of gravity. The lander’s size and weight were manageable for deployment from a small boat by hand or davit. The lander provided the most protected means of lowering the camera to the seafloor. The hand-held configuration resulted in some slight spinning of the camera despite the use of a swivel between the eyebolt and the line (not shown in Figure 2), and an occasional rocking/bobbing due to the motion of the boat or the line handler. While the tripod was easily deployed by a scuba diver in Sequim Bay and by wading in the shallow water near Astoria, it could not be safely lowered from a boat or ballasted with enough weight to firmly set it on a relatively flat section of the bottom to avoid tipping over in dynamic water.
The deployments at the three different locations were in depths ranging from 1 to 22 m and water visibility from 1 to 18 m. Videos recorded in shallow water (1 to 4 m) were affected by a sun glare on the water surface that resulted in over-exposure of the upper third to half of the footage and obscured the lower part. Over-exposure due to the sun’s reflection on the surface was also visible on videos recorded at greater depths (8 to 22 m) but did not affect the quality of the lower half of the videos.
The ability to detect the target on the video footage was very dependent on the water visibility at each site. In Sequim Bay, the target (toggle floats) was visible at about 5 m from the lander despite a heavy phytoplankton bloom (Figure 5a). Targets (wood logs) were barely visible on videos from the deployments near Astoria because of the murkiness of the water and the over-exposure of the video footage due to the sun’s reflection (Figure 5b). Underwater targets were more visible when shade was provided at the water surface.
In La Jolla, the target (a WEC anchor) was estimated to be visible at about 2 to 7 m from the video lander during daylight, but not visible during the darker hours of dawn or dusk when beyond the illumination footprint provided by the external lighting rig, estimated by the light manufacturer to be about 5 m wide (Figure 6). The distance of the camera from the target affected the ability to detect and identify organisms in the near proximity of the wood logs (Astoria) or WEC anchor (La Jolla). In addition, the ability to detect and identify organisms during dawn and dusk in La Jolla depended on whether the external lights functioned well. Identifications were made easier when the lights illuminated the side of a fish, revealing distinctive markings. While the SOLA lights were a lot brighter than the Rugo lights and made the illumination footprint wider, they sometimes ran out of battery and turned off before the lander could be recovered, decreasing the illumination toward the end of the dusk deployments.
Video stitching had to be done manually for each set of 360-degree videos to best accommodate differences in lighting, water visibility, and parallax effect conditions, resulting in greater video processing time than if automatically stitched by the software. The stitch line was often visible on the final product (Figure 6) for three main reasons. Because of uneven underwater lighting, adjusting and matching exposure in post-production between every three cameras was imperfect. In addition, the Boxfish has wide-angle camera lenses, which cause dark edges relative to the center of the frame and can create inconsistencies in a stitch. Lastly, the metal bars of the lander frame were aligned directly in the stitch line, just far enough away from the camera that the stitch line was affected, and the parallax effect problem needed to be solved. If organisms were not tracked carefully, the inconstancies and flawed stitch lines could very slightly affect the fish counts by making fishes disappear in the stitch lines.

3.2. Animal Observations

A total of 155 marine animals and fish schools were documented in 9 h of video data from the deployments in La Jolla by the WEC anchor. Six taxa were identified, as well as unidentified fish and an invertebrate (Table 2; Figure 7). Large schools of bait fish (potentially anchovies or sardines) swarmed the lander during each dusk deployment, but individual fishes could not be enumerated within each school due to their fast swimming behavior. After the bait fish, the most abundant taxon was the sand bass, either barred or spotted (Paralabrax spp.). Few mega-invertebrates were observed: only one octopus and an unidentified invertebrate, possibly a crab.
Fishes were observed at each deployment but not equally in each viewing frame. More fish were observed in the middle viewing frame due to 33 fish, including 21 sand bass, counted on the last dusk video (drop ID 120221-03; Table 3). Few fish were observed during the noon deployments, even though water visibility was the greatest during mid-day (see Secchi disc measurements in Table 1). The highest counts of individual fishes came from the dusk deployments (Table 3).

4. Discussion

Monitoring the presence of benthic, demersal, and/or pelagic fish, and other mobile organisms around marine energy devices is essential to understand whether the animals take advantage of the new habitat provided by these structures, which therefore could act as ARs and/or FADs [2]. Fish may be drawn to marine energy devices because of novel opportunities for shelter or new food sources, but this close proximity to the devices could put them at greater risk of collision with the moving parts of tidal turbines, or of being affected by underwater noise or electromagnetic fields emitted by the devices and their associated structures [20]. Whether ARs and FADs displace local fish populations from natural reefs or enhance the production of reef fishes is an ongoing debate [3,21,22,23], and stakeholders frequently enquire about marine energy devices’ potential to displace or enhance local fish stocks (e.g., [24,25]).

4.1. 360-Degree Video Lander

Strong currents and high wave profiles of the environments in which marine energy devices are deployed can make in situ observations of fish around devices challenging [20]. Common means of monitoring are passive visual or acoustic technologies, the latter being more useful in low-visibility environments for observing fish behavior but lacking the ability to identify species [8,26,27]. Static video landers or drop cameras are often favored for stationary observations over moving ROVs or scuba divers because of their logistical simplicity and to avoid affecting fish behavior with the motion of the monitoring apparatus, and they have proven efficient under the difficult conditions associated with temperate nearshore reefs [16,18,28]. Previous authors have mounted three video cameras at 120° from each other on a midwater platform or a video lander in order to have at least one camera pointing toward the target or for a greater likelihood of unobstructed footage; however, they did not stitch the videos together to analyze a 360-degree field of view and instead selected the best video from their three cameras to analyze the footage [10,18]. Others have used four video cameras at 90° from each other as baited remote underwater video stations and analyzed the footage from all four cameras separately [29] or stitched it into a four-pane video [17]. Another study deployed a low-cost 360-degree camera attached to a tripod from a small boat onto soft sediment, but close to 10% of the recordings were invalidated because of unstable positioning due to seafloor relief [15]. In another example, the authors directly attached their 360-degree camera to a tidal turbine to observe animals interacting with the rotating blades of the device [19]. The present study is, to our knowledge, the first to assess the use of a 360-degree underwater camera mounted on a video lander to monitor fish around artificial structures, such as the anchors of a marine energy device.
In recent years, 360-degree cameras have seen lots of innovation, especially for sports action and underwater cameras, and low-cost COTS 360-degree cameras are becoming more common, accessible, and perfected [19]. However, the more-expensive Boxfish was the most suitable option for the goal of the present study because the depth rating and runtime were superior to any of the low-cost options, and the shape enabled the attachment of external lights. The Boxfish camera and custom-made lander were easy to transport, assemble, deploy, and recover from an unstable surface such as a small boat. Unlike the tripod, the lander provided great stability on the seafloor while remaining light enough for two persons to handle. Based on our calculation, at least 20 kg of ballast weight could be added to the grate if needed (we used 12 kg), without compromising the lander’s integrity or maneuverability. While glare from the lights on the aluminum lander frame affected the video quality of the first deployment in Sequim Bay, it was easily resolved by applying matte dark paint. Electrical tape would also suffice (e.g., [16]). While not used in our study, sets of sizing lasers can be mounted to each side of the lander, below the camera plate, to help with measuring the lengths of and distances from objects. Unlike the stable lander configuration, the hand-held and tripod configurations are not ideal for deployments of the 360-degree camera at high-energy sites because of the increased risk of spinning and rocking because of surface swell motion (hand-held), or because of the fact that someone must physically secure it to the seafloor (tripod). This limits the approach to deployments of the video lander lowered onto the seafloor unless quiet sea states are targeted for a hand-held deployment by a surface structure (e.g., to survey biofouling and aggregating assemblages on the underwater side of a floating marine energy device). However, the quality of video footage from any close-to-surface deployment would be diminished by the glare from sunlight at the surface affecting more than the zenith area of the footage, unless some shadow is artificially provided.
During the field trial in La Jolla, the WEC’s anchor was visible on seven of nine stitched videos, indicating the 360-degree approach is an efficient way to visually monitor animal activity around the seafloor structures associated with marine energy devices. Water visibility is a limitation for any underwater video survey, especially between dusk and dawn, and external lights are often necessary, adding the potential to attract or deter fish [26]. The external lights mounted on top of the Boxfish provided good illumination and helped with detecting and identifying animals because they reflected the light when moving around. External lights were not always necessary because of good water visibility in southern California, even though animals’ coloration and markings were less apparent without the lights. Magenta filters could have been added to the camera lenses to enhance colors [18]. However, the lights used at dusk during the field trial in La Jolla attracted large schools of bait fish, which obstructed the view. In addition, even though fish could be observed between the WEC’s anchor and the video lander, the anchor itself was not very visible during the dusk deployments, despite the lights; and interactions between fish and the anchor were near impossible to observe. An acoustic camera could be added to the lander to complement the optical footage [27]. The ability to observe the interaction between animals and the artificial structure greatly depended on the distance from the anchor. The farther the video lander was from the anchor, the more difficult it was to determine the proximity of the fish to the anchor from the footage.

4.2. Challenges and Opportunities of the 360-Degree Approach

In addition to the deployment aspects listed above, a 360-degree video approach involves specific considerations to keep in mind. While several conventional underwater video cameras used for marine energy environmental monitoring have been designed to provide live-stream footage, with the video data sets stored in dedicated recording locations and power provided from an outside source through an umbilical [26,30], no similar option yet exists with COTS 360-degree cameras [19]. Interfacing with COTS 360-degree cameras and their high-pressure-rated housings remains a technical challenge that camera companies have not addressed yet because of the lack of market demand. Projects must rely on internal batteries and data storage, and plan accordingly, especially since most cameras need to be powered on before closing the housing, which can occur at a non-negligible time before deployment and consume battery time. Moreover, underwater high-definition video surveys produce large data files [26], and 360-degree video surveys produce even larger datasets, especially when multiple cameras are used to produce the stitched spherical footage (e.g., three cameras with the Boxfish, six with 360RIZE 360Abyss). Data storage in the hundreds of gigabytes (GB) to terabytes becomes necessary to archive video data sets. For example, nine deployments of the Boxfish for about an hour each in La Jolla resulted in 77 video files totaling close to 850 GB. Once stitched and rendered, this resulted in another 50 GB of video files. While the processing of 360-degree video footage requires more effort than conventional footage because of the time-consuming stitching step (e.g., about 3 h per 30 min video in the present case), the time needed for video review and analysis is not much different and depends more on the actual length of the videos as well as the study goals rather than the file size. A review of the video footage can benefit from the progress in machine learning and big data analysis with the implementation of autonomous object detection and classification algorithms [30,31]. If rendered in an equirectangular format, 360-degree videos are not different from conventional footage and can be processed by automatic image processing algorithms. Such algorithms use machine learning or deep learning methods such as convolutional neural networks to detect and classify fish species and even track fish trajectories [32,33,34]. Tracking fish trajectories with such algorithms would greatly help with enumerating individuals in fish schools.
Nevertheless, the benefits that the 360-degree approach brings to video lander deployments, especially for monitoring specific targets, outweigh the limitations identified above. Despite the remaining challenges, 360-degree cameras are useful tools for monitoring animal interactions with marine energy devices. With the proper amount of ballast weight to counteract drag forces due to currents, a lander like ours can be deployed at either wave or tidal energy sites to record videos of the bottom structures. The lander structure is versatile enough that additional instruments can be attached, such as sizing lasers or a CTD (conductivity–temperature–depth sensor). Although our sample size was small, the results suggest that the time of day the camera is deployed needs to be carefully considered to balance water visibility and fish observations. If using 360-degree cameras to assess the AR and FAD effects of marine energy devices and/or associated structures, a large-enough number of video deployments needs to be planned (e.g., using a power analysis) to provide statistical power to the data analyses while accounting for deployment failures (e.g., lander too far from the target, camera misfunction). In addition, the sampling design should identify whether to review the whole video footage or evenly spaced time points, based on study goals and budget. Overall, the 360-degree video lander approach provides a useful solution to monitoring the AR and FAD effects of marine energy installations, without the constraints and limits inherent to ROVs and scuba divers, and is a great technology to add to the diverse toolbox available for monitoring changes in benthic and pelagic habitats [8].

Author Contributions

Conceptualization, L.G.H.; methodology, L.G.H.; formal analysis, K.F.M. and L.G.H.; data curation, L.G.H., K.F.M. and E.B.P.; writing—original draft preparation, L.G.H.; writing—review and editing, L.G.H., K.F.M., C.M.G. and E.B.P.; visualization, L.G.H. and K.F.M. All authors have read and agreed to the published version of the manuscript.

Funding

This research was possible because of the generous support of the U.S. Department of Energy EERE Water Power Technologies Office (WPTO) to the Pacific Northwest National Laboratory (PNNL) under contract DE-AC05-76RL01830.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Raw video and environmental data files are available on the Portal and Repository for Information on Marine Renewable Energy (PRIMRE) Marine and Hydrokinetic Data Repository (MHKDR), under accession number 396: https://mhkdr.openei.org/submissions/396 (accessed on 6 March 2022).

Acknowledgments

We are grateful for the ongoing support and guidance from PNNL staff with the Triton Initiative: Alicia Amerson, Joe Haxel, John Vavrinec, James McVey, Linnea Weicht, Garrett Staines, and Ioana Bociu. The authors also thank Bob Mueller of PNNL for lending the Aquaview Scout XL camera, Heida Diefenderfer and Amy Borde of PNNL for the opportunity to test the camera in the field during their “Columbia Estuary Ecosystem Restoration Program Science Support” project, and PNNL’s permitting team. Fieldwork in La Jolla would not have been possible without support from Brett Pickering and Christian McDonald of Scripps Institution of Oceanography, and from Dan Petcovic and Thomas Boerner of CalWave. We thank Samantha Eaves of WPTO for the helpful feedback and Susan Ennor of PNNL for assistance with the manuscript.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. Hemery, L.G. Changes in Benthic and Pelagic Habitats Caused by Marine Renewable Energy Devices. In OES-Environmental 2020 State of the Science Report: Environmental Effects of Marine Renewable Energy Development around the World; Copping, A.E., Hemery, L.G., Eds.; Ocean Energy Systems (OES): Richland, WA, USA, 2020; pp. 104–125. [Google Scholar]
  2. Kramer, S.; Hamilton, C.; Spencer, G.; Ogston, H. Evaluating the Potential for Marine and Hydrokinetic Devices to Act as Artificial Reefs or Fish Aggregating Devices, Based on Analysis of Surrogates in Tropical, Subtropical, and Temperate U.S. West Coast and Hawaiian Coastal Waters; OCS Study BOEM 2015-021; U.S. Department of Energy, Energy Efficiency and Renewable Energy: Golden, CO, USA, 2015; p. 90. [Google Scholar]
  3. Langhamer, O. Artificial reef effect in relation to offshore renewable energy conversion: State of the art. Sci. World J. 2012, 2012, 386713. [Google Scholar] [CrossRef] [PubMed]
  4. Langhamer, O. The location of offshore wave power devices structures epifaunal assemblages. Intern. J. Mar. Energy 2016, 16, 174–180. [Google Scholar] [CrossRef]
  5. Brickhill, M.J.; Lee, S.Y.; Connolly, R.M. Fishes associated with artificial reefs: Attributing changes to attraction or production using novel approaches. J. Fish Biol. 2005, 67, 53–71. [Google Scholar] [CrossRef]
  6. Witt, M.J.; Sheehan, E.V.; Bearhop, S.; Broderick, A.C.; Conley, D.C.; Cotterell, S.P.; Crow, E.; Grecian, W.J.; Halsband, C.; Hodgson, D.J.; et al. Assessing wave energy effects on biodiversity: The wave hub experience. Philos. Trans. A Math. Phys. Eng. Sci. 2012, 370, 502–529. [Google Scholar] [CrossRef] [Green Version]
  7. Coastal Virginia Offshore Wind. (Dominion Energy, Richmond, VA, USA). 2021. Available online: https://player.vimeo.com/video/572244218 (accessed on 6 March 2022).
  8. Hemery, L.G.; Mackereth, K.F.; Tugade, L.G. What’s in My Toolkit? A Review of Technologies for Assessing Changes in Habitats Caused by Marine Energy Development. J. Mar. Sci. Eng. 2022, 10, 92. [Google Scholar] [CrossRef]
  9. Cruz-Marrero, W.; Cullen, D.W.; Gay, N.R.; Stevens, B.G. Characterizing the benthic community in Maryland’s offshore wind energy areas using a towed camera sled: Developing a method to reduce the effort of image analysis and community description. PLoS ONE 2019, 14, e0215966. [Google Scholar] [CrossRef]
  10. Sheehan, E.V.; Bridger, D.; Nancollas, S.J.; Pittman, S.J. PelagiCam: A novel underwater imaging system with computer vision for semi-automated monitoring of mobile marine fauna at offshore structures. Environ. Monit. Assess. 2020, 192, 11. [Google Scholar] [CrossRef]
  11. Taylor, J.C.; Paxton, A.B.; Voss, C.M.; Sumners, B.; Buckel, C.A.; Pluym, J.V.; Ebert, E.B.; Viehman, T.S.; Fegley, S.R.; Pickering, E.A.; et al. Benthic Habitat Mapping and Assessment in the Wilmington-East Wind Energy Call Area: Final Report; OCS Study BOEM 2016-003 and NOAA Technical Memorandum 196; U.S. Bureau of Ocean Energy Management: Sterling, VI, USA, 2016; p. 171. [Google Scholar]
  12. Page, H.M.; Dugan, J.E.; Dugan, D.S.; Richards, J.B.; Hubbard, D.M. Effects of an offshore oil platform on the distribution and abundance of commercially important crab species. Mar. Ecol. Prog. Ser. 1999, 185, 47–57. [Google Scholar] [CrossRef]
  13. Thuringer, P.; Reidy, R. Summary Report on Environmental Monitoring Related to the Pearson College-ENCANA-Clean Current Tidal Power Demonstration Project at Race Rocks Ecological Reserve; Lester B. Pearson College of the Pacific: Victoria, BC, Canada, 2006; p. 54. [Google Scholar]
  14. Campbell, M.D.; Huddleston, A.; Somerton, D.; Clarke, M.E.; Wakefield, W.; Murawski, S.; Taylor, C.; Singh, H.; Girdhar, Y.; Yoklavich, M. Assessment of attraction and avoidance behaviors of fish in response to the proximity of transiting underwater vehicles. Fish. Bull. 2021, 119, 216–230. [Google Scholar] [CrossRef]
  15. Mallet, D.; Olivry, M.; Ighiouer, S.; Kulbicki, M.; Wantiez, L. Nondestructive Monitoring of Soft Bottom Fish and Habitats Using a Standardized, Remote and Unbaited 360° Video Sampling Method. Fishes 2021, 6, 50. [Google Scholar] [CrossRef]
  16. Hannah, R.W.; Blume, M.T.O. Tests of an experimental unbaited video lander as a marine fish survey tool for high-relief deepwater rocky reefs. J. Exp. Mar. Biol. Ecol. 2012, 430–431, 1–9. [Google Scholar] [CrossRef]
  17. Campbell, M.D.; Salisbury, J.; Caillouet, R.; Driggers, W.B.; Kilfoil, J. Camera field-of-view and fish abundance estimation: A comparison of individual-based model output and empirical data. J. Exp. Mar. Biol. Ecol. 2018, 501, 46–53. [Google Scholar] [CrossRef]
  18. Watson, J.L.; Huntington, B.E. Assessing the performance of a cost-effective video lander for estimating relative abundance and diversity of nearshore fish assemblages. J. Exp. Mar. Biol. Ecol. 2016, 483, 104–111. [Google Scholar] [CrossRef]
  19. Pattison, L.; Serrick, A.; Brown, C. Testing 360 Degree Imaging Technologies for Improved Animal Detection around Tidal Energy Installations; Offshore Energy Research Association: Halifax, NS, Canada, 2020; p. 97. [Google Scholar]
  20. Copping, A.E.; Hemery, L.G.; Viehman, H.; Seitz, A.C.; Staines, G.J.; Hasselman, D.J. Are fish in danger? A review of environmental effects of marine renewable energy on fishes. Biol. Conserv. 2021, 262, 109297. [Google Scholar] [CrossRef]
  21. Bohnsack, J. Are High Densities of Fishes at Artificial Reefs the Result of Habitat Limitation or Behavioral Preference? Bull. Mar. Sci. 1989, 44, 631–645. [Google Scholar]
  22. Claisse, J.; Pondella, D.; Love, M.; Zahn, L.; Williams, C.; Williams, J.; Bull, A. Oil Platforms off California are Among the Most Productive Marine Fish Habitats Globally. Proc. Natl. Acad. Sci. USA 2014, 111, 15462–15467. [Google Scholar] [CrossRef] [Green Version]
  23. Lima, J.; Zalmon, I.; Love, M. Overview and trends of ecological and socioeconomic research on artificial reefs. Mar. Environ. Res. 2019, 145, 81–96. [Google Scholar] [CrossRef]
  24. Federal Energy Regulatory Commission (FERC). Environmental Assessment for Hydropower License: PacWave South Project; Federal Energy Regulatory Commission: Washington, DC, USA, 2020; p. 407. [Google Scholar]
  25. Naval Facilities Engineering Command (NAVFAC). Wave Energy Test Site Environmental Assessment: Marine Corps Base Hawaii; Naval Facilities Engineering Command: Honolulu, HI, USA, 2014; p. 412. [Google Scholar]
  26. Hasselman, D.; Barclay, D.; Cavagnaro, R.; Chandler, C.; Cotter, E.; Gillespie, D.; Hastie, G.; Horne, J.; Joslin, J.; Long, C.; et al. Environmental Monitoring Technologies and Techniques for Detecting Interactions of Marine Animals with Turbines. In OES-Environmental 2020 State of the Science Report: Environmental Effects of Marine Renewable Energy Development around the World; Copping, A.E., Hemery, L.G., Eds.; Ocean Energy System (OES): Richland, WA, USA, 2020; pp. 177–213. [Google Scholar]
  27. Staines, G.J.; Mueller, R.P.; Seitz, A.C.; Evans, M.; Wosnik, M.; O’Byrne, P. Capabilities of an acoustic camera to inform fish collision risk with current energy converter turbines. J. Mar. Sci. Eng. 2022, 10, 483. [Google Scholar] [CrossRef]
  28. Easton, R.R.; Heppell, S.S.; Hannah, R.W. Quantification of Habitat and Community Relationships among Nearshore Temperate Fishes Through Analysis of Drop Camera Video. Mar. Coast. Fish. 2015, 7, 87–102. [Google Scholar] [CrossRef]
  29. Whitmarsh, S.K.; Huveneers, C.; Fairweather, P.G. What are we missing? Advantages of more than one viewpoint to estimate fish assemblages using baited video. R. Soc. Open Sci. 2018, 5, 171993. [Google Scholar] [CrossRef] [Green Version]
  30. Hutchison, I.; Tait, C.; Sheehy, J.; Morgan, P. Review of Underwater Video Data Collected around Operating Tidal Stream Turbines 1225; NatureScot: Perth, Scotland, UK, 2020; p. 33. [Google Scholar]
  31. Xu, W.; Matzner, S. Underwater Fish Detection using Deep Learning for Water Power Applications. In Proceedings of the International Conference on Computational Science and Computational Intelligence (CSCI 2018), Las Vegas, NV, USA, 12–14 December 2018; pp. 313–318. [Google Scholar]
  32. Knausgård, K.M.; Wiklund, A.; Sørdalen, T.K.; Halvorsen, K.T.; Kleiven, A.R.; Jiao, L.; Goodwin, M. Temperate fish detection and classification: A deep learning based approach. Appl. Intell. 2022, 52, 6988–7001. [Google Scholar] [CrossRef]
  33. Agarwal, A.K.; Tiwari, R.G.; Khullar, V.; Kaushal, R.K. Transfer learning inspired fish species classification. In Proceedings of the 2021 8th International Conference on Signal Processing and Integrated Networks (SPIN), Noida, India, 26–27 August 2021; pp. 1154–1159. [Google Scholar]
  34. Li, X.; Liu, M.; Zhang, S.; Zheng, R. Fish Trajectory Extraction Based on Object Detection. In Proceedings of the 2020 39th Chinese Control Conference (CCC), Shenyang, China, 27–29 July 2020; pp. 6584–6588. [Google Scholar]
Figure 1. The six 360-degree cameras initially considered (from left to right): GoPro Max 360, Insta360 One R, Ricoh Theta, Kandao QooCam, Boxfish 360, and 360RIZE 360Abyss.
Figure 1. The six 360-degree cameras initially considered (from left to right): GoPro Max 360, Insta360 One R, Ricoh Theta, Kandao QooCam, Boxfish 360, and 360RIZE 360Abyss.
Jmse 10 00555 g001
Figure 2. Boxfish in the different configurations: from left to right, hand-held, mounted on tripod, and mounted on lander with external lights on top of the camera.
Figure 2. Boxfish in the different configurations: from left to right, hand-held, mounted on tripod, and mounted on lander with external lights on top of the camera.
Jmse 10 00555 g002
Figure 3. U.S. west coast deployment locations of the Boxfish camera in the hand-held, tripod, and lander configurations.
Figure 3. U.S. west coast deployment locations of the Boxfish camera in the hand-held, tripod, and lander configurations.
Jmse 10 00555 g003
Figure 4. Example of a lander deployment with position of the left, middle, and right viewing frames (white rectangles) used for reviewing the 360-degree video footage, with the middle frame showing the targeted wave energy converter anchor (yellow oval). The frame and base of the lander are visible in the foreground.
Figure 4. Example of a lander deployment with position of the left, middle, and right viewing frames (white rectangles) used for reviewing the 360-degree video footage, with the middle frame showing the targeted wave energy converter anchor (yellow oval). The frame and base of the lander are visible in the foreground.
Jmse 10 00555 g004
Figure 5. Screenshots from test deployments: (a) lander deployment in Sequim Bay, WA, on a day with a heavy phytoplankton bloom, with the live-stream camera centered in the foreground and the artificial target centered in the background; and (b) hand-held deployment in Seal Slough near Astoria, OR, in very turbid water, with a wood log target centered in the foreground.
Figure 5. Screenshots from test deployments: (a) lander deployment in Sequim Bay, WA, on a day with a heavy phytoplankton bloom, with the live-stream camera centered in the foreground and the artificial target centered in the background; and (b) hand-held deployment in Seal Slough near Astoria, OR, in very turbid water, with a wood log target centered in the foreground.
Jmse 10 00555 g005
Figure 6. Selection of screenshots from the deployments in La Jolla, CA, showing the distance of the anchor from the lander and the variability in target visibility between deployments: (ac) three noon deployments without external lights; (d) late dawn deployment with external lights; (e) early dusk deployment with external lights; and (f) late dusk deployment with external lights. The stitch marks between the three videos, delimiting the viewing frames, are visible as vertical lines on the images, and the anchor is centered in the background.
Figure 6. Selection of screenshots from the deployments in La Jolla, CA, showing the distance of the anchor from the lander and the variability in target visibility between deployments: (ac) three noon deployments without external lights; (d) late dawn deployment with external lights; (e) early dusk deployment with external lights; and (f) late dusk deployment with external lights. The stitch marks between the three videos, delimiting the viewing frames, are visible as vertical lines on the images, and the anchor is centered in the background.
Jmse 10 00555 g006
Figure 7. Screenshots of fish and mega-invertebrates observed on the video footage from La Jolla, CA: (a,b) sand bass; (c) kelp bass; (d) California scorpionfish; (e,f) flatfish, circled in white; (g) Unidentified invertebrate, circled in white; (h) octopus; and (i,j) bait fish schools.
Figure 7. Screenshots of fish and mega-invertebrates observed on the video footage from La Jolla, CA: (a,b) sand bass; (c) kelp bass; (d) California scorpionfish; (e,f) flatfish, circled in white; (g) Unidentified invertebrate, circled in white; (h) octopus; and (i,j) bait fish schools.
Jmse 10 00555 g007
Table 1. Field deployment details, with date, latitude (Lat.), longitude (Lon.), bottom depth (in meters), time at which the camera reached the bottom and was retrieved, water visibility (in meters) before a deployment (pre-Secchi) and after a deployment (post-Secchi), and surface and bottom temperatures (temp.; in degrees Celsius).
Table 1. Field deployment details, with date, latitude (Lat.), longitude (Lon.), bottom depth (in meters), time at which the camera reached the bottom and was retrieved, water visibility (in meters) before a deployment (pre-Secchi) and after a deployment (post-Secchi), and surface and bottom temperatures (temp.; in degrees Celsius).
Drop IDTime of DayDateLat.Lon.DepthTime at BottomTime RetrievedPre SecchiPost SecchiSurface Temp.Bottom Temp.
113021-01Dawn30/11/202132°52.048′ N117°15.782′ W20.7 m06:3607:4411 m11.5 m20.9 °C15.9 °C
113021-02Noon30/11/202132°52.045′ N117°15.783′ W21.8 m11:2312:2412.5 m10.5 m15.3 °C15.5 °C
113021-03Dusk30/11/202132°52.044′ N117°15.780′ W21.6 m17:0317:5610 mN/A17 °C17.4 °C
120121-01Dawn01/12/202132°52.049′ N117°15.782′ W20.7 m06:3007:3310.5 m14.5 m19.2 °C16.7 °C
120121-02Noon01/12/202132°52.048′ N117°15.780′ W22.4 m11:1412:2416 m15.5 m14.9 °C15.3 °C
120121-03Dusk01/12/202132°52.046′ N117°15.780′ W21.5 m16:3017:3012.5 mN/A18.5 °C17.2 °C
120221-01Dawn02/12/202132°52.046′ N117°15.781′ W21.9 m06:3007:368.5 m11.5 m22.4 °C16.2 °C
120221-02Noon02/12/202132°52.049′ N117°15.782′ W22.4 m11:1212:2618 m14 m16.7 °C15.9 °C
120221-03Dusk02/12/202132°52.047′ N117°15.782′ W21.4 m16:3417:409.5 mN/A17.5 °C15.9 °C
Table 2. Occurrences of each taxon in the nine video deployments, identified by their drop ID.
Table 2. Occurrences of each taxon in the nine video deployments, identified by their drop ID.
Taxon113021-01113021-02113021-03120121-01120121-02120121-03120221-01120221-02120221-03
Sand bass (Paralabrax spp.)4 292 1914239
Kelp bass (Paralabrax clathratus) 1 1
California scorpionfish
(Scorpaena guttata)
1
Flatfish 2
Bait fish school >1 >1 >1
Unidentified fish 121158615
Octopus 1
Invertebrate 1
Table 3. Number of individual fish (# fish) and taxa (# taxa) observed on the three different viewing frames of each video deployment (drop ID).
Table 3. Number of individual fish (# fish) and taxa (# taxa) observed on the three different viewing frames of each video deployment (drop ID).
Drop IDTime of DayLeft FrameMiddle FrameRight Frame
# Fish# Taxa# Fish# Taxa# Fish# Taxa
113021-01Dawn211111
113021-02Noon003200
113021-03Dusk17 + schools38 + schools37 + schools2
120121-01Dawn112100
120121-02Noon000011
120121-03Dusk4 + schools218 + schools32 + schools2
120221-01Dawn10210221
120221-02Noon008222
120221-03Dusk7 + schools233 + schools311 + schools3
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Hemery, L.G.; Mackereth, K.F.; Gunn, C.M.; Pablo, E.B. Use of a 360-Degree Underwater Camera to Characterize Artificial Reef and Fish Aggregating Effects around Marine Energy Devices. J. Mar. Sci. Eng. 2022, 10, 555. https://doi.org/10.3390/jmse10050555

AMA Style

Hemery LG, Mackereth KF, Gunn CM, Pablo EB. Use of a 360-Degree Underwater Camera to Characterize Artificial Reef and Fish Aggregating Effects around Marine Energy Devices. Journal of Marine Science and Engineering. 2022; 10(5):555. https://doi.org/10.3390/jmse10050555

Chicago/Turabian Style

Hemery, Lenaïg G., Kailan F. Mackereth, Cailene M. Gunn, and Edward B. Pablo. 2022. "Use of a 360-Degree Underwater Camera to Characterize Artificial Reef and Fish Aggregating Effects around Marine Energy Devices" Journal of Marine Science and Engineering 10, no. 5: 555. https://doi.org/10.3390/jmse10050555

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop