Introduction

With the advent of molecular genetics and techniques allowing to manipulate neuronal physiology with unprecedented versatility and precision, the number of animal models is growing considerably, supporting a renewed interest for integrative physiology and behavioral phenotyping. However, the presumably limited introspection and language capabilities of laboratory animals promote the need for designing sophisticated behavioral readout of internal cognitive states.

The extent of the behavioral repertoire we can identify largely depends on the technologies available for the acquisition of relevant biological information. In line with pioneering investigation of coordinated sniffing, whisking, and nose and head movements in the rat using frame-by-frame manual analysis of video recordings [1], the development of video hardware and image processing algorithms sustains fast progress in behavioral phenotyping. Recent examples include 2D [2,3,4] and 3D [5, 6] video acquisition, which combined with machine learning algorithms allowed to identify a number of basic postures and dynamics of spontaneous behavior [2,3,4, 6,7,8], providing a new vision of the ethogram at the sub-second timescale [6]. Simultaneous video recordings from several angles, including through a transparent floor plate, could successfully identify the positions of the paws and other body parts, providing detailed information about the dynamic coordination of paws and body movement during locomotion [3, 4, 9,10,11]. However, most (if not all) internal movements underlying behavior, such as heartbeat, shivering, or the dynamics of momentum during locomotion, remain out of reach from purely visual inspection. Electromyogram or electrocardiogram, as all invasive approaches, are likely to seriously interfere with spontaneous behavior. Piezoelectric technology however offers sensors of exquisite sensitivity, which positioned below the floor plate can be used to collect the dynamics of movement with very high temporal precision in a totally non-invasive manner [12, 13]. Plates resting on piezo sensors or accelerometers have successfully been used to build automated ethograms, distinguishing various behaviors such as sleep, rest, grooming, etc… [9, 14,15,16,17,18]. But none of them provides detailed and quantified information about the precise dynamics and forces involved in single movements during spontaneous, ongoing behavior. We now report on a novel device, the Phenotypix, which consists in an open-field platform resting on highly sensitive piezoelectric (electromechanical) pressure-sensors. The combination of such electromechanical (EM) acquisition with video recordings and signal analysis based on time-frequency decomposition, clustering, and machine learning algorithms, allowed us to automatically or semi-automatically detect and quantify various behavioral components with high accuracy, such as individual heartbeats and breathing cycles during rest, shaking in response to fear, and the dynamics of balance within individual footsteps during spontaneous locomotion. We believe that this novel device represents a significant progress and offers new opportunities for the awaited advance of behavioral phenotyping.

Materials and methods

Animals

Adult (age 2–4 months) male rats and mice of different strains were recorded:

  • 2 Sprague–Dawley OFA adult rats, recorded during resting immobility and sleep

  • 51 adult wild-type mice (23 3xTg-AD-WT, 12 C57B1-6J, 16 FMR1-WT), including 31 recorded in control condition, 10 after exposure to fearful conditions, 7 after pre-treatment with Diazepam, and 3 after local injection of CFA in a rear paw

  • 5 transgenic mice of the Ts65Dn line, a mouse model of Down Syndrome [19]

  • 11 transgenic Fmr1-KO mice (including 7 for grooming quantification and 5 for heart-beat monitoring, in control and after injection of BMS-204352), deficient for both FMR1 RNA and FMR Protein, a model of Fragile-X Syndrome [20].

For validation purposes, invasive electrophysiological recordings were performed on an additional 2 rats and 2 mice (age 2–4 months), equipped with electrode-wires implanted against the diaphragm (to collect ECG and EMG) and accelerometers (XYZ) on top of the head (to collect head movement), and recorded on the Phenotypix (to collect EM signal and synchronous video), in freely moving conditions. These recordings are described in detail in Supplementary Methods, and illustrated in Fig. 3 and supplementary Figs. S1, S2, S4, and S5.

The BKCa agonist BMS-204352 (Tocris, 2 mg per kg of body weight, IP), was dissolved in 0.9% NaCl, 1.25% DMSO, 1.25% Tween-80, 10 ml per kg of body weight), as in prior studies using BMS-204352 [21, 22].

All animals were bred in the laboratory animal facility in collective cages, and transferred to individual cages for the duration of the experiments. Animals were kept on a 12 h/12 h light/dark cycle, provided with nesting material and food and water ad libitum. All experiments were performed during the light period under constant mild luminosity (60-70 Lux). All experimental procedures were performed in accordance with the EU directives regarding the protection of animals used for experimental and scientific purposes (86/609/EEC and 2010/63/EU), with the French law, and approved by the Ethical committee CEEA50 (saisines #15349, 15350, 10897, 50120156-A and 30823-2021033102337869 v2).

Behavioral data acquisition

The mice and rats were introduced individually onto the recording platform (Phenotypix, Roddata, Quinsac, France), a dimly illuminated open-field environment (45 x 35cm), surrounded by 60-cm-high walls and equipped with video monitoring. The epoxy floor-plate and the walls were sprayed and wiped clean with 70% ethanol before the introduction of each animal. Spontaneous behavior was recorded continuously for durations ranging from 5 min to 3 h. In this system, the floor plate is resting on three evenly distributed (as a triangle) piezoelectric pressure sensors, all connected together to a single charge amplifier (hence receiving the sum of the signals from the three sensors), providing a continuous voltage analog signal (bandpass 0.1 Hz–9 KHz) proportional to the pressure exerted on the sensors underlying the floor plate, so that any subtle changes in floor-plate pressure due to animal movement could readily be detected. Unlike other existing phenotyping systems based on analyzing the vibrational pattern of the floor-plate to identify ongoing behavior [14, 18], the Phenotypix rather collects the minute pressure changes resulting from individual animal movements, which requires optimal signal preservation and was achieved by minimizing dampening and resonance. For this purpose, the platform laid on an antivibration table (TMC, CleanBench Laboratory, Peabody, MA, USA), made of a plain stainless steel table top (weight about 150 kg) resting on Gimbal Pistons using air pressure to keep the table top above a heavy (~100 kg) 4-legs frame. Lighter isolation platforms with pneumatic isolators (Newport Benchtop, Irvine, CA, USA) did not prove efficient enough to preserve the pressure-derived signal from vibrations, and the performance of the Phenotypix were seriously degraded. Recordings were always made with a single animal at a time to prevent interference between signals from multiple animals. Video signal was acquired at a sample rate of 25 frames/s with a webcam placed 1 m above the platform, and the electromechanical signal was recorded continuously at a sampling rate of 20 kHz. Both signals were acquired synchronously using a Power1401 digitizer and Spike2 software (CED, Cambridge, UK) and stored on a PC for offline analysis with EthoVision XT software (Noldus, Wageningen, the Netherlands) and custom-made matlab scripts (Mathworks, Natik, MA, USA).

EEG recording in freely moving rats

Rats were deeply anesthetized with isoflurane (2–5%) and implanted with individual 50-µm-diameter insulated tungsten-wires connected to an Omnetics connector fixed to the cranial bone with dental cement. For EEG recordings, the electrodes were placed within the hippocampus under EEG monitoring, at the following coordinates (Paxinos atlas): AP −3.3, L 2.5, V −2.5 to −2.8. The skin was put back on the skull and maintained with surgical glue (Gluture, WPI, Sarasota, FL, USA). Before recording, the animals were kept under daily monitoring for 1 week for healing and recovery. For recording, the animal was plugged to the recording system (L8 amplifiers, Neuralynx, Bozeman, MT, USA) with a tethered headstage (HS-16, Neuralynx), and the wide band (0.1Hz-9kHz) digitized signal continuously recorded with CED-Spike2 (in synchrony with the behavioral data) and stored on PC for offline analysis.

Fear conditioning

On day 1, WT mice were transferred from their housing room to the recording room for a fear conditioning session. After a 5 min acclimation period to the conditioning chamber, 5 trials (intertrial interval 5–10 min) were performed, each consisting of 10 intermittent white tones (80 dB, 500 ms duration separated by 1 s), the last five of which paired with electrical footshocks (duration 1 ms each, intensity 0.075J, delivered through a metallic floor mesh, Ugo Basile). The mice were brought back to their housing room after the conditioning session. On day 2, the mice were tested for contextual fear by being inserted in the recording chamber (different from the conditioning chamber but in the same experimental room with the same contextual configuration) for 15 min of free exploration after which 3 series of tones (conditioned stimulus, CS) were presented.

Exposure to predator

Individual WT mice could freely explore the recording chamber of the Phenotypix for 1 h. On the next day, they were placed again in the recording chamber, from which one of the arena walls had been removed and replaced by a cage containing an adult Sprague–Dawley rat, with a grid separating the rat (separated compartment, not in contact with the piezo sensors) from the mouse (placed on the Phenotypix platform). The spontaneous behavior of the mouse was recorded during 15 min.

Data analysis

The piezosensor-derived signal was first explored visually together with the video for identification of spontaneous behaviors. For monitoring of fast behavioral events (e.g. paw movements during grooming), the video (and synchronized EM trace) was replayed frame-by-frame (cf. Fig. 1). After visualization of the raw data, manual tagging of the different behaviors was performed by a trained expert, using Spike2 software. For further processing, the EM raw data was downsampled from 20KHz to 1250 Hz using ndmanager plugins [23]. Sonic Visualizer software was used to explore visually the time-frequency domain of the piezo derived-signal. Running and immobility periods were selected based on the animal velocity, calculated from the XY coordinates obtained through offline automatic animal tracking with Ethovision XT software (Noldus).

Fig. 1: Pressure-sensor-derived monitoring of various body movements with the Phenotypix system.
figure 1

The Phenotypix system is an open-field platform lying on pressure sensors allowing to detect various body movements of a freely moving rat or mouse with high sensitivity and fine time resolution. The video signal (25 frames per second) is recorded simultaneously with the digitized (20 kHz) electromechanical signal from the pressure sensors, allowing synchronized offline replay for analysis. A Schema of the Phenotypix acquisition system and the sensor-derived electromechanical (EM) signal elicited by a jumping mouse. Example tracking of the mouse with Noldus Ethovision software: yellow shape, detected mouse; red dot, central body point; vertical and horizontal blue lines, XY coordinates; red curve, mouse trajectory. B Spectral composition (power spectral density) of the EM signal recorded during 1 h of free open-field exploration. C–F Typical EM signal generated by different behaviors (within manually identified behavioral periods). Each trace is shown with the corresponding time-frequency spectrogram. Time of occurence of specific behaviors (C, locomotion >13 cm/s; D, nose grooming, each cycle of front paw movement appears as a separate deflection in the mechanical signal; E, grooming of the back; F, sniffing, each deflection corresponds to an individual nose-movement) is indicated with an horizontal red line, illustrated with pictures taken from the video signal, and decomposed as power spectral density (PSD). Note the main component around 10 Hz (and presumed harmonics in the 20–30 Hz range) during active behavior.

Full size image

Grooming momentum was quantified automatically using custom matlab scripts, on manually selected periods (i.e. momentum quantification was automatic, but the periods of grooming behavior were manually identified from the video), as the peak-to-through amplitude of each body movement-related EM-signal deflection.

Breathing and heartbeat-related movements were identified in the EM signal based on their shape, kinetics and frequency of occurrence, either manually or using MiniAnalysis software (Synaptosoft, Decatur, GA), during manually determined and carefully selected periods of immobility presenting the highest and clearest signal-to-noise ratio for series of at least 20 consecutive events for heartbeat, 10 for breathing. For validation purposes, the EM signal time tags were compared to those independently obtained by an expert observer, blind to the EM signal tags, from diaphram-EMG or video signals (see Figs. 3 and S4).

Shaking detection was done only during periods of immobility (speed <2.5 cm/s during a least 1 s). Automatic detection of shaking events was performed as threshold crossing on the bandpass filtered (65–130 Hz), squared and normalized signal. Automatic detection of freezing events was performed as threshold crossing on the 5–130 Hz bandpass filtered, squared and normalized EM signal. The thresholds for event detection were adjusted manually for each recording, and the outcome of the detection verified manually using Neuroscope visualization software [23]. Signal decomposition (Fourier Analysis) and quantification were performed using custom matlab (The Mathworks) code, available upon request.

The dynamics of locomotion were investigated within a similar speed range in WT and DS mice, during periods of intermediate running speed selected from Noldus Ethovision XY coordinates by a custom written matlab script, between 13 and 30 cm/s for a minimum duration of 700 ms. The corresponding EM signal, downsampled to 1250 Hz, was extracted using a sliding window of 800 samples. In order to discriminate between the features of locomotion signal issued from control vs CFA treated animals, we have used Support Vector Machine (SVM). The regression coefficients obtained from an autoregressive model [24] of order p were used as input features for k-means clustering [25]. We next identified “pure” clusters, containing samples from only either CFA or control mice, and performed on this sub-dataset a classification process using a linear support vector machine (SVM) [26] approach. In order to find the largest feature dataset with the highest classification performance, we conducted a grid search on the following parameters: AR order (p), window size (w), and number of clusters (k) maximizing an objective function defined by the product of the feature dataset size and the classification performance measured by F-score [27]. From the feature dataset maximizing the objective function we retrieved the corresponding signal windows, merging them to obtain the signal chunks where discrimination had been detected. Finally, we carried out the extraction of manually defined descriptive features on these signal chunks, which showed significant differences. Moreover, predictive SVM models were built by crossvalidation experiments using distinct data sets for training and testing, ensuring that there is no double dipping effect [28] in the estimation of the classification performance. Our 10-fold crossvalidation experiments in which the dataset was split into 10 equal subsets, each alternatively used as test-set while the remaining 9 subsets were used for training the linear SVM model, produced an F-score of 0.80 (data not shown), which is well above random choice classification.

Because visual inspection of the signal chunks identified by the discriminating clusters guided our attention towards the amplitude and time course of the EM signature of individual footsteps, we have performed an automatic, systematic quantification of amplitude and half width of all footsteps emitted by control and CFA mice during locomotion at comparable running speed (Fig. 5B, C). Periods of locomotion were selected during which the animal was moving between 13 and 30 cm/s without interruption and reaching at least 20 cm/s. Individual footsteps were identified as consecutive suprathreshold peak-trough-peak sequences from the EM signal, bandpass filtered at various frequencies using zero-phase distortion filters (i.e. filtering in the forward and backward direction to prevent phase-distortion). Peaks and troughs were detected as local extremas in the 0–300 Hz passband filtered EM-signal, within 50 ms of either the minima detected from the 0 to 50 Hz passband filtered signal (approximative troughs) or of the maxima detected from the 0 to 20 Hz passband filtered EM-signal (approximative peaks), respectively. Bandpass filtered 0–5 Hz signal was taken as baseline, and only local minima (troughs) of amplitude >1 SD from baseline were selected for further footstep analysis. The amplitude of footsteps was measured as the difference between the trough and the mean of its pre- and post-peaks. The half-width was measured as the width at half amplitude.

Locomotion and gait were also analyzed at the more global level of footsteps dynamics (Fig. 5D–F) by comparing the envelopes of locomotion-related EM signal across conditions. Periods of locomotion were selected during which the animal was moving for at least 500 ms between 13 and 30 cm/s. The linear correlation coefficient between upper and lower envelopes was computed automatically for each locomotor period.

Statistics

Data processing was performed with custom-made scripts and functions from the Matlab statistics toolbox (Mathworks). Data were systematically tested for Normal distribution, either with the Lilliefors test, a modification of the Kolmogorov–Smirnov test recommended for small sample sizes [29], or with the Kolmogorov–Smirnov test for sample sizes >50. Homoscedasticity was assessed using the Levene’s test. Data following Normal distributions and with homogenous variances were analyzed using parametric tests: ANOVA (F) with genotype or condition as factors (and post hoc Tukey test) for independent data sets, and Student’s t test (t) for paired data. Data following non-normal distributions were analyzed using the non-parametric tests Kruskal–Wallis (X2) for multiple groups comparisons (post hoc, Dunn–Sidak test) or Wilcoxon rank-sum test (z) for pairwise comparison of paired data. Results were considered significant for values of p < 0.05. Data are presented as mean ± SEM.

Results

Resolution of individual movements during spontaneous behavior in the freely moving mouse

The behavioral phenotyping device (Phenotypix, Roddata, Bordeaux, France) was designed to transmit any pressure applied on the open-field platform (35x45cm) to the underlying piezoelectric sensors, with minimal dampening and resonance for a faithful transmission of any movement of the animal (see Online Methods section for details). The output signal of the piezoelectric sensors was recorded in synchrony with the video (cf. Fig. 1A), but at much higher sampling rate (20 kHz instead of 25 frames/s). The dynamics of animal movement can therefore be resolved with high temporal precision. As illustrated in Fig. 1B, frequency decomposition (power spectral density) of the electromechanical (EM) signal retracing the spontaneous behavior of a wild-type (WT) mouse during a 1 h open-field session suggests that animal movements are mostly expressed at frequencies between 0 and 10 Hz. A closer examination of specific behaviors such as walking, self grooming or sniffing (which periods were manually defined from the video) revealed that frequency decomposition of the signal indeed showed a common expression of main temporal dynamics around 10 Hz. Nevertheless, the signal amplitude and shape was different for each behavior, presumably resulting in signal harmonics in the 20–30 Hz range (Fig. 1C–F). Another interesting observation is that the high-frequency response of the device and sampling rate of the signal allowed to resolve individual movements within complex behaviors. As illustrated in Fig. 1C–F, frame-by-frame analysis of movements related to specific behaviors revealed that individual footsteps during locomotion, paw movements during self grooming of the nose, body twitches during grooming of the back, or coordinated head and nose movements during sniffing, could be identified and quantified, providing the time course and momentum of individual movements within complex behaviors. As a first result of interest, these observations suggest that most common behavioral motor components share the same frequency range, from 5 to 15 Hz. We hypothesize that this frequency range may be a common tuning of a variety of neuronal networks involved in the control of voluntary movement. The second interest of these results is that the resolution of the EM-signal might be used in future studies as a complement to video data to improve the automatic detection of behavior and to build ethograms with much improved precision compared to existing systems [9, 14,15,16,17,18]. An example is provided in the last section of results, using a combination of video data, time-frequency decomposition of the EM-signal and machine learning algorithms for the automatic detection of limp during locomotion. But in addition to behavior detection, one information readily obtained from the EM signal and most likely out of reach of visual inspection, is the strength (momentum) or amplitude of individual movements. This may be of interest in specific applications such as the experimental study of self grooming behavior.

Quantification of grooming movement

Recent work suggests that Fmr1-KO mice, a model of Fragile-X syndrome related with Autism Spectrum Disorders (ASD), express subtle changes in grooming behavior as a sign of stress when exposed to a novel environment [30]. Altered grooming might be related with deficits in motor coordination [31]. On the other hand, some ASD patients express scratching with excessive strength, even sometimes resulting in self injury [32]. This aspect would be interesting to investigate in animal models, but the available phenotyping devices do not offer, to our knowledge, the possibility to easily quantify grooming movement strength/amplitude (momentum) and frequency. Even though movement frequency and amplitude (e.g. the 3D trajectories of individual paws) can be investigated from video recordings using proper machine learning algorithms [33], movement strength/momentum still remains out of reach of video investigation. On the other hand, as described in Supplementary Methods and illustrated in Figs. S1 and S2, such quantitative aspects of behavior might be readily captured by the Phenotypix.

As illustrated in Fig. 2, we have here quantified the amplitude and frequency of the EM signal associated with grooming of the back and belly in Fmr1-KO mice (n = 6 and 7 animals, respectively), and found that they were differentially affected. The behavioural periods specifically spent in grooming either the back or the belly were manually defined offline from the video, and custom matlab scripts used to automatically detect the precise time of occurrence and quantify the amplitude of the EM signal associated with each grooming movement within grooming periods. The main change regarding grooming of the back was an increase in frequency (z = −19.338, p < 0.001) while amplitude was hardly affected (WT vs Fmr1-KO, z = 2.193, p = 0.028). On the other hand, the main change regarding grooming of the belly was an increase in amplitude (WT vs Fmr1-KO, z = 29.147, p < 0.001) while the frequency was hardly affected (z = −5.080, p < 0.001). Therefore, subtle changes associated with specific movements are readily detectable from the EM-signal.

Fig. 2: Pressure-sensor-derived monitoring of grooming of the back and belly in WT and Fmr1-KO mice.
figure 2

A Typical example of sensor-derived (EM) signal related with grooming periods of different body parts identified manually from the video recording (horizontal bar: white, grooming of the nose, black, of the back, and gray, of the belly) in a WT mouse. B–D Electromechanical signal typical of a back-grooming episode, in WT mice (B, WT) and in Fmr1-KO mice (B, KO). The normalized cumulative distributions (C, D, left; black curves, WT; red curves, KO; superimposed bar histograms, difference between WT and KO) and averages across animals (C, D, right) of grooming signal amplitude (C) and inter-movement interval (D). E–G Same as in C, D but for grooming of the belly, indicating the amplitude and frequency of grooming of the belly in Fmr1-KO compared to WT mice. Note the clear increases in amplitude for grooming of the belly and in frequency (i.e. decreased inter-event intervals) for grooming of the back in Fmr1-KO compared to WT mice (*p < 0.05; ***p < 0.001).

Full size image

Non-invasive monitoring of breathing and heartbeat for the detection of sleep and emotional states

Other very subtle movements, that can also serve as an index of emotional regulation, are breathing and heartbeat. Interestingly, heart rate is also an index of vagal/parasympathetic tonus, considered to be misregulated in some pathologies such as Fragile-X syndrome (FXS).

Within the EM signal obtained from a rat during sleep and immobility, we actually noticed events that seemed to correspond to breathing and heartbeat (Fig. S3). The signal-to-noise ratio was highest during sleep and lower during rest, probably because the movements issued from the heart and chest were transferred less directly to the sensors when the animal was resting on his paws than when his chest was in direct contact with the floor-plate. As a validation procedure, we did observe that, as described in the literature, breathing was more regular during slow wave sleep (SWS) than during REM sleep, the two main brain states here identified by the theta/delta ratio of the EEG, simultaneously recorded from the hippocampus. But as described in Supplementary Methods and illustrated in Figs. 3A–C and S4, more direct evidence was provided by concomitant recording of heart and breathing activity using invasive electrocardiogram (ECG) and electromyogram (EMG) monitoring in freely moving animals, which demonstrated reliable monitoring of heartbeat and breathing from the EM signal. Because the animals (in particular mice, due to their small size) are likely to be perturbed by the presence of implanted devices (e.g. internal pressure sensors or ECG electrodes), we have used the EM signal as a non-invasive measure of heart rate in control WT and Fmr1-KO mice (a transgenic mouse model of FXS) at rest in a familiar environment. As illustrated in Fig. 3G, H, we have found that Fmr1-KO mice had an increased basal heart rate compared to WT mice (t(17) = −7.26; p < 0.001), rescued to normal levels by the BKCa channel agonist BMS-204352 (paired t-test, KO vs KO + BMS: t(4) = 6.62; p = 0.003, and independent samples t-test WT vs KO + BMS: t(17) = −1.39; p = 0.18). This is compatible with the clinical hypothesis that abnormal sympathetic/parasympathetic regulation may participate in the anxiety, behavioral distress, and gaze avoidance typically observed in FXS children [34,35,36], and confirms BKCa as a potentially relevant molecular target for the development of drug medication against FXS/ASD [30].

Fig. 3: Pressure-sensor-derived monitoring of breathing and heart-beat.
figure 3

A Schematic display of the periods with favorable signal-to-noise ratio for breathing (top, red) and heartbeat (bottom, gray) analysis, identified manually from the EM signal recorded from a freely moving mouse. The vertical black bar above the asterisk corresponds to the period which EM signal and ECG/EMG traces are shown in C. B, C Simultaneously recorded electromechanical signal (EM, upper traces) and electrocardiogram/electromyogram (ECG/EMG, lower traces, also shown as integrated signal in red) from the freely moving rat (B) and mouse (C). Note the slow and higher frequency components of the EM-signal derived from breathing and heartbeat, represented in middle traces (upper middle, linearized breathing cycles in red, lower middle, linearized heartbeat cycles in black) as broken lines. D–G Comparison of heart and breathing rates measured from the EM signal (X-axis) and ECG/EMG (Y-axis) in the rat (D, F) and in the mouse (E, G) over discrete periods of at least 20 contiguous heartbeat/10 breathing events. Note the tight correlation (Pearson correlation coefficient, r > 0.99) between EM and ECG/EMG-derived evaluation of heart and breathing rates. H Electromechanical signal and linearized heartbeat cycles during spontaneous immobility in a familiar environment from a representative WT (upper trace), Fmr1-KO (middle trace), and Fmr1-KO mouse injected with BMS (lower trace). I Average heart rate during awake immobility in WT, Fmr1-KO and Fmr1-KO mice injected with BMS (*p < 0.05.).

Full size image

High-frequency shivering associated with fear

Although the temporal dynamics of normal movements seem to be mostly confined to frequencies within the 0–15 Hz range, we did notice faster components in specific experimental conditions. A behavioral condition of interest in terms of motor expression is that of fear, characterized either by the active suppression of movement (freezing), or by the expression in various body parts of high-frequency uncontrolled muscle contractions (shaking/shivering). While freezing behavior is classically quantified manually from video recordings, non-invasive detection of shaking/shivering has to our knowledge not been reported, and its behavioural expression is hardly investigated at all. Because freezing and shaking may interestingly represent distinct but overlooked aspects of fear, we have looked for their potential specific signatures in the time-frequency composition of the EM signal of mice after contextual fear conditioning. Episodes of total immobility could be automatically detected with high efficiency and reliability (cf. Supplementary Methods and Fig. S5A–B) using a script collecting periods of EM signal below a manually set threshold of power in the 5–130 Hz frequency range (Fig. 4A). In addition, we noticed the high incidence of high-frequency (80–120 Hz) shaking events when the animal was inserted in the recording arena after fear conditioning (Fig. 4B). The detection of shaking/shivering from the piezo signal is based on the expression of high-frequency components in the piezo signal which are hardly expressed during immobility in control condition. Nevertheless, during active behavior (eg grooming, locomotion, etc…), movements with fast kinetics and large amplitude do carry a significant amount of high-frequency components in control conditions, which overlap with that of shaking. We have thus compared the expression of high-frequency components during the whole recording vs specific periods of either immobility or movement/locomotion. As illustrated in Fig. S5C–D (complementary to Fig. 4E), the increased expression of high-frequency components during fearful conditions could be detected from the whole recording (i.e. all behaviors included). But the main contributor to shaking-related high-frequency components is immobility, because contrary to immobility, there was no statistically significant difference between experimental conditions to reveal increased expression of shaking-related high-frequency components during movement periods. We have therefore restricted our detection analysis of shaking/shivering to immobility periods.

Fig. 4: Pressure-sensor-derived detection of fear-related freezing and shaking.
figure 4

A Electromechanical signal (top trace, raw data) and associated time-frequency spectrogram (underneath color plot, left scale is frequency range, color scale indicates power) show the presence of transient immobility periods (freezing, blue asterisk) during spontaneous exploration of a novel environment in a naive mouse. Below are shown a single freezing event (blue box, ii) vs basal resting condition (gray box, i), displayed at wider time scale above the corresponding Power Density Spectra. Note the strong drop in power at all frequencies during freezing. B. Electromechanical signal (top trace, raw data) and associated time-frequency spectrogram (underneath color plot, left scale is frequency range, color scale indicates power) show the presence of spontaneous transient 80–120 Hz oscillations in the signal (shaking, red asterisk) during behavioural rest from a mouse after contextual fear conditioning. Below are shown a single shaking event (red box, i) vs basal resting condition (gray box, ii), displayed at wider time scale above the corresponding Power Density Spectra. The signal in the blue box (blue asterisk) corresponds to a freezing episode. Note the high power in the 80–120 Hz frequency range during shaking. C, D Time course of occurrence of freezing (color coded horizontal time tags, freezing periods), detected as infra-threshold power in the 5–130 Hz frequency range) and shaking (blue line, detected as supra-threshold power in the 65–130 Hz frequency range and displayed as number of events per min, time bin 1 min) events over 13 min of recording before (C, Control) and after (D, Fear) contextual fear conditioning. Freezing episodes are indicated according to their duration: brief (<1 s, in cyan), intermediate (1–2 s, in magenta), long (>2 s, in green). E, F Bar plots of global expression of spontaneous freezing (E) and 65–130 Hz shaking events (F) either in control conditions (novel environment, NE), before and after pre-treatment with the anxiolytic Diazepam (+DZ), or after contextual fear conditioning (Fear context/CS, conditioned stimulus). Note that while freezing is the dominant behavioural expression of fear in response to CS, shaking is the dominant expression of context-related fear. ***p < 0.001.

Full size image

When we compared the times of occurrence of shaking and freezing in different behavioral conditions (Fig. 4C–F, n = 7 animals), we found that shaking was predominantly expressed as a behavioral response to context but not to the conditioned stimulus (Control vs Fear Context, F(2,18) = 35.639, p < 0.001; Control vs CS, F(2,18) = 35.639, p = 0.953), while it was the opposite for freezing (Control vs Fear context, F(2,18) = 24.493, p = 0.614; Control vs CS, F(2,18) = 24.493, p < 0.001), raising the possibility that shaking may be a behavioral response to diffuse threat and freezing to imminent threat. A confounding factor with freezing is that it was difficult from visual inspection to distinguish between immobility periods due to the real expression of fear from behavioral immobility associated with brief rest or active scanning of the environment. Short immobility periods (<2 s) were detected by our algorithm during the first minutes of exploration. In the literature, a classical approximation is to consider immobility periods > 2 s as freezing and to ignore those of shorter durations. Accordingly, mice inserted in a novel environment (n = 7 animals) expressed shaking during the first few minutes while little freezing behavior (> 2 s) could be identified. Nevertheless, pre-treatment with the anxiolytic Diazepam (at the non-sedative dose of 1.5 mg/kg, n = 6 animals) fully abolished the shaking events expressed during the first few minutes in the environment (Control vs DZ, t(11) = 5.052, p < 0.001), but did not significantly reduce the number of freezing episodes (Control vs DZ, t(11) = −0.517, p = 0.6153). Further investigation may evaluate whether these brief immobility periods are related with mild anxiety and increased attention to potential alerts in an unfamiliar environment. The possibility that shaking and freezing might be distinct signatures of fear was further suggested by the behavioral reaction of mice exposed to the presence of a rat, one of their natural predators, which induced at times the remarkable expression of shaking with little freezing (Fig. S6). Therefore, the Phenotypix proved efficient to detect and quantify internal movements such as heartbeat or shaking/shivering, hardly detectable by the eye of a human observer.

Gait and time course of individual footsteps

Another interesting aspect of pressure sensors is the possibility to evaluate the dynamics of the coordination and strengths of limb movements (momentum) involved in locomotion. Locomotion has been extensively studied using various experimental paradigms associated with image processing tools allowing to gather increasingly sophisticated spatio-temporal information about stride or stance. Nevertheless, it has remained quite out of reach to get non-invasive information about the dynamics of momentum, which can not be evaluated by visual inspection, however sophisticated. We observed that the EM signal provides some information about the dynamics of locomotion.

Frame by frame analysis of the synchronously recorded video signal allows to depict the EM signature of individual footsteps. In the short sample of spontaneous locomotion illustrated in Fig. 5A, one can distinguish a few initial footsteps of small amplitude, that correspond to orienting behavior (the mouse was changing direction but not moving forward). The EM signature of locomotion, with the animal really starting to move ahead, then becomes much more visible, as series of 5–10 footsteps of increasing and then decreasing amplitude, displaying a spindle pattern that turned out to be very typical of mouse locomotion. Because the EM signal is the result of the dynamic distribution of weight and of all the forces generated by a multitude of muscles within the animal’s body, it is a complex mixture that depends on the coordination of the various limbs and strengths (momentum) involved in movement. Nevertheless, we think that the very stereotypical signature of spontaneous locomotion in WT mice may serve as a reference template for the detection of motor impairment in various models of pathology.

Fig. 5: Pressure-sensor-derived signature of gait during spontaneous locomotion.
figure 5

A. Typical Electromechanical signature of mouse locomotion. Images 1–5, successive video frames (white asterisk, right rear paw). Note the very light electromechanical signal generated by orientation footsteps (#1–2, when the mouse is changing orientation) compared to the strong signature of forward footsteps (#3–5, each footstep appears as a fast downward/upward deflection when the mouse is giving impulsion to move forward). B, C The signature of individual footsteps during locomotion is affected in mice with a sore paw. Electromechanical signal typical of a locomotor episode (speed 13–23 cm/s; right, zoom on an individual footstep), either in normal conditions (Ct, upper traces) or after CFA-induced inflammatory pain in the left rear paw (CFA, lower traces). The normalized cumulative distributions (C, left; black curves, Ct; red curves, CFA; superimposed bar histograms, difference between Ct and CFA) and averages across animals (C, right) of footsteps amplitude (upper plots) and half-width (lower plots) indicate the presence of footsteps of statistically significant (***p < 0.001) lower amplitude and slower kinetics in CFA compared to control mice. D–F The correlation coefficient between the upper and lower EM signal envelopes (D, orange and yellow lines respectively) was computed as an index of the timecourse profile of successive footsteps during locomotor events (homogenous speed range, 13–30 cm/s) in control (WT) and Ts65Dn (DS) mice. The distribution of correlation coefficients of all locomotor episodes is presented as a normalized cumulative histogram (E), as a moustache plot (F, left) and as mean and SEM (F, right). Note the statiscally different (*p < 0.05) correlation coefficients between upper and lower envelopes in WT vs DS mice.

Full size image

As a first example, we recorded the EM signal of mice injected with CFA in one of the rear paws, producing local inflammation so that the animal tended to avoid pressing on the sore paw. Using a machine learning approach combining linear Support Vector Machine (SVM) classifier, autoregressive model (AR) feature extraction, and k-means clustering (see Online Methods section), we identified clusters of footsteps that differentiated control from CFA mice. Visual inspection of the signal chunks identified by the discriminating clusters guided our attention towards the amplitude and time course of the EM signature of individual footsteps. We therefore performed a systematic quantification of amplitude and half width of all footsteps emitted by control and CFA mice during locomotion at comparable running speed (n = 9 animals in each condition). As illustrated in Fig. 5B, C, some footsteps of CFA mice indeed turned out to be smaller and of slower time course than those of control mice (control vs CFA: amplitude, z = 7.781, p < 0.001; half-width, z = −10.354, p < 0.001), which reflects in the distributions of amplitude and half-width of the EM signal underlying individual footsteps during steady locomotion (15–30 cm/s). This is compatible with the likely consequence that CFA mice tend to avoid pressing on their sore paw.

Down Syndrom (DS) is also associated with motor impairment. Upon visual inspection of locomotion in Ts65Dn (a model of DS) and WT mice (n = 5 animals in each group), we noticed a disruption of the typical spindle pattern characteristic of WT mice. As illustrated in Fig. 5D–F, this was confirmed by the automatic quantification of the correlation coefficients between the lower and upper envelopes of the EM signal associated with steady locomotion (13–30 cm/s), suggesting altered balance and movement coordination in DS (WT vs DS, t(8) = 4.2212, p = 0.0029).

Discussion

A novel device (the Phenotypix), made of an open-field platform resting on highly sensitive piezoelectric pressure sensors, provided access in a totally non-invasive manner to very fine components of rat or mouse spontaneous behavior. Existing systems [14, 18] based on similar principles, combined with spectral decomposition and automatic classification, are used to generate ethograms, attributing each time bin of the recording to the most likely ongoing behavior, such as walking, eating, drinking, seizures, etc… But in contrast to these systems, the fine sensitivity and high sampling rate of the Phenotypix, combined with highly efficient antivibration system to minimize dampening and resonance, allow to resolve individual movements such as individual breathing cycles, heartbeats, or single footsteps during locomotion. Through the study of various behavioral conditions and transgenic models, we could identify and quantify novel behavioral components that can be useful for the study of several fields of behavioral neuroscience such as sleep, stress, or motor symptoms of neurodevelopmental diseases and locomotion.

Although existing devices were shown to have good performance for the detection of various kinds of self-grooming behavior [14, 18], they are not used, to our knowledge, for the quantification of the frequency and of the strength/momentum or amplitude of individual self-grooming body movements. The observation of increased amplitude and frequency of the EM signal underlying body movement in self-grooming of the back and belly in Fmr1-KO mice is an interesting complement to recent studies pointing at fine alterations in self-grooming behavior in these mice under stressful conditions [37,38,39], because it may help better understand repetitive and self-injury behavior in FXS/ASD patients [30, 32, 40, 41].

The direct and non-invasive evaluation of breathing and heart rate may prove useful for the study of sleep apneas, a pathological condition we still poorly understand. A previous report [13] described the use of piezo sensors to detect breathing movements and heartbeats of a mouse placed on a small platform (7 × 13 cm). Our system reached comparable sensitivity and resolution with dimensions more compatible with the study of spontaneous behavior for mice and rats. Although the dimensions of the system described here are 45 × 35 cm, it can be extended to larger environments by the apposition of several platforms, providing a multi-compartment environment best suited to the expression of complex behavior of both rats and mice. Breathing and heartbeat are vital parameters, but also strongly related to emotions, an aspect of behavior difficult to identify in animal studies. Anxiety is classically evaluated as the avoidance of situations of innate aversion such as exposed or bright areas (e.g. center of an open field, open arms of a maze) [42]. Fear on the other hand, a more acute and stronger behavioral reaction to perceived threat, is classically quantified as freezing immobility in rat and mouse studies. From our results, we propose high-frequency (80–120 Hz) shaking as a complementary index of fear in the mouse, expressed during exposition to a fearful situation such as a novel environment, the presence of a predator or a context previously associated with fear conditioning.

Our device also allowed the detection of abnormalities in the execution of locomotion, a fundamental motor function. While a number of systems are available to measure the spatio-temporal organization of gait, analyzing the sequence of positions of the various limbs during locomotion [3, 4, 9,10,11, 43], the Phenotypix allowed to reveal subtle alterations in the pressure signature of individual footsteps. This compound output is the result of complex interactions, that we can not yet dissociate, between the muscular strengths and the coordination of the individual limbs involved in each footstep. Nevertheless, we could access the time course of the momentum that corresponds to individual footsteps, and identify its reduced amplitude and slower time course in limping mice. Moreover, global analysis of the dynamics of successive footsteps revealed that mouse locomotion is typically organized as series of 5–10 footsteps of increasing and then decreasing amplitude, a pattern that was disrupted in a mouse model of Down Syndrome. Our system therefore allowed to detect alterations of locomotion in different mouse models, suggesting access to novel criteria for gait analysis that may shed new light in the understanding of various forms of ataxia.

Compared to existing approaches used in behavioral studies, the specificities of the Phenotypix present some advantages and limitations, guiding its fields of application. The main thing about the Phenotypix is to be at the same time global and very sensitive. This is advantageous to detect fine components potentially overlooked by other approaches (e.g. shaking, momentum), but a limitation when it comes to underpin the specific behavioral factor responsible for the identified EM-signal (e.g. to distinguish the relative contributions of movement amplitude, speed, or strength in the EM signature of grooming). The global nature of the signal can also be a disadvantage in applications that require uninterrupted monitoring of a specific behavioral component. For example, the Phenotypix allows to detect heartbeat and breathing in a majority of animals, but only during some periods of immobility, and alternative methods such as ECG/EMG or nasally implanted thermocouples [44] may thus be preferred for uninterrupted monitoring, although these approaches also have their own disadvantages (e.g. to be invasive). Overall, rather than an alternative to video approaches, EM signals might prove very helpful as a complement to image processing algorithms by bringing complementary non-visual information to feed machine learning algorithms, which progress at tremendous pace. It is thus very likely that the combination of EM signal and video monitoring, as provided in the Phenotypix system, should considerably improve the performance in building automated ethograms in the future.