Palomar Gattini-IR: Survey Overview, Data Processing System, On-sky Performance and First Results

The requirement of a large areal survey speed required the use of a fast focal beam to achieve a large field view when feeding the single, moderate-sized, detector that we had available for the project (Section 2.2). The aperture of the telescope was set to 30 cm by a requirement of a single epoch 5 point source sensitivity of ≈16.0–16.5 AB mag in J band. Gattini-IR uses a 30 cm aperture, f/1.44 catadioptric optical telescope assembly (OTA) with 6 all-spherical elements, commercially available as the Terebizh TEC300VT by Telescope Engineering Inc. The optical design was evaluated for performance at J band combined with the 18 micron pixel size of the available detector. Fortuitously, with only a slight detector focus adjustment, sub-pixel performance was simulated to be possible across the entire field over a temperature range from 0°C to 30°C over a maximum field of 7°, corresponding to a detector size of 52 mm (Moore et al. 2016). The OTA was assembled and tested to be diffraction limited on-axis at a wavelength of 633 nm prior to delivery to Caltech. The telescope, mount and cryogenic system are located inside a clam-shell dome at Palomar Observatory, which also houses a compute server controlling the robotic OS ("scheduler node" hereafter).

However, design specifications of sub-pixel imaging and athermal focus quality over the entire detector plane have not been achieved during on-sky tests (Section 5). The original focusing mechanism relies on three variable screws that to be manually adjusted to best focus. The manual operation makes it difficult to settle such a fast telescope on a stable position. In addition, temporal variations in the image quality as a function of ambient temperature are not corrected dynamically during nightly operation or even revisited at regular intervals. This issue will be corrected in the final quarter of 2019 with the addition of a robotic focusing mechanism. The system is based around a circular flexure element that is pre-stressed against three linear actuators. This design provides a level of anti-backlash to the system as well as helping with variable gravity vector effects during operation at various zenith angles. The three linear actuators will be remotely controlled and adjusted on regular basis to optimize the image PSF. The focus has been fully tested in the laboratory and gives a focus range of motion of 3 mm and a resolution of 10 μm.

The telescope and the cryogenic system housing the detector are attached to a to a GM3000 HPS robotic equatorial mount made by 10Micron Technology. The mount is designed to support up to 100 kg of instrument weight with a slew speed of up to 12° s⁻¹. A pointing model was constructed for the mount after telescope installation and mount balancing, using a sample of 100 bright stars placed randomly across the visible sky. The resulting pointing model produces rms residuals of ≈12'' (≈1.5 native detector pixels).

Gattini-IR uses an engineering grade 2 K × 2 K Hawaii-2RG (H2RG) detector from Teledyne with a cut-off at 1.7 μm to be capable of J-band imaging and avoid the thermal infrared background beyond 2 μm. The pixel size is 18 μm, which provides a pixel scale on sky of 873 pixel⁻¹ and a FOV of 496 × 496 when attached to the F/1.44 focal beam of the telescope. The mean quantum efficiency in J band is ≈70% (Blank et al. 2011). The gain and read noise of the detector were measured in the laboratory and found to be ≈4.5 e⁻/ADU and 25 e⁻ respectively. The read noise together with the measured J band sky background with the instrument (at least ∼1000 e⁻ s⁻¹ pixel⁻¹) allows for background limited imaging in exposures as short as ≈1 s (the minimum allowed by the detector electronics). The detector does not have a shutter mechanism and is thus parked horizontally (facing the dome walls) during day time and poor weather conditions.

The detector nonlinearity was measured using a constant light source in the laboratory and found to be <3% up to 30,000 counts (136,000 e⁻), which we nominally adopt as the linearity limit for the purposes of the data processing system. The detector saturates at ≈36,000 counts (≈160,000 e⁻). The number of hot pixels in the detector are measured periodically using darks inside the telescope dome. Due to the absence of a shutter mechanism, darks are periodically acquired in the presence of observatory staff by covering the telescope tube with an aluminum coated cap and recording images. Due to the high background in our imaging application, the requirement for low dark current is not substantial. Nevertheless, the detector has been found to exhibit dark current levels of ≈0.9 e⁻ s⁻¹ under nominal operating conditions while hot pixels amount to 0.1% of the detector pixels.

Dead (nonlinear and unresponsive) pixels are identified in the array using sky flats with different exposure times taken during twilight at the start and end of every observing night. Dead pixels amount to ≈2.7% of the total number of pixels in the detector, including the intentionally non-responsive reference pixels that are 4 deep on each edge of the detector. In addition, the detector has a triangular corner region of lower QE (measuring 700 × 740 pixels on the perpendicular sides, amounting to ≈12.4% of the detector area) due to the absence of an anti-reflection (AR) coating that was layered on the rest of the detector during a previous experimental phase. The region missing the AR coating was left to experimentally measure the change of the QE due to the presence of the AR coating, and has been verified to have a different zero-point (i.e., lower sensitivity) from the rest of the detector in commissioning data.

The detector is read out using a detector controller system supplied by Astronomical Research Cameras, Inc. (ARC). The ARC controller chassis houses four eight-channel infrared video processor boards, a clock driver board, and a 250 MHz fiber optic timing board. Each video board contains eight identical video processors, with each processor consisting of multiple stages having adjustable gains and offsets and an 18 bit analog to digital converter. Together, these four eight-channel video boards read all 32 outputs of the H2RG simultaneously. The video boards also contain programmable DC supplies to supply the bias voltages to the detector. The clock driver board translates digital input signals into analog output signals for driving the clocks required to control the H2RG detector. The fiber optic timing board contains a digital signal processor which generates the timing waveforms and communicates between the ARC controller and the host computer using a duplex fiber optic link to a PCIe interface board in the host.

The H2RG detector is read out non-destructively using conventional readout, though the pixel time has been reduced from what is typically used because previous studies (Wizinowich et al. 2014) have shown improved noise performance. The pixel readout time is 6 μs, which represents 2 μs settling and 4 μs integration per pixel. The line overhead of 21 μs contains "pre-charge" pulses required by the H2RG as well as pulses required for initializing the ARC controllers video board electronics. This amounts to a minimum frame time of about 834 ms.

The Gattini-IR cryostat is a custom built in-house fabricated assembly that was designed to minimize weight and volume by using 6061-T6 aluminum (Figure 1). The H2RG detector is mounted to a molybdenum block to match the CTE of the detector package which is cooled to about 100 K by using a Brooks Poly Cold Compact Compressor charged with P-14 refrigerant. The cold head cools the activated carbon getter and heat is conducted away from the detector by using three braided copper thermal straps. In addition, the vacuum volume uses two room-temperature zeolite desiccants, freshly baked before pumping down the vacuum volume. Electrical connections to the vacuum volume is accomplished with the Vacuum Interface board that carries both the detector and thermal management wiring.

Zoom In Zoom Out Reset image size

Two heaters are used—a low power heater to stabilize the detector temperature, and a higher power (up to 50 W, currently set to 8 W) heater near the entrance window to guard against dew. The latter was installed toward the end of the commissioning period to remedy the accumulation of condensation on the window plate during periods of high humidity. The H2RG detector is optically filtered by a cold, J band interference filter. The dewar is connected to the compressor by 50 feet of armored flex hose with particular attention paid to maximizing the bend radius in an effort to protect the system from fatigue failure due to the observing cadence and all-sky motion envelope needed for Gattini-IR. To prevent the compressor from becoming too cold in the winter, a wooden enclosure with a thermostat controlled fan surrounds the compressor and uses the compressor waste heat to maintain the box temperature at 21°C.

Nightly operations are controlled by an automated telescope scheduler adapted from the publicly-available software¹² for the ZTF (Bellm et al. 2019a). Unless the Palomar 200 inch telescope operator sets a weather override, the scheduler opens the dome and observes nightly between the times of nautical twilight. The celestial sphere north of −28° is divided into 1329 fields with overlaps of 6' between fields. The fields are separated by an average of ≈486 in the N–S direction and up to ≈49 in the E–W direction, depending on decl. Under nominal survey operations, Gattini-IR observes fields over the entire visible celestial sphere from Palomar Observatory.

Each field visit consists of a set of 8 dithered exposures with an exposure time of 8.1 s each (total exposure time of 64.8 s per field visit). Multiple dithers were required to facilitate longer exposure times on the bright sky background, and to allow PSF reconstruction in data processing (Section 4) using the Drizzle algorithm (Fruchter & Hook 2002). The amplitude of the dither is set to ≈3', which is randomized by a uniform distribution of 1' amplitude to sample random sub-pixel phases for individual point sources. Figure 3 shows a distribution of the pointing rms in the R.A. and decl. direction for fields distributed over the sky. As the dither amplitude is larger than the typical pointing rms and median offset, the entire field region is covered during the dither sequence. A minimum number of 8 dithers was selected to obtain uniform coverage of the sub-sampled pixels across the drizzled images such that σ_w/m_w < 0.15 in the output drizzled images (Gonzaga et al. 2012), where σ_w is the standard deviation of the output weight image from image reconstruction using Drizzle and m_w is the median weight. The exposure time per dither and number of dithers were balanced as a trade-off between maximizing the volumetric survey speed and cadence over the sky (Section 3.2).

Zoom In Zoom Out Reset image size

The scheduler currently employs a greedy algorithm that minimizes slew time and airmass while prioritizing fields that have not been observed as recently. This is done by selecting for the next field for observation that has the highest value of the following metric:

$$\begin{eqnarray}&&\displaystyle \frac{{t}_{\exp }+{t}_{{\mathrm{OH}}_{\min }}}{{t}_{\exp }+{t}_{\mathrm{OH}}}\times {\rm{\Delta }}{t}^{0.1}\times \displaystyle \frac{{z}_{\min }}{z}{\mathrm{HA}}_{\mathrm{weight}}\end{eqnarray} \tag{ 1 }$$

where t_exp is the exposure time (in seconds), t_OH is the overhead (in seconds, including slew, settle, and camera initialization/readout) to slew from the current field to the next, ${t}_{{\mathrm{OH}}_{\min }}$ is the minimum field slew overhead measured in operations, Δt is the time since the field was last observed (in days), z is the airmass, z_min is the minimum observable airmass allowed by the mount, and HA_weight is a weight factor based on the hour angle of the field. The exposure time and overhead factors serve to maximize the areal survey rate and the Δt prioritizes fields in proportion to the time elapsed since the prior observation. The airmass factors are used to encourage observations to occur at the local meridian. The hour angle factor prioritizes setting fields because this would maximize the single-pass sky coverage with the original survey rate which was intended to be faster than sidereal. Figure 4 shows a distribution of the slew distances and observation airmasses from data taken during six months of the commissioning period using this scheduling algorithm. As shown, the scheduling algorithm prioritizes observations that minimize the slew distance and airmass of the observation.

Zoom In Zoom Out Reset image size

The primary overhead during observing sequences is the time taken to move between individual dither positions in a dither sequence given the settling time of the mount, since the detector readout time is small (≈0.9 s). Figure 5 shows a distribution of the time between the start of subsequent dithered exposures in dither sequences taken over several nights. The distribution has a narrow peak around the median time of ≈13.0 s, including the exposure time of 8.1 s amounting to a dither overhead of ≈60%. Figure 5 also shows a distribution of the time between the start of exposures of successive fields during a night, which includes the exposure time inside the field (64.8 s), the time taken to dither between dither positions inside the field and the slew time to the next field. The distribution also has a sharp peak near the median time of ≈115 s. Figure 6 shows a distribution of the observing efficiency of the system, defined as the total exposure time in a night as a fraction of the time the telescope was observing. The overall observing efficiency distribution has a median of ≈61%, accounting for all the overheads for dithers and slewing between fields.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

We show the volumetric survey speed (Bellm 2016) for a SN Ia (peaking at M = −19) as a function of the exposure time per dither in Figure 7, for different choices of the number of dithers folding in the measured dither and field slew overheads. The volumetric survey speed increases with a smaller number of dithers and hence we select 8 dithers as a minimum to support the PSF re-construction downstream. For the choice of 8 dithers per field, the exposure time that maximizes the volumetric survey speed is ≈17s per dithered exposure, and a corresponding areal survey rate of ≈470 sq. deg. hr⁻¹ with an all sky cadence of ≈4 nights. Adopting an exposure time of ≈8.1 s (equal to 9 times the default frame readout time) instead, we found the volumetric survey speed to be only ≈5% smaller, while providing an areal survey speed of ≈800 sq. deg. hr⁻¹. This configuration allows coverage of the entire visible sky (≈15,000 square degrees) over two nights (assuming an average of ≈9 hr per night). On the other hand, increasing the cadence to cover the entire sky over a single night would require reducing the exposure time per dither to be <2 s, where the volumetric survey speed would be 50% smaller than the maximum volumetric speed.

Zoom In Zoom Out Reset image size

We thus adopt eight dithers of 8.1 s each as the nominal observing strategy for the survey. The choice of all sky coverage is motivated by the aim of performing a completely untargeted time domain survey as the first infrared counterpart to ongoing optical time domain surveys. The 2 day cadence enables early discovery and follow-up of SNe in nearby galaxies and large amplitude transients such as classical nova outbursts within the Galaxy. This cadence also allows the survey to obtain well sampled NIR light curves of these transients, while building up a large number (≈100) of epochs for variable star science in each year of operations. Figure 8 shows a distribution of the average cadence over the visible sky for a typical month of observing. The cadence is <3 nights for 90% of visible fields in the sky, while the median cadence of two days (for ≈60% of fields). A small fraction of fields (8%) near the north celestial pole have typically shorter cadence of ≈1 day owing to their longer visibility during the night. The higher cadence in these fields allows the survey to probe the phase space of shorter timescale phenomena in the infrared time domain sky. Figure 9 shows the sky distribution of the cumulative number of field visits since the start of the survey commissioning period. Fields near the north pole have the largest number of visits due to their long visibility window from Palomar Observatory.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

The scheduler is designed to respond to target of opportunity (ToO) triggers to respond to time critical events such as gravitational wave triggers and neutrino alerts. ToO requests are submitted as a list of fields to be observed for a specified integration time. In the case of gravitational wave and neutrino triggers, the field tiling is optimally determined using the algorithm presented in Coughlin et al. (2019b) and forwarded to the scheduling system via the GROWTH Marshal (Kasliwal et al. 2019), including a start and expiry date for the request. A cron job on the scheduler server periodically checks for new ToO requests every 5 minutes. In case a new ToO request is found, the scheduler interrupts the nightly survey after finishing the set of dithers on the field being observed.

Once triggered, the ToO scheduler checks if the list of requested fields in the ToO are observable (above airmass 2.5). If this condition is satisfied, the telescope slews to the field and begins dithered exposures of the nominal 8.1 s exposure time such that the total integration time equals the requested exposure time on the field. Each field in the submitted ToO is observed until no more observable fields are left, after which the scheduler resumes the nightly survey. The list of fields in each submitted ToO is checked every 5 minutes until all the fields in the ToO are observed or the ToO has reached its expiry date.

Figure 10 provides an overview of the data processing flow within the GDPS. The data reduction flow for the system proceeds in five steps—(i) image pre-processing, (ii) astrometric solutions, (iii) stacking of dithered exposures, (iv) photometric solutions and (v) difference imaging, which are performed sequentially when a new set of raw images are received. Although each step is performed sequentially on the raw incoming data, the processing of multiple observed fields within each step is parallelized with 30 threads to support the requirement of real-time processing.

Overall, the GDPS is controlled by a watchdog program that looks for new incoming images from the telescope and performs the data reduction through each of the steps mentioned above, while recording metadata for raw and intermediate data products at each step of the pipeline. At the end of the night, the watchdog accumulates the metadata and quality metrics for the data acquired during the night, including nightly sky coverage, image depths and PSF quality, while performing accountability checks on the number of files received and ingested through each step in the pipeline. These metrics are sent to the members of the project in a summary email. We summarize the processing architecture below and provide detailed descriptions in the following sections.

• 1.  
  Raw data received from the scheduler node are initially de-trended and then digitally split into four quadrants of 1044 × 1044 pixels each (corresponding to a size of 25 × 25 on sky), including an overlap (of 10 pixels =87'') between the common edges to avoid missing sources that are split between the quadrant edges (Section 4.3).
• 2.  
  An astrometric solution is derived for all the image quadrants produced from a single field visit i.e., 32 images from 8 dithers in four quadrants in the nominal observing strategy¹⁴ (Section 4.4).
• 3.  
  The astrometric solutions are used to stack the processed images (Section 4.5) on a per-quadrant basis resampled to a pixel scale half of the raw pixel scale of the detector (thus producing stacks which are 2088 × 2088 pixels in size) using the Drizzle algorithm (Fruchter & Hook 2002).
• 4.  
  The stacked quadrant images are again digitally split up into four sub-quadrants containing 1044 × 1044 resampled pixels each (≈125 × 125 on sky), including the same overlap between the sub-quadrants (of 87'' or 20 pixels in drizzled images). Figure 11 shows the layout of the detector plane with respect to the sky. Photometric solutions are derived on the split sub-quadrants (Section 4.6).
• 5.  
  Transient candidates are identified from difference imaging using the ZOGY algorithm (Zackay et al. 2016). These sources are passed through a machine learning (ML) classifier and cross-matched to several all-sky catalogs and known solar system objects from the Minor Planet Center. Candidates are subsequently uploaded to a web-portal for human vetting (Section 4.8).

Zoom In Zoom Out Reset image size

The step-wise design for splitting up the raw images into 16 sub-images as the final products was motivated by several reasons that were tested during the commissioning phase:

• 1.  
  Typical images exhibit a large variation of the PSF and sky-background over the large 5° × 5° FOV, and hence better photometric solutions and difference imaging is obtained by splitting up the image into smaller sections covering 1/16 of the total detector area, where the PSF and background are locally uniform.
• 2.  
  Splitting up the raw images into four quadrants before the astrometric solution derivation produces images that are small enough such that the distortion in the field is small but also large enough so that sufficient number of sources remain to derive a robust solution (this is particularly important under non-ideal observing conditions with low sky transparency). Dividing the images into 16 pieces at the start of the processing (before astrometry) increased both the failure rate and processing time for the reductions.

Raw images from Gattini are recorded as FITS images, along with comprehensive header information including meteorological conditions (temperature, pressure, humidity, wind speed/direction) and ephemeris information (airmass, Sun and moon positions, estimated sky brightness). A quick astrometric and photometric solution is performed, to allow real-time correction of the telescope position and an estimate of the sky transparency. The raw FITS images contain 2048 × 2048 pixels stored as 2 byte short integers, resulting in an image size of ≈8.1 MB each. The total amount of raw data acquired in a full night of observing is ≈20 GB. Raw data are robotically acquired by the OS under safe observing conditions and saved to disk on the scheduler node inside the telescope dome. Subsequently, the OS transfers the images to the data processing server housed at Caltech (using an rsync-based synchronization running every minute) via the NSF-funded High Performance Wireless Research and Education Network (HPWREN) administered by the University of California San Diego. The volume of raw data is small enough to be transferred in real-time to Caltech (transfer time of ≲1 s per image compared to the image acquisition time of ≈10 s).

The metadata for the raw images are ingested into the PSQL DB followed by de-trending of the raw images. De-trending involves subtraction of the most recently acquired dark frame from the raw image followed by flat-fielding with the most recently made sky flat (Section 4.11) by querying metadata for calibration frames stored in the DB. Images are then digitally split into four overlapping quadrants, with an overlap size of 10 pixels (corresponding to ≈15 on sky) between quadrants sharing common edges. The image quadrants are stored to disk and metadata are recorded to the DB. Since the data processing was designed such that each quadrant corresponds to a fixed position in sky coordinates, the position of each quadrant has to be rotated by 180° depending on the side of the meridian of the equatorial mount i.e., the raw image is rotated by 180° for all images taken on the rising side of the meridian, while the orientation is left as is for observations taken on the setting side of the mount (Figure 11).

The pre-processed quadrants as stored as 32 bit floating point numbers, and are fed to the Astromatic package SExtractor (Bertin & Arnouts 1996) to create a source catalog for the image and produce a corresponding background subtracted image. The detection threshold for point sources is set to 5σ in the de-trended image using the prescription in Zackay & Ofek (2017). Background subtraction is performed using a spatial median filter using a box size of 32 × 32 pixels (≈5 × 5' on sky) to remove the bright spatially varying background in J band. The source catalog is saved to support astrometric calibration in the subsequent steps. A background rms map is also generated during the same SExtractor run and saved to disk with locations of known bad pixels (see Section 4.11) masked. The background rms maps are used to generate inverse variance weight maps for stacking individual dithers into a single image.

Astrometric calibration is performed with respect to the Gaia DR2 catalog (Gaia Collaboration et al. 2018), using the pre-existing cross-match table between 2MASS and Gaia DR2 available as a part of the Gaia DR2 release (Marrese et al. 2019). The list of astrometric calibrators are stored as pre-partitioned static binary FITS tables ordered by field numbers since the observing is performed on a fixed sky grid. Only sources with 2MASS J magnitudes between 7 and 13 are used for astrometric calibration. In order to select a pure sample of stars (i.e., point sources), we require that all astrometric calibrators have a non-zero proper motion (>3σ) in the Gaia DR2, do not have a corresponding counterpart in the 2MASS Extended source catalog (Jarrett et al. 2000), and are not confused in the Gaia–2MASS cross-match solutions (i.e., number_of_mates = 0 and number_of_neighbours = 1). Figure 12 shows a distribution of the number of astrometric calibrators per field quadrant, showing a minimum of a few hundred astrometric calibrators.

Zoom In Zoom Out Reset image size

All image quadrants acquired as part of a single dither sequence on a field are astrometrically solved in a single run of Scamp (Bertin 2006). For the nominal observing strategy of 8 dithers per field,¹⁵ this involves the solution of 32 image quadrants per execution of Scamp. The astrometry is solved to derive a common astrometric solution for all the dithers in a given quadrant using a third order distortion solution. The distortion coefficients are stored using the TPV convention and written to the headers of the image quadrants, and recorded in the DB. Images for which the astrometric solutions fail are not processed further downstream. Astrometric failures are nearly zero for 70% of nights, while the highest observed failure rate can be ≈25% for images acquired through clouds and non-photometric conditions. We discuss the accuracy of the astrometric solutions in Section 5.2.

Image quadrants acquired in each dither sequence are stacked using the Drizzle algorithm (Fruchter & Hook 2002). Briefly, Drizzle resamples input images on to a user specified output grid with the option of shrinking each pixel by a user-defined parameter called pixfrac. pixfrac specifies the linear fraction by which each side of the input pixels are shrunk before co-addition. While smaller values of pixfrac produce sharper PSFs, smaller values of pixfrac result in uneven coverage in the output pixels if the dithers are not placed ideally. The pixel scale of the output grid is determined by the scale parameter, which determines the linear size of the output pixels with respect to the input pixels. Drizzle also produces an output weight image, which reflects the number of images that were sampled into each pixel during the resampling. In order to reduce artifacts associated with pixel shrinking and uneven coverage, Gonzaga et al. 2012 recommend σ_w/m_w < 0.15, where σ_w is the standard deviation in the resampled weight image and m_w is the median of the resampled weight image.

In the GDPS, the input images are resampled on to a fixed output grid with a pixel size half of the native pixels (≈43), corresponding to a scale of 0.5. The output grid is determined from the fixed sky grid of fields using astrometric solutions of on-sky images from the telescope. We use a pixfrac parameter of 0.9, that controls the shrinkage of the pixels before resampling on to the output grid (i.e., the raw pixels are shrunk to 90% of the size on each side before adding on the output grid). Smaller values of pixfrac produce uneven coverage over the smaller pixels in the output grid, resulting in a larger dispersion in the weight over the output pixels (see Gonzaga et al. 2012 for a discussion). As the mount pointing is not accurate enough to provide sub-pixel pointing adjustments during the dither sequence, we resort to using a random sampling of sub-pixel phases during the dither sequence. A pixfrac of 0.9 was found to be adequate to produce Nyquist sampled images that are limited by the focusing of the optics.

The astrometry-solved and background subtracted input quadrant images are resampled to the fixed output WCS grid using a python implementation of the Drizzle algorithm.¹⁶ In order to remove effects from cosmic rays, hot pixels, moving planes and satellites, the Drizzle code was modified to perform sigma-clipping on the resampled images to reject outliers on a per-pixel basis. The stacking of the images is performed using a sigma-clipped (at 2.5σ), inverse-variance weighted mean of the resampled input images. The stacked image quadrants are then split further into four sub-quadrants for photometry and image-subtraction downstream. The splitting is done as shown in Figure 11, including a 20 pixel (10 raw pixel) overlap between the sub-quadrants, while transforming the WCS between the parent quadrant and child sub-quadrants. Both the resampled image sub-quadrants and their corresponding weight images are stored on disk as floating point 32-bit images for long term archiving in a single Multi-Extension FITS image, and quality metrics are recorded in the DB.

Photometric calibration is performed against the 2MASS point source catalog (Cutri et al. 2003) cross-matched to Gaia DR2, using a subset of the sources that were used for astrometric calibration. The photometric catalog includes additional filters to select a list of isolated stars with J magnitudes between 9 and 16 that are not saturated in images. "Isolated" stars were defined to be sources that did not have any neighbors in 2MASS (regardless of any cuts in the astrometric catalog) within a radius of 12'' (3 Gattini drizzled pixels) to avoid confused sources. The stacked image sub-quadrants are fed as inputs to the photometric calibration pipeline. Figure 12 shows a distribution of the number of photometric calibrators per field quadrant. As in the case of the astrometric calibrators, the photometric calibrators are also stored as static pre-partitioned FITS binary tables per field on sky.

The stacked image sub-quadrants are fed as inputs to the photomteric calibration pipeline, which first creates a SExtractor catalog of detected sources along with a set of 15 × 15 pixel cutouts for each detected source. The SExtractor catalog is fed to PSFEx (Bertin 2011) to generate a PSF model of 15 × 15 drizzled pixels using the sources detected in the field. The PSF model is saved to disk for supporting difference imaging further downstream. The PSFEx model is then fed to a second run of SExtractor to generate a PSF-fit photometry catalog for the drizzled stack. Aperture corrections are computed between PSF-fit magnitudes and apertures of different sizes and recorded in the FITS headers. The PSF-fit catalog is used to select a list of unsaturated sources for photometric calibration, that are at least 40 pixels away from the edges of the image, and with SExtractor parameter FLAGS = 0 and FLAGS_MODEL = 0 and S/N > 10. The crossmatch proceeds only if there are at least 5 good cross-matched sources in the image.

The instrumental PSF and catalog magnitudes are fit with a linear solution of the form:

$$\begin{eqnarray}&&{m}_{\mathrm{TM},{\rm{J}}}-{m}_{\mathrm{ins}}=\mathrm{ZP}+c\,({m}_{\mathrm{TM},{\rm{J}}}-{m}_{\mathrm{TM},{\rm{H}}})\end{eqnarray} \tag{ 2 }$$

where m_TM,J and m_TM,H are the 2MASS magnitudes in J and H filters respectively, m_ins is the instrumental magnitude, ZP is the zero-point of the image and c is the color coefficient to convert from the Gattini system to the 2MASS (TM) system. The solution is derived by fitting a linear polynomial to the magnitude differences as a function of the source color (so the intercept is the zero-point and the slope is color coefficient). The extreme outliers (1 percentile) in the fit are eliminated first and a solution is derived. Subsequently, outliers that are more than 4 sigma away (typically <2% of the total number of stars) from the best-fit solution are clipped again and the final solution is re-derived. The photometric solution is recorded in the header of the image sub-quadrant and the DB, including quality metrics for data quality filtering. The PSF-fit source catalog is saved to disk and used to support light curve generation for sources detected across multiple-epochs. We discuss the accuracy of the photometric solutions in Section 5.3.

The pixel reconstruction and resampling procedure used in Drizzle leads to correlated noise between adjacent pixels in the output image. The noise correlation leads to underestimation of the photometric uncertainties in the PSF-fit source catalogs and correspondingly, an overestimation of the depths of the images. In order to estimate the correction to the noise rms in the images due to correlated noise, we use the prescription used for the WISE survey.¹⁷ The GDPS uses simulated white noise images that are drizzled to an output grid using the same dither pattern, pixfrac and scale parameters as in the GDPS operations to correct for the correlated pixel noise in the output photometric catalogs, estimates of image depths and the noise in the difference images produced downstream.

Briefly, a Monte-Carlo simulation is performed for estimating the PSF-flux uncertainty correction by calculating PSF-fit fluxes at random locations in the simulated drizzled images using the PSF model for the input image. The variance in these measurements are then compared to the flux uncertainties expected from uncorrelated pixel noise, and a noise rms rescaling factor is computed to inflate the uncertainties for the PSF-fit photometry fluxes. The same simulation is used to estimate the correction for the aperture photometry fluxes by measuring the aperture summed fluxes at random locations in the simulated drizzled images and computing the rescaling factor expected from uncorrelated noise. Figure 13 shows the estimated scaling of the noise rms of the image as a function of the aperture size used for photometry. The correction factor increases for large aperture sizes but flattens beyond a radius of ≈5 pixels due to the correlation length of the resampling process. The correction factors are used to correct the estimated limiting magnitude of the epochal stacked images and stored in the FITS headers of the calibrated image sub-quadrants and the DB.

Zoom In Zoom Out Reset image size

Photometrically calibrated science images are fed to the image differencing pipeline if a good quality reference image exists for the field and if the photometric solution quality flags suggest that the image was acquired under good observing conditions (see Section 5). The difference imaging pipeline starts by preparing the input science and reference images by cross-matching the PSF-fit source catalogs to compute the relative flux-scaling and astrometric uncertainty between the two images. Since both the science and reference images are resampled on to the same fixed sky grid, only a spatial differential background is subtracted from the input images before subtraction with the ZOGY algorithm (Zackay et al. 2016). The science and reference image uncertainty maps, the corresponding PSF models and the astrometric registration uncertainty are fed as additional inputs to the image subtraction.

The uncertainty maps are prepared by computing robust standard deviations (background noise) of the science and reference images and scaling them in regions of the image with low weight (smaller number of dithers) to inflate the noise maps accordingly. Source noise from the sources in the science and reference image are added to the uncertainty maps. Since the images fed into the ZOGY do not have uncorrelated pixel noise (due to the resampling performed by Drizzle), the uncertainty (rms) maps are multiplied with the rms scaling factor derived for scaling the PSF-fit photometric uncertainties. This factor reflects the noise correlation over the size scale of a PSF and hence is appropriate for the match filtering performed to produce the Scorr image, which denotes the statistical significance for point source detection (Zackay et al. 2016). A saturated source mask is generated using a 25 pixel box at the location of any sources in the science image with pixel values above 0.95 times the saturation limit of the image. We adopt a lower value than true saturation to account for nonlinearity effects near the saturation limit. An additional mask for bright sources in the field is prepared by querying the 2MASS point source catalog for sources brighter than 5th magnitude in the field. The size of the mask is adjusted such that brighter stars have larger masks around them, ranging from 81 pixel box masks for 5th magnitude sources to 181 pixel box masks for sources brighter than magnitude 0. A final bright source mask is produced by combining the saturation mask and the bright source mask from the 2MASS catalog.

The image subtraction produces a difference image, a difference image PSF and a corresponding Scorr image (Equation (25) in Zackay et al. 2016), which is a match-filtered S/N image optimized for point source detection. An initial quality check of the Scorr image is performed to ensure the ZOGY run did not fail before proceeding (i.e., there are not a significant number of NANs in the image¹⁸ ). Further quality checks of the subtraction are determined later in the pipeline using candidate counts. The final bright source mask is applied to the difference and Scorr image to remove saturation artifacts. The Scorr image is then fed to SExtractor to generate a list of sources with S/N > 5 in the match-filtered image, corresponding to sources that have peak Scorr values of >=5 in at least one pixel. The candidates are filtered to exclude image pixels with low weights (e.g., if the dither pattern did not sample the edge of a field uniformly, or if the region of the image is populated with many bad pixels). Additionally, candidates within 20 pixels of the edges of the image are excluded since they are already included in the adjoining sub-quadrant (or field) due to the overlap between the images earlier in the processing.

Quality metrics for the candidates are computed to detect bad subtractions from PSF-variation and astrometric residuals. Bad subtractions from positive–negative (yin–yang) residuals are found to be particularly severe in parts of the field where there are large gradients in the PSF. This arises due to a combination of poorly defined source positions in the presence of elongated and asymmetric PSFs, which leads to systematic residuals in the astrometric solution. Combined with the poor quality of PSF matching during the subtraction cross-convolution, this can produce a large number of subtraction artifacts unless filtered appropriately. We thus use the following criteria:¹⁹

• 1.  
  The ratio of magnitudes measured in a 4 pixel aperture and 8 pixel aperture is required to be between [0.4, 1.5] for a candidate to be saved. This criterion rejects yin–yang residuals resulting from large PSF variation across the images, as well as hot pixels.
• 2.  
  The ratio of the sum of pixels over the sum of their absolute values in a 3 × 3 median filtered image using a 7 × 7 pixel cutout centered at the source location is required to be ≥0.4 for a candidate to be saved. This criterion is effective at rejecting yin-yang residuals from large PSF variation in the images. An additional ratio of pixel sums using a variable box size which depends on the image FWHM, PSF asymmetry, and relative astrometric uncertainty at the source location is also calculated and stored for later filtering.
• 3.  
  The number of pixels 5σ below the median value of the difference image contained in a 9 × 9 pixel box centered at the source location is determined. Candidates are rejected if the count is >1 and the ratio of pixel sums with variable box size in #2 is <0.8. This cut is only applied for sources with measured magnitudes >10 to avoid rejecting bright sources which can produce significant "ringing" in the difference image as a result of the noise decorrelation process in ZOGY.
• 4.  
  The ratio of the flux of the candidate in the difference image and the reference image. Cross matching between sources in the difference and reference image is performed using a variable radius that is a function of the image FWHM, PSF asymmetry, and relative asymmetric uncertainty at the source location. No cuts are performed solely using this metric.

Values for these metrics were determined by performing tests with injected fake sources, which is described in Section 5.6. This process is also performed for the corresponding negative difference and Scorr image to find sources that have faded from the reference image. An additional filtering of positive–negative source pairs is performed to remove extended residuals caused by astrometric residuals or extreme cases of PSF-variation between the new and reference image. Cross matching between the positive and negative source catalog is performed using a variable radius that is a function of the image FWHM, PSF asymmetry, and relative asymmetric uncertainty at the source location. Candidates are discarded if a positive–negative cross match is found and the ratio of the source flux in the difference and reference image (item 4) is less than 1. Together, these quality metrics reject ≈75% of total candidates generated on a typical night.

PSF-fit fluxes are measured for each source in the Scorr image by fitting the difference image PSF model on the location of the source detected in SExtractor. The position of the source is refined in the fit by χ² minimization of the residuals from the PSF model. Although the ZOGY algorithm is designed to produce difference images with uncorrelated pixel noise, this does not hold in cases where the input images have correlated noise in them. Hence, in order to correctly estimate the difference photometric uncertainties, we perform a Monte Carlo simulation by estimating the variance of PSF-fit fluxes over a simulated difference image containing only noise, and scale the PSF-fit photometric uncertainties from the difference image accordingly. As an additional filter of bad quality candidates, we require the absolute value of the difference between the measured PSF-fit magnitude and 8 pixel aperture magnitude be ≤0.4. Last, a final quality check is performed on the difference image using the total number of "good" candidates, both positive and negative, found as compared to the total number of objects in the source catalog of the science image. If this value is >0.2, the image is flagged as poor quality and no candidates are saved. Otherwise, the values for aperture photometry, PSF fitting, and other quantities described in this section are recorded in the DB for downstream filtering. Image cutouts (61 × 61 pixels; 44 on each side) are recorded in the DB around the location of the source in the science, reference and difference image for ML based classification and human vetting externally.

To automatically distinguish between an astrophysical source and image subtraction artifacts, we use a ML based real-bogus (RB) classification scheme. The GDPS uses a RB classifier scheme implemented through supervised Deep Learning where features are extracted from an input set of candidate sources using many-layered perceptrons (artificial neural networks). The Deep Learning based classifier design was adopted from the classification system implemented for the ZTF (Duev et al. 2019; Mahabal et al. 2019). The classifier was trained by assembling a training set of separately labeled real and bogus data by human classifiers. Bogus candidates were compiled using a labeling scheme on Zooniverse, a citizen science web portal which allows set up of individual projects usually pertaining to classification and data visualization.²⁰ Real sources were selected based on a sample of known variable stars, SNe and asteroids found with human vetting during the commissioning period.

We used a two-layer Convolutional Neural Network (CNN) for our Deep Learning model, as CNNs are commonly used in analyzing visual imagery and have numerous benefits over standard multi-layer perceptrons (Krizhevsky et al. 2012). This model is implemented using the TensorFlow package (Abadi et al. 2016) and the high-level Keras API.²¹ The model has two convolutional layers, one flatten layer and two fully-connected layers. The first convolutional layer uses 32 3 × 3 filters with a Refined Linear Unit (ReLU) activation function, and is followed by a maxpooling layer of size 2 × 2. The second convolutional layer uses 64 5 × 5 filters with a ReLU activation function, and is followed by a maxpooling layer of size 4 × 4. Dropout layers with rates of 0.25 are included after each convolutional layer to minimize over-fitting. After a flatten layer, there is one fully-connected layer of size 32 with a ReLU activation function, followed by a dropout layer of rate 0.40. The final output layer is a fully-connected layer with a sigmoid activation function for binary classification (real or bogus), amounting to a total of 5 layers.

The model is trained and run on stacks of images of shape 61 × 61 × 3, consisting of science, reference, difference cutouts (of size 61 × 61 each). We utilized a training/validation/test split of a 72%/8%/20%. While training the model, we used a batch size of 30 and utilized the early stop method at 20 epochs as there was no improvement in validation accuracy and an increase in validation loss past that point. We used K-fold cross validation technique with k = 10 to reduce bias and prevent over-fitting. The performance of the model was evaluated using the following metrics: accuracy on the test set of 0.975, a Matthews correlation coefficient of 0.949 and an F1 score of 0.977. Figure 14 presents the normalized confusion matrices for the model, showing a false positive rate under 5% and false negative rate under 1.5%. The ML model has been deployed for regular use after extensive testing on unseen survey data. A complete description of the ML architecture will be presented in a forthcoming publication (M. Sharma et al. 2019, in preparation).

Zoom In Zoom Out Reset image size

The ML model is currently being tested on unseen data and will be refined before final deployment.

Transient sources recorded in the DB are cross-matched to external catalogs for supporting follow-up prioritization using the a dedicated database server at Caltech (Duev et al. 2019). In order to filter on variable stars, extracted sources are cross-matched to the 2MASS point source catalog (Cutri et al. 2003) and the distances and J magnitudes of the three nearest sources are recorded. The same is recorded for the three nearest sources in the reference image for the respective field sub-quadrant. Sources are also cross-matched to the PS1 DR2 catalog (Chambers et al. 2016) including the star–galaxy classification scheme described in Tachibana & Miller (2018), to store the distances, magnitudes and star–galaxy classification score of the three nearest objects. Additional recorded metadata include cross-matches to known solar system objects using the astcheck²² utility, the ZTF public alert archive (Masci et al. 2019), the Census of the Local Universe (CLU) catalog of nearby galaxies (Cook et al. 2019), Gaia DR2 and previously known objects in SIMBAD.

The cross-matches are performed on the database node and recorded in the DB, which is accessible across the Caltech internal network for subsequent human filtering and vetting. Human vetting is performed via a dedicated scanning page on the GROWTH marshal (Kasliwal et al. 2019) showing the science, reference and difference image cutouts of detected sources together with their metadata. The scanning page is created via read-access to the DB on the database node, and allows the user to enable several metadata-based filters on the detected transients, including the distance to the nearest 2MASS source (to reject variable stars), ML RB score, the presence of a ZTF counterpart and proximity to a nearby galaxy in the CLU catalog. Candidates are assigned a survey name once they are saved by a human scanner, and are followed up with imaging and spectroscopy via requests on the GROWTH Marshal.

4.11.1. Reference Image Generation

4.11.2. Flat Image and Dead Pixel Mask Generation

4.11.3. Dark Image and Hot Pixel Map Generation

4.11.4. Match File Generation

Figure 15 shows a distribution of the success rate of the real-time pipeline processing in producing (i) photometrically calibrated stacked data products on disk, and (ii) in producing a usable subtraction for transient extraction. The success rate distribution was measured from all nights in 2019 August. The median success rate of the production of photometrically calibrated stacks is 99.2%, while it is 99.4% for subtractions and transient extraction. The primary issue affecting the success rate of photometric stacks was observations taken under non-photometric conditions (e.g., through clouds, or through high humidity) leading to a significant fraction of astrometry and photometry failures (up to ≈30% of images on one of the nights in this period). The situation with regard to periods of high humidity is expected to improve with the recent installation of a window heater to avoid the accumulation of condensation.

Zoom In Zoom Out Reset image size

Figure 15 also shows a distribution of the elapsed time between the end of a field observation at the telescope and (i) a photometrically calibrated stacked image being available on disk and (ii) availability of transient candidates in the DB for human vetting. The median time between the end of an observation and the generation of a photometrically calibrated image is ≈2 hr, while all images are generally processed within ≈4 hr from the end of an observation. Transient candidates are available within a median time of ≈3.8 hr, although the distribution of elapsed time has a long tail extending out to ≈12 hr. This primarily occurs due to fields in the Galactic plane where the high source density leads to the detection of a large number of candidates that strain the subsequent steps of PSF-fitting, photometry and external cross-matches. However, all candidates are available well before the start of the next night (the pipeline completes no later than noon of the following day on a typical night), and thus the processing is well suited for human vetting of transient candidates within a day of the observation and subsequent follow-up assignment. By benchmarking the data processing rate, we find that the time delays are limited by the availability of compute power so that the system processes data ≈1.5× slower than the data acquisition rate. Thus, the complete data processing chain takes ≈12–13 hr to complete for a typical ≈8–9 hr observing night, leading to the median ≈4 hr delay between data acquisition and candidate availability.

The astrometric calibration method described in Section 4.4 leads to astrometric solutions which have typical accuracies of ≈08 (≈0.1 pixels) over the entire sky. Figure 16 (left panel) shows the distribution of the astrometric rms (per axis) achieved over a range of airmasses and observing conditions during all nights in the month of 2019 August, as measured from the astrometric solutions of the native images (after de-trending) using Scamp for sources with S/N > 10. We show an equivalent plot for the stacked images produced using Drizzle on the right panel of Figure 16, plotting the distribution of the median radial separation of sources detected in the Drizzled images cross-matched to the Gaia DR2 reference catalog. The median radial separation is ≈07.

Zoom In Zoom Out Reset image size

Note that Figure 16 shows the astrometric accuracy of all sources detected above S/N > 10, where the astrometric measurements are prone to Poisson errors of centroiding for sources near S/N ≈ 10. The true achievable precision is higher for the case of brighter sources and is depicted in Figure 17, showing the median radial separation of sources from the reported Gaia DR2 position as a function of the source magnitude, both for a high Galactic latitude and a low Galactic latitude field. For sources brighter than ≈13 Vega mag, the astrometric precision achievable is better than ≈04 (0.05 native pixels) and is representative of the achievable astrometric precision. The astrometric precision is likely limited by the measurement of accurate source positions in the presence of asymmetric and variable PSFs in the final stacked images.

Zoom In Zoom Out Reset image size

Section 4.6 describes the photometric calibration procedure for stacked images against the 2MASS catalog. Figure 18 shows the distribution of the difference between 2MASS and Gattini calibrated magnitudes (including the zero-point and a color term) measured from the epochal PSF-fit source catalogs for a stacked field sub-quadrant. The residuals are plotted as a function of the source magnitude combined over 20 visits of a high and low Galactic latitude field respectively. The left panels show the distributions for a high Galactic latitude field (low source density) and the right panels are for a low Galactic latitude field (high source density). The photometric scatter rms increases for fainter sources and up to ≈20% for sources near the 5σ limiting magnitude. The orange squares denote the median residual in bins of 1 magnitude, which show evidence of a small systematic deviation at the brightest and faintest ends of the distribution, likely due to uncorrected nonlinearity in the detector pixel response.

Zoom In Zoom Out Reset image size

The bottom panels show the dependence of the median flux rms residuals against 2MASS as a function of the source magnitude, binned into groups of 1 magnitude. The best achievable photometric accuracy is ≈3% (30 mmag) at the bright end (brighter than ≈11 mag), while it is better than ≈5% for sources brighter than ≈13 mag. The scatter is believed to be largely dominated by errors in background estimation, flat-fielding and PSF fitting over the field sub-quadrant (note that the PSF variation over a single field sub-quadrant can be significant at the edges of the focal plane).

The Nσ limiting magnitude of each stacked image can be estimated using either an estimate of the median background rms in the image and the size of the PSF, or measuring the observed magnitudes for sources in a narrow range of S/N around Nσ. The GDPS measures the limiting magnitude of each image using both methods, while including the corrections for correlated noise in the image pixels (Section 4.7). Figure 19 shows the relative flux uncertainty for sources detected in a single epoch field visit of a high Galactic latitude field (left) and a low Galactic latitude field (right). The 5σ limiting magnitude corresponds to a relative flux uncertainty of 20%. The vertical red dashed lines correspond to the limiting magnitude estimated using the background rms and PSF size information, while the horizontal red dashed line marks the location of 20% flux uncertainty. The intersection of the two lines overlap with the contour of sources in the flux uncertainty plane suggesting consistent limiting magnitude estimates from the two methods.

Zoom In Zoom Out Reset image size

As seen in Figure 19, the depth of the image for the low Galactic latitude field is shallower than that for the high Galactic latitude field as a result of confusion noise in regions of very high source density, given the large pixel scale of the detector. Confusion noise generally degrades the limiting magnitudes (estimated using the the magnitude of sources near S/N = 5) of observations in the Galactic plane fields. In Figure 20, we show a distribution of the 5σ limiting magnitudes of images taken over nights in 2019 August near the Galactic plane ($\left|b\right|\lt 20^\circ$) and outside the plane ($\left|b\right|\gt 20^\circ$). The median 5σ limiting magnitude of images outside the plane is ≈15.7 AB mag while the same for low Galactic latitude fields is ≈15.3 AB mag. The low Galactic latitude fields show a long tail in the distribution extending to shallow limiting magnitudes, corresponding to the most crowded fields in the Galactic plane.

Zoom In Zoom Out Reset image size

The limiting magnitude of the stacked field sub-quadrants also depends on its position in the plane of the detector due to large variations in the PSF size and shape across the detector. Larger PSFs lead to shallower limiting magnitudes given the high sky background. The top panels in Figure 21 show the variation of the PSF FWHM and ellipticity measured as a function of position on the detector, averaged over all nights of observations in 2019 August. Sub-pixel image quality has not been achieved with the designed optical system. In the absence of a focusing mechanism (due to be installed in 2019), variations in the PSF FWHM and ellipticity have been observed as a function of the ambient temperature. The typical PSF FWHM size is ≈12''–14'' (≈1.5–1.7 detector pixels), although it is worse (up to ≈17'' ≈2 detector pixels) at the edges of the focal plane. As a result, the use of Drizzle with an output pixel scale half of the native pixel scale has been adequate to reconstruct Nyquist sampled images.

Zoom In Zoom Out Reset image size

The lower panels in Figure 21 show the median limiting magnitude (including both low and high Galactic latitude observations) and zero-point as a function of position in the detector plane. The zero-point of the detector is largely uniform across the plane except for the bottom right corner, where the zero-point (sensitivity) is lower by ≈0.05 mag due to the absence of a AR coating. The distribution of limiting magnitude as a function of position in the detector plane is influenced by a combination of the local zero-point and PSF size, such that larger zero-points and smaller PSF FWHMs lead to deeper limiting magnitudes. Since the sky background from the PSF footprint is the dominant noise contribution limiting the image depths, the depths of the images are expected to improve with the installation of a focus mechanism in the second half of 2019.

Section 4.11.1 discusses the generation of reference images by stacking all exposures of a given field acquired under photometric conditions. Although reference image availability was limited during the commissioning phase of the survey due to extended periods of bad weather and poor observing conditions, reference maps were re-built at the end of 2019 June with data acquired between 2018 November and 2019 June. The resulting reference maps have 99.3% coverage of the visible sky from Palomar and the sky distribution of the reference limiting magnitudes are shown in Figure 22. The number of images stacked in each field varies between 40 individual dithers (5× the nominal survey field visits) and 300 individual dithers, with fields near the north pole having the largest number of images stacked due to their near continuous visibility from Palomar Observatory. The resulting limiting magnitudes of the reference stacks vary between ≈14.5 AB mag in crowded regions in the Galactic plane (where the depths are limited by confusion) to ≈18 AB mag outside the plane. The reference images are deeper than 16.5 AB mag for 60% of the sky.

Zoom In Zoom Out Reset image size

After applying the quality cuts to the candidates generated from the subtractions, the difference imaging pipeline typically generates ≈25,000 candidates each night (median ≈7 candidates per sub-quadrant), noting that the total number of candidates generated can have a large dispersion depending on the Galactic plane coverage and total observing time (which changes over the year). In order to evaluate the efficacy of the difference imaging pipeline we performed various tests with injected fake sources. First, a set of test data was created using a copy of the nightly survey data. A total of 992 fake sources were inserted in these images using a PSF model for the image generated using PSFex and scaled to a specific magnitude. The magnitude of the fake sources were randomly drawn from a uniform brightness distribution between 0 and 5 magnitudes above the limiting magnitude of the science image. The positions for the fake sources were randomly drawn from a uniform distribution of x and y coordinates in the image, with checks to ensure that sources were not placed in low-weight portions of the maps (i.e., regions near the edge which were not sampled by at least half the dithers of the dither sequence). The subtraction pipeline was run on these test images to check the fractional source recovery, which are shown in Figures 23 and 24.

Zoom In Zoom Out Reset image size

Injected sources which were missed by the pipeline can be broadly characterized into six groups: sources that were flagged as saturated (2), sources that were coincident with the masked area of a nearby bright star (11), sources masked as possible artifacts from the missing AR coating on a corner of the detector (23), sources which failed to achieve a peak Scorr value of 5 (38), sources contained in images that failed quality criteria for candidate extraction (2), and sources that were initially detected but cut as part of candidate filtering criteria (17). The values listed after each category indicate the number of missed sources in that category. In total, 899 sources out of 992 possible sources were recovered, yielding an overall recovery fraction of 90.6%.

Zoom In Zoom Out Reset image size

The recovered magnitude for the injected sources were compared with their injected magnitude to ensure consistency. Additionally, the coordinates of the injected sources were compared with their corresponding recovered candidate coordinates. The results of these tests can be found in Figures 25 and 26. The photometric precision for sources brighter than 13th Vega mag is ≈3%, while the astrometric rms is 09 in each axis for all recovered sources (down to S/N = 5).

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

In addition to the first set of test data, we performed a second evaluation of the subtraction pipeline to assess its efficacy for finding sources in close proximity to nearby galaxies. An additional set of test data was generated using the same procedure described above, except that the positions for the fake sources were randomly selected to be within a 30'' radius from nearby galaxies that are part of the CLU (Cook et al. 2019) catalog. The subtraction pipeline was run on this test data to check the rate of injected sources recovered as a function of the surface brightness at the location of the injected source. The test yielded what was at first an unexpected result, because the sky background level at J-band dominates over the galaxy light. This makes the "standard" test of surface brightness versus recovery somewhat more difficult to interpret. Overall, the performance of this test was comparable to the earlier randomly placed fake sources. The pipeline found 366 out of 382 possible sources. This slightly improved rate of 95.8% sources found is likely due to the fact that CLU galaxies are found in regions where the J-band source density is not particularly high, unlike the Galactic plane fields which were included in the first test. Regions with high source densities make transient detection much more difficult because the data is typically not as deep due to the effects of confusion and there is higher chance of being masked by coincidence. Taking these into account, the improved performance of the second test makes sense and bodes well for finding transients in nearby galaxies with Palomar Gattini-IR.

Since the start of the commissioning period, several bright SNe and extragalactic transients were recovered in the Gattini transient stream. These include the SNe II 2018hna, 2019hsw and SN Ia 2019np. In addition, large amplitude NIR flaring was detected from several high redshift blazars, a subset of which were followed up with optical/NIR spectroscopy on the Palomar 200 inch telescope. In Figure 27, we show a collage of the J band light curves and spectra of extragalactic transients detected in the commissioning phase. SN 2018hna was reported by K. Itagaki to the Transient Name Sever (TNS) early in the commissioning phase of Gattini-IR and brightened to be detectable in the NIR soon after explosion. Gattini observed the field as a part of regular survey operations with photometry capturing the entire rise to the radioactive peak and subsequent decline of this SN 1987A-like SN II (Figure 27, top left panel). SN 2019hsw (ASASSN 19pn/PGIR 19jg) is a SN II at 25 Mpc detected in the Gattini-IR commissioning data, and shows a slow J band decline at early time (Figure 27, top left panel). The SN went close to the Sun soon after discovery and was not covered as a part of the nightly Gattini-IR observations. SN 2019np (PGIR 19ayh) was a nearby SN Ia reported to TNS (Itagaki 2019) and subsequently detected in Gattini data. Only two detections of the SN were recovered due to an extended period of poor weather surrounding the detection of the SN. PGIR 19c is a large amplitude flaring blazar (B1420+326) detected as a transient over several weeks of operations (Figure 27, top right panel). The fast variability and large amplitude flaring detected in the Gattini-IR data were announced via the Astronomer's Telegram (De et al. 2019c). Gattini-IR detected a new NIR flare of the blazar CTA 102 (Figure 27, top right panel), which was saved with the internal name PGIR 19ayd and announced publicly (De et al. 2019b).

Zoom In Zoom Out Reset image size

Figure 27 also shows light curves of several Galactic transients detected in the data—highly reddened Galactic novae AT 2019qwf and AT 2019owg (Figure 27, bottom left panel), the outbursting X-ray binary MAXI J1820+07/ASASSN-18ey (Tucker et al. 2018; the NIR brightening was reported in Hankins et al. 2019d) and the dwarf nova PGIR 19tf (Figure 27, bottom right panel). AT 2019qwf (PGIR 19brv) was first discovered and reported as a bright (≈11 mag) NIR transient at Galactic latitude of 02 by Gattini-IR, and classified as a Galactic nova with optical spectroscopy (De et al. 2019e). Another reddened classical nova AT 2019owg (initially reported as Gaia 19dum to TNS) was detected as a bright NIR transient (at ≈8th mag), close to the saturation magnitude of the instrument (De et al. 2019a). Between 2019 July and September, Gattini-IR detected a total of four Galactic classical novae—V3890 Sgr, Gaia 19dum/AT 2019owg, PGIR 19brv/AT 2019qwf and V2860 Ori. Both MAXI J1820+07 and MAXI J1807+32 were detected as NIR transients coincident with a brightening detected in the optical wavebands. After re-processing older data taken during initial commissioning, a previous outburst of MAXI J1820+07 in 2019 was also recovered. Several recurrent outbursts from the dwarf nova Ay Lyr (PGIR 19tf) were detected in the commissioning, and are shown in Figure 27. Gattini-IR detected a reddened binary microlensing event in the Galactic plane (PGIR 19btb/Gaia 19dqj/AT 2019odt) which was announced in De et al. (2019d).

Figure 27 also shows a collage of optical (from P60 + SED Machine/P200 + DBSP) and NIR spectra (from P200 + TripleSpec) obtained for these transients detected during the commissioning phase. PGIR 19jg and PGIR 19hj exhibit typical features of SNe II, including broad P-Cygni lines of H, He, O i and Fe ii. A peak light spectrum of PGIR 19ayh shows typical features of SNe Ia near peak—Si ii, S ii, Ca ii and Fe ii. PGIR 19brv (AT 2019qwf) was followed up with rapid low resolution spectroscopy on the SED Machine spectrograph on the Palomar 60 inch telescope (Blagorodnova et al. 2018). The spectrum showed a reddened continuum and strong emission lines of H and O, classifying this source as a reddened Galactic classical nova (De et al. 2019e). We also obtained a NIR spectrum of the reddened Galactic nova AT 2019owg, which shows broad emission lines of He and H along with strong emission lines of O I. The NIR spectrum of Ay Lyr (PGIR 19tf) shows several narrow absorption lines of H, typical of dwarf nova outbursts.

Given the large FOV of Gattini, it has also been performing targeted follow-up of the localization regions of several alerts announced by LIGO/Virgo in O3. Gattini has demonstrated the capability to tile large fractions of the error regions of the localization regions, ranging from 32% of the poorly localized single detector detection of LIGO/Virgo S190425z (Coughlin et al. 2019a), and of the localization region of the candidate NS-BH mergers S190426c (92%; Hankins et al. 2019a, 2019b) and S190814bv (89.5%; Hankins et al. 2019c). In the case of S190426c, Gattini tiled the localization region a total of ≈20 times over the course of one week after the merger, while each field in the localization region of S190814bv was observed for ≈2.5 hr during one week after the merger. Given the longer timescale (∼1 week) of the infrared emission in kilonova counterparts (Kasen et al. 2017), stacking multiple epochs of data will allow the first constraints on infrared emission from compact binary mergers independent of optical searches.

As a part of nominal survey operations, Gattini-IR will obtain J-band light curves of sources brighter than J ≈ 16 AB mag. The photometric measurements from these observations are readily available in the epochal PSF-fit source catalogs, which are cross-matched across epochs to produce match files for every detected source. In order to demonstrate the quality of light curves and the variable science potential of Gattini, we ran a blind period search algorithm on all sources detected more than 30 times. In Figure 28, we show a sample of periodic variables recovered from the blind period search, some of which already had known variable counterparts in SIMBAD. These include a candidate short period (≈6 hr) eclipsing binary, an RR-Lyrae type variable showing a distinct saw-tooth shaped light curve, a Cepheid variable and a BY Draconis type variable, demonstrating the photometric quality of the data and the capability for blind period searches.

Zoom In Zoom Out Reset image size

Figure 29 also shows light curves of Galactic variables that were detected in the subtraction pipeline—a candidate brown dwarf and a known Young Stellar Object (YSO). While the brown dwarf is undetected in the optical due to its red color, the variability from the YSO is undetected in the optical due to extinction. Operating in J band, Gattini will probe variability in the coolest and dustiest stars in the galaxy (such as brown dwarfs and asymptotic giant branch stars) that are bright in the infrared but faint in the optical. It will also be particularly sensitive to stellar variability in the most dust extinguished lines of sight in the Galactic plane where optical time domain surveys become insensitive.

Zoom In Zoom Out Reset image size