Multiparametric scaling of diffraction intensities

Otwinowski, Z.; Borek, D.; Majewski, W.; Minor, W.

doi:10.1107/S0108767303005488

research papers

FOUNDATIONS
ADVANCES

ISSN: 2053-2733

Volume 59| Part 3| May 2003| Pages 228-234

doi:10.1107/S0108767303005488

Multiparametric scaling of diffraction intensities

Zbyszek Otwinowski,^a ^* Dominika Borek,^a Wladyslaw Majewski ^a and Wladek Minor ^b

^aDepartment of Biochemistry, University of Texas, Southwestern Medical Center at Dallas, TX 75390-9038, USA, and ^bDepartment of Molecular Physiology and Biological Physics, University of Virginia, Charlottesville, VA 22908, USA
^*Correspondence e-mail: zbyszek@work.swmed.edu

(Received 18 December 2002; accepted 1 March 2003)

A novel and general approach to scaling diffraction intensities is presented. The method minimizes the disagreement among multiple measurements of symmetry-related reflections using a stable refinement procedure. The scale factors are described by a flexible exponential function that allows different scaling corrections to be chosen and combined according to the needs of the experiment. The scaling model presented here includes: scale and temperature factor per batch of data; temperature factor as a continuous function of the radiation dose; absorption in the crystal; uneven exposure within a single diffraction image; and corrections for phenomena that depend on the diffraction peak position on the detector. This scaling model can be extended to include additional corrections for various instrumental and data-collection problems.

Keywords: scaling; absorption; diffraction; exponential modelling.

1. Introduction

A natural starting point in the complex analysis of measurements is to define the data model that predicts observations based on the physics of the experiment. The purpose of data analysis is to solve the inverse problem – how to obtain the parameters of the physical model from the measurements. In crystallography, the inverse problem has many steps, and this article addresses one of them – determination of the scaling model in the diffraction data reduction process.

The basic equation that describes measured intensity of an hkl reflection is (Guinier, 1994 $[Guinier, A. (1994). X-ray Diffraction in Crystals, Imperfect Crystals, and Amorphous Bodies. New York: Dover.]$ )

$[I(hkl) = I_b r_e ^2 {{\lambda ^2 } \over {| {\bf S} \times \boldomega '|}}PT{V \over {v_u ^2 }}| {{\bf{F}}({hkl})} |{}^2 D_A D_S, \eqno(1)]$

where I_b is the flux density of the primary beam; r_e = e² /m_e c² is the classical electron radius (2.818 × 10⁻¹⁵ m); $[\lambda]$ is the wavelength; $[{\bf{S}} \times {{\boldomega '}}]$ is the cross product between the diffraction vector $[{\bf{S}}]$ and $[{{\boldomega '}}]$ (projection of the crystal rotation-speed vector $[{{\boldomega }}]$ on the plane perpendicular to the primary beam), term $[| \boldomega | / | {\bf S} \times \boldomega ' |]$ is known as the Lorentz factor; P is the polarization factor (Azároff, 1955 ); T is the transmission [absorbance is defined as A = - lnT, T = exp( - A)]; v_u is the volume of a primitive crystal unit cell; V is the volume of the crystal exposed by the beam; $[| {{\bf{F}}({hkl}) } |{}^2]$ is the square of the structure-factor amplitude for a given reflection hkl; D_A is the X-ray absorption in the detector's active material; D_S is the detector's response to a single absorbed X-ray photon; it may depend on the wavelength, the position in the detector, the incidence angle etc.

To derive $[| {{\bf{F}}({hkl})} |{}^2]$ , all components of (1) need to be determined. The product of all factors that multiply the structure-factor amplitude squared is the total scale factor:

$[\eqalignno{I_{\rm measured} &= K | {{\bf{F}}({hkl})} |{}^2, &(2)\cr I_{\rm scaled} &= (1 /K) I_{\rm measured}, &(3)}]$

where I is the intensity and K is the total scale factor.

In principle, all scaling components in (1) can be calculated from non-diffraction measurements and calibration of the data-collection system. The total scale factor of a particular observation can be expressed as

$[K = k_o k_r = k_o ({k_{\rm beam} k_{\rm polarization} k_{{\rm{detector}}} \ldots}). \eqno(4)]$

Typically, only relative scale factors, k_r, are calculated and an overall scale factor k_o, common to all measurements, is determined by comparing the scaled data to the squared structure-factor amplitudes predicted from an atomic model. The overall scale factor describes the typical lack of knowledge about the absolute calibration of the whole system (Evans, 1993 ).

We always assume that some components of scale factors are known from detector calibration, beam monitoring and the diffraction geometry; k_c represents their product. The scaling procedure described here determines only the remaining part of the relative scale factors, k_s:

$[K = k_o k_c k_s. \eqno(5)]$

In many cases, quantities needed to describe k_s are calculated based on the knowledge from subsequent stages of crystallographic analysis. In such cases, scaling and subsequent steps should be done iteratively, with reasonable guesses for starting values (see §§4.1 and 4.3).

2. General scaling method

The method presented here is a generalization of scaling methods where parameterized scaling corrections were applied, for example, using scale and temperature factor per batch of data (Fox & Holmes, 1966 ; Stuart & Walker, 1979 ; Evans, 1993). The presented method applies to crystallographic scaling the concept of exponential modeling (Della-Pietra et al., 1997 ). Scale factors for individual measurements are calculated using a set of parameters p_i:

$[k_s({\rm observation}) =e^{\left[{\textstyle\Sigma}_i \,p_i\, f_i ({\rm observation})\right ]}. \eqno(6)]$

f_i is a pre-defined modeling function of experimental conditions for a given observation and p_i specify the a priori unknown parameters. The i is shorthand for the hierarchical index, which describes here both the type of correction for a given physical effect and indices of parameters for functions describing this correction. Some of the functions f_i in (6) can be derived directly from the description of a particular physical effect, for example an error of individual amplifier gain in multireadout channel CCD detectors (see §2.1.2). To represent an unknown but smooth-shaped function, for example one describing absorption in the crystal (see §2.1.3), the combination of the basis functions f_i and a group of parameters can be used. The values of smooth functions rather than the values of the p_i coefficients can be interpreted in physical terms.

The advantage of the exponential modeling approach is its flexibility when dealing with correlations among parameters and a uniform description of the parameter optimization process for various scaling models.

2.1. Parameterization

Scale factors are used to describe crystal diffraction, the experimental system and to correct for potential approximation errors during the integration step.

On the crystal diffraction level, the main problem is absorption in the crystal, before and after diffraction (Kopfmann & Huber, 1968 ; Huber & Kopfmann, 1969 ; Stuart & Walker, 1979; Schutt & Evans, 1985 ).

Typical instrumental problems are: beam fluctuations (Stupakov & Heifets, 2002 ); the error in the position of the rotation axis; inaccuracy of detector calibration (Tate et al., 2000 ); mechanical problems resulting in non-uniform crystal rotation and shutter errors (Evans, 1993).

Integration programs make certain assumptions about diffraction spot shape and size. Crystal mosaicity and changing spot shape may result in systematic underestimation of the diffraction intensity owing to reflection tails extending past the assumed shape. Imperfect profile shape predictions also result in systematic effects (Diamond, 1969 ; Leslie, 1999 ).

Parameterization of a particular source of an effect described by a scale factor may be quite effective in describing other physical phenomena or approximation errors, and for this reason refined values of parameters may not have a simple physical interpretation. For example, beam intensity corrections may also correct for changes in exposed volume of the crystal.

2.1.1. Error in the Lorentz factor

An example of the application of a modeling function is correcting for a small discrepancy between the actual and assumed directions of the crystal rotation axis. The inaccuracy results in an error in the calculated value of the Lorentz factor. The scale factor to correct this error can be described using the parameter p_l, whose value represents a small angular error, and the corresponding function is

$[f_l = {\bf S} \cdot {\boldomega '} / | {\bf S} \times \boldomega ' |. \eqno(7)]$

2.1.2. Corrections that depend on the Bragg-peak position on the detector

Detector-specific corrections are important for measurements of a very weak phasing signal. A very small correction, of about 1% in magnitude, can be critical for phasing from weak anomalous scatterers. Even if the detector sensitivity has been properly calibrated, it may have changed with time. For detectors with multichannel read-out, the most likely source of the above problem is a relative change in amplifier gains. The amplifier corrections are represented by characteristic functions corresponding to CCD areas read out by separate channels:

$[\eqalign{f_{da,j} = \cases{1 &for data read out by channel {\it j}\cr 0 &for data read out by other channels.}}\eqno(8)]$

The parameters p_da,j are logarithms of relative changes in amplifier gains.

Other detector corrections require a smooth function that can be described by a series of basis functions. One possibility is to use two-dimensional Fourier–Bessel series (Weissman, 1982 ); alternatively, one can use two-dimensional Chebyshev polynomials or two-dimensional cosines (Boyd, 1991 ). The last option transforms detector coordinates (x,y) onto the square $[[{({0,0}),({\pi, \pi })}]]$ . For transformed values of coordinates, the modeling function is

$[f_{d,n,m} = \cos ({nx})\cos ({my}), \eqno(9)]$

where d,n,m are components of the hierarchical index i from (6).

The above functions have been used to effectively correct for the signal decay during the imaging plate read-out. To correct for differences between plates in two-plate scanners, an additional index is necessary to describe separate corrections for individual plates.

Smooth functions dependent on the spot position might correct also for other phenomena, e.g. the non-rotating component of absorption, the incorrect polarization and systematic errors in integration. In diffraction data collected with a large oscillation range from mosaic crystals, reflections diffracting along the rotation axis have a distorted spot shape. Such spot-shape variability results in an integration error that decreases with the distance from the projection of the rotation axis onto the detector. A function that was found useful in correcting for this effect is

$[f_c = {{a|\, {f_l }|} /({a + | \,{f_l }|}}), \eqno(10)]$

where f_l is a function describing error in the Lorentz factor [equation (7)] and a is a stabilizing factor (reasonable value a = 5).

In the case of anisotropic mosaicity, this correction can be made asymmetric with respect to the sign of f_l (left/right distinction for horizontal rotation axis). This can be accomplished by using two functions instead of the function in (10):

$[\eqalign{ f_{c1} &= step({f_l }) f_c ,\cr f_{c2}&= step({ - f_l }) f_c ,} \eqno(11)]$

where step is a step function with zero value for negative arguments and one for positive.

2.1.3. Correction for the absorption component that rotates with the crystal

Owing to a different geometry of diffraction for every reflection, the absorption in the crystal is not uniform. A frequently used approximation of crystal absorbance is the average of absorbances in the incoming- and diffracted-beam directions (Kopfmann & Huber, 1968). The crystal absorption can be parameterized by real spherical harmonics in the rotating crystal coordinate system (Katayama, 1986 ; Blessing, 1995 ):

$[\eqalign{ f_{as,lm} & = \textstyle{1 \over 2}\displaystyle\left [{{{({2l + 1})({l - m})!} \over {4\pi ({l + m})!}}}\right]^{1/2} \!\left(\matrix{ P_{lm}({\cos \theta _i })\sin ({2\pi m\Phi _i }) \hfill \cr \quad+ P_{lm}({\cos \theta _o })\sin ({2\pi m\Phi _o }) \hfill \cr} \right) \cr f_{ac,lm} & = \textstyle{1 \over 2}\displaystyle\left[{{{({2l + 1})({l - m})!} \over {4\pi ({l + m})!}}}\right]^{1/2}\! \left(\matrix{ P_{lm} ({\cos \theta _i })\cos ({2\pi m\Phi _i }) \hfill \cr \quad+ P_{lm}({\cos \theta _o })\cos({2\pi m\Phi _o }) \hfill \cr} \right), } \eqno(12)]$

where as,lm and ac,lm are parts of the hierarchical index i from (6); l,m are indices of spherical harmonics; P_lm are Legendre polynomials; $[\theta, \Phi]$ are polar coordinates of the incoming (index i) and the outgoing (index o) directions in the crystal coordinate system.

Owing to absorption being the same in opposite directions, the odd-order spherical harmonics should have zero coefficients and can be omitted in the description of pure absorption. Nevertheless, owing to correlation with other effects, the scaling results can be improved by the inclusion of low-order odd harmonics.

2.1.4. Overall scale per batch of data

This scale factor describes the product of different physical effects: beam intensity, illuminated crystal volume and average absorption. It has a simple modeling function:

$[f_{s_j } = \cases{1 &for data in batch {\it j} \cr 0 &for other data.} \eqno(13)]$

The parameter p_{s_j} is the logarithm of the scale factor for batch j.

2.1.5. Decay described by B factor

Resolution-dependent crystal decay can be described by a temperature factor that is a function of the accumulated radiation dose. This dependence can be stated explicitly:

$[f_{pb,n} = ({{| {{\bf{S}} \cdot {\bf{S}}} |} / 2}) {\rm dose}^n, \eqno(14)]$

where f_pb for n = 1 describes a linear dependence and higher-order terms can be added to describe more complex behavior of radiation decay. Alternatively, the traditional approach is to apply temperature factors separately to batches of data:

$[f_{b_j } = \cases{ {{| {{\bf{S}} \cdot {\bf{S}}}|} /2} &for data in batch {\it j}\cr 0 &for other data. }\eqno(15)]$

Parameter p_{b_j} is the temperature factor of batch j.

2.1.6. Uneven crystal rotation and/or exposure

The rotation method (Arndt & Wonacott, 1977 ) assumes that the exposure is constant in the angular range. The factors that can result in non-uniform exposure are: gear misalignment in the crystal goniostat, beam fluctuations and X-ray shutter timing errors. Uneven exposure can be reproducible from image to image and may also have a random component. The correction for the reproducible part of exposure fluctuations is well determined and its description needs relatively few parameters. The correction for the random component can be problematic owing to a large number of parameters needed to describe it. Random fluctuations should be eliminated rather than corrected for by scaling. The main value of correcting for random changes may lie in its usefulness as a powerful diagnostic tool to detect under-recognized instrumental problems.

Uneven exposure can be described by a function series based on the sine and cosine of the goniostat angle $[\alpha]$ corresponding to a diffraction condition for the center of a Bragg peak. The correction of uneven exposure has to average the exposure variations over the angular range in which a particular reflection diffracts. The angular range depends not only on mosaicity but also on the geometry of a reflection crossing the Ewald sphere.

The reproducible unevenness of exposure due to mechanical backlash or a shutter error can be described by a series of frequencies g = nt, where $[t = {{2\pi } / \tau }]$ and $[\tau]$ is the rotation range. For a particular periodicity g, the modeling functions are:

$[\eqalign{ f_{u,g,\sin } & = A\sin ({2\pi \alpha g}) ,\cr f_{u,g,\cos } & = A\cos ({2\pi \alpha g}), } \eqno(16)]$

where

$[A = {1 \over {1 + ac^2 + bc^4 }},\quad c = {{mg| {\bf{S}} || {{\boldomega }}|} \over {2\pi | {\bf S} \times \boldomega ' |}}.]$

For a Gaussian mosaicity profile, the averaging factor A should be a Gaussian function that depends on the coefficient c. The expression for A approximates the Gaussian function effectively but it can also approximate averaging resulting from other mosaicity shapes. The averaging resulting from the mosaicity shape function is described by two parameters, a and b, that tend to have values close to one and zero, respectively. For unusual shapes of intensity profiles, a and b may need to be optimized. To describe random fluctuations, one can use the term $[f_{u,g,\cos } = A\cos ({2\pi \alpha g})]$ , where g = nt /2 and n goes from 1 to a maximum value. Parameters p_u,g,c,j are independent for every frame of data j. The term corresponding to n = 0 is the same as the scale factor per batch of data (see §2.1.4).

The consequence of averaging of the exposure variations is the reduced impact of the fluctuations with very high frequencies. For example, a stepper motor creates fluctuations corresponding to the step size of the motor divided by the gear ratio, typically 0.01° or less. For crystals of typical mosaicity, the consequences of such high-frequency fluctuations are negligible. Fluctuations of very low frequencies are corrected by the scale factor per batch j. The fluctuations of intermediate angular scale are the ones most affected by (16).

3. Optimization of scale factors

The scaling procedure determines the parameters p_i [equation (6)] by a method where a $[\chi ^2]$ -like target function is minimized. The target function describes the agreement between symmetry-related reflections weighted by the expectation of measurement errors. Two similar target functions can be generated, depending on whether the intensities or their logarithms are compared.

3.1. Logarithmic target function resulting in a fully linear equation

A function that works very well in practice and whose optimization always converges in one cycle is obtained from comparing the logarithms of scaled measurements (Rae, 1965 ). First, we define r as the logarithm of the unknown scale factor k_s:

$[r = \ln ({k_s }), \eqno(17)]$

then P_m is the logarithm of the measured intensity I_m for a given observation:

$[P_s = \ln ({I_s }) = P_m - r, \eqno(18)]$

where P_s is the logarithm of a scaled observation.

The intensity error estimate can be used to calculate the error estimate of the logarithm of the intensity and the corresponding weight u in the least-squares method:

$[u = {{({I_m }){}^2 } / {({\sigma _m }){}^2 }}. \eqno(19)]$

The logarithm of the geometrical average of symmetry-related observations P_av [equation (20)] is

$[P_{\rm av} = \textstyle\sum\limits_n u_n (P_{m,n} - r_n) \Big/ \sum\limits_n {u_n }, \eqno(20)]$

where n is the index of symmetry-related observations of a unique hkl. In the target function, we minimize the squared difference between the logarithms of scaled intensity and the scaled geometrical average of symmetry-related measurements:

$[\chi _{\ln }^2 = \textstyle\sum\limits_{hkl} \sum\limits_n ({I_{m,n} ^2 } /\sigma _{m,n} ^2)({P_{m,n} - r_n - P_{\rm av} }){}^2 = \sum\limits_{hkl} \sum\limits_n {u_n } \Delta _n^2. \eqno(21)]$

The minimum of the target function can be found by using multivariate Newton methods. The first and second derivatives are straightforward to calculate. The first derivative is calculated with respect to r and then, using the chain rule, the derivative with respect to parameters p_i is calculated:

$[\eqalignno{{{\delta \chi _{\ln }^2 } / {\delta r_p }} & = 2\textstyle\sum\limits_{hkl} \sum\limits_n {u_n } \Delta _n {{\delta \Delta _n } / {\delta r_p }}\cr {{\delta \Delta _n } / {\delta r_p }} & = {{u_p } \Big/{\textstyle\sum\limits_n {u_n } }} - \delta _{np} &(22) \cr{{\delta \chi _{\ln }^2 } / {\delta r_p }} & = - 2u_p \Delta _p.}]$

The second derivative with respect to r is

$[{{\delta ^2 \chi _{\ln }^2 } / {\delta r_p \delta r_q }} = 2\left({\delta _{nq} u_p - {{u_p u_q } \Big/ {\textstyle\sum\limits_n {u_n } }}} \right). \eqno(23)]$

Because r is a linear function of unknown parameters p_i:

$[r = \ln ({k_s }) = \textstyle\sum\limits_i {p_i \, f_i }, \eqno(24)]$

the first derivative of the target function with respect to p_i is

$[{{\delta \chi _{\ln }^2 }/ {\delta p_i }} = \textstyle\sum\limits_p ({{{\delta \chi _{\ln }^2 } / {\delta r_p }}})({{\delta r_p }/{\delta p_i }}) = - \textstyle\sum\limits_p {2u_p \Delta _p f_{i,p} }. \eqno(25)]$

It is important to note that the second derivative with respect to p_i for the logarithmic target function is constant – independent of the parameters p_i [equations (23) and (25)]. In such a case, the solution is given by a single matrix equation, and there is no need to make initial guesses for parameter values.

3.2. Alternative refinement of scale factors

The traditional approach to finding scale factors is based on an equation where the inverse rather than the logarithm of the scale factor appears and where we use the arithmetic rather than the geometric average of intensities:

$[\chi _{}^2 = \textstyle\sum\limits_{hkl} \sum\limits_n (k_{s,n} ^2 /\sigma _n ^2) (I_s - \langle I \rangle _{hkl}){}^2, \eqno(26)]$

where k_s is the scaling factor, $[\sigma _n]$ is the estimated error of I_m and $[\langle I \rangle _{hkl}]$ is the mean intensity of reflections for a given hkl.

The calculation of the first and then the second derivative with respect to k_s follows the same logic as in the case of a logarithmic function. First derivative:

$[{{\delta \chi ^2 } /{\delta k_{s,n} }} = - ({2/ {\sigma _n^2 }})\langle I \rangle _{hkl} ({k_{s,n} \langle I\rangle _{hkl} - I_{m,n} }), \eqno(27)]$

and second derivative:

$[\eqalignno{ {{\delta ^2 \chi ^2 } /{\delta k_{s,p} \delta k_{s,q} }} & = 2\delta _{pq} a_p^{} \langle I \rangle _{hkl}^2 - \left({{2a_p^{} a_q^{} } \Big/ \textstyle\sum\limits_n {a_n^{} k_{s,n} ^2 } }\right)\cr &\quad\times({2k_q\langle I \rangle _{hkl}^{} - I_q })({2k_p \langle I \rangle _{hkl}^{} - I_p }), & (28)}]$

where $[a = {1/ {\sigma ^2 }}]$ .

However, the formula

$[2\delta _{pq} a_p^{} \langle I\rangle _{hkl}^2 - \left({{2a_p^{} a_q^{} } \Big/{\textstyle\sum\limits_n {a_n^{} k_{s,n} ^2 } }}\right)({k_q \langle I \rangle _{hkl}^{} - I_q })({k_p \langle I \rangle _{hkl}^{} - I_p }), \eqno(29)]$

when used as a second derivative in the Newton method, gives better convergence because it better approximates the second derivative at the minimum of the target function.

Since in this case the second derivative is not constant with respect to k_s, the Newton method has to be used iteratively.

It is not clear if or when scale factors determined by logarithmic or traditional approaches are better, but even in the case when one prefers the traditional approach, it is preferable to run the first cycle with the logarithmic function owing to its superior convergence.

3.3. The description of stabilization based on the prior knowledge of the magnitude of corrections

For reflections with no redundancy, scale factors are extrapolated from other measurements. Generally, without additional restraints, the extrapolation is unstable for highly correlated parameters. In the method presented here, the restraints can be obtained from the prior knowledge about the reasonable magnitude of refined parameters. The logic behind it is the same as that behind the restraints in the atomic refinement. For example, one can assume that logarithms of scale factors typically do not fluctuate more than w_s between frames, where expectation about w_s is a function of the data-collection stability (beam stability, goniostat and/or crystal vibrations). This knowledge is described by adding a penalty term to the functions being optimized:

$[({1 /{w_s ^2 }})({p_{s_j } - p_{s_{j + 1} } }){}^2, \eqno(30)]$

where p_{s_j} is a logarithm of the scale factor for batch j.

We can similarly treat the expectation about the magnitude of absorption coefficients. For a smooth absorption with the expectation of decreasing magnitude of parameters for high orders of spherical harmonics [equation (21)], a reasonable penalty term added to the target function [equation (21)] is

$[{{l^2({p_{as,lm}^2 + p_{ac,lm}^2 })} /{w_a ^2 }}{{.}} \eqno(31)]$

If we do not want to penalize high-order terms more, we can use the following term:

$[{{({p_{as,lm}^2 + p_{ac,lm}^2 })} / {w_a ^2 }}{{.}} \eqno(32)]$

In a similar fashion, penalty terms can be created and used to restrain all the other scaling parameters.

4. Discussion

4.1. Success criteria

The main criteria of scaling effectiveness come from subsequent stages of analysis: merging, phasing etc., described elsewhere. During scaling itself, we can only judge the convergence and the value of the target function – overall $[\chi ^2]$ .

The method described above, based on the logarithmic target function, always converges in one cycle. Thus, the next step – merging – can always be accomplished, even with problematic data. A high value of overall $[\chi ^2]$ in itself does not differentiate whether it is due to anomalous signal, low quality of data or an unreasonable error model. Comparing scaled symmetry-related data allows for the calculation of various statistics in order to identify potential problems with data quality, with the error model or the existence of non-isomorphism in the diffraction.

Potentially, merging analysis may separate sources of variability in the structure factors owing to anomalous and dispersive effects, radiation-induced changes, non-isomorphism among crystals, and pseudosymmetry. In the case of structure factors not being constant, it is usually better to ignore this variability during determination of the scale factors and to scale the reflections but separate them during merging (Evans, 1993; Otwinowski & Minor, 2000 ). Redetermination of the error model, postrefinement and rejection of outliers may impact the assumptions about the data to be scaled, so it maybe worthwhile to redo the scaling after the execution of these steps.

In the following example, the structure factors were stable and the measurement errors were lower than in most experiments. Traditional scaling with scale and temperature factor per batch of data resulted in better than average merging statistics, but still with a rather low value of the significance ratio for anomalous scattering from intrinsic sulfur. Application of the above-described scaling corrections improved the merging statistics and resulted in a significant anomalous signal for medium resolution (Table 1).

Table 1
Comparison of the traditional scaling, defined as a scale and temperature factor per diffraction image, with scaling based on the method presented

The crystal of β-hydroxydecanoyl thiol ester dehydrase (Leesong et al., 1996 ) with 2 × 171 amino acids in an asymmetric unit, unit cell of P2₁2₁2₁ symmetry (a = 59.7, b = 66.9, c = 86.0 Å), was measured at Cu Kα wavelength with an R-axisII detector. An anomalous signal comes from 2 × 9 single sulfur atoms. The same outlier list and error model were used in both cases. The following statistics are presented: R_mM is the R_merge for merged Friedel pairs, χ²M is the goodness of fit for merged Friedel pairs, U refers to the same statistics calculated for unmerged Friedel pairs, AS is the statistical significance coefficient of the anomalous signal defined as a ratio χ²M to χ²U. When the value of AS is close to 1, the anomalous diffraction signal is lost in the noise.

	Traditional scaling					After corrections
Resolution shell (Å)	R_mM	χ²M	R_mU	χ²U	AS	R_mM	χ²M	R_mU	χ²U	AS
20.00–4.33	0.014	3.75	0.017	2.71	1.38	0.010	2.57	0.006	0.97	2.65
4.33–3.44	0.019	4.20	0.020	4.42	0.95	0.009	1.60	0.006	0.94	1.71
3.44–3.01	0.024	3.26	0.025	3.74	0.87	0.012	1.47	0.009	1.05	1.39
3.01–2.73	0.028	2.57	0.030	2.88	0.89	0.017	1.36	0.013	0.98	1.39
Overall	0.023	1.98	0.027	1.98	–	0.017	1.39	0.012	1.03	–

4.2. Global and local scaling

Global scaling can be followed by local scaling (Matthews & Czerwinski, 1975 ). Local scaling is mostly applied to calculate differences of the phasing signal, where it is assumed that a group of measurements, for example those close together in the reciprocal space or in the detector space, should be on a similar scale. A flexible parameterization by the exponential modeling allows for a good description of all kinds of smooth corrections. Local scaling is much more limited in terms of what type of smooth variation is being corrected for, so it is unlikely to provide additional benefit to the general scaling method described here.

4.3. Related problems non-correctable by multiplicative scaling

The basic equation (1) assumes a crystal with a level of imperfection that results in negligible extinction. More perfect crystals require a non-linear correction for extinction. However, crystals often are more than `ideally imperfect' (Guinier, 1994 $[Guinier, A. (1994). X-ray Diffraction in Crystals, Imperfect Crystals, and Amorphous Bodies. New York: Dover.]$ ). The departure from crystallinity may result in diffuse scattering, extra modulation (incommensurate structure), stacking disorders, large mosaicity that produces spot overlap, twinning etc. The sample may also be contaminated with other crystals or ice. These problems may require further analysis with already scaled data but with results that can impact the assumption about the data to be scaled. As in the case of changes in the error model or outliers rejection, iterations may be needed.

One of the purposes of scaling is to correct for a reasonable level of imperfection in the measurement system. Such a procedure may also attempt to correct severe problems resulting in non-obvious behavior of the determined scale factors. For example, an incorrect or missing description of the beamstop produces similar effects to a very large absorption in an area of reciprocal space. The smoothly varying absorption correction will produce a high estimate of absorption not only for the area behind the beamstop but also for reflections at some distance from the beamstop. Such problems should not be corrected by scaling, but rather by applying the proper experimental procedure.

4.4. Detector calibration

The purpose of scale determination can be the calibration of the instrument rather than the structure determination. For this purpose, one can use a diffraction pattern from a very high quality crystal, but it is even better to use artificial patterns. In particular, grid masks can be used not only for distortion calibration but also for the calibration of the detector sensitivity in addition to the flood field method. Sensitivity calibration can also be defined as a problem of finding a multiplicative scale function, so it can be parameterized and determined in analogous fashion as described above. This may help to solve problems in detector calibration discussed before in the literature (Tate et al., 2000).

5. Summary

The structure determination from a single-crystal diffraction experiment requires scale factors that relate measured and calculated intensities. We describe here a scaling procedure based on exponential modeling that is very effective in correcting for factors that are difficult to calculate from the physical description of the experiment. Because such factors always exist in macromolecular crystallography owing to the complex geometry of absorption around the crystal and owing to radiation decay, this kind of scaling is in practice necessary. It is also effective in small-molecule crystallography.

The power of the described method comes from the absolute stability of the refinement for virtually any description of experimental problems. All so-far proposed parameterizations (Fox & Holmes, 1966; Rossmann et al., 1979 ; Evans, 1993) can be accommodated into an exponential approach. The parameterizations described here are being implemented in Scalepack (Otwinowski & Minor, 1997 $[Otwinowski, Z. & Minor, W. (1997). Processing of X-ray Diffraction Data Collected in Oscillation Mode. In Methods in Enzymology, Vol. 276, edited by J. C. W. Carter & R. M. Sweet. New York: Academic Press.]$ , 2000). This program corrects for a large group of problems that have been observed to be significant, though typically not all problems in every case. The development of the parameterization was driven by the recognition of problems encountered in practice. This process is likely to continue. The remarkable success of applying the corrections discussed in this article requires a description of the problems in the particular experiments, a subject of separate publications.

The parameters in the scaling model should be chosen on the basis of the aim of the experiment and the understanding of what is the most significant scaling problem likely to be encountered. The choice of parameterization is particularly important when obtaining the phasing signal is the target of the experiment. For typically sized crystals, errors in the measurements of strong reflections are mostly multiplicative, so in principle an appropriate scaling model can correct them. Carefully conducted experiments with proper scaling should allow for SAD structure solution from sulfur and other weak scatterers to be more than a rare occurrence.

Acknowledgements

This work was supported by grant GM53163 from the National Institutes of Health. The authors would like to thank Janet Smith and MinSun LeeSong for providing diffraction data presented here, Halszka Czarnocka for her help with the preparation of the manuscript and Mischa Machius for his extensive comments and stimulating discussions.

References

Arndt, U. W. & Wonacott, A. J. (1977). The Rotation Method in Crystallography. Amsterdam: North Holland. Google Scholar
Azároff, L. (1955). Acta Cryst. 8, 701–704. CrossRef CAS IUCr Journals Web of Science Google Scholar
Blessing, R. H. (1995). Acta Cryst. A51, 33–38. CrossRef CAS Web of Science IUCr Journals Google Scholar
Boyd, J. P. (1991). Chebyshev and Fourier Spectral Methods. Mineola, NY: Dover. Google Scholar
Della-Pietra, S., Della-Pietra, V. & Lafferty, J. (1997). IEEE Trans. Pattern Anal. 19, 380–393. CrossRef Web of Science Google Scholar
Diamond, R. (1969). Acta Cryst. A25, 43–55. CrossRef CAS IUCr Journals Web of Science Google Scholar
Evans, P. R. (1993). Data Reduction: Data Collection and Processing, edited by L. Sawyer, N. Isaacs & S. Bailey. Proceedings of the Daresbury CCP4 Study Weekend, pp. 114–123. Daresbury Laboratory, Warrington, England. Google Scholar
Fox, G. C. & Holmes, K. C. (1966). Acta Cryst. 20, 886–891. CrossRef CAS IUCr Journals Web of Science Google Scholar
Guinier, A. (1994). X-ray Diffraction in Crystals, Imperfect Crystals, and Amorphous Bodies. New York: Dover. Google Scholar
Huber, R. & Kopfmann, G. (1969). Acta Cryst. A25, 143–152. CrossRef CAS IUCr Journals Web of Science Google Scholar
Katayama, C. (1986). Acta Cryst. A42, 19–23. CrossRef CAS Web of Science IUCr Journals Google Scholar
Kopfmann, G. & Huber, R. (1968). Acta Cryst. A24, 348–351. CrossRef IUCr Journals Web of Science Google Scholar
Leesong, M., Henderson, B. S., Gillig, J. R., Schwab, J. M. & Smith, J. L. (1996). Structure, 4, 253–264. CrossRef CAS PubMed Web of Science Google Scholar
Leslie, A. G. W. (1999). Acta Cryst. D55, 1696–1702. Web of Science CrossRef CAS IUCr Journals Google Scholar
Matthews, B. W. & Czerwinski, E. W. (1975). Acta Cryst. A31, 480–487. CrossRef CAS IUCr Journals Web of Science Google Scholar
Otwinowski, Z. & Minor, W. (1997). Processing of X-ray Diffraction Data Collected in Oscillation Mode. In Methods in Enzymology, Vol. 276, edited by J. C. W. Carter & R. M. Sweet. New York: Academic Press. Google Scholar
Otwinowski, Z. & Minor, W. (2000). Denzo and Scalepack. In International Tables for Crystallography, Vol. F, edited by M. G. Rossman & E. Arnold. Dordrecht: Kluwer. Google Scholar
Rae, A. (1965). Acta Cryst. 19, 683–684. CrossRef IUCr Journals Web of Science Google Scholar
Rossmann, M. G., Leslie, A. G. W., Abdel-Meguid, S. S. & Tsukihara, T. (1979). J. Appl. Cryst. 12, 570–581. CrossRef CAS IUCr Journals Web of Science Google Scholar
Schutt, C. E. & Evans, P. R. (1985). Acta Cryst. A41, 568–570. CrossRef CAS Web of Science IUCr Journals Google Scholar
Stuart, D. & Walker, N. (1979). Acta Cryst. A35, 925–933. CrossRef CAS IUCr Journals Web of Science Google Scholar
Stupakov, G. & Heifets, S. (2002). Phys. Rev. Spec. Top. Ac. 5, No. 054402. Google Scholar
Tate, M. W., Eikenberry, E. F. & Gruner, S. M. (2000). CCD Detectors. In International Tables for Crystallography, Vol. F, edited by M. G. Rossman & E. Arnold. Dordrecht: Kluwer. Google Scholar
Weissman, L. (1982). Strategies for Extracting Isomorphous and Anomalous Signals. Computational Crystallography, edited by D. Sayre. Oxford: Clarendon Press. Google Scholar

© International Union of Crystallography. Prior permission is not required to reproduce short quotations, tables and figures from this article, provided the original authors and source are cited. For more information, click here.

FOUNDATIONS
ADVANCES

ISSN: 2053-2733

Volume 59| Part 3| May 2003| Pages 228-234

doi:10.1107/S0108767303005488