LATLINE

By Vinzenz Unger and Anchi Cheng

This program performs fitting of curves to the experimental amplitude and phase data for individual lattice lines.

Fitting of curves to a set of scattered input data allows the generation of discrete and uniformly sampled structure factors which can be used to calculate a 3-D density map by Fourier summation. For a more detailed description of the underlining concepts, the readers are encouraged to read the original paper on the program (Agard, 1983) and a good description of the sampling theorem (e.g. Ch.7 by Moody p168-170 in “Biophysical Electron Microscopy”, Hawkes and Valdrè editors, Academic press). The sampling theorem says that we can reconstruct the transform of a real-space object of finite width (D), by sampling the transform at 0, ±1/D, ±2/D,…., followed by a convolution of the sampled transform with the transform of an envelope function that has the shape of a square-pulse (i.e., a uniform height for all points within width D but zero everywhere else). Accordingly, for the data from a 2-D crystal, the task of obtaining a lattice line can be broken down into two parts. The first is to estimate the width of the envelope function within which the map density will be confined, and the second is to determine the amplitudes and phases of the complex transform at 0, ±1/D, ±2/D,…. A least square fitting approach in LATLINE is used to minimize errors between the data and the fitted curves by optimizing the amplitudes and phases of the sampled points. The success of the least squares minimization largely depends on the proper width of the envelope function that is manually defined by the user. Finding the proper width is mainly based on existing knowledge about the sample and sometimes a trial-and-error approach to produce the most reasonable fit. Too small a width will result in a fit that does not faithfully follow the most rapid changes in the transform. However, choosing the width too large will cause a fit that attempt to follow the noisy scatter of the data too closely. While the latter will always result in smaller overall fitting errors, the final map will be noisier due to the low amount of averaging.

The following parameters need to be adjusted:

ZMIN ≤ 1/RMIN and ZMAX ≥ 1/RMAX

where RMIN and RMAX are the limits of the real-space envelope function (see below).

The five parameters in the following input card, DELPRO, RMIN, RMAX, RCUT, and PFACT, define the shape and size of the envelope function. As explained earlier, the envelope function contributes to the lattice line as a convolution of its transform with the discretely sampled structure factors. Hence, its effects are found in all parts of the lattice line and the least square fitting procedure does not “correct” for the effects of a wrongly chosen envelope function. Accordingly, a “trial-and-error” approach may be necessary to find the best values based on the overall fitting statistics.