An in-house software program called AutoProcess has been developed at the CMCF. It automatically generates appropriate input files for, and runs XDS. It uses POINTLESS to identify the space group from systematic absences and BEST to determine a data collection strategy. All generated input and output files are saved in a new directory, including .mtz, .shelx and .cns reflection output files which can be used directly in your favourite structure solution/refinement programs.
Full documentation for AutoProcess can be found here. The following presents a summary of the most often-used features.
Running AutoProcess
Run AutoProcess either from the MxDC data collection software or from a Data Processing terminal command line.
In MxDC, running AutoProcess is a very easy and seamless process. Simply go to the Processing Results tab, select the dataset(s), select the processing options and click Run. Processing in MxDC will automatically upload the results to MxLIVE, associated with the sample on which the data were collected. Additional information is available in the MxDC Processing Results tab section.
To run AutoProcess from the command line, it is required to first log-in to one of our high-performance computing nodes of our Data Processing cluster. You can do that by double-clicking on the Data Processing icon on the Desktop.
Input
Note that AutoProcess will create a directory for output files within the current working directory. Therefore, change the directory appropriately before issuing the commands.
Start AutoProcess by typing "auto.process" followed by any options and representative image file(s). Some options include -s (screen/characterize crystal and obtain a collection strategy), -a (to indicate anomalous data) and -m (MAD data). Images do not have to be sequential.
example: process a native dataset
auto.process any_image_file.img
example: process an anomalous dataset
auto.process -a any_image_file.img
example: obtain strategy for a native data collection, requires at least 4 degrees of data to produce reliable results
auto.process -s any_image_file.img
example: obtain strategy for an anomalous data collection
auto.process -as any_image_file.img
example: processing a MAD dataset
auto.process -m any_peak_file.img any_infl_file.img any_remo_file.img
example: merging two or more datasets
auto.process set1_image_file.img set2_image_file.img ...
A full list of options is displayed by typing "autoprocess" with no options:
auto.process [options] /path/to/set1.img /path/to/set2.img ... /path/to/setn.img
options:
--mad, -m : Process each set, scale together and generate separate reflection files.
--screen, -s : Process a few frames and characterize crystal from each set.
--anom, -a : Process with Friedel's law False.
--backup, -b : Backup previous output directory if it exists.
--prefix=p1,p2,p3 : Comma separated list of prefixes to use for output files.
Default is first part of image name.
Prefix order should correspond to the order of the data sets.
For example for MAD data, use --prefix=peak,infl,remo.
--dir=/path : Directory to store processed results.
Default is to create a new one in the current directory.
--inputs, -i : Generate XDS.INP only and quit.
--help, -h : Display this message.
Default (no option) : Process each set, scale together and merge into one reflection file.
data sets:
Each data set can be represented by any frame from that set.
Output
AutoProcess by default creates a new working directory that contains the XDS input and output files. The new directory name depends on the requested mode of operation and will be named either "proc-native" (native data processing output) or "proc-screen"(crystal screening/strategy), or "proc-mad" (MAD data processing), or "proc-merge" (merging multiple datasets), or "proc-anom" (anomalous data processing). Alternatively, a specific directory name can be forced by adding the "--dir=/required/output/directory" option to the command. For multi-dataset processing such as MAD or Merging, sub-directories will be created within the working directory described above, for individual datasets. The output .mtz, .shelx and .cns reflection files can be found in the top-level working directory. This directory will also contain output as report files in HTML (report.html) and TEXT (report.txt) formats, summarizing the results. The HTML file can be viewed using a browser. TEXT files, and various other .log files can also be viewed using your favorite text editor/viewer such as "gedit" or "more".
To view the web browser compatible report type
firefox report.html
Modifying the Processing
AutoProcess in addition to providing the "auto.process" command, also provides additional commands to be used at times when you want to specify certain parameters, such as space group, resolution limit or image range. The extra commands work by starting AutoProcess from a given step within the processing chain and can be used to repeat a given step. Each of the commands, while starting from a different step, will run all subsequent steps to the end, using the modified parameter(s).
All the commands provided by AutoProcess, including the main "auto.process" command, accept a "-h" or "--help" option which prints out detailed help about available options and examples.
Since the extra commands can only be used to modify a previously run AutoProcess session, it is required to change into the top-level AutoProcess working directory before running any of these commands.
- auto.process -- When run from within a working directory without any parameters, it means try to continue an interrupted session from where it was interrupted.
- auto.inputs -- Generate XDS.INP file for running XDS manually
- auto.integrate -- Repeat or optimize a previous processing from the integration step onward, with the same or different settings for resolution, data range or Friedel's law
- auto.report -- Generate HTML and TEXT report files report.html, report.txt
- auto.scale -- Repeat a previous processing from the scaling step onward, with the same or different settings for resolution range or Friedel's law
- auto.strategy -- Repeat previous processing from the strategy determination step onward with the same or different settings for resolution and Friedel's law
- auto.symmetry -- Repeat a previous processing from the spacegroup determination step onward, with the same or different settings for resolution range or spacegroup
example: to modify the processing to use spacegroup P2(1)2(1)2(1), change into the working directory and type:
auto.symmetry --spacegroup=19 OR auto.symmetry -g 19
example: use a different data range for integration
auto.integrate --frames=20-120
example: set high resolution limit for scaling to 2.2
auto.scale --res=2.2
Advanced Usage
Besides the reflection files and reports, XDS creates numerous other files, including
- image files (.cbf)
- XDS input files (.INP)
- XDS output files (.LP)
Viewing .cbf Files
You may view the .cbf files with imgview or xds-viewer. For example, to view spots on the last frame processed, type
imgview FRAME.cbf OR xds-viewer FRAME.cbf
Using XDS Input Files
There may be instances when you want to run XDS manually. You can view a full list of XDS input parameters on the XDS homepage.
auto.inputs any_image_file.img
- An example of an input file appears below:
!- XDS.INP ----------- Generated by AutoProcess
JOB= ALL ! CORRECT
!------------------- Dataset parameters
X-RAY_WAVELENGTH= 0.97934
DETECTOR_DISTANCE= 200.0
STARTING_ANGLE= 100.0
STARTING_FRAME= 100.0
OSCILLATION_RANGE= 0.50
NAME_TEMPLATE_OF_DATA_FRAMES=/users/school/insulin/insulin_???.img
FRIEDEL'S_LAW= TRUE
DATA_RANGE= 1 120
DELPHI= 8.00
SPOT_RANGE= 1 8
SPOT_RANGE= 91 98
BACKGROUND_RANGE= 1 10
SPACE_GROUP_NUMBER= 197
UNIT_CELL_CONSTANTS= 78.000 78.000 78.000 90.000 90.000 90.000
!----------------- Beamline parameters
DETECTOR= CCDCHESS
NX=3072 NY= 3072
QX=0.07324 QY=0.07324
ORGX= 1516 ORGY= 1536
SENSOR_THICKNESS= 0.000
MINIMUM_VALID_PIXEL_VALUE= 1
OVERLOAD= 65535
STRONG_PIXEL= 6
MINIMUM_ZETA= 0.05
TRUSTED_REGION=0.00 1.414
TEST_RESOLUTION_RANGE= 50.0 1.0
TOTAL_SPINDLE_ROTATION_RANGES= 90 360 30
STARTING_ANGLES_OF_SPINDLE_ROTATION= 0 180 15
VALUE_RANGE_FOR_TRUSTED_DETECTOR_PIXELS= 6000 30000
INCLUDE_RESOLUTION_RANGE=50.0 0.0
ROTATION_AXIS= 1.0 0.0 0.0
INCIDENT_BEAM_DIRECTION=0.0 0.0 1.0
FRACTION_OF_POLARIZATION=0.95
POLARIZATION_PLANE_NORMAL= 0.0 1.0 0.0
DIRECTION_OF_DETECTOR_X-AXIS= 1.000 0.000 0.000
DIRECTION_OF_DETECTOR_Y-AXIS= 0.000 1.000 0.000
!----------------- Extra parameters