The CMCF supports a variety of data processing, data analysis, and model building programs. These programs can be run from a local terminal from any user workstation at the beamline or NX session. Some programs can also be run on our high-capacity data processing servers for fastest data processing.
The following sections detail the programs available to academic users, and how best to run each program.
| Program Name | Startup Command |
|---|---|
| Autoprocess | auto.process any_frame |
| XDS | xds |
| Mosflm | imosflm |
| xia2 | xia2 /directory_where_images_are |
| HKL2000 | not supported, users must contact hkl@hkl-xray.com for up-to-date site files |
| PHENIX | phenix |
| Solve/Resolve | phenix or (various) |
| CCP4 | ccp4i |
| SHELX | ccp4i or (various) |
| ARP/wARP | ccp4i or (various) |
| CNS | (various) |
| XPREP | xprep |
| AutoPartha | runPartha |
| Coot | coot |
|---|---|
| PyMOL | pymol |
| mosgo (strategy determination) | msg |
|---|---|
| xdsgo (data processing) | xdsgo |
| sms (structure solution) | sms |
Data Processing
The CMCF has high-capacity data processing servers dedicated to serve users. To access these servers, open a Data Processing console by clicking on the icon on the Desktop of the user workstation at the beamline or NX session. Or, enter the command hpc from terminal. Graphical consoles and commands run outside of the HPC will run on the local machine which may impact the responsiveness of your local session.
Note: Frame data from the EIGER 9M detector (installed on CMCF-ID) is stored in a compressed HDF5 (.h5) format. Up-to-date versions of all major processing programs (including XDS, iMosflm, HKL2000, DIALS) handle .h5 files natively and is the process supported by CMCF. Users choosing to process their data as CBF can convert on-site using the command cbf <yourdata>_master.h5 from within the data directory, or using your own converter programs off-site like eiger2cbf.
XDS & Autoprocess
Using XDS is made easy at the CMCF through an in-house script called Autoprocess which generates the appropriate input files and runs XDS automatically. You can also use XDS in the traditional manner. We recommend running Autoprocess to generate the appropriate input files which you can then edit manually.
Off-site processing of .h5 files within XDS requires the Neggia library provided by Dectris. Compiled libraries are available from the Dectris website (a free account required) or can be compiled directly from github. Key parameters can be extracted from the master.h5 file using any HDF5 viewer or via command line, although it is frequently much easier to just copy parameters from the xds.inp file generated by Autoprocess.
Mosflm
imosflm
xia2
xia2 [options] [/directory/where/images/are]
HKL2000/3000
- HKL2000/3000 is not supported at this time
- Running HKL with HDF5 data collected at CMCF-ID will require a site file corresponding to the appropriate beamline. Users should contact the HKL support team directly for an up to date copy (hkl@hkl-xray.com), or characterize their own site file using a known good high resolution lysozyme dataset. CMCF can provide lysozyme dataset upon request.
- Historical site files can be found below. (click to download site file):
- 08ID1-300 site file for the MarCCD 300 which was in use at beamline 08ID-1 April 2010 - February 2017
- 08ID1-225 site file for the MarCCD 225 which was in use at beamline 08ID-1 before April 2010
- 08B1-300HE site file for the MarCCD 300HE which was in use at beamline 08B1-1 before May 2020
Data Analysis
The following data analysis programs can either be run through a local terminal or in a Data Processing console (accessed by clicking on the icon on the desktop of the user workstation at the beamline or NX session). In some cases, speed will be improved by running in a data processing console. Some features, such as those which include a graphical UI, will only run locally.
PHENIX
phenix
CCP4
ccp4i
SHELX
ccp4i
You may also wish to make use of the hkl2map application
hkl2map
CNS
Solve/Resolve
ARP/wARP
ccp4i
XPREP
xprep
AutoPartha
Partha is a program developed by Peter Zwart [Acta Cryst. (2005) D61, 1437-1448] that simulates SAD data and estimates the likelihood of success of SAD phasing, given a set of parameters. "AutoPartha" is an application for using Partha, developed and adapted for use at the CMCF. It includes easily accessible choices of energies available at the facility, automatic calculations of theoretical f' and f" and an adaptive scoring mechanism based on the literature and experience.
runPartha
Model Building
The following model building programs are graphics applications that should be run from a local terminal on a user workstation at the beamline or NX session. Note: These applications are not available from the data processing servers.
Coot
coot
PyMOL
pymol
Chemical Crystallography
MSG
MSG (Mosflm-Go) is a strategy determination program developed at CMCF. MSG uses Mosflm for cell indexing and strategy determination. To invoke MSG, type:
Examples:
running msg with a specific frame name:
msg frame_name * Only supports .cbf file type.
to enforce a specific space group:
msg frame_name -s 19 (or P212121)
to access recently collected frames in MxDC:
msg -x
to change default oscillation angle for strategy determination:
msg frame_name -o 0.2
to access recently created frames via lsl:
lsl -m(works best in subdirectories)
All the above options could also be accessed with lsl. For example, to calculate a data collection strategy with the last created frames, specific space group, and oscillation angle type:
lsl -m -s 19 -o 0.5
Example of output:
Please cite mosflm in your publications: T.G.G. Battye, L. Kontogiannis, O. Johnson, H.R. Powell and A.G.W. Leslie 2011, Acta Cryst. D67, 271-281.
XDSGO
XDSGO is a data reduction program, which was developed at CMCF. It utilises the XDS package for data reduction and Sadabs program for data correction and merging. To invoke XDSGO, navigate to your data folder and type:
xdsgo
To integrate multiple datasets the following command can be used:
xdsgo -m /path/to/first/data_directory /path/to/second/data_directory /path/to/N/data_directory
XDSGO creates input files for XDS, XSCALE, and SADABS and run those programs in sequence. Data conversion between XDS file format and SADABS file format is accomplished via the XDS2SAD program. XDSGO is designed to recover from all the posible errors and usualy does not require user input. It creates standard XDS output files and provides a data statistics report in html format. All the above mentioned files can be found in the "proc" direcotry created in the dataset directory from which it is run.
Example of data processing:
SMS
SMS is a structure solution program. It utilizes XPREP and SHELXT for data solution. It supports XDS, Scalepack, and shelx input file formats, and outputs the solution in a shelx compatible format. To invoke sms, navigate to your "proc" folder, created by XDSGO, and type:
sms -xds XDS_ASCII.HKL -f FORMULA -o OUTPUT_FILE_NAME
Both XDSGO and SMS can be run in sequence from your data directory using the following command:
sms -a -f FORMULA -o OUTPUT_FILENAME
Other SMS options can be accessed by typing:
sms -h
which will result in the following output:
-h Print this help message
-o Output file name
-xds XDS input file
-sca Scalepack input file
-sx Shelx input file
-f Formula
-c Cell parameters (required by shelx file format)
-w Wavelength in Angstrom
-a Automatic data reduction (XDS), scaling (SADABS) and solution (SHELXT
References
- Kabsch, W. Automatic processing of rotation diffraction data from crystals of initially unknown symmetry and cell constants. J. Appl. Cryst. 1993, 26, 795-800.
- Sheldrick, G. M. Sadabs; University of Gottingen, Germany Program for Empirical Absorption Correction of Area Detector Data, 1996.
- Sheldrick, G. M. 2008 XDS2SAD. University of Gottingen, Germany
- XPREP 2014; X-ray data Preparation and Reciprocal space Exploration Program.Bruker AXS Inc., Madison, Wisconsin, USA.