This section describes the computational facilities at the CMCF, including computer accounts, computer locations, image storage and processing capabilities. We also describe data transfer options, image viewing options and include a list of helpful Linux commands.
To see details about our software, please visit:
- MxDC - CMCF data collection software
- MxLIVE - CMCF sample and experiment tracking software
- Autoprocess - CMCF automated data processing pipeline
- Crystallography Software - external software that Users have access to.
New user groups to the CMCF will be issued an account used to log in to:
- beamline user computers,
- NX client for remote control, and
- MxLIVE on the web.
Any images generated on the beamline will be saved under the logged-in user's account and will only be visible to that user.
There are two user workstations available in the CMCF-ID Control Room and two workstations in the CMCF-BM Control Room (see the map for locations) Additional computers are also available in the following locations:
- Experimental Hutch - This may be used to centre samples before locking up the hutch, but should not be used to collect data.
- Sample Prep Room - This should not be used for data collection.
Workstations marked "Staff" should only be used by beamline staff.
As data are collected on a CMCF beamline, images are saved and stored as follows:
- Images are saved under the user's personal account. Generally images are kept in user accounts for up to a month, or until storage space is needed for new data.
- Images are backed up to an additional storage server used for archives. Files will remain in the archives for up to three months, or until the space is needed for new data.
Data Transfer Options
The most convenient method of data transfer is using MxLIVE. Login with your beamline user account and download individual datasets, reports (including all processing files), or your complete session directory. Find the "Download" icon at the top-right hand corner when viewing your dataset, report, or session, or download individual diffraction frames from a dataset by finding the download icon on the right-hand side of each frame in the frame list.
An archive of the entire dataset, report, or session will be created and streamed to you through your browser. Note that session downloads include all datasets and reports from that session, as long as they are available in our on-site data storage.
Data, including image files and processing files, can be transferred to a user-supplied USB external hard drive and shipped along with your samples. An archiver script has been set up to automate the task of copying data. Set up a directory in which all of your data will be collected. Connect your drive to a User Workstation (we recommend a different workstation than the one you are collecting data on). From the User Workstation your drive is connected to, open a local terminal (not a data processing terminal), and simply type
archiver [somedir] /run/media/[user_name]/[drivename]
where somedir is the directory in your users directory that you want to transfer/backup and drivename is the name of your hard drive. The archiver script will then automatically backup all of the data collected in subsequent subdirectories as you collect.
• Any USB drive should be NTFS or FAT32. EXT2-4 drives are supported provided the permissions on the target directory have been previously set to be world-writable. This can be done with the following command using administrator privileges
chmod 777 <target_directory>• HDF5 image files generated by the EIGER X 9M can be quite large (upwards of 30 GB for a single dataset in 0.5 GB partitions). An average compressed dataset (3600 images) is ~10-15 GB in size. Please ensure your drive has sufficient space for your entire session.
• FAT32 hard drives formatted using Windows XP (or before) are limited to partitions below 32, 768 MB. Please reformat your drive using an x64 system (note: reformatting drives will delete all data currently stored on the disk).
• Firewire is not supported at this time.
Data Transfer via SSH
This is the command to recursively copy a whole directory (not allowed in SFTP) from the local computer to a remote computer using secure copy SCP:
scp -r /data/directory001 firstname.lastname@example.org:/home/user/directory
scp -r LocalDirectory [username]@[remotecomputer]:/[absolutepath/][/remotecomputer][/username]
(The command will prompt for a password.)
Users who have uploaded an ssh-key to their profile on MxLIVE can also request
ssh access to the data transfer server by emailing their support contact. Data transfer can then be completed using either of the following commands;
scp -r -C -P [port] [username]@[cmcf_remotecomputer]:/[absolutepath]/[to]/[remotedirectory]/* /[location]/[on]/[localmachine]
(Ideal for only a few files.)
rsync -avzhe -e 'ssh -p [port]' [username]@[cmcf_remotecomputer]:/[absolutepath]/[to]/[datadirectory]/* /[location]/[on]/[localmachine]
(Ideal for large downloads or unstable connections. This command can also be repeated as new data is generated)
Double check that you have enough disk space and have correctly entered the file path prior to running
scp. Files with the same name can be easily overwritten if incorrectly executed.
Viewing Diffraction Images
Viewing Images at CMCF
- To view diffraction images outside of the MXDC data collection software, you may use the icon to launch the Diffraction Frame Viewer
- Or simply type
- To view the header information stored in an image file, type
- Image viewing is also supported using
Viewing Images on Your PC
Diffraction images collected at CMCF are generated as single frame .cbf (for CMCF-BM) or multi-frame .h5 (for CMCF-ID) files. Viewing of images is generally supported in up-to-date versions of MosFlm and HKL2000, provided the data metadata master file is also included. HDF5 files contain image data in a hierarchical format, and will often require a separate viewing program to interpret. More details on the HDF5 format can be found here.
Note: Most commands in Linux are case-sensitive.
Execute command but pause after each screenful of output
Launch command in the background
(useful for GUI programs such as gedit)
Alternative way of running programs in the background
Press CTRL-Z after running command and then type bg
Launch an easy-to-use text-based file editor
[If you know how to use Vi or Emacs, why are you reading this page?]
Display the contents of filename
Display the last ten lines of filename
Reset the current terminal
(useful if you cat a binary file by mistake and the terminal gets scambled)
List all files in the current directory, sorting by date
List all files in the current directory, sorting by size
Search the current directory and all subdirectories for filename
find . -name
See the free space remaining on all mounted partitions
To see the size of directories
du -sh *