User Guide –
Computing & Data Transfer
This section describes the computational facilities at the CMCF, including computer accounts, computer locations, image storage and processing capabilities. We also describe data transfer options, image viewing options and include a list of helpful Linux commands.
To see details about our software, please visit:
- MxDC - CMCF data collection software
- MxLIVE - CMCF sample and experiment tracking software
- Autoprocess - CMCF automated data processing pipeline
- Crystallography Software - external software that Users have access to.
New user groups to the CMCF will be issued an account used to log in to:
Any images generated on the beamline will be saved under the logged-in user's account and will only be visible to that user.
There are two user workstations available in the 08ID-1 Control Area and two workstations in the 08B1-1 Control Room (see the map for locations) Additional computers are also available in the following locations:
- Experimental Hutch - This may be used to centre samples before locking up the hutch, but should not be used to collect data.
- Sample Prep Room - This should not be used for data collection.
Workstations marked "Staff" should only be used by beamline staff.
As data are collected on a CMCF beamline, images are saved and stored as follows:
- Images are first saved to the local disk of the detector computer (this disk is cleared daily) and then saved under the user's personal account. Generally images are kept in user accounts for up to a month, or until storage space is needed for new data.
- Images are backed up each night to an additional storage server used for archives. Files will remain in the archives for up to three months, or until the space is needed for new data.
Data Transfer to a Hard Drive
The most convenient method of transferring image files is with a user-supplied USB external hard drive. An archiver script has been set up to automate the task of copying data. Set up a directory in which all of your data will be collected. Connect your drive to a User Workstation (we recommend a different workstation than the one you are collecting data on). From the User Workstation your drive is connected to, open a local terminal (not a data processing terminal), and simply type
archiver [somedir] /run/media/[user_name]/[drivename]
where somedir is the directory in your users directory that you want to transfer/backup and drivename is the name of your hard drive. The archiver script will then automatically backup all of the data collected in subsequent subdirectories as you collect.
- Any USB drive should be NTFS or FAT32. EXT2-4 drives are supported provided the permissions on the target directory have been previously set to be world-writable. This can be done with the following command using administrator privileges
chmod 777 <target_directory>
- While Windows XP can read and write to FAT32-formatted drives without any difficulty, its Disk Management facility cannot create individual FAT32 partitions larger than 32,768 MB. Instead, you can use this FAT32 format utility; please note that formatting the drive will erase all data currently on it. (See the author's page for more details.)
- Firewire is not supported at this time.
Data Transfer via SSH
This is the command to recursively copy a whole directory (not allowed in SFTP) from the local computer to a remote computer using secure copy SCP:
scp -r /data/directory001 email@example.com:/home/user/directory
scp -r LocalDirectory [username]@[remotecomputer]:/[absolutepath/][/remotecomputer][/username]
(The command will prompt for a password.)
Viewing Images at CMCF
- To view diffraction images outside of the MXDC data collection software, you may use the icon to launch the Diffraction Frame Viewer
- Or simply type
- To view the header information stored in an image file, type
Viewing Images on Your PC
Diffraction images collected at CMCF (.img files) are TIFF-formatted, but not all image viewing applications will be able to open them. Those applications that can (such as Irfanview) will not display the images properly.
The CCP4 image display program, idiffdisp, runs under Windows, but requires the download of the entire 500 MB CCP4 Windows package.
Alternatively, you can download the CMCF Image Viewer for Windows (13.5 MB). Unzip the zip file to any convenient folder (the zip password is CMCF) and run imageviewer2.exe (displayed simply as imageviewer2 if extensions are hidden). Note that the size of diffraction images will make them difficult to view on screens with a resolution of 1024x768 or smaller; please use a larger resolution when running the program. (You may require MSVCR71.DLL, the Microsoft Visual C++ 2005 runtime, available here.)
Note: Most commands in Linux are case-sensitive.
Execute command but pause after each screenful of output
Launch command in the background
(useful for GUI programs such as gedit)
Alternative way of running programs in the background
Press CTRL-Z after running command and then type bg
Launch an easy-to-use text-based file editor
[If you know how to use Vi or Emacs, why are you reading this page?]
Display the contents of filename
Display the last ten lines of filename
Reset the current terminal
(useful if you cat a binary file by mistake and the terminal gets scambled)
List all files in the current directory, sorting by date
List all files in the current directory, sorting by size
Search the current directory and all subdirectories for filename
find . -name filename
See the free space remaining on all mounted partitions
To see the size of directories
du -sh *