Spreadsheets

A spreadsheet represents a container of crystal samples that you bring to the beamline. Spreadsheets are the primary way to provide sample information that is then stored in the Crystal Server database.

Note - Currently an uploaded spreadsheet is required to see automated data processing results in Blu-Ice and the Crystal Server.

Create a New Spreadsheet

New spreadsheets can be created by clicking the Create New subtab under the Spreadsheets main tab (Fig. 1) and then entering an optional spreadsheet name and selecting a container type (SAM Cassette, Uni-puck, or SSRL Plate). Spreadsheets can then be downloaded, modified and uploaded to the Crystal Server database or uploaded directly to the database.

The Create New tab
Figure 1. The Create New tab can be used to create a sample spreadsheet.

There are two choices to create a spreadsheet:

  • Download Template: click Download Template and fill in your sample information, then click Upload Spreadsheet to upload the spreadsheet to the Crystal Server database.
  • Quick Start: click Quick Start. The system creates a spreadsheet directly in the datasbase with container Port names (i.e. A1, A2) as default sample names, ready for immediate use.

List Spreadsheets

Click on the List Spreadsheets subtab under the main Spreadsheets tab to view a list of your spreadsheets (Fig. 2).

The List Spreadsheets tab
Figure 2. The List Spreadsheet tab displays a list of all the user's spreadsheets.

The table displays the following information:

  • Upload Time: When the spreadsheet was uploaded.
  • Last Modified: The last time the file was modified.
  • Spreadsheet Name: Name.
  • Spreadsheet ID: Internal spreadsheet ID.
  • Container Type: SAM Cassette, Uni-Puck, or SSRL Plate.
  • Processed Datasets: Number of samples with processing results.
  • Assigned Beamline: The beamline that the spreadsheet is assigned to for data collection.
  • Assigned Position: Assinged dewar position (l, m, r).

Options

Click Options to see a list of avaiable spreadsheet actions (Fig. 3):

  • Rename: Change the spreadsheeet name.
  • Download: Export to Excel with all data and processing results.
  • Delete: Permanently remove the spreadsheet and all related sample information from the database.
The Options Menu
Figure 3. The Options menu for spreadsheet actions.

Screening Results

Clicking a spreadsheet name opens two additional tables: Screening Results and Processing Results. The Screening Results tab shows results for samples that have been automatically screened, indexed and assigned a quality score in BluIce (Fig. 4).

Screening results view
Figure 4. Screening results and relevant output parameters are displayed for each sample.

Column Views

Use the Views selection to show more or less parameters:

Minimum:

  • Port
  • Crystal ID
  • Protein
  • I/σ(I) Estimate
  • Mosaicity
  • Bravais Lattice
  • Unit Cell
  • Index Warning
  • Resolution
  • Score
  • Spot Warning

Less adds:

  • Spots
  • Bragg Spots
  • Spot Strength
  • Spot Resolution
  • Spot Score
  • Ice Rings

More adds:

  • Sample ID
  • Directory
  • Index Images
  • Spot Quality
  • Spot Shape

All adds:

  • Container ID
  • Container Type
  • Multicrystal Project
  • Index Directory
  • Overloads

First Column Choice

Samples are organized by port, however the first colmun can be changed using the First Column drop down menu (Fig. 5).

  • Port
  • CrystalID
  • Protein
  • Bravais Lattice
  • Resolution
  • Score
The First Column Drop Down
Figure 5. The First Column menu dispalys a list of parameters that can be used for the first column.

In addition, any column can be used to sort the results by clicking the column label and toggling the black arrow.

The Restore Defaults button will select the Less view, set default column widths and set the First Column to Ports. The Export to Excel button will dump the screening tab results into an excel spreadsheet.

Processing Results

The Processing Results tab appears next to theScreening Results tab, showing autoprocessing data for samples in the spreadsheet (Fig. 6).

Spreadsheet processing results
Figure 6. The Processing Results display for a specific spreadsheet.

Column Views

Minimum

  • Start Time
  • Status
  • Port
  • Run
  • Mosaicity
  • Space Group
  • Unit Cell
  • Cutoff Method
  • Anisotropy
  • Resolution
  • Completeness
  • Multiplicity
  • I/s(I)
  • Rpim
  • CC1/2
  • Anomalous
  • Error

Less adds:

  • Summary File
  • Observations
  • B-factor
  • Completeness (anom)
  • Mult (anom)
  • CC (anom)
  • S/N (anom)
  • MTZ File

More adds:

  • Crystal ID
  • Protein
  • Multicrystal Project
  • Filename
  • Image Directory
  • Processing Directory
  • Phi Range
  • Source
  • Pipeline
  • Pipeline Version
  • 3rd Party Pipeline
  • 3rd Party Version
  • Run Time
  • Sample ID

All adds:

  • I/σ(I) Est.
  • Job ID
  • Slurm Job ID
  • Slurm Job Name
  • Slurm Partition
  • Hostname
  • Slurm Run Time

Buttons

The "Restore Defaults" button returns View to Less, Restores the First Column and all column spacing and sorting.

The "Export to Excel" button allows the user to download an excel file with all the processing results associated with the spreadsheet. Each processing method is shown on a different sheet.

The “?” button opens a webpage with documentation for the Processing Tab.

Status

The "Status" column indicates the current status of the auto-processing job:

  • Pending - the job has been added to the queue but not started yet
  • Submitted - the job has been started
  • Running - the job is currently in progress
  • Error (highlighted in red) - the job has exited with an error. The associated error message can be found in the column labeled "Error". Processing error messages are extracted from the general log file.
  • Completed - the job has finished without errors

Anomalous Signal Detection

When autoPROC detects an anomalous signal, it is reported in the Anomalous column and another processing job is automatically spawned with the -ANO flag and the ExpectLargeHeavyAtomSignal=yes flag to optimize anomalous data processing. The anomalous multilicity can be found in the Multiplicity (anom) column (the Multiplicity column assumes no anomaolous).

Anisotropy

In addition to the standard processing method, autoPROC will run an anisotropic analysis. The processing statistics can be flound in the summary.html file under "anisotropic". the program STARANISO is used to determine anisotropic difraction by fitting an ellipse. The "Anisotropy" parameter in the Processing tab is calculated from the 3 axis of the ellipse:

Anisotropy = [ [max(res1,res2,res3) − min(res1,res2,res3)] / ave(res1,res2,res3) ]

Zero indicates no anisotropy. However, anisotropies above 5% have made improvements (sometimes significant) to electron density maps compared to the standard processing method. If Anisotropy is detected, 3 resolution values will be shown representing the fitted ellipse.

Results Summary File

The "Summary" column shows a View button that will open the summary file in a browser.

  • Periodically refreshing the page will show the updated statistics as the processing job runs.
  • The file will also display any error messages and warnings that come up.

How to Interpret Error Messages

If an error occurs, the Status column will indicate an error and the message will be displayed in the Error column on the Processing Tab. There are two general types of errors:

  • ICEflow errors (these should be labeled "ICEFLOW ERROR")
  • Processing software errors (these should be labeled "AUTOPROC ERROR" for processing with autoPROC)

If there are ICEflow errors, please contact beamline support.

autoPROC errors most often reflect issues with processing the data; some of the most common types are:

  • indexing errors in XDS
  • integration errors in XDS
  • scaling errors in apScale or XSCALE

ICEflow extracts autoPROC error messages from the "top" log file (typically named out-{cutoff}.log); these errors most often point to the log files for specific processes (e.g. indexing) and provide relative paths to them. Inspect these logs if you need more detailed information about what went wrong.

The autoPROC manual lists a few common errors that can be encountered when running autoPROC as well as a few general suggestions for how to handle them. ICEflow is designed to avoid the more basic errors (for example, all SSRL beamline-specific settings have been implemented already), but if any of these errors crop up, please contact beamline support.

  • CC1/2
  • I/sigma(I)