Spreadsheets

A spreadsheet represents a container of crystal samples that you bring to the beamline. Spreadsheets are the primary way to provide sample information that is then stored in the Crystal Server database.

Note - Currently a spreadsheet must be uploaded and assinged to a beamline to see automated data processing results in Blu-Ice and the Crystal Server.

Create a New Spreadsheet

New spreadsheets can be created by clicking the Create New subtab under the Spreadsheets main tab (Fig. 1) and then entering an optional spreadsheet name and selecting a container type (SAM Cassette, Uni-puck, or SSRL Plate). Spreadsheets can then be downloaded, modified and uploaded to the Crystal Server database or uploaded directly to the database for quick use.

The Create New tab

Figure 1. The Create New tab can be used to create a sample spreadsheet.

There are two choices to create a spreadsheet:

  • Download Template: click Download Template and fill in your sample information, then click Upload Spreadsheet to upload the spreadsheet to the Crystal Server database.
  • Quick Start: click Quick Start. The system creates a spreadsheet directly in the datasbase with container Port names (i.e. A1, A2) as default sample names, ready for immediate use.

List Spreadsheets

Click on the List Spreadsheets subtab under the main Spreadsheets tab to view a list of your spreadsheets (Fig. 2).

The List Spreadsheets tab

Figure 2. The List Spreadsheet tab displays a list of all the user's spreadsheets.

The table displays the following information:

  • Upload Time: When the spreadsheet was uploaded.
  • Last Modified: The last time the file was modified.
  • Spreadsheet Name: Name.
  • Spreadsheet ID: Internal spreadsheet ID.
  • Processed Datasets: Number of samples with processing results.
  • Container Type: Cassette, Uni-puck or Plate.
  • Assigned Location: The beamline and dewar location (left middle or right) that the spreadsheet is assigned to for data collection.
  • Options: Options menu.

Options

Click Options to see a list of avaiable spreadsheet actions (Fig. 3):

  • Download: Export to Excel with all data and processing results.
  • Rename: Change the spreadsheeet name.
  • Delete: Permanently remove the spreadsheet and all related sample information from the database.
The Options Menu

Figure 3. The Options menu for spreadsheet actions.

Screening Results

Clicking a spreadsheet name opens two additional tabs: Screening Results and Processing Results for the selected spreadsheet. The Screening Results tab shows results for samples that have been automatically screened, indexed and assigned a quality score in BluIce (Fig. 4).

Screening results view

Figure 4. Screening results and relevant output parameters are displayed for each sample.

Column Views

Use the Views selection to show more or less parameters:

Minimum:

  • Port
  • Protein
  • Bravais Lattice
  • Unit Cell
  • Mosaicity
  • Resolution
  • Ice Rings
  • Score
  • Spot Warnings
  • Index Warnings

Less adds:

  • Crystal ID
  • Index Images

More adds:

  • Multicrystal Project
  • Overloads

All adds:

  • Sample ID
  • Container ID
  • Container Type

First Column Choice

Samples are organized by port, however the first colmun can be changed using the First Column drop down menu (Fig. 5).

  • Port
  • CrystalID
  • Protein
  • Bravais Lattice
  • Resolution
  • Score
The First Column Drop Down

Figure 5. The First Column menu dispalys a list of parameters that can be used for the first column.

In addition, any column can be used to sort the results by clicking the column label and toggling the black arrow.

The Restore Defaults button will select the Less view, set default column widths and set the First Column to Port. The Export to Excel button will dump the screening tab results into an excel spreadsheet.

Processing Results

The Processing Results tab appears next to theScreening Results tab, showing autoprocessing data for samples in the spreadsheet (Fig. 6).

Spreadsheet processing results

Figure 6. The Processing Results display for a specific spreadsheet.

Column Views

Minimum

  • Start Time
  • Status
  • Port
  • Run
  • Mosaicity
  • Space Group
  • Unit Cell
  • Cutoff Method
  • Anisotropy
  • Resolution
  • Completeness
  • Multiplicity
  • I/s(I)
  • Rpim
  • CC1/2
  • Anomalous
  • Error

Less adds:

  • Summary File
  • Observations
  • B-factor
  • Completeness (anom)
  • Mult (anom)
  • CC (anom)
  • S/N (anom)
  • MTZ File

More adds:

  • Crystal ID
  • Protein
  • Multicrystal Project
  • Filename
  • Image Directory
  • Processing Directory
  • Phi Range
  • Source
  • Pipeline
  • Pipeline Version
  • 3rd Party Pipeline
  • 3rd Party Version
  • Run Time
  • Sample ID

All adds:

  • I/σ(I) Est.
  • Job ID
  • Slurm Job ID
  • Slurm Job Name
  • Slurm Partition
  • Hostname
  • Slurm Run Time

Buttons

The "Restore Defaults" button returns View to Less, Restores the First Column and all column spacing and sorting.

The "Export to Excel" button allows the user to download an excel file with all the processing results associated with the spreadsheet. Each processing method is shown on a different sheet.

The “?” button opens a webpage with documentation for the Processing Tab.

Status

The "Status" column indicates the current status of the auto-processing job:

  • Pending - the job has been added to the queue but not started yet
  • Submitted - the job has been started
  • Running - the job is currently in progress
  • Error (highlighted in red) - the job has exited with an error. The associated error message can be found in the column labeled "Error". Processing error messages are extracted from the general log file.
  • Completed - the job has finished without errors

Anomalous Signal Detection

When autoPROC detects an anomalous signal, it is reported in the Anomalous column and another processing job is automatically spawned with the -ANO flag and the ExpectLargeHeavyAtomSignal=yes flag to optimize anomalous data processing. The anomalous multilicity can be found in the Multiplicity (anom) column (the Multiplicity column assumes no anomaolous).

Anisotropy

In addition to the standard processing method, autoPROC will run an anisotropic analysis. The processing statistics can be flound in the summary.html file under "anisotropic". the program STARANISO is used to determine anisotropic difraction by fitting an ellipse. The "Anisotropy" parameter in the Processing tab is calculated from the 3 axis of the ellipse:

Anisotropy = [ [max(res1,res2,res3) − min(res1,res2,res3)] / ave(res1,res2,res3) ]

Zero indicates no anisotropy. However, anisotropies above 5% have made improvements (sometimes significant) to electron density maps compared to the standard processing method. If Anisotropy is detected, 3 resolution values will be shown representing the fitted ellipse.

Results Summary File

The "Summary" column shows a View button that will open the summary file in a browser.

  • Periodically refreshing the page will show the updated statistics as the processing job runs.
  • The file will also display any error messages and warnings that come up.

How to Interpret Error Messages

If an error occurs, the Status column will indicate an error and the message will be displayed in the Error column on the Processing Tab. There are two general types of errors:

  • ICEflow errors (these should be labeled "ICEFLOW ERROR")
  • Processing software errors (these should be labeled "AUTOPROC ERROR" for processing with autoPROC)

If there are ICEflow errors, please contact beamline support.

autoPROC errors most often reflect issues with processing the data; some of the most common types are:

  • indexing errors in XDS
  • integration errors in XDS
  • scaling errors in apScale or XSCALE

ICEflow extracts autoPROC error messages from the "top" log file (typically named out-{cutoff}.log); these errors most often point to the log files for specific processes (e.g. indexing) and provide relative paths to them. Inspect these logs if you need more detailed information about what went wrong.

The autoPROC manual lists a few common errors that can be encountered when running autoPROC as well as a few general suggestions for how to handle them. ICEflow is designed to avoid the more basic errors (for example, all SSRL beamline-specific settings have been implemented already), but if any of these errors crop up, please contact beamline support.

  • CC1/2
  • I/sigma(I)