Live Data Monitoring with Interceptor and ResoNet AI

General Information

Experimental efficiency greatly depends on the researchers' ability to evaluate whether the experiment is successful, so that time is not wasted collecting unusable data. In the past, such analysis was performed by visual inspection of collected diffraction patterns, but the current high data rates make this not feasible. Furthermore, the human eye cannot evaluate diffraction resolution with accuracy, or detect subtle but deleterious effects, such as closely overlapped multiple lattices, also known as "split spots". An automated approach to data analysis is thus required.

Interceptor is a live data-collection monitoring program designed to (i) balance the load among many distributed image-analysis workers, (ii) minimize disk I/O, (iii) handle scenarios of worker shortage at peak capacity and (iv) provide the workers with immediate access to individual images before they are incorporated into an aggregate file format, for example HDF5. The Interceptor architecture is based on the ZeroMQ messaging library, with available workers requesting images using the ZeroMQ REQ protocol and the data-collection software forwarding the data via the REP protocol while images are written to disk.

While early versions of the Interceptor utilized DIALS data processing backends, recently these backends were replaced with the ResoNet data analysis AI. Trained on synthetic X-ray diffraction datasets, the ResoNet models estimate resolution, count reflections, and detect spot splitting, thus providing real-time analysis of diffraction data, which is displayed to users in readable format via the strip chart in the BLU-ICE Collect tab.

Obtaining BluIce

The Interceptor and the ResoNet AI are open-source projects, distributed under the Stanford open-source license. For any inquiries on the Interceptor, contact Art Lyubimov; for ResoNet, contact Derek Mendez.

Both codebases are available on GitHub:

More Information

ResoNet AI publication: Mendez, et al., Deep residual networks for crystallography trained on synthetic data. Acta Cryst. (2024). D80, 26-43. https://doi.org/10.1107/S2059798323010586
Art Lyubimov's talk on Interceptor.

Last modified:Wednesday, 17-Jan-2024 13:41:15 PST.

Home User Guide User Resources Schedule Forms Our Research News Staff Visiting
Live Data Monitoring with Interceptor and ResoNet AI General Information Experimental efficiency greatly depends on the researchers' ability to evaluate whether the experiment is successful, so that time is not wasted collecting unusable data. In the past, such analysis was performed by visual inspection of collected diffraction patterns, but the current high data rates make this not feasible. Furthermore, the human eye cannot evaluate diffraction resolution with accuracy, or detect subtle but deleterious effects, such as closely overlapped multiple lattices, also known as "split spots". An automated approach to data analysis is thus required. Interceptor is a live data-collection monitoring program designed to (i) balance the load among many distributed image-analysis workers, (ii) minimize disk I/O, (iii) handle scenarios of worker shortage at peak capacity and (iv) provide the workers with immediate access to individual images before they are incorporated into an aggregate file format, for example HDF5. The Interceptor architecture is based on the ZeroMQ messaging library, with available workers requesting images using the ZeroMQ REQ protocol and the data-collection software forwarding the data via the REP protocol while images are written to disk. While early versions of the Interceptor utilized DIALS data processing backends, recently these backends were replaced with the ResoNet data analysis AI. Trained on synthetic X-ray diffraction datasets, the ResoNet models estimate resolution, count reflections, and detect spot splitting, thus providing real-time analysis of diffraction data, which is displayed to users in readable format via the strip chart in the BLU-ICE Collect tab. Obtaining BluIce The Interceptor and the ResoNet AI are open-source projects, distributed under the Stanford open-source license. For any inquiries on the Interceptor, contact Art Lyubimov; for ResoNet, contact Derek Mendez. Both codebases are available on GitHub: Interceptor: https://github.com/ssrl-px/interceptor Instructions for installation of Interceptor with ResoNet AI. ResoNet AI: https://github.com/ssrl-px/resonet Installation and tutorials for standalone ResoNet AI More Information ResoNet AI publication: Mendez, et al., Deep residual networks for crystallography trained on synthetic data. Acta Cryst. (2024). D80, 26-43. https://doi.org/10.1107/S2059798323010586 Art Lyubimov's talk on Interceptor.

Technical questions: Webmaster Content questions: Scott McPhilips
Last modified:Wednesday, 17-Jan-2024 13:41:15 PST.