CytoGuard: detecting cellular injury in Cell Painting screens

High-throughput screens routinely produce "hits" whose activity comes from cellular injury rather than on-target engagement: redox cycling, nonselective covalent reactivity, membrane disruption, cytoskeletal collapse or DNA damage. These nuisance compounds consume follow-up resources and, if missed, resurface later as toxicity liabilities. Dahlin et al. (2023) assembled 218 prototypical cytotoxic and nuisance compounds, profiled them with Cell Painting in U-2 OS cells across a dose range, and showed that injury mechanisms form distinct morphological clusters. Serrano et al. (2025) used the processed profiles as a case study for Pycytominer, training a multi-class logistic regression to predict 15 injury categories.

This project turns those reference data into a tool a screening scientist can use to triage new compounds. That takes three things beyond a classifier: an evaluation that reflects never-seen compounds, a pipeline that works from raw images rather than pre-computed features, and an interface that shows the evidence behind each call.

Contributions.

1. A leakage-aware evaluation of injury classification, with compound-grouped cross-validation, a permutation control and decision-level (per-compound) metrics, together with a copairs-based validation of profile quality that is independent of any classifier.
2. A classifier that outperforms the published baseline under the baseline's own protocols and reaches a macro-F1 of 0.462 on held-out compounds.
3. An image-to-decision pipeline (OME-Zarr, Cellpose-SAM, cp_measure, Pycytominer) whose outputs are bridged into the reference feature space and validated against the profile track.
4. A deployable platform (FastAPI, job queue, plugin system, web viewer) with tests at every stage and a single-command reproducible workflow.

Reference set. IDR study idr0133, screen A: 84 plates of U-2 OS cells, Cell Painting (DNA, ER, RNA/nucleoli, AGP, mitochondria), 20x Opera Phenix, 24 h and 48 h treatments. I used the labeled per-well CellProfiler profiles released with the Pycytominer case study (labeled_cell_injury_profile.csv.gz, WayScience/predicting-cell-injury-compounds v2.1.1): 16,701 wells, 372 features, 144 annotated compounds plus DMSO, 1,111 distinct treatments (compound x concentration x time).

Raw images. For the pixel track I retrieved raw 16-bit TIFFs (2160 x 2160, one per channel and field) for 25 wells of plate BR00114096 from the EBI mirror of IDR, verifying each file against the published SHA-1 checksums. The selection comprises 8 randomly chosen DMSO wells and, for each injury class present on the plate, the highest-concentration well of up to two distinct compounds (17 wells, 11 classes). The IDR OMERO rendering service could not serve pixels for this study at the time of writing, so the file mirror was used directly.

3.1 Profile processing

Profiles were renamed into a tidy Metadata_* schema and cast to float32. Pycytominer feature_select removed low-variance features, one feature from each pair with Pearson correlation above 0.9, features with missing values, and features on the standard blocklist, leaving 352 features. The released profiles are already plate-normalized, so no further normalization was applied in the profile track.

3.2 Profile quality with mean average precision

I used copairs to compute average precision for every well and aggregated to mean average precision (mAP) per group, with significance from a 10,000-permutation null and Benjamini-Hochberg correction at 5%.

• Phenotypic activity. Positives are replicate wells of the same treatment. Negatives are DMSO wells on the same plate, which controls for plate effects and keeps the comparison local. A treatment is "active" if its mAP is significant.
• Injury consistency. Treatment-level consensus profiles (median) of active treatments were compared with positives defined as different compounds sharing an injury label, and negatives as treatments with a different label. A class scores well only if its phenotype generalizes across chemistry.

3.3 Injury classification and evaluation design

Two model families were evaluated: an L2-regularized multinomial logistic regression (standardized inputs, C = 0.05, balanced class weights), matching the form of the published baseline, and LightGBM (300 trees, learning rate 0.05, 31 leaves, row subsampling 0.8, feature subsampling 0.5, balanced class weights). Hyperparameters were fixed a priori rather than tuned on test folds.

In the compound holdout every well receives an out-of-fold prediction from a model that saw no well, dose or plate of that compound. A permutation control shuffles labels at the compound level and repeats the full procedure. Metrics are macro-F1 over the classes present, accuracy, and an injury AUROC that uses 1 - P(Control) as a score for "injured vs DMSO". Because triage decisions are made per compound, I also report decision-level accuracy: each compound's class probabilities are averaged over its wells with an injury score of at least 0.5 (falling back to all its wells), and I check whether the true mechanism is ranked first or within the top three.

Baseline. Per-class F1 of the published model was taken from the released fs_all_f1_scores.csv.gz and averaged over its 15 classes, as in my macro-F1. Inspection of the baseline's split code showed that its treatment holdout picks, for each injury with at least 10 compounds, the compound returned by df.min().iloc[1], which is the alphabetically first compound name rather than the one with the fewest wells. That holdout therefore contains only a few compounds, and its score is compared with my compound holdout only directionally.

3.4 Pixel pipeline

1. Ingest. The five channel planes of a field are stacked, 2x2 mean-binned (to 1080 x 1080) and written as a three-level multiscale OME-Zarr (NGFF 0.4) with ngff-zarr, including channel names and physical scale.
2. Segmentation. Nuclei are segmented from the DNA channel and cells from a cytoplasm composite (normalized RNA + AGP, with DNA as a second channel) using Cellpose-SAM (cpsam_v2) on Apple MPS. A classical backend (Otsu threshold with distance-transform watershed for nuclei, then nucleus-seeded watershed on the cytoplasm composite) is provided as a fallback and comparison. Nuclei and cells are matched one-to-one by greedy maximal overlap. Unmatched nuclei receive a 6-pixel grown cell and cells without a nucleus are discarded, following CellProfiler's primary/secondary object logic. The cytoplasm is the cell minus the nucleus. Masks are written back into the OME-Zarr as NGFF labels.
3. Measurement. cp_measure computes intensity, size and shape, Zernike, Feret, radial distribution and Haralick texture per object for nuclei, cells and cytoplasm. Texture was computed at binned scales 3, 5 and 10 pixels, which correspond to the reference's full-resolution scales 5, 10 and 20. Features were renamed into CellProfiler's Compartment_Module_Feature_Channel[_Scale_Direction] scheme. 329 of the 352 selected reference features were recovered; the remainder are Gabor features, which cp_measure does not implement.
4. Profiling. Per-cell features are median-aggregated per well and z-scored against the DMSO wells in the selection (median and standard deviation, with a floor on the standard deviation).

3.5 Bridge model

Absolute feature values differ between CellProfiler 4 and cp_measure, and between the full-resolution reference and the binned demo images. I therefore express both domains relative to their own vehicle controls: reference wells are z-scored per plate against that plate's DMSO wells, and demo wells against the demo DMSO wells. A LightGBM model with the settings above is trained on the reference wells restricted to the 329 shared features, excluding the demo plate and every demo compound. Every pixel-track prediction is therefore a never-seen-compound prediction on features from a different software stack.

3.6 Platform

The library exposes each stage as a function. Analyses are plugins implementing run(PlateContext) -> PluginResult, discovered through the cytoguard.plugins entry-point group. Three are built in: injury flags (per-compound worst-well triage), similarity search (cosine similarity after per-feature scaling, excluding the query's own compound), and plate spatial QC. The spatial plugin tests DMSO wells for edge-versus-interior (Mann-Whitney U), row and column effects (Kruskal-Wallis), with Bonferroni correction and a minimum effect size of 0.05 in injury score, because with about 100 DMSO wells per plate trivially small shifts are otherwise significant. A FastAPI service serves plates, wells, similarity, jobs and decisions. Jobs run on Redis/RQ when configured (with separate gpu and default queues) or on an in-process pool otherwise. The web client renders the plate map, a canvas-based five-channel compositor with mask overlays, a triage table with persisted decisions, and the QC and model views.

4.1 Profiles carry strong, mechanism-specific signal

Of 1,111 treatments, 88% were retrieved against plate-matched DMSO with significant mAP. Activity was highest for mTOR, HDAC and Hsp90 inhibitors (100%) and lowest for saponins (69%), redox cyclers (79%) and tannins (79%), consistent with the weaker or dose-limited phenotypes of those classes.

Cross-compound injury consistency was significant for 13 of 14 classes. The strongest were HDAC (mAP 0.67), genotoxins (0.56), Hsp90 (0.44) and cytoskeletal agents (0.36). The only non-significant class was Miscellaneous (0.27), a heterogeneous catch-all label for which consistency is not expected. Mitochondrial toxicants (0.05) and nonspecific reactives (0.08) were significant but weak, and they are also among the hardest classes to predict (Section 4.3).

4.2 Classification

My logistic regression already matches or exceeds the baseline under its own protocols, and LightGBM adds a further 0.14 to 0.18 macro-F1. The drop from plate holdout (0.819) to compound holdout (0.462) quantifies how much of the "easy" performance comes from recognizing specific compounds. Even so, performance on never-seen compounds is far above the permutation control, and injured wells are separated from healthy ones with an AUROC of 0.947. In triage the injury score drives the keep or drop decision, and the mechanism label guides interpretation, so this separation matters more than mechanism accuracy.

4.3 Where the model fails, and why

Per-class F1 on held-out compounds (LightGBM): Control 0.91, Cytoskeletal 0.91, HDAC 0.90, Genotoxin 0.61, Proteasome 0.61, Hsp90 0.49, Ferroptosis 0.49, Miscellaneous 0.44, Kinase 0.42, Saponin 0.34, Tannin 0.28, Redox 0.26, Nonspecific reactive 0.22, Mitochondria 0.02, mTOR 0.00. The main error patterns are:

4.4 From raw pixels

Cellpose-SAM processed each 1080 x 1080 field in about 29 s on an Apple M4 GPU (MPS); cp_measure took 2 to 22 s per field depending on cell number.

Cellpose-SAM beats the classical segmenter on every accuracy row, and the gap is largest in the separation between treated and control wells.

Concordance with the profile track. On the same 25 wells, pixel-track injury scores correlate with the out-of-fold profile-track scores at r = 0.95, and the two agree on flagged versus healthy for 24 of 25 wells. The treated wells called healthy by both tracks, DZNep (relative cell count 0.94), BSI-201 (1.10) and (R)-MG132 (0.89), show near-normal cell counts, so these compounds had no visible effect in this experiment. (R)-MG132 also serves as an internal check. Its stereoisomer (S)-MG132, tested on the same plate at the same concentration, is flagged with an injury score of 1.00 and an 89% loss of cells, and both tracks and the cell counts agree on this difference. The single disagreement is wortmannin (pixel 0.28, profile 1.00), a compound with a subtle phenotype and near-normal cell count (0.85).

Mechanism naming across the software gap is weaker (5 of 17 top-1, 9 of 17 top-3) and skews toward Miscellaneous. Detection depends on large, robust shifts (cell loss, rounding, nuclear changes) that survive differences in feature implementation and resolution. Mechanism discrimination depends on finer texture and distribution features, which are the ones most affected by the CellProfiler-to-cp_measure translation, the missing Gabor features, single-field sampling and binning.

4.5 Plate quality control

On the reference plate BR00114096, DMSO wells showed a statistically significant edge-versus-interior difference of -0.011 in injury score, which the effect-size criterion correctly does not flag, and a row effect with a spread of 0.082 across row medians, which is flagged for review. Without the effect-size criterion both would be reported. With about 100 DMSO wells per plate, significance alone flags shifts too small to matter.

Cellular injury is detectable from Cell Painting with high reliability even for never-seen compounds, and the detection holds up when the whole pipeline is rebuilt from raw pixels. Mechanism assignment is useful but more fragile: excellent for cytoskeletal, HDAC and genotoxic agents, weak for mechanisms with small reference sets or diffuse phenotypes. In screening triage, the injury score gates the decision, and the mechanism probabilities, similar reference compounds and images give the scientist evidence for the final call.

The gap between plate-holdout and compound-holdout performance (0.82 vs 0.46 macro-F1) shows how much of the easier splits' performance comes from recognizing known compounds. Reporting only random or plate splits overstates how well a model will behave on a new library. Compound-grouped evaluation, permutation controls and decision-level metrics should be the default for this task.

• Label granularity. Mechanism labels are per compound and ignore dose response, which penalizes correct "healthy" calls at inactive doses and caps achievable well-level accuracy.
• Class imbalance and size. Five classes have four or fewer compounds. Estimates for these classes are high-variance.
• Pixel-track scope. One field per well, 2x binning, one plate, and a selection of the highest concentrations. The pixel-track numbers are a feasibility and concordance demonstration, not a benchmark.
• Feature-space mismatch. The bridge model reduces the differences between CellProfiler 4 and cp_measure but does not remove them. The fix is to re-profile the full reference set with the same stack used at inference.
• Single cell line and assay. All data are U-2 OS Cell Painting from one laboratory. Transfer to other cell types, stains and microscopes is untested.

All results are produced by snakemake -s workflow/Snakefile -c4, which downloads and checksums the data, runs the profile benchmark, selects and processes the demo wells, and renders the results page. Random seeds are fixed. The test suite (20 tests, about 3 s) runs on a synthetic dataset with the same schema, so continuous integration never depends on external downloads. Software versions used: Python 3.11, Cellpose 4.2, cp_measure 0.2, Pycytominer 1.7, copairs 0.5, ngff-zarr 0.47, scikit-learn 1.9, LightGBM 4.7, PyTorch 2.14.

Compute. The profile benchmark (mAP, three splits, two models, permutation control) takes about 3.5 minutes on an Apple M4 laptop. The pixel track for 25 wells takes about 20 minutes with Cellpose-SAM on MPS, or about 6 minutes with the classical segmenter.