{ "cells": [ { "cell_type": "markdown", "id": "c000", "metadata": {}, "source": [ "# From a CellProfiler run\n", "\n", "CellProfiler's `ExportToSpreadsheet` writes one CSV per object plus an `Image.csv`. mantispy joins those into a\n", "single AnnData and parses what every column means: which object it describes, which measurement family it belongs\n", "to and which channel it was measured in. Later steps, such as selecting intensity features or building feature\n", "sets, read this annotation instead of re-parsing names.\n", "\n", "{func}`~mantispy.ds.jump_export` downloads one real export, the directory the JUMP pipeline wrote for one field of\n", "view of plate `BR00121438`." ] }, { "cell_type": "code", "execution_count": 1, "id": "c001", "metadata": { "execution": { "iopub.execute_input": "2026-10-22T03:31:57.988471Z", "iopub.status.busy": "2026-10-22T03:31:57.988001Z", "iopub.status.idle": "2026-10-22T03:32:02.318901Z", "shell.execute_reply": "2026-10-22T03:32:02.317923Z" } }, "outputs": [ { "data": { "text/plain": [ "['Cells.csv', 'Cytoplasm.csv', 'Experiment.csv', 'Image.csv', 'Nuclei.csv']" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import tempfile\n", "from pathlib import Path\n", "\n", "import mantispy as mt\n", "\n", "directory = mt.ds.jump_export()\n", "sorted(path.name for path in directory.iterdir())" ] }, { "cell_type": "markdown", "id": "c002", "metadata": {}, "source": [ "`Cells.csv`, `Cytoplasm.csv` and `Nuclei.csv` hold one row per object. `Image.csv` holds one row per field of\n", "view: the plate, well and site it came from, the file names of each channel, and the `MeasureImageQuality`\n", "statistics. `Experiment.csv` records the pipeline itself.\n", "\n", "## Reading\n", "\n", "`primary_object` sets what a row of the result is, and the other objects are joined onto it. The JUMP pipeline\n", "gives `Cytoplasm` a parent cell and a parent nucleus and gives `Cells` no parent among the exported objects, so the cytoplasm is the\n", "object that joins all three tables here, and one row is one cell." ] }, { "cell_type": "code", "execution_count": 2, "id": "c003", "metadata": { "execution": { "iopub.execute_input": "2026-10-22T03:32:02.323615Z", "iopub.status.busy": "2026-10-22T03:32:02.323424Z", "iopub.status.idle": "2026-10-22T03:32:05.028974Z", "shell.execute_reply": "2026-10-22T03:32:05.027981Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ ":179: PerformanceWarning: DataFrame is highly fragmented. This is usually the result of calling `frame.insert` many times, which has poor performance. Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`\n", " table[f\"Metadata_Center_{axis}\"] = table[source].to_numpy()\n", ":179: PerformanceWarning: DataFrame is highly fragmented. This is usually the result of calling `frame.insert` many times, which has poor performance. Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`\n", " table[f\"Metadata_Center_{axis}\"] = table[source].to_numpy()\n", ":182: PerformanceWarning: DataFrame is highly fragmented. This is usually the result of calling `frame.insert` many times, which has poor performance. Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`\n", " table[\"Metadata_ObjectType\"] = primary_object\n", ":183: PerformanceWarning: DataFrame is highly fragmented. This is usually the result of calling `frame.insert` many times, which has poor performance. Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`\n", " table[\"Metadata_ImageID\"] = make_image_id(table)\n" ] }, { "data": { "text/plain": [ "AnnData object with n_obs × n_vars = 239 × 5839\n", " obs: 'Metadata_ImageNumber', 'Metadata_ObjectNumber', 'Metadata_CellsObjectNumber', 'Metadata_NucleiObjectNumber', 'Metadata_AbsPositionZ', 'Metadata_AbsTime', 'Metadata_BinningX', 'Metadata_BinningY', 'Metadata_ChannelID', 'Metadata_ChannelName', 'Metadata_Col', 'Metadata_ExposureTime', 'Metadata_FieldID', 'Metadata_ImageResolutionX', 'Metadata_ImageResolutionY', 'Metadata_ImageSizeX', 'Metadata_ImageSizeY', 'Metadata_MainEmissionWavelength', 'Metadata_MainExcitationWavelength', 'Metadata_MaxIntensity', 'Metadata_ObjectiveMagnification', 'Metadata_ObjectiveNA', 'Metadata_PlaneID', 'Metadata_Plate', 'Metadata_PositionX', 'Metadata_PositionY', 'Metadata_PositionZ', 'Metadata_Row', 'Metadata_Site', 'Metadata_Well', 'Metadata_Center_X', 'Metadata_Center_Y', 'Metadata_ObjectType', 'Metadata_ImageID'\n", " var: 'object', 'feature_group', 'feature', 'channel', 'scale', 'angle', 'gray_levels', 'radial_bin', 'params', 'is_feature', 'feature_kind'\n", " uns: 'mantispy'\n", " layers: None (.X)" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "adata = mt.io.read_profiles(directory, primary_object=\"Cytoplasm\")\n", "adata" ] }, { "cell_type": "code", "execution_count": 3, "id": "a5396615", "metadata": { "execution": { "iopub.execute_input": "2026-10-22T03:32:05.032960Z", "iopub.status.busy": "2026-10-22T03:32:05.032783Z", "iopub.status.idle": "2026-10-22T03:32:05.036811Z", "shell.execute_reply": "2026-10-22T03:32:05.035797Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "239 single-cell objects x 5839 features, resolution=object\n", "obs carries 34 columns, e.g. ['Metadata_ImageNumber', 'Metadata_ObjectNumber', 'Metadata_CellsObjectNumber', 'Metadata_NucleiObjectNumber', 'Metadata_AbsPositionZ']\n" ] } ], "source": [ "print(\n", " f\"{adata.n_obs:,} single-cell objects x {adata.n_vars} features, \"\n", " f\"resolution={adata.uns['mantispy'].get('resolution')}\"\n", ")\n", "print(f\"obs carries {adata.obs.shape[1]} columns, e.g. {list(adata.obs.columns[:5])}\")" ] }, { "cell_type": "markdown", "id": "c004", "metadata": {}, "source": [ "## What landed where\n", "\n", "`X` holds the measurements, one row per cell, as float32." ] }, { "cell_type": "code", "execution_count": 4, "id": "c005", "metadata": { "execution": { "iopub.execute_input": "2026-10-22T03:32:05.039886Z", "iopub.status.busy": "2026-10-22T03:32:05.039715Z", "iopub.status.idle": "2026-10-22T03:32:05.043674Z", "shell.execute_reply": "2026-10-22T03:32:05.042808Z" } }, "outputs": [ { "data": { "text/plain": [ "((239, 5839), dtype('float32'))" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "adata.X.shape, adata.X.dtype" ] }, { "cell_type": "markdown", "id": "c006", "metadata": {}, "source": [ "`obs` holds the columns that identify where a cell came from, all with the `Metadata_` prefix. Among them are the\n", "centroid, `Metadata_Center_X` and `Metadata_Center_Y`. It is not a profile feature, but QC uses it to find cells\n", "on the edge of a field and crowding needs it, so it is kept here." ] }, { "cell_type": "code", "execution_count": 5, "id": "c007", "metadata": { "execution": { "iopub.execute_input": "2026-10-22T03:32:05.045269Z", "iopub.status.busy": "2026-10-22T03:32:05.045105Z", "iopub.status.idle": "2026-10-22T03:32:05.126830Z", "shell.execute_reply": "2026-10-22T03:32:05.125147Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Metadata_PlateMetadata_WellMetadata_SiteMetadata_Center_XMetadata_Center_Y
0BR00121438J041138.50369721.141035
1BR00121438J041522.71934331.477202
2BR00121438J041954.10758236.398345
3BR00121438J041730.24058225.867151
4BR00121438J041626.72803833.968995
\n", "
" ], "text/plain": [ " Metadata_Plate Metadata_Well Metadata_Site Metadata_Center_X \\\n", "0 BR00121438 J04 1 138.503697 \n", "1 BR00121438 J04 1 522.719343 \n", "2 BR00121438 J04 1 954.107582 \n", "3 BR00121438 J04 1 730.240582 \n", "4 BR00121438 J04 1 626.728038 \n", "\n", " Metadata_Center_Y \n", "0 21.141035 \n", "1 31.477202 \n", "2 36.398345 \n", "3 25.867151 \n", "4 33.968995 " ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "adata.obs[[\"Metadata_Plate\", \"Metadata_Well\", \"Metadata_Site\", \"Metadata_Center_X\", \"Metadata_Center_Y\"]].head()" ] }, { "cell_type": "markdown", "id": "c008", "metadata": {}, "source": [ "`var` is the parsed annotation, with one row per feature describing what it measures." ] }, { "cell_type": "code", "execution_count": 6, "id": "c009", "metadata": { "execution": { "iopub.execute_input": "2026-10-22T03:32:05.130757Z", "iopub.status.busy": "2026-10-22T03:32:05.130246Z", "iopub.status.idle": "2026-10-22T03:32:05.146601Z", "shell.execute_reply": "2026-10-22T03:32:05.145409Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
objectfeature_groupfeaturechannelscaleanglegray_levelsradial_binparamsis_featurefeature_kind
Cytoplasm_AreaShape_AreaCytoplasmAreaShapeAreaNaNNaNNaNNaNNaNNaNTruemeasurement
Cytoplasm_AreaShape_BoundingBoxAreaCytoplasmAreaShapeBoundingBoxAreaNaNNaNNaNNaNNaNNaNTruemeasurement
Cytoplasm_AreaShape_CompactnessCytoplasmAreaShapeCompactnessNaNNaNNaNNaNNaNNaNTruemeasurement
Cytoplasm_AreaShape_EccentricityCytoplasmAreaShapeEccentricityNaNNaNNaNNaNNaNNaNTruemeasurement
Cytoplasm_AreaShape_EquivalentDiameterCytoplasmAreaShapeEquivalentDiameterNaNNaNNaNNaNNaNNaNTruemeasurement
\n", "
" ], "text/plain": [ " object feature_group \\\n", "Cytoplasm_AreaShape_Area Cytoplasm AreaShape \n", "Cytoplasm_AreaShape_BoundingBoxArea Cytoplasm AreaShape \n", "Cytoplasm_AreaShape_Compactness Cytoplasm AreaShape \n", "Cytoplasm_AreaShape_Eccentricity Cytoplasm AreaShape \n", "Cytoplasm_AreaShape_EquivalentDiameter Cytoplasm AreaShape \n", "\n", " feature channel scale \\\n", "Cytoplasm_AreaShape_Area Area NaN NaN \n", "Cytoplasm_AreaShape_BoundingBoxArea BoundingBoxArea NaN NaN \n", "Cytoplasm_AreaShape_Compactness Compactness NaN NaN \n", "Cytoplasm_AreaShape_Eccentricity Eccentricity NaN NaN \n", "Cytoplasm_AreaShape_EquivalentDiameter EquivalentDiameter NaN NaN \n", "\n", " angle gray_levels radial_bin params \\\n", "Cytoplasm_AreaShape_Area NaN NaN NaN NaN \n", "Cytoplasm_AreaShape_BoundingBoxArea NaN NaN NaN NaN \n", "Cytoplasm_AreaShape_Compactness NaN NaN NaN NaN \n", "Cytoplasm_AreaShape_Eccentricity NaN NaN NaN NaN \n", "Cytoplasm_AreaShape_EquivalentDiameter NaN NaN NaN NaN \n", "\n", " is_feature feature_kind \n", "Cytoplasm_AreaShape_Area True measurement \n", "Cytoplasm_AreaShape_BoundingBoxArea True measurement \n", "Cytoplasm_AreaShape_Compactness True measurement \n", "Cytoplasm_AreaShape_Eccentricity True measurement \n", "Cytoplasm_AreaShape_EquivalentDiameter True measurement " ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "adata.var.head()" ] }, { "cell_type": "markdown", "id": "c010", "metadata": {}, "source": [ "Channel names are not hardcoded. CellProfiler writes one `Intensity_MeanIntensity_` measurement per channel\n", "it measured, and the names are read from those. The file names in `Image.csv` can differ: this run loaded `OrigDNA`,\n", "corrected it with `IllumDNA` and measured the result as `DNA`. A two-channel assay with channels called `Hoechst`\n", "and `GFP` parses the same way as a Cell Painting run {cite:p}`Bray_2016`. This one measured five fluorescence channels and three\n", "brightfield planes." ] }, { "cell_type": "code", "execution_count": 7, "id": "c011", "metadata": { "execution": { "iopub.execute_input": "2026-10-22T03:32:05.149276Z", "iopub.status.busy": "2026-10-22T03:32:05.148974Z", "iopub.status.idle": "2026-10-22T03:32:05.155856Z", "shell.execute_reply": "2026-10-22T03:32:05.154772Z" } }, "outputs": [ { "data": { "text/plain": [ "['AGP', 'BFHigh', 'BFLow', 'Brightfield', 'DNA', 'ER', 'Mito', 'RNA']" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sorted(c for c in adata.var[\"channel\"].dropna().unique() if \"|\" not in c)" ] }, { "cell_type": "markdown", "id": "c012", "metadata": {}, "source": [ "## Columns that are not features\n", "\n", "CellProfiler also writes columns that are not measurements: object numbers, parent links, child counts and\n", "positions. The parser marks them `is_feature = False` and keeps them out of `X`. CellProfiler 4 writes the centroid\n", "and the bounding box under `AreaShape`, beside the shape measurements, and those are positions as well." ] }, { "cell_type": "code", "execution_count": 8, "id": "c013", "metadata": { "execution": { "iopub.execute_input": "2026-10-22T03:32:05.158394Z", "iopub.status.busy": "2026-10-22T03:32:05.158178Z", "iopub.status.idle": "2026-10-22T03:32:05.167863Z", "shell.execute_reply": "2026-10-22T03:32:05.166108Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
objectfeature_groupfeatureis_feature
Cytoplasm_AreaShape_AreaCytoplasmAreaShapeAreaTrue
Cytoplasm_AreaShape_BoundingBoxAreaCytoplasmAreaShapeBoundingBoxAreaTrue
Cytoplasm_AreaShape_CompactnessCytoplasmAreaShapeCompactnessTrue
Cytoplasm_AreaShape_EccentricityCytoplasmAreaShapeEccentricityTrue
Cytoplasm_AreaShape_EquivalentDiameterCytoplasmAreaShapeEquivalentDiameterTrue
...............
Nuclei_Texture_Variance_RNA_3_03_256NucleiTextureVarianceTrue
Nuclei_Texture_Variance_RNA_5_00_256NucleiTextureVarianceTrue
Nuclei_Texture_Variance_RNA_5_01_256NucleiTextureVarianceTrue
Nuclei_Texture_Variance_RNA_5_02_256NucleiTextureVarianceTrue
Nuclei_Texture_Variance_RNA_5_03_256NucleiTextureVarianceTrue
\n", "

5839 rows × 4 columns

\n", "
" ], "text/plain": [ " object feature_group \\\n", "Cytoplasm_AreaShape_Area Cytoplasm AreaShape \n", "Cytoplasm_AreaShape_BoundingBoxArea Cytoplasm AreaShape \n", "Cytoplasm_AreaShape_Compactness Cytoplasm AreaShape \n", "Cytoplasm_AreaShape_Eccentricity Cytoplasm AreaShape \n", "Cytoplasm_AreaShape_EquivalentDiameter Cytoplasm AreaShape \n", "... ... ... \n", "Nuclei_Texture_Variance_RNA_3_03_256 Nuclei Texture \n", "Nuclei_Texture_Variance_RNA_5_00_256 Nuclei Texture \n", "Nuclei_Texture_Variance_RNA_5_01_256 Nuclei Texture \n", "Nuclei_Texture_Variance_RNA_5_02_256 Nuclei Texture \n", "Nuclei_Texture_Variance_RNA_5_03_256 Nuclei Texture \n", "\n", " feature is_feature \n", "Cytoplasm_AreaShape_Area Area True \n", "Cytoplasm_AreaShape_BoundingBoxArea BoundingBoxArea True \n", "Cytoplasm_AreaShape_Compactness Compactness True \n", "Cytoplasm_AreaShape_Eccentricity Eccentricity True \n", "Cytoplasm_AreaShape_EquivalentDiameter EquivalentDiameter True \n", "... ... ... \n", "Nuclei_Texture_Variance_RNA_3_03_256 Variance True \n", "Nuclei_Texture_Variance_RNA_5_00_256 Variance True \n", "Nuclei_Texture_Variance_RNA_5_01_256 Variance True \n", "Nuclei_Texture_Variance_RNA_5_02_256 Variance True \n", "Nuclei_Texture_Variance_RNA_5_03_256 Variance True \n", "\n", "[5839 rows x 4 columns]" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "adata.var[[\"object\", \"feature_group\", \"feature\", \"is_feature\"]]" ] }, { "cell_type": "markdown", "id": "c014", "metadata": {}, "source": [ "## How objects are joined\n", "\n", "`Cells` and `Nuclei` measurements are joined onto the cytoplasm they belong to rather than added as extra rows, so\n", "one row still means one cell and `Nuclei_AreaShape_Area` sits next to `Cells_AreaShape_Area`. The link is read\n", "from the `Parent_` columns. Matching on object number instead would pair unrelated objects that happen to share a\n", "number, without any warning, so a missing link raises an error, and by default the join must be one-to-one." ] }, { "cell_type": "code", "execution_count": 9, "id": "c015", "metadata": { "execution": { "iopub.execute_input": "2026-10-22T03:32:05.170313Z", "iopub.status.busy": "2026-10-22T03:32:05.170141Z", "iopub.status.idle": "2026-10-22T03:32:05.178833Z", "shell.execute_reply": "2026-10-22T03:32:05.177877Z" } }, "outputs": [ { "data": { "text/plain": [ "['Nuclei_AreaShape_Area',\n", " 'Nuclei_AreaShape_BoundingBoxArea',\n", " 'Nuclei_AreaShape_Compactness',\n", " 'Nuclei_AreaShape_Eccentricity']" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sorted(name for name in adata.var_names if name.startswith(\"Nuclei\"))[:4]" ] }, { "cell_type": "markdown", "id": "c016", "metadata": {}, "source": [ "## Image quality\n", "\n", "`MeasureImageQuality` describes a field of view rather than a cell, so it does not belong in `X`. It is stored\n", "separately, keyed by image, together with the plate and well, so that image QC can set thresholds per plate." ] }, { "cell_type": "code", "execution_count": 10, "id": "c017", "metadata": { "execution": { "iopub.execute_input": "2026-10-22T03:32:05.182231Z", "iopub.status.busy": "2026-10-22T03:32:05.182094Z", "iopub.status.idle": "2026-10-22T03:32:05.193822Z", "shell.execute_reply": "2026-10-22T03:32:05.192586Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
ImageQuality_FocusScore_OrigAGPImageQuality_FocusScore_OrigBrightfieldImageQuality_FocusScore_OrigBrightfield_HImageQuality_FocusScore_OrigBrightfield_LImageQuality_FocusScore_OrigDNAImageQuality_FocusScore_OrigERImageQuality_FocusScore_OrigMitoImageQuality_FocusScore_OrigRNAImageQuality_LocalFocusScore_OrigAGP_10ImageQuality_LocalFocusScore_OrigAGP_20...ImageQuality_LocalFocusScore_OrigER_5ImageQuality_LocalFocusScore_OrigER_50ImageQuality_LocalFocusScore_OrigMito_10ImageQuality_LocalFocusScore_OrigMito_20ImageQuality_LocalFocusScore_OrigMito_5ImageQuality_LocalFocusScore_OrigMito_50ImageQuality_LocalFocusScore_OrigRNA_10ImageQuality_LocalFocusScore_OrigRNA_20ImageQuality_LocalFocusScore_OrigRNA_5ImageQuality_LocalFocusScore_OrigRNA_50
Metadata_ImageID
BR00121438|J04|1|19720.0209280.0001150.0003310.0002090.1038670.0591470.0031040.0587020.037780.038886...0.0184650.0569210.0221520.0134220.0262180.0053060.046270.0743920.0291230.070208
\n", "

1 rows × 40 columns

\n", "
" ], "text/plain": [ " ImageQuality_FocusScore_OrigAGP \\\n", "Metadata_ImageID \n", "BR00121438|J04|1|1972 0.020928 \n", "\n", " ImageQuality_FocusScore_OrigBrightfield \\\n", "Metadata_ImageID \n", "BR00121438|J04|1|1972 0.000115 \n", "\n", " ImageQuality_FocusScore_OrigBrightfield_H \\\n", "Metadata_ImageID \n", "BR00121438|J04|1|1972 0.000331 \n", "\n", " ImageQuality_FocusScore_OrigBrightfield_L \\\n", "Metadata_ImageID \n", "BR00121438|J04|1|1972 0.000209 \n", "\n", " ImageQuality_FocusScore_OrigDNA \\\n", "Metadata_ImageID \n", "BR00121438|J04|1|1972 0.103867 \n", "\n", " ImageQuality_FocusScore_OrigER \\\n", "Metadata_ImageID \n", "BR00121438|J04|1|1972 0.059147 \n", "\n", " ImageQuality_FocusScore_OrigMito \\\n", "Metadata_ImageID \n", "BR00121438|J04|1|1972 0.003104 \n", "\n", " ImageQuality_FocusScore_OrigRNA \\\n", "Metadata_ImageID \n", "BR00121438|J04|1|1972 0.058702 \n", "\n", " ImageQuality_LocalFocusScore_OrigAGP_10 \\\n", "Metadata_ImageID \n", "BR00121438|J04|1|1972 0.03778 \n", "\n", " ImageQuality_LocalFocusScore_OrigAGP_20 ... \\\n", "Metadata_ImageID ... \n", "BR00121438|J04|1|1972 0.038886 ... \n", "\n", " ImageQuality_LocalFocusScore_OrigER_5 \\\n", "Metadata_ImageID \n", "BR00121438|J04|1|1972 0.018465 \n", "\n", " ImageQuality_LocalFocusScore_OrigER_50 \\\n", "Metadata_ImageID \n", "BR00121438|J04|1|1972 0.056921 \n", "\n", " ImageQuality_LocalFocusScore_OrigMito_10 \\\n", "Metadata_ImageID \n", "BR00121438|J04|1|1972 0.022152 \n", "\n", " ImageQuality_LocalFocusScore_OrigMito_20 \\\n", "Metadata_ImageID \n", "BR00121438|J04|1|1972 0.013422 \n", "\n", " ImageQuality_LocalFocusScore_OrigMito_5 \\\n", "Metadata_ImageID \n", "BR00121438|J04|1|1972 0.026218 \n", "\n", " ImageQuality_LocalFocusScore_OrigMito_50 \\\n", "Metadata_ImageID \n", "BR00121438|J04|1|1972 0.005306 \n", "\n", " ImageQuality_LocalFocusScore_OrigRNA_10 \\\n", "Metadata_ImageID \n", "BR00121438|J04|1|1972 0.04627 \n", "\n", " ImageQuality_LocalFocusScore_OrigRNA_20 \\\n", "Metadata_ImageID \n", "BR00121438|J04|1|1972 0.074392 \n", "\n", " ImageQuality_LocalFocusScore_OrigRNA_5 \\\n", "Metadata_ImageID \n", "BR00121438|J04|1|1972 0.029123 \n", "\n", " ImageQuality_LocalFocusScore_OrigRNA_50 \n", "Metadata_ImageID \n", "BR00121438|J04|1|1972 0.070208 \n", "\n", "[1 rows x 40 columns]" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "adata.uns[\"mantispy\"][\"image_table\"].filter(like=\"FocusScore\")" ] }, { "cell_type": "markdown", "id": "c018", "metadata": {}, "source": [ "## What each well received\n", "\n", "An export knows its plate, well and site, and nothing about what the well was treated with. For a JUMP plate the\n", "annotation is a join, once the laboratory is named. `annotate_jump` expects well-level input, so on this\n", "per-cell object it warns; that is expected here, where the well's annotation is simply repeated for every\n", "cell:" ] }, { "cell_type": "code", "execution_count": 11, "id": "c019", "metadata": { "execution": { "iopub.execute_input": "2026-10-22T03:32:05.196148Z", "iopub.status.busy": "2026-10-22T03:32:05.195795Z", "iopub.status.idle": "2026-10-22T03:32:07.122151Z", "shell.execute_reply": "2026-10-22T03:32:07.119932Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ ":2: UserWarning: this function expects 'well' resolution but the object is annotated 'object'; results may not mean what you expect\n", " mt.pp.annotate_jump(adata)\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Metadata_WellMetadata_PerturbationMetadata_Control
0J04JCP2022_033924True
\n", "
" ], "text/plain": [ " Metadata_Well Metadata_Perturbation Metadata_Control\n", "0 J04 JCP2022_033924 True" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "adata.obs[\"Metadata_Source\"] = \"source_4\"\n", "mt.pp.annotate_jump(adata)\n", "adata.obs[[\"Metadata_Well\", \"Metadata_Perturbation\", \"Metadata_Control\"]].drop_duplicates()" ] }, { "cell_type": "markdown", "id": "c020", "metadata": {}, "source": [ "For your own screen, `read_profiles(directory, platemap=\"platemap.csv\")` joins a table mapping wells to treatments,\n", "and `mt.pp.annotate_controls(adata, negcon=(\"DMSO\",))` marks the negative controls, which several later steps use.\n", "\n", "## Checking the contract\n", "\n", "`validate` returns a report instead of raising, so you can inspect a partly formed object." ] }, { "cell_type": "code", "execution_count": 12, "id": "c021", "metadata": { "execution": { "iopub.execute_input": "2026-10-22T03:32:07.125005Z", "iopub.status.busy": "2026-10-22T03:32:07.124569Z", "iopub.status.idle": "2026-10-22T03:32:07.155101Z", "shell.execute_reply": "2026-10-22T03:32:07.154072Z" } }, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "report = mt.io.validate(adata)\n", "report.ok" ] }, { "cell_type": "code", "execution_count": 13, "id": "c022", "metadata": { "execution": { "iopub.execute_input": "2026-10-22T03:32:07.157718Z", "iopub.status.busy": "2026-10-22T03:32:07.157296Z", "iopub.status.idle": "2026-10-22T03:32:07.176985Z", "shell.execute_reply": "2026-10-22T03:32:07.175946Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ERROR: obs is missing required column 'Metadata_Plate' (resolution 'object')\n" ] } ], "source": [ "broken = adata.copy()\n", "broken.obs = broken.obs.drop(columns=\"Metadata_Plate\")\n", "print(mt.io.validate(broken))" ] }, { "cell_type": "markdown", "id": "c023", "metadata": {}, "source": [ "## Writing\n", "\n", "`write` validates first, then writes h5ad (or zarr for a `.zarr` suffix). The schema\n", "version is stored in the file and checked on read, so a file written by a future\n", "incompatible version raises an error instead of loading incorrectly." ] }, { "cell_type": "code", "execution_count": 14, "id": "c024", "metadata": { "execution": { "iopub.execute_input": "2026-10-22T03:32:07.179309Z", "iopub.status.busy": "2026-10-22T03:32:07.178917Z", "iopub.status.idle": "2026-10-22T03:32:07.661075Z", "shell.execute_reply": "2026-10-22T03:32:07.659567Z" } }, "outputs": [ { "data": { "text/plain": [ "AnnData object with n_obs × n_vars = 239 × 5839\n", " obs: 'Metadata_ImageNumber', 'Metadata_ObjectNumber', 'Metadata_CellsObjectNumber', 'Metadata_NucleiObjectNumber', 'Metadata_AbsPositionZ', 'Metadata_AbsTime', 'Metadata_BinningX', 'Metadata_BinningY', 'Metadata_ChannelID', 'Metadata_ChannelName', 'Metadata_Col', 'Metadata_ExposureTime', 'Metadata_FieldID', 'Metadata_ImageResolutionX', 'Metadata_ImageResolutionY', 'Metadata_ImageSizeX', 'Metadata_ImageSizeY', 'Metadata_MainEmissionWavelength', 'Metadata_MainExcitationWavelength', 'Metadata_MaxIntensity', 'Metadata_ObjectiveMagnification', 'Metadata_ObjectiveNA', 'Metadata_PlaneID', 'Metadata_Plate', 'Metadata_PositionX', 'Metadata_PositionY', 'Metadata_PositionZ', 'Metadata_Row', 'Metadata_Site', 'Metadata_Well', 'Metadata_Center_X', 'Metadata_Center_Y', 'Metadata_ObjectType', 'Metadata_ImageID', 'Metadata_Source', 'Metadata_JCP2022', 'Metadata_InChIKey', 'Metadata_Perturbation', 'Metadata_Perturbation_Type', 'Metadata_Control_Type', 'Metadata_Control'\n", " var: 'object', 'feature_group', 'feature', 'channel', 'scale', 'angle', 'gray_levels', 'radial_bin', 'params', 'is_feature', 'feature_kind'\n", " uns: 'mantispy'\n", " layers: None (.X)" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "path = Path(tempfile.mkdtemp()) / \"cells.h5ad\"\n", "mt.io.write(adata, path)\n", "mt.io.read(path)" ] }, { "cell_type": "markdown", "id": "c025", "metadata": {}, "source": [ "## Already have profiles?\n", "\n", "[Published profiles and JUMP](profiles.ipynb) reads well-level tables from pycytominer {cite:p}`Serrano_2025`,\n", "CytoTable, the Cell Painting Gallery {cite:p}`Weisbart_2024` and JUMP." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.15" } }, "nbformat": 4, "nbformat_minor": 5 }