.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/09-analysis_record.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_09-analysis_record.py: Analysis record and replay ========================== Every call to :func:`massphoto.fit`, :func:`massphoto.detect`, and :func:`massphoto.process` automatically attaches an analysis record to the returned DataFrame. This tutorial shows how to: 1. Run an analysis with custom parameters on one movie. 2. Inspect the attached ``analysis_record``. 3. Resolve the record into a replay callable and apply it to a second movie. .. seealso:: :ref:`sphx_glr_auto_examples_02-mass_distribution.py` example for details on custom detection and fitting parameters. :ref:`Analysis Record and Reproducibility ` section in the usage guide. .. GENERATED FROM PYTHON SOURCE LINES 21-22 .. code-block:: Python :dedent: 1 .. GENERATED FROM PYTHON SOURCE LINES 24-30 Record once, replay on another movie ---------------------------------------- An analysis record captures the settings used for Movie 1. Replay applies those settings to Movie 2, producing a separate results table for the new data. The orange path shows how the settings pass between the two analyses. .. GENERATED FROM PYTHON SOURCE LINES 30-32 .. code-block:: Python :dedent: 1 .. image-sg:: /auto_examples/images/sphx_glr_09-analysis_record_001.png :alt: 09 analysis record :srcset: /auto_examples/images/sphx_glr_09-analysis_record_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 80-83 Imports ------- We first import necessary packages and modules. .. GENERATED FROM PYTHON SOURCE LINES 83-90 .. code-block:: Python from pprint import pprint from zenodozipdownloader import ZenodoZipDownloader import massphoto as mp .. GENERATED FROM PYTHON SOURCE LINES 91-98 Download example data --------------------- This tutorial uses two short movies from Zenodo: a calibration movie (``005_mfp1_500x.mp``) and a dynamin movie (``006_dynamin_6.mp``). We analyse the calibration movie with custom parameters and replay the same analysis on the dynamin movie. .. GENERATED FROM PYTHON SOURCE LINES 98-114 .. code-block:: Python downloader = ZenodoZipDownloader(doi="10.5281/zenodo.15800729", download_dir="example_data") files_calib = downloader.download(inner_pattern="005_mfp1_500x.mp") files_dynamin = downloader.download("Figure 2.zip/006_dynamin_6.mp") if not files_calib or not files_dynamin: raise RuntimeError( "Failed to download example data. Please download it manually from https://zenodo.org/record/15800729." ) path_movie_1 = files_calib[0] path_movie_2 = files_dynamin[0] print(f"Movie 1 (calibration): {path_movie_1}") print(f"Movie 2 (dynamin) : {path_movie_2}") .. rst-class:: sphx-glr-script-out .. code-block:: none Movie 1 (calibration): /home/runner/work/massphoto/massphoto/docs/source/_examples/example_data/005_mfp1_500x.mp Movie 2 (dynamin) : /home/runner/work/massphoto/massphoto/docs/source/_examples/example_data/006_dynamin_6.mp .. GENERATED FROM PYTHON SOURCE LINES 115-121 Analyse Movie 1 with custom parameters ---------------------------------------- We run :func:`massphoto.fit` with a shorter averaging window and lower detection thresholds. The ``analysis_record`` is attached to the result DataFrame by default (``attach_record=True``). .. GENERATED FROM PYTHON SOURCE LINES 121-139 .. code-block:: Python # Disable progress bars for cleaner output in this tutorial mp.defaults.set("progress_bar", False) movie_1, _ = mp.read_movie(path_movie_1) fitres_1 = mp.fit( movie_1, ratiometric_args={"navg": 4}, detection_args={ "thresholds": [0.5, 0.4, 0.001], "correlation_step_args": {"navg": 10}, }, movie_correct_drift=True, ) fitres_1[["frame", "x", "y", "contrasts"]].head() .. raw:: html
frame x y contrasts
0 4 42.732599 28.582098 0.002919
1 13 48.013501 40.574262 0.015468
2 21 94.475353 32.253934 0.002621
3 25 136.769493 45.390146 0.001615
4 38 93.386788 21.537367 0.002861


.. GENERATED FROM PYTHON SOURCE LINES 140-148 Inspect the analysis record --------------------------- The record is stored in ``DataFrame.attrs`` under the key ``"analysis_record"``. Schema version 2 records the origin (``fit``, ``detect``, or ``process``), captured call kwargs, a snapshot of ``massphoto.defaults``, and basic movie metadata. Replay requires the movie and any precomputed events again. .. GENERATED FROM PYTHON SOURCE LINES 148-155 .. code-block:: Python record = fitres_1.attrs["analysis_record"] print("schema :", record["schema_version"]) print("origin :", record["origin"]) print("movie_info :", record["movie_info"]) .. rst-class:: sphx-glr-script-out .. code-block:: none schema : 2 origin : fit movie_info : {'shape': (14949, 59, 150), 'dtype': 'uint16'} .. GENERATED FROM PYTHON SOURCE LINES 156-158 The ``origin_kwargs`` entry captures effective arguments to :func:`massphoto.fit`, including resolved defaults, while excluding the movie and precomputed events. .. GENERATED FROM PYTHON SOURCE LINES 158-161 .. code-block:: Python print("\norigin_kwargs:") pprint(record["origin_kwargs"]) .. rst-class:: sphx-glr-script-out .. code-block:: none origin_kwargs: {'attach_record': True, 'chunk_overlap': -1, 'chunk_size': 1000, 'compute_movie_metrics': False, 'detection_args': {'correlation_step_args': {'navg': 10}, 'thresholds': [0.5, 0.4, 0.001]}, 'detection_func': , 'fit_args': None, 'fit_func': , 'movie_correct_drift': True, 'movie_normalize_frame_means': False, 'movie_template': None, 'postprocess': (,), 'progress_bar': False, 'ratiometric_args': {'navg': 4}, 'ratiometric_func': , 'return_ratiometric': False, 'scheduler': None} .. GENERATED FROM PYTHON SOURCE LINES 162-165 ``defaults_state`` is a full snapshot of ``massphoto.defaults`` at the time the analysis ran — including any package-wide settings such as PSF parameters or mass calibration. Here we just confirm the key is present: .. GENERATED FROM PYTHON SOURCE LINES 165-167 .. code-block:: Python print(f"\ndefaults_state: {len(record['defaults_state'])} keys") .. rst-class:: sphx-glr-script-out .. code-block:: none defaults_state: 45 keys .. GENERATED FROM PYTHON SOURCE LINES 168-177 Replay the analysis on Movie 2 -------------------------------- :func:`massphoto.resolve_record` converts the stored record into: - ``defaults_overrides``: the merged defaults state (defaults snapshot + origin kwargs) that will reproduce the run. - ``replay``: a defaults-wrapped :func:`massphoto.fit` callable preloaded with the stored state. Only the new movie needs to be supplied. .. GENERATED FROM PYTHON SOURCE LINES 177-185 .. code-block:: Python overrides, replay = mp.resolve_record(fitres_1) movie_2, _ = mp.read_movie(path_movie_2) fitres_2 = replay(movie_2) fitres_2[["frame", "x", "y", "contrasts"]].head() .. raw:: html
frame x y contrasts
0 4 146.287735 16.464759 0.011452
1 4 182.339746 16.222570 0.006088
2 4 96.719994 21.519629 0.002191
3 4 150.222602 27.896460 0.003025
4 4 62.213479 32.808530 0.003230


.. GENERATED FROM PYTHON SOURCE LINES 186-187 Confirm that the same custom parameters were used. .. GENERATED FROM PYTHON SOURCE LINES 187-191 .. code-block:: Python record_2 = fitres_2.attrs["analysis_record"] print("Replayed origin_kwargs match original:", record_2["origin_kwargs"] == record["origin_kwargs"]) .. rst-class:: sphx-glr-script-out .. code-block:: none Replayed origin_kwargs match original: True .. GENERATED FROM PYTHON SOURCE LINES 192-233 Persistence note ---------------- ``DataFrame.attrs`` is an in-memory dictionary. Tabular exports such as CSV, XLSX, and ``DataFrame.to_hdf`` **do not** preserve it. To save the DataFrame together with its ``attrs`` (including ``analysis_record`` and ``frame_metrics``), use :meth:`pandas.DataFrame.to_pickle` and :func:`pandas.read_pickle`: .. code-block:: python import pandas as pd fitres_1.to_pickle("fitres.pkl") loaded = pd.read_pickle("fitres.pkl") _, replay = mp.resolve_record(loaded) fitres_replayed = replay(movie_1) If you export the event table to another format and want to preserve the record, save the record dictionary alongside the data explicitly: .. code-block:: python import pickle # Save the record separately with open("fitres_record.pkl", "wb") as f: pickle.dump(fitres_1.attrs["analysis_record"], f) # Later, load and replay with open("fitres_record.pkl", "rb") as f: saved_record = pickle.load(f) _, replay = mp.resolve_record(saved_record) fitres_replayed = replay(movie_1) To opt out of attaching the record entirely, pass ``attach_record=False``: .. code-block:: python fitres = mp.fit(movie_1, attach_record=False) .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 56.149 seconds) .. _sphx_glr_download_auto_examples_09-analysis_record.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: 09-analysis_record.ipynb <09-analysis_record.ipynb>` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: 09-analysis_record.py <09-analysis_record.py>` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: 09-analysis_record.zip <09-analysis_record.zip>` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_