First Approach
We initially approached this problem by merging the channels, finding the cell nuclei through blob detection (Laplacian of Gaussian) and tracking them with a simple Linear Assignment Problem (LAP) tracker [8]. Although this gave reasonable results, it couldn't differentiate cells in dense areas and didn't find cells with low intensity.
ImageJ visualization, using blob detection and LAP tracking.
Second Approach
To improve on this, we independently segment each channel and do the tracking on a combination of these segmentations. For this we rely on recent work by Bragantini et al. [1] which introduced a novel method of using segmentation hypothesis to create a hierarchy of segmentations, optimizing tracking and the final segmentation jointly. They do this by formulating the problem as an Integer Linear Programming (ILP) problem which maximizes the intersection of the union of cell segmentations. This problem considers not just pairs of frames, but all the frames, allowing optimization on a global level. Moreover, the authors embed biological constraints related to cell division, disappearing or appearing in the ILP.We start with a preprocessing step to remove background noise. This is done by performing morphological reconstruction through dilation to find the “background” and subtracting that out of the original image. We then do a simple normalization while also eliminating low intensity values which remove leftover noise in the image. Next, the frames are segmented to find the cells. We performed an exhaustive search of all combinations of segmentation inputs to the tracking algorithm and have found that using StarDist and Cellpose on each of the preprocessed fluorescence channels independently works best. Finally, we apply a postprocessing step wherein we detect discontinuities in the color composition of a tracked cell. If we detect a large jump in intensity, we split the track at the given time. Finally, we iterate through each of the split tracks to prune tracks that are too short.
Napari visualization, using Stardist and Cellpose segmentation and LIP formulated
tracking.
References
[1] J. Bragantini, M. Lange and L. Royer, "Large-Scale Multi-Hypotheses Cell Tracking Using Ultrametric Contours Maps," August 2023.[2] U. Schmidt, M. Weigert, C. Broaddus and G. Myers, "Cell Detection with Star-Convex Polygons," in Lecture Notes in Computer Science, Springer International Publishing, 2018, p. 265–273.
[3] C. Stringer, T. Wang, M. Michaelos and M. Pachitariu, "Cellpose: a generalist algorithm for cellular segmentation," Nature Methods, vol. 18, p. 100–106, December 2020.
[4] M. Maška, V. Ulman, P. Delgado-Rodriguez, E. Gómez-de-Mariscal, T. Nečasová, F. A. Guerrero Peña, T. I. Ren, E. M. Meyerowitz, T. Scherr, K. Löffler, R. Mikut, T. Guo, Y. Wang, J. P. Allebach, R. Bao, N. M. Al-Shakarji, G. Rahmon, I. E. Toubal, K. Palaniappan, F. Lux, P. Matula, K. Sugawara, K. E. G. Magnusson, L. Aho, A. R. Cohen, A. Arbelle, T. Ben-Haim, T. R. Raviv, F. Isensee, P. F. Jäger, K. H. Maier-Hein, Y. Zhu, C. Ederra, A. Urbiola, E. Meijering, A. Cunha, A. Muñoz-Barrutia, M. Kozubek and C. Ortiz-de-Solórzano, "The Cell Tracking Challenge: 10 years of objective benchmarking," Nature Methods, vol. 20, p. 1010–1020, May 2023.
[5] D. Reid, "An algorithm for tracking multiple targets," IEEE Transactions on Automatic Control, vol. 24, p. 843–854, December 1979.
[6] F. Jug, E. Levinkov, C. Blasse, E. W. Myers and B. Andres, "Moral lineage tracing," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[7] E. Turetken, X. Wang, C. J. Becker, C. Haubold and P. Fua, "Network Flow Integer Programming to Track Elliptical Cells in Time-Lapse Sequences," IEEE Transactions on Medical Imaging, vol. 36, p. 942–951, April 2017.
[8] K. Jaqaman, D. Loerke, M. Mettlen, H. Kuwata, S. Grinstein, S. L. Schmid and G. Danuser, "Robust single-particle tracking in live-cell time-lapse sequences," Nature Methods, vol. 5, p. 695–702, July 2008.