Designing a single cell RNA sequencing benchmark dataset to compare protocols and analysis methods (RNAmix_Sort-seq)

Single cell RNA sequencing (scRNA-seq) technology has undergone rapid development in recent years and brings new challenges in data processing and analysis. This has led to an explosion of tailored analysis methods for scRNA-seq to address various biological questions. However, the current lack of gold-standard benchmarking datasets makes it difficult for researchers to evaluate the performance of the many methods available in a systematic manner. Here, we designed and generated a cross-platform benchmark dataset that has in-built truth in various forms and varying levels of biological noise. We used this dataset to compare different protocols and data analysis methods. We found that different protocols have different data quality and ERCC spike-in works independently to endogenous RNA. We found significant differences in the results from the methods compared and we associated the results with data characteristics to identify methods that perform well in different situations. Our dataset and analysis provide a valuable resource for algorithm selection in different biological settings. Overall design: our experiment utilized the 3 human lung adenocarcinoma cell lines H2228, H1975 and HCC827. The experiment included mixtures of RNA and single cells from these cell lines. For the single cell designs, the three cell lines were mixed equally and processed by 10X chromium, Drop-seq and CEL-seq2, referred to as sc_10X, sc_Drop-seq and sc_CEL-seq2 respectively in analysis that follows. For the mixture designs, we used plate-based protocols to mix and dilute samples in 2 different ways. 9 cell mixtures from the 3 cell lines were sorted in different combinations in the cell mixture experiment and data were generated by CEL-seq2, the material after pooling from 384 wells were subsampled in either 1/9 or 1/3 to simulate cells of different sizes, with different PCR product clean up ratios ranging from 0.7 to 0.9, referred to as cellmix1 to cellmix4. For the cell mixture experiment, we also sorted wells with 10 times more cells (90 cells) to provide a pseudo bulk reference for each mixture (referred to as cellmix5). Distinct RNA mixtures which were diluted down to create single cell equivalents (ranging from 3.75, 7.5, 15 to 30 pg per well) were generated using CEL-seq2 and SORT-seq (referred to as RNAmix_CEL-seq2 and RNAmix_Sort-seq.

1

Tian L, Su S, Dong X, Amann-Zalcenstein D, Biben C, Seidi A, Hilton DJ, Naik SH, Ritchie ME