{ "cells": [ { "cell_type": "markdown", "id": "8bc02404-8cd1-46d9-8237-2d035ebb3e79", "metadata": {}, "source": [ "# **[Project] Cancer Subtype Classification**" ] }, { "cell_type": "markdown", "id": "0c5076f4", "metadata": {}, "source": [ "# Introduction" ] }, { "cell_type": "markdown", "id": "8a599748", "metadata": {}, "source": [ "The [TCGA Kidney Cancers Dataset](https://archive.ics.uci.edu/dataset/892/tcga+kidney+cancers) is a bulk RNA-seq dataset that contains transcriptome profiles (i.e., gene expression quantification data) of patients diagnosed with three different subtypes of kidney cancers.\n", "This dataset can be used to make predictions about the specific subtype of kidney cancers given the normalized transcriptome profile data.\n", "\n", "The normalized transcriptome profile data is given as **TPM** and **FPKM** for each gene.\n", "\n", "> TPM (Transcripts Per Million) and FPKM (Fragments Per Kilobase Million) are two common methods for quantifying gene expression in RNA sequencing data.\n", "> They both aim to account for the differences in sequencing depth and transcript length when estimating gene expression levels.\n", ">\n", "> **TPM** (Transcripts Per Million):\n", "> - TPM is a measure of gene expression that normalizes for both library size (sequencing depth) and transcript length.\n", "> - The main idea behind TPM is to express the abundance of a transcript relative to the total number of transcripts in a sample, scaled to one million.\n", ">\n", "> **FPKM** (Fragments Per Kilobase Million):\n", "> - FPKM is another method for quantifying gene expression, which is commonly used in older RNA-seq analysis pipelines. It's similar in concept to TPM but differs in the way it's calculated.\n", "> - FPKM also normalizes for library size and transcript length, but it measures gene expression as the number of fragments (i.e., reads) per kilobase of exon model per million reads.\n", ">\n", "> TPM is generally considered more robust to variations in library size, making it a preferred choice in many modern RNA-seq analysis workflows.\n", "\n", "We provide one dataset for each kidney cancer subtype:\n", "\n", "- [TCGA-KICH](https://portal.gdc.cancer.gov/projects/TCGA-KICH): kidney chromophobe (renal clear cell carcinoma)\n", "- [TCGA-KIRC](https://portal.gdc.cancer.gov/projects/TCGA-KIRC): kidney renal clear cell carcinoma\n", "- [TCGA-KIRP](https://portal.gdc.cancer.gov/projects/TCGA-KIRP): kidney renal papillary cell carcinoma\n", "\n", "> This and _much_ more data is openly available on the [NCI Genomic Data Commons (GDC) Data Portal](https://portal.gdc.cancer.gov/)." ] }, { "cell_type": "markdown", "id": "16712787", "metadata": {}, "source": [ "# Data access" ] }, { "cell_type": "markdown", "id": "6421ef6c", "metadata": {}, "source": [ "There are two ways to access the data: via the TNT homepage or the GDC Data Portal." ] }, { "cell_type": "markdown", "id": "b977e8b8", "metadata": {}, "source": [ "## Download from the TNT homepage (_recommended_)" ] }, { "cell_type": "markdown", "id": "800fa7bd", "metadata": {}, "source": [ "The download from the TNT homepage is straightforward:" ] }, { "cell_type": "code", "execution_count": 1, "id": "dda97b16", "metadata": { "tags": [] }, "outputs": [ { "ename": "SyntaxError", "evalue": "invalid syntax (2666948873.py, line 6)", "output_type": "error", "traceback": [ "\u001b[0;36m Cell \u001b[0;32mIn[1], line 6\u001b[0;36m\u001b[0m\n\u001b[0;31m from IPython.display import clear_output(wait=True)\u001b[0m\n\u001b[0m ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n" ] } ], "source": [ "! wget http://www.tnt.uni-hannover.de/edu/vorlesungen/AMLG/data/project-cancer-classification.tar.gz\n", "! tar -xzvf project-cancer-classification.tar.gz\n", "! mv -v project-cancer-classification/ data/\n", "! rm -v project-cancer-classification.tar.gz" ] }, { "cell_type": "markdown", "id": "bc2db880", "metadata": {}, "source": [ "In the `data/` folder you will now find many files in the [TSV format](https://en.wikipedia.org/wiki/Tab-separated_values) ([CSV](https://en.wikipedia.org/wiki/Comma-separated_values)-like with tabs as delimiter) containing the normalized transcriptome profile data.\n", "\n", "To start, you can read a TSV file into a [pandas](https://pandas.pydata.org) [`DataFrame`](pandas dataframe to dict) using the [`pandas.read_csv()`](https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html#pandas-read-csv) function with the `sep` parameter set to `\\t`:" ] }, { "cell_type": "markdown", "id": "ed50d396-fe33-47a7-ad19-8eb975ef0fa5", "metadata": {}, "source": [ "## Lesen der DNA-Sequenz Dateien und speichern in einer Datei" ] }, { "cell_type": "code", "execution_count": 41, "id": "2adae4ff", "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Es wurden 1034 Dateien eingelesen.\n" ] } ], "source": [ "import numpy as np\n", "import pandas as pd\n", "import pickle\n", "\n", "\n", "import os\n", "#'./data/tcga-kirp-geq'\n", "\n", "labels = [\"kirp\", \"kirc\", \"kich\"] # Setzen Sie hier Ihren Ordnerpfad ein\n", "n_files = 0\n", "y = list()\n", "x = list()\n", "\n", "rick = list()\n", "data = []\n", "\n", "for l in labels:\n", " root_folder = f\"./data/tcga-{l}-geq\"\n", " for root, dirs, files in os.walk(root_folder):\n", " for file in files:\n", " if file.endswith('.tsv'):\n", " n_files += 1\n", " # Vollständiger Pfad zur Datei\n", " file_path = os.path.join(root, file)\n", " # Hier können Sie etwas mit der Datei machen, z.B. einlesen\n", " df = pd.read_csv(filepath_or_buffer=file_path, sep=\"\\t\", header=1)\n", " df = df['tpm_unstranded']\n", "\n", " df = df[4:]\n", " df = np.array(df)\n", " rick.append(df)\n", " \n", " data.append([df, l])\n", "\n", "print(f\"Es wurden {n_files} Dateien eingelesen.\")\n", "#tsv_file_path = \"data/tcga-kich-geq/0ba21ef5-0829-422e-a674-d3817498c333/4868e8fc-e045-475a-a81d-ef43eabb7066.rna_seq.augmented_star_gene_counts.tsv\"\n", "\n", "# Read the TSV file into a DataFrame\n", "#df = pd.read_csv(filepath_or_buffer=tsv_file_path, sep=\"\\t\", header=1)\n", "\n", "# Display the first few rows of the DataFrame\n", "#print(df.head(n=20))\n", "rick = np.array(rick)\n", "\n", "# Speichern der 'kirp' Liste in einer Pickle-Datei\n", "with open('rick.pickle', 'wb') as f:\n", " pickle.dump(rick, f)\n" ] }, { "cell_type": "code", "execution_count": 42, "id": "dfe4f964-6068-46da-8103-194525086f01", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", " | genome_frequencies | \n", "cancer_type | \n", "
---|---|---|
0 | \n", "[20.331, 0.0, 25.1806, 1.1301, 0.4836, 7.3269,... | \n", "kirp | \n", "
1 | \n", "[37.0405, 0.5002, 77.4246, 4.2188, 1.0408, 29.... | \n", "kirp | \n", "
2 | \n", "[45.4456, 0.0903, 74.9545, 4.843, 1.5188, 11.8... | \n", "kirp | \n", "
3 | \n", "[15.2345, 0.3393, 62.0003, 2.4412, 0.932, 2.66... | \n", "kirp | \n", "
4 | \n", "[35.0709, 0.2333, 62.8022, 2.8872, 1.0547, 18.... | \n", "kirp | \n", "