{ "cells": [ { "cell_type": "markdown", "id": "aca70817", "metadata": {}, "source": [ "# SkillNet AI Scientist: From Task to Discovery\n", "\n", "This notebook demonstrates the **SkillNet** workflow where an AI Agent autonomously plans and executes a scientific mission.\n", "\n", "## Workflow Lifecycle\n", "1. **Task Definition**: User provides a high-level research goal.\n", "2. **AI Planning**: The Agent decomposes the goal into logical steps.\n", "3. **Skill Discovery**: The Agent searches the **SkillNet** for specialized skills matching each step.\n", "4. **Acquisition & Orchestration**: Skills are downloaded, validated, and composed into a pipeline.\n", "5. **Execution**: The skills interact to generate scientific results.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "bc3bc61a", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "^C\n" ] } ], "source": [ "# Install SkillNet-AI package and dependencies\n", "%pip install skillnet-ai\n", "%pip install scanpy requests leidenalg matplotlib seaborn pandas\n" ] }, { "cell_type": "code", "execution_count": null, "id": "d8f7f1be", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "āœ… SkillNet AI Package loaded successfully.\n" ] } ], "source": [ "import shutil\n", "from IPython.display import display, Markdown\n", "\n", "# Import the actual SkillNet Client\n", "from skillnet_ai import SkillNetClient\n", "\n", "print(\"āœ… SkillNet AI Package loaded successfully.\")\n" ] }, { "cell_type": "code", "execution_count": null, "id": "6196c679", "metadata": {}, "outputs": [], "source": [ "import scanpy as sc\n", "import pandas as pd\n", "import numpy as np\n", "import requests\n", "import json\n", "import matplotlib.pyplot as plt\n", "import seaborn as sns\n", "import os\n", "\n", "# Configure Scanpy settings for reproducibility\n", "sc.settings.verbosity = 3\n", "sc.settings.set_figure_params(dpi=80, facecolor='white')\n", "\n", "# Set OpenAI API Key for SkillNet Evalaution & Analysis\n", "# Replace with your actual key if available\n", "os.environ[\"API_KEY\"] = \"your-api-key-here\" \n", "os.environ[\"BASE_URL\"] = \"https://api.openai.com/v1\"\n" ] }, { "cell_type": "code", "execution_count": 40, "id": "4ec62f8f", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "šŸ¤– **AI Scientist Agent Initialized**\n", "\n", "šŸ“ **User Mission**: \"Analyze single-cell RNA-seq data to identify potential cancer therapeutic targets, validate them against clinical databases, and write a summary report.\"\n", "\n", "šŸš€ **SkillNet Orchestration Started**\n", "\n", "--- [Phase 1: Data Processing] ---\n", "šŸ” Searching SkillNet for: 'cellxgene'...\n", " āœ… Found Skill: cellxgene-census (Stars: 15976)\n", " ā¬‡ļø Downloading to: ./active_skills_library\\cellxgene-census...\n", " āš–ļø Evaluating Skill Quality...\n", " Safety: Good | Completeness: Good | Executability: Good | Modifiability: Good | Cost_awareness: Good\n", "\n", "--- [Phase 2: Mechanism Analysis] ---\n", "šŸ” Searching SkillNet for: 'kegg'...\n", " āœ… Found Skill: kegg-database (Stars: 17598)\n", " ā¬‡ļø Downloading to: ./active_skills_library\\kegg-database...\n", " āš–ļø Evaluating Skill Quality...\n", " Safety: Good | Completeness: Good | Executability: Average | Modifiability: Good | Cost_awareness: Good\n", "\n", "--- [Phase 3: Target Validation] ---\n", "šŸ” Searching SkillNet for: 'gget'...\n", " āœ… Found Skill: gget (Stars: 16081)\n", " ā¬‡ļø Downloading to: ./active_skills_library\\gget...\n", " āš–ļø Evaluating Skill Quality...\n", " Safety: Good | Completeness: Good | Executability: Good | Modifiability: Good | Cost_awareness: Good\n", "\n", "--- [Phase 4: Reporting] ---\n", "šŸ” Searching SkillNet for: 'scientific writing'...\n", " āœ… Found Skill: citation-management (Stars: 16204)\n", " ā¬‡ļø Downloading to: ./active_skills_library\\citation-management...\n", " āš–ļø Evaluating Skill Quality...\n", " Safety: Average | Completeness: Good | Executability: Average | Modifiability: Good | Cost_awareness: Good\n", "\n", "šŸ”— **Analyzing Skill Relationships** (SkillNet Analyzer)...\n", " Found 2 dependencies between skills.\n", " - gget --[compose_with]--> kegg-database\n", " - cellxgene-census --[compose_with]--> gget\n", "\n", "āœØ **Orchestration Complete**: 4 skills ready for execution.\n" ] } ], "source": [ "# 1. Initialize SkillNet Agent\n", "client = SkillNetClient()\n", "print(\"šŸ¤– **AI Scientist Agent Initialized**\")\n", "\n", "# 2. Task Definition & AI Planning\n", "USER_MISSION = \"Analyze single-cell RNA-seq data to identify potential cancer therapeutic targets, validate them against clinical databases, and write a summary report.\"\n", "print(f\"\\nšŸ“ **User Mission**: \\\"{USER_MISSION}\\\"\")\n", "\n", "# (Simulated AI Planning Step) - Updated plan to match available/searchable skills\n", "plan = [\n", " {\n", " \"step\": 1, \n", " \"phase\": \"Data Processing\", \n", " \"query\": \"cellxgene\", # Search query for data source\n", " \"expected_skill\": \"cellxgene-census\" \n", " },\n", " {\n", " \"step\": 2, \n", " \"phase\": \"Mechanism Analysis\", \n", " \"query\": \"kegg\", \n", " \"expected_skill\": \"kegg-database\"\n", " },\n", " {\n", " \"step\": 3, \n", " \"phase\": \"Target Validation\", \n", " \"query\": \"gget\", \n", " \"expected_skill\": \"gget\"\n", " },\n", " {\n", " \"step\": 4, \n", " \"phase\": \"Reporting\", \n", " \"query\": \"scientific writing\", \n", " \"expected_skill\": \"citation-management\"\n", " }\n", "]\n", "\n", "# Directory to store our skills\n", "SKILLS_LIB = \"./active_skills_library\"\n", "if os.path.exists(SKILLS_LIB): shutil.rmtree(SKILLS_LIB)\n", "os.makedirs(SKILLS_LIB, exist_ok=True)\n", "\n", "active_skills = {}\n", "\n", "print(\"\\nšŸš€ **SkillNet Orchestration Started**\")\n", "\n", "for task in plan:\n", " print(f\"\\n--- [Phase {task['step']}: {task['phase']}] ---\")\n", " \n", " # A. SEARCH\n", " print(f\"šŸ” Searching SkillNet for: '{task['query']}'...\")\n", " try:\n", " results = client.search(q=task['query'], limit=1)\n", " except Exception as e:\n", " results = []\n", " print(f\" (Search unavailable: {e})\")\n", "\n", " if results:\n", " best_skill = results[0]\n", " print(f\" āœ… Found Skill: {best_skill.skill_name} (Stars: {best_skill.stars})\")\n", " skill_url = best_skill.skill_url\n", " skill_name = best_skill.skill_name\n", " else:\n", " # Fallback if no network or local skill not indexed\n", " print(f\" āš ļø Skill not found via search. Retrieving from local registry: '{task['expected_skill']}'\")\n", " skill_name = task['expected_skill']\n", " skill_url = None \n", "\n", " # B. DOWNLOAD\n", " target_path = os.path.join(SKILLS_LIB, skill_name)\n", " if skill_url:\n", " print(f\" ā¬‡ļø Downloading to: {target_path}...\")\n", " try:\n", " client.download(url=skill_url, target_dir=SKILLS_LIB)\n", " except Exception as e:\n", " print(f\" (Download failed: {e})\")\n", " else:\n", " # Simulate checking local library\n", " if not os.path.exists(target_path):\n", " os.makedirs(target_path, exist_ok=True)\n", " print(f\" ā¬‡ļø Acquired local skill: {target_path}\")\n", "\n", " # C. EVALUATE\n", " print(f\" āš–ļø Evaluating Skill Quality...\")\n", " try:\n", " report = client.evaluate(target=target_path) \n", " \n", " # Output format based on user request: All 5 dimensions\n", " # {'safety': {'level': '...'}, 'completeness': {'level': '...'}, ...}\n", " \n", " dims = ['safety', 'completeness', 'executability', 'modifiability', 'cost_awareness']\n", " results_str = []\n", " for dim in dims:\n", " level = report.get(dim, {}).get('level', 'N/A')\n", " results_str.append(f\"{dim.capitalize()}: {level}\")\n", " \n", " print(f\" {' | '.join(results_str)}\")\n", " \n", " except Exception as e:\n", " # Add logic to determine error cause\n", " error_msg = str(e)\n", " if \"API_KEY\" in error_msg or \"api_key\" in error_msg:\n", " print(f\" āš ļø Evaluation skipped: Missing OpenAI API_KEY. Please set os.environ['API_KEY'].\")\n", " else:\n", " print(f\" āš ļø Evaluation failed: {error_msg}\")\n", "\n", " active_skills[task['step']] = skill_name\n", "\n", "\n", "# D. ANALYZE\n", "print(f\"\\nšŸ”— **Analyzing Skill Relationships** (SkillNet Analyzer)...\")\n", "try:\n", " relationships = client.analyze(skills_dir=SKILLS_LIB, save_to_file=False)\n", " \n", " if relationships and isinstance(relationships, list):\n", " print(f\" Found {len(relationships)} dependencies between skills.\")\n", " for r in relationships[:3]:\n", " # Parsing: {\"source\": \"A\", \"target\": \"B\", \"type\": \"depend_on\", \"reason\": \"...\"}\n", " src = r.get('source', 'Unknown')\n", " rtype = r.get('type', 'related_to')\n", " tgt = r.get('target', 'Unknown')\n", " print(f\" - {src} --[{rtype}]--> {tgt}\")\n", " else:\n", " print(\" No complex dependencies detected.\")\n", " \n", "except Exception as e:\n", " # Add logic to determine error cause\n", " error_msg = str(e)\n", " if \"API_KEY\" in error_msg or \"api_key\" in error_msg:\n", " print(f\" āš ļø Analysis skipped: Missing OpenAI API_KEY. Relationship inference requires LLM.\")\n", " else:\n", " print(f\" āš ļø Analysis failed: {error_msg}\")\n", "\n", "print(f\"\\nāœØ **Orchestration Complete**: {len(active_skills)} skills ready for execution.\")\n" ] }, { "cell_type": "markdown", "id": "1a190080", "metadata": {}, "source": [ "## Execution Phase 1: Data Processing\n", "**Active Skill**: `cellxgene-census`\n", "\n", "The Agent uses the `cellxgene-census` skill to query and retrieve single-cell data.\n", "*(Note: For this demo, we simulate the retrieval of a dataset containing specific marker genes)*\n" ] }, { "cell_type": "code", "execution_count": 36, "id": "4ab6495c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ā–¶ļø **Running Skill**: [cellxgene-census]...\n", "normalizing counts per cell\n", " finished ({time_passed})\n", "computing PCA\n", " with n_comps=50\n", " finished (0:00:00)\n", "computing neighbors\n", " using 'X_pca' with n_pcs = 50\n", " finished: added to `.uns['neighbors']`\n", " `.obsp['distances']`, distances for each pair of neighbors\n", " `.obsp['connectivities']`, weighted adjacency matrix (0:00:00)\n", "computing UMAP\n", " finished: added\n", " 'X_umap', UMAP coordinates (adata.obsm)\n", " 'umap', UMAP parameters (adata.uns) (0:00:00)\n", "running Leiden clustering\n", " finished: found 3 clusters and added\n", " 'leiden', the cluster labels (adata.obs, categorical) (0:00:00)\n", "ranking genes\n", " finished: added to `.uns['rank_genes_groups']`\n", " 'names', sorted np.recarray to be indexed by group ids\n", " 'scores', sorted np.recarray to be indexed by group ids\n", " 'logfoldchanges', sorted np.recarray to be indexed by group ids\n", " 'pvals', sorted np.recarray to be indexed by group ids\n", " 'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:00)\n", "āœ… **[cellxgene-census] Completed**. Retrieved and processed dataset. Identified Target: 'EGFR'\n" ] } ], "source": [ "current_skill = active_skills[1] \n", "print(f\"ā–¶ļø **Running Skill**: [{current_skill}]...\")\n", "\n", "# --- Skill Implementation: cellxgene-census ---\n", "# In a fully autonomous run, the Agent would execute:\n", "# > cellxgene_census.get_anndata(organism=\"Homo sapiens\", measurement_name=\"RNA\")\n", "\n", "# Simulating Data Retrieval & Processing\n", "adata = sc.datasets.blobs(n_variables=1000, n_observations=300, n_centers=3, cluster_std=1.0, random_state=42)\n", "adata.X = np.abs(adata.X) # Ensure non-negative\n", "\n", "# 1. Quality Control (Mock QC)\n", "# FIX: Define 'mt' (mitochondrial) genes for QC metrics to avoid KeyError\n", "adata.var['mt'] = np.random.choice([True, False], size=adata.n_vars, p=[0.1, 0.9])\n", "\n", "# Standard Scanpy Pipeline (Process)\n", "sc.pp.calculate_qc_metrics(adata, qc_vars=['mt'], percent_top=None, log1p=False, inplace=True)\n", "sc.pp.filter_cells(adata, min_genes=20)\n", "sc.pp.filter_genes(adata, min_cells=3)\n", "sc.pp.normalize_total(adata, target_sum=1e4)\n", "sc.pp.log1p(adata)\n", "sc.tl.pca(adata)\n", "sc.pp.neighbors(adata)\n", "sc.tl.umap(adata)\n", "sc.tl.leiden(adata, resolution=0.5)\n", "sc.tl.rank_genes_groups(adata, 'leiden', method='t-test')\n", "\n", "# Identify Marker\n", "try:\n", " top_marker_synthetic = sc.get.rank_genes_groups_df(adata, group='0').iloc[0]['names']\n", "except KeyError:\n", " top_marker_synthetic = sc.get.rank_genes_groups_df(adata, group=adata.obs['leiden'].cat.categories[0]).iloc[0]['names']\n", "\n", "target_gene_symbol = \"EGFR\" # Simulating finding EGFR in the census data\n", "# --- End Skill Implementation ---\n", "\n", "print(f\"āœ… **[{current_skill}] Completed**. Retrieved and processed dataset. Identified Target: '{target_gene_symbol}'\")\n" ] }, { "cell_type": "markdown", "id": "8e7c4cf4", "metadata": {}, "source": [ "## Execution Phase 2: Mechanism Analysis\n", "**Active Skill**: `kegg-database`\n", "\n", "The Agent uses `scripts/kegg_api.py` from the `kegg-database` skill to map the gene to biological pathways.\n" ] }, { "cell_type": "code", "execution_count": 37, "id": "97850037", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ā–¶ļø **Running Skill**: [kegg-database] (Input: EGFR)...\n", "āœ… **[kegg-database] Completed**. Mapped EGFR to 50 pathways.\n" ] } ], "source": [ "current_skill = active_skills[2]\n", "print(f\"ā–¶ļø **Running Skill**: [{current_skill}] (Input: {target_gene_symbol})...\")\n", "\n", "# --- Skill Implementation: kegg-database ---\n", "# Emulating logic from `active_skills_library/kegg-database/scripts/kegg_api.py`\n", "def run_kegg_lookup(gene):\n", " try:\n", " # 1. Look up Gene ID\n", " find_url = f\"http://rest.kegg.jp/find/genes/{gene}\"\n", " resp = requests.get(find_url).text\n", " kegg_id = resp.split('\\t')[0] if \"hsa:\" in resp else None\n", " \n", " if not kegg_id: return []\n", " \n", " # 2. Retrieve Pathways\n", " link_url = f\"http://rest.kegg.jp/link/pathway/{kegg_id}\"\n", " pathways = [line.split('\\t')[1] for line in requests.get(link_url).text.split('\\n') if line]\n", " return pathways\n", " except:\n", " return []\n", "\n", "associated_pathways = run_kegg_lookup(target_gene_symbol)\n", "# --- End Skill Implementation ---\n", "\n", "print(f\"āœ… **[{current_skill}] Completed**. Mapped {target_gene_symbol} to {len(associated_pathways)} pathways.\")\n" ] }, { "cell_type": "markdown", "id": "db60ed65", "metadata": {}, "source": [ "## Execution Phase 3: Target Validation\n", "**Active Skill**: `gget`\n", "\n", "The Agent uses `gget` (Gene enhancement tool) to validate the target's associations in the Open Targets Platform.\n" ] }, { "cell_type": "code", "execution_count": 38, "id": "e37294ba", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ā–¶ļø **Running Skill**: [gget]...\n", "āœ… **[gget] Completed**. Validated target. Top association: non-small cell lung carcinoma\n" ] } ], "source": [ "current_skill = active_skills[3]\n", "print(f\"ā–¶ļø **Running Skill**: [{current_skill}]...\")\n", "\n", "# --- Skill Implementation: gget (Open Targets Module) ---\n", "# Emulating `gget.opentargets` behavior\n", "def run_gget_opentargets(gene_symbol):\n", " # gget typically wraps the GraphQL API for Open Targets\n", " url = \"https://api.platform.opentargets.org/api/v4/graphql\"\n", " \n", " # 1. ID Mapping\n", " search_q = \"\"\"query($q:String!){search(queryString:$q,entityNames:[\"target\"],page:{index:0,size:1}){hits{id}}}\"\"\"\n", " r1 = requests.post(url, json={\"query\": search_q, \"variables\": {\"q\": gene_symbol}})\n", " \n", " try:\n", " ensembl_id = r1.json()['data']['search']['hits'][0]['id']\n", " except:\n", " return None\n", "\n", " # 2. Association Query\n", " details_q = \"\"\"query($id:String!){target(ensemblId:$id){approvedSymbol approvedName associatedDiseases(page:{index:0,size:5}){rows{disease{name} score}}}}\"\"\"\n", " r2 = requests.post(url, json={\"query\": details_q, \"variables\": {\"id\": ensembl_id}})\n", " \n", " return r2.json().get('data', {}).get('target')\n", "\n", "ot_data = run_gget_opentargets(target_gene_symbol)\n", "# --- End Skill Implementation ---\n", "\n", "if ot_data:\n", " top_d = ot_data['associatedDiseases']['rows'][0]['disease']['name']\n", " print(f\"āœ… **[{current_skill}] Completed**. Validated target. Top association: {top_d}\")\n", "else:\n", " print(f\"āš ļø **[{current_skill}] Completed**. No data found.\")\n" ] }, { "cell_type": "markdown", "id": "0b8247ea", "metadata": {}, "source": [ "## Execution Phase 4: Reporting\n", "**Active Skill**: `citation-management`\n", "\n", "The Agent generates the final report using the collected data, utilizing the `citation-management` skill to ensure proper referencing.\n" ] }, { "cell_type": "code", "execution_count": 39, "id": "870a51a7", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ā–¶ļø **Running Skill**: [citation-management]...\n", "āœ… **[citation-management] Completed**. Report with citations generated.\n" ] }, { "data": { "text/markdown": [ "\n", "# Scientific Discovery Report: EGFR Analysis\n", "*Generated by SkillNet AI Scientist*\n", "\n", "## 1. Data Processing\n", "**Skill**: `cellxgene-census`\n", "Using standardized single-cell analysis pipelines [Wolf et al., 2018], we identified **EGFR** as a significant marker.\n", "\n", "## 2. Biological Mechanism\n", "**Skill**: `kegg-database`\n", "Pathway enrichment analysis [Kanehisa, 2000] identified 50 associated pathways, linking EGFR to key biological processes:\n", "- path:hsa01521\n", "- path:hsa01522\n", "- path:hsa03272\n", "\n", "## 3. Therapeutic Validation\n", "**Skill**: `gget`\n", "Cross-referencing with clinical databases [Open Targets, 2024] confirms therapeutic relevance for:\n", "- non-small cell lung carcinoma\n", "- lung adenocarcinoma\n", "- cancer\n", "\n", "## 4. References (Managed by citation-management)\n", "1. Wolf, F. A., et al. (2018). Scanpy: large-scale single-cell gene expression data analysis. Genome biology.\n", "2. Kanehisa, M. & Goto, S. (2000). KEGG: kyoto encyclopedia of genes and genomes. Nucleic acids research.\n", "3. Open Targets Platform (2024). version 24.03.\n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "current_skill = active_skills[4]\n", "print(f\"ā–¶ļø **Running Skill**: [{current_skill}]...\")\n", "\n", "# --- Skill Implementation: citation-management ---\n", "# Simulating `scripts/format_bibtex.py` and `scripts/validate_citations.py` from the skill\n", "def format_citation(source, year, title):\n", " return f\"[{source}, {year}]\"\n", "\n", "def generate_report_with_citations(gene, pathways, ot_info):\n", " # Mock Citations\n", " ref_scanpy = format_citation(\"Wolf et al.\", 2018, \"Scanpy\")\n", " ref_kegg = format_citation(\"Kanehisa\", 2000, \"KEGG\")\n", " ref_ot = format_citation(\"Open Targets\", 2024, \"Platform\")\n", " \n", " disease_list = [d['disease']['name'] for d in ot_info['associatedDiseases']['rows'][:3]] if ot_info else []\n", " \n", " report_md = f\"\"\"\n", "# Scientific Discovery Report: {gene} Analysis\n", "*Generated by SkillNet AI Scientist*\n", "\n", "## 1. Data Processing\n", "**Skill**: `{active_skills[1]}`\n", "Using standardized single-cell analysis pipelines {ref_scanpy}, we identified **{gene}** as a significant marker.\n", "\n", "## 2. Biological Mechanism\n", "**Skill**: `{active_skills[2]}`\n", "Pathway enrichment analysis {ref_kegg} identified {len(pathways)} associated pathways, linking {gene} to key biological processes:\n", "{chr(10).join([f'- {p}' for p in pathways[:3]])}\n", "\n", "## 3. Therapeutic Validation\n", "**Skill**: `{active_skills[3]}`\n", "Cross-referencing with clinical databases {ref_ot} confirms therapeutic relevance for:\n", "{chr(10).join([f'- {d}' for d in disease_list])}\n", "\n", "## 4. References (Managed by {active_skills[4]})\n", "1. Wolf, F. A., et al. (2018). Scanpy: large-scale single-cell gene expression data analysis. Genome biology.\n", "2. Kanehisa, M. & Goto, S. (2000). KEGG: kyoto encyclopedia of genes and genomes. Nucleic acids research.\n", "3. Open Targets Platform (2024). version 24.03.\n", " \"\"\"\n", " return report_md\n", "\n", "report_content = generate_report_with_citations(target_gene_symbol, associated_pathways, ot_data)\n", "# --- End Skill Implementation ---\n", "\n", "print(f\"āœ… **[{current_skill}] Completed**. Report with citations generated.\")\n", "display(Markdown(report_content))\n" ] } ], "metadata": { "kernelspec": { "display_name": "skillnet", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.19" } }, "nbformat": 4, "nbformat_minor": 5 }