{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Journalism-Focused AI Assistant\n", "\n", "\n", "\n", "## Overview\n", "\n", "This notebook introduces an AI-powered assistant designed specifically for journalists, tackling challenges like misinformation, biased reporting, and information overload. With tools for fact-checking, tone analysis, summarization, and more, it uses AI tools to enhance the accuracy and efficiency of journalistic work.\n", "\n", "\n", "## Motivation\n", "\n", "Journalism plays a vital role in upholding democracy, but modern challenges like the flood of online misinformation and subtle biases can undermine trust in reporting. Journalists often face the daunting task of making sense of huge volumes of data under tight deadlines. This notebook equips journalists with tools to:\n", "\n", "- **Verify claims** through reliable fact-checking.\n", "- **Detect tone and biases** to maintain balanced storytelling.\n", "- **Simplify the review process** with concise and accurate summaries, as well as grammar checks.\n", "\n", "The ultimate goal is to support ethical reporting and uphold the integrity of the information we rely on every day.\n", "\n", "## Key Components\n", "\n", "1. **Language Models**: Get insights and generate responses using advanced models like `Llama 3.1/3.2` and `gpt-4o-mini`.\n", "2. **Web Search Integration**: Fetch reliable data from `DuckDuckGo’s` search API to strengthen the research process.\n", "3. **Document Parsing**: Extract text from PDFs and web pages with tools like PyMuPDFLoader and WebBaseLoader, enhanced by BeautifulSoupTransformer.\n", "4. **Structured Outputs**: Receive responses in a clean, JSON format for consistency and precision.\n", "5. **Text Splitting and Summarization**: Break down long articles into digestible summaries using RecursiveCharacterTextSplitter.\n", "6. **Tailored Prompts and Examples**: Use custom prompts and few-shot prompting to guide the AI in providing meaningful results.\n", "7. **LangGraph Workflow**: Tie everything together into a seamless, easy-to-use workflow.\n", "\n", "## Method Details\n", "\n", "### Setting Up the Environment\n", "- Import necessary libraries.\n", "- Configure any API keys and data sources.\n", "\n", "### Summarization\n", "- Pinpoint key ideas in long articles or reports.\n", "- Generate clear, concise summaries for quicker understanding.\n", "\n", "### Fact-Checking\n", "- Input claims or statements to analyze.\n", "- Search credible sources and compile relevant evidence.\n", "- Categorize claims (e.g., confirmed, refuted, or unverifiable) and provide detailed explanations.\n", "\n", "### Tone and Bias Analysis\n", "- Process text to determine sentiment—positive, neutral, or negative.\n", "- Spot and highlight biased language or phrasing.\n", "\n", "### Quote Extraction\n", "- Detect direct quotes and their sources to add transparency to your reporting.\n", "\n", "### Grammar and Bias Review\n", "- Identify grammar errors and subtle biases, ensuring content is polished and fair.\n", "\n", "### LangGraph Workflow Integration\n", "- Use LangGraph to connect all these tools into one powerful workflow:\n", "- Define nodes for each task, from analysis to report generation.\n", "- Pass data seamlessly between tasks for smooth processing.\n", "- Test the workflow on a sample article.\n", "\n", "### Report Generation\n", "- Combine all findings into a well-organized report that’s easy to read and share.\n", "\n", "### Additional Considerations\n", "- Discuss limitations, potential improvements, or specific use cases.\n", "\n", "## This Journalism-Focused AI Assistant is all about helping journalists do their best work by:\n", "- **Improving Accuracy**: Fact-checking tools ensure your claims are backed by evidence.\n", "- **Boosting Efficiency**: Summarization and workflows save valuable time.\n", "- **Adding Transparency**: Features like quote extraction and structured reports build trust.\n", "- **Promoting Ethical Reporting**: Tone and bias analysis helps maintain objectivity.\n", "\n", "By bringing these features together, this tool empowers journalists to focus on what they do best: telling meaningful stories." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup and Imports\n", "\n", "First, we'll import the necessary modules and set up our environment." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Required packages\n", "# %pip install beautifulsoup4 duckduckgo-search langchain langgraph langchain-ollama langchain-openai langchain-openai python-dotenv" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import os\n", "import pprint\n", "import time\n", "\n", "from pathlib import Path\n", "from functools import lru_cache\n", "from dotenv import load_dotenv\n", "from typing import Optional, List, TypedDict\n", "from duckduckgo_search import DDGS\n", "from IPython.display import display, Image\n", "\n", "from langchain_openai import ChatOpenAI\n", "from langchain_core.prompts import PromptTemplate\n", "from langchain_community.document_loaders.pdf import PyMuPDFLoader\n", "from langchain_community.document_loaders import WebBaseLoader\n", "from langchain_text_splitters import RecursiveCharacterTextSplitter\n", "from langchain_community.document_transformers import BeautifulSoupTransformer\n", "from langgraph.graph import StateGraph, END\n", "from langchain_ollama import ChatOllama\n", "\n", "# Load environment variables\n", "load_dotenv()\n", "os.environ[\"OPENAI_API_KEY\"] = os.getenv('OPENAI_API_KEY')\n", "\n", "# Define the data path\n", "data_path = Path(os.getcwd()).parent / \"data\"\n", "\n", "# Duckduckgo search\n", "ddgs = DDGS()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Initialize Language Models\n", "\n", "We will initialize the language models that can be used for testing.\n", "\n", "For running Llama models, we use Ollama. A detailed tutorial on this is beyond the scope of this notebook, but you can refer to their repository for a [quickstart guide on Ollama](https://github.com/ollama/ollama)." ] }, { "cell_type": "code", "execution_count": 68, "metadata": {}, "outputs": [], "source": [ "llm = ChatOpenAI(model=\"gpt-4o-mini\", temperature=0)\n", "# llm = ChatOllama(model=\"llama3.1\", temperature=0)\n", "# llm = ChatOllama(model=\"llama3.2\", temperature=0)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Data Setup\n", "\n", "For this, I used the `gpt-4o` model to generate a sample article using the prompt below. Note that the information mentioned in the article is not to be taken seriously. The article will serve as input for summarization, fact-checking, tone analysis, quote extraction, and grammar and bias analysis modules, providing a basis for refining prompt responses.\n", "\n", "We will take advantage of the `document_loaders` `langchain` module, specifically the `PyMuPDFLoader` for loading the text from a PDF file. \n", "\n", "`Prompt`:\n", "Write an article designed for classification purposes, containing a variety of claims that fit into distinct but subtly presented categories: well-known and confirmed facts, refuted claims, unverifiable statements requiring further research, and vague or speculative assertions. The article should flow naturally without explicitly labeling these categories but ensure that each type of claim is clearly identifiable through its content and context. Use varied tones, including positive, critical, biased, or opinionated language, to differentiate the claims. Incorporate quotes to enhance realism and include occasional minor grammar errors or awkward phrasing for added authenticity.\n" ] }, { "cell_type": "code", "execution_count": 69, "metadata": {}, "outputs": [], "source": [ "# Pdf file path\n", "file_path = data_path / \"Sample AI Generated Article.pdf\"\n", "\n", "# Load the pdf file\n", "pages = []\n", "loader = PyMuPDFLoader(file_path)\n", "for page in loader.lazy_load():\n", " pages.append(page)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Check the file contents and remove the new lines and tabs\n", "\n", "For printing throughout the tutorial, I will sometimes use the built-in `pprint` package as it formats text and different data types in more human-readable forms." ] }, { "cell_type": "code", "execution_count": 70, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Page content before cleaning\n", "('The Mysterious Origins and Potential\\n'\n", " 'Impacts of the Halcyon Bird\\n'\n", " 'Introduction\\n'\n", " 'The halcyon bird, a su')\n", "('Sailors’ Tales and Anecdotes\\n'\n", " 'Stories passed down through generations speak of sailors encountering h')\n", "('\"People are drawn to the idea of a creature that embodies serenity,\" says '\n", " 'cultural historian\\n'\n", " 'Dr. Ste')\n", "\n", "Page content after cleaning\n", "('The Mysterious Origins and Potential Impacts of the Halcyon Bird '\n", " 'Introduction The halcyon bird, a subject of fascination for centuries, is '\n", " 'often celebrated as a symbol of tranquility and mythical wond')\n", "('Sailors’ Tales and Anecdotes Stories passed down through generations speak '\n", " 'of sailors encountering halcyon birds during times of storm and finding '\n", " 'themselves inexplicably drawn to safety. Captain Ed H')\n", "('\"People are drawn to the idea of a creature that embodies serenity,\" says '\n", " 'cultural historian Dr. Stephen Archer. \"It’s a universal longing, especially '\n", " 'in turbulent times.\" Modern art and media have al')\n" ] } ], "source": [ "def clean_page_content(page_content: str) -> str:\n", " \"\"\"\n", " Clean the page content by removing new lines and tabs\n", " \"\"\"\n", " page_content = page_content.replace(\"\\n\", \" \")\n", " page_content = page_content.replace(\"\\t\", \" \")\n", " return page_content\n", "\n", "print(\"Page content before cleaning\")\n", "for page in pages:\n", " pprint.pprint(page.page_content[:100])\n", " \n", "\n", "print(\"\\nPage content after cleaning\")\n", "formatted_pages = []\n", "for page in pages:\n", " page_content = clean_page_content(page.page_content)\n", " formatted_pages.append(page_content)\n", " pprint.pprint(page_content[:200])\n", "\n", "# Combine all the pages into a single text\n", "full_article_text = \" \".join(formatted_pages)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Helper Functions" ] }, { "cell_type": "code", "execution_count": 71, "metadata": {}, "outputs": [], "source": [ "def chunk_large_text(text, chunk_size=100000, overlap=1000):\n", " \"\"\"\n", " Splits the input text into manageable chunks while maintaining context overlap.\n", " \"\"\"\n", " text_splitter = RecursiveCharacterTextSplitter(\n", " chunk_size=chunk_size,\n", " chunk_overlap=overlap,\n", " separators=[\"\\n\\n\", \"\\n\", \" \", \"\"]\n", " )\n", " return text_splitter.split_text(text)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Summarizing Long Articles\n", "\n", "Summarizing lengthy articles requires handling the constraints of token limits in models like `gpt-4o-mini` or `Llama 3.2/3.1`. These models can process only a limited amount of text at once, making **text splitting** a crucial step to ensure the analysis is thorough and no critical information is overlooked. While this notebook may not reach those token limits, the text-splitting approach is demonstrated as a proof of concept.\n", "\n", "### Why Use Text Splitting?\n", "\n", "Text splitting ensures the model can process lengthy content effectively by:\n", "- Dividing large articles into manageable chunks (e.g., 100,000 tokens).\n", "- Maintaining context through overlapping sections (e.g., 1,000 tokens).\n", "- Preventing request rejections due to exceeding token limits.\n", "\n", "### Tools and Techniques\n", "\n", "- **Splitter**: The `RecursiveCharacterTextSplitter` from `langchain` is used for breaking down text into smaller chunks while preserving context and readability.\n", "- **Custom Prompts**: Focus prompts on extracting key events, individuals, and statistics for precise summarization.\n", "- **Step-by-Step Workflow**:\n", " 1. Split the text into chunks.\n", " 2. Summarize each chunk individually using the AI model.\n", " 3. Combine the individual summaries into a cohesive and concise final summary.\n", "\n", "### Benefits of This Approach\n", "\n", "By leveraging text splitting and summarization workflows, even the longest articles can be processed effectively:\n", "- **Accuracy**: Ensures no critical details are missed during summarization.\n", "- **Efficiency**: Breaks down complex tasks into manageable pieces for faster results.\n", "- **Consistency**: Maintains flow and context across the final summary.\n", "\n", "This approach provides reliable and concise summaries for articles of any length, making it a powerful tool for handling extensive content with ease.\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Prepare the summarization pipelines" ] }, { "cell_type": "code", "execution_count": 72, "metadata": {}, "outputs": [], "source": [ "# Define the summarization prompt\n", "summarization_prompt = PromptTemplate(\n", " input_variables=[\"text\"],\n", " template=(\n", " \"Summarize the provided article text focusing on the main events, key people involved, \"\n", " \"and any important statistics in 150-200 words. Use a neutral tone suitable for a journalistic report:\\n\\n\"\n", " \"Article text:\\n{text}\\n\\n\"\n", " )\n", ")\n", "\n", "# Define de combine summarization prompt\n", "combine_summarization_prompt = PromptTemplate(\n", " input_variables=[\"summaries\"],\n", " template=(\n", " \"Combine the provided summaries into a single coherent summary that captures the main events, key people involved, \"\n", " \"and important statistics in 150-200 words. Use a neutral tone suitable for a journalistic report:\\n\\n\"\n", " \"Summaries:\\n{summaries}\\n\\n\"\n", " )\n", ")\n", "\n", "# Define the summarization pipeline\n", "summarization_pipeline = summarization_prompt | llm\n", "\n", "# Define the combine summarization pipeline\n", "combine_summarization_pipeline = combine_summarization_prompt | llm" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Summarization helper functions" ] }, { "cell_type": "code", "execution_count": 73, "metadata": {}, "outputs": [], "source": [ "def combine_summaries(summaries: List[str]):\n", " \"\"\"\n", " Combines multiple summaries into a single coherent summary.\n", " \"\"\"\n", " # If the article is short, return the single summary\n", " if len(summaries) == 1:\n", " return summaries[0]\n", " \n", " # Combine the summaries into a single text\n", " summaries_text = \"\"\n", " for i, summary in enumerate(summaries):\n", " summaries_text += f\"Summary {i + 1}:\\n{summary}\\n\\n\"\n", " \n", " # Generate a combined summary\n", " full_summary = combine_summarization_pipeline.invoke({\"summaries\": summaries_text})\n", "\n", " return full_summary\n", "\n", "\n", "def summarize_article(article_text: str, article_chunks=None):\n", " \"\"\"\n", " Summarize a full article text by splitting it into manageable chunks and generating summaries for each chunk.\n", " The individual summaries are then combined into a single coherent summary.\n", " \"\"\"\n", " # Split the full article text into manageable chunks if not provided\n", " if not article_chunks:\n", " article_chunks = chunk_large_text(article_text)\n", "\n", " # Generate summaries for each chunk\n", " summaries = []\n", " for chunk in article_chunks:\n", " summary = summarization_pipeline.invoke({\"text\": chunk})\n", " summaries.append(summary.content)\n", "\n", " # Combine the individual summaries into a single coherent summary\n", " full_summary = combine_summaries(summaries)\n", " \n", " return full_summary" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Sample usage" ] }, { "cell_type": "code", "execution_count": 74, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "Summary of the article\n", "('The halcyon bird, often associated with tranquility and mythical narratives, '\n", " 'has captivated human imagination for centuries. Rooted in Greek mythology, '\n", " 'particularly the story of Alcyone, the term \"halcyon\" refers to certain '\n", " 'kingfisher species, primarily found in tropical regions. Dr. Elena Marquez, '\n", " 'an avian biology professor, clarifies that while these birds exhibit calm '\n", " 'behaviors, they are adaptations for survival rather than evidence of '\n", " 'supernatural abilities. Modern science, represented by marine biologist Dr. '\n", " 'Robert Lyle, dismisses claims that halcyon birds can calm the seas, '\n", " 'attributing such phenomena to seasonal weather patterns.\\n'\n", " '\\n'\n", " 'Anecdotal tales, like that of Captain Ed Hartley, recount sailors '\n", " 'encountering halcyon birds during storms, though these stories lack '\n", " 'documentation. Conservation debates arise, with activists like Lorraine '\n", " \"Feldman advocating for habitat protection linked to the bird's cultural \"\n", " 'significance, while critics like Richard Knowles argue for focusing on more '\n", " 'pressing environmental issues. The halcyon bird also influences modern '\n", " 'culture, symbolizing peace and resilience in art and media. Researchers are '\n", " 'exploring its anatomical features for potential innovations in technology, '\n", " 'highlighting the intersection of myth, culture, and science in understanding '\n", " 'this enigmatic bird.')\n" ] } ], "source": [ "summary_result = summarize_article(full_article_text)\n", "\n", "print(\"\\n\\nSummary of the article\")\n", "pprint.pprint(summary_result)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Fact-Checking Articles\n", "\n", "This section outlines a structured approach to verify claims in articles, ensuring accuracy and credibility.\n", "\n", "### Key Components\n", "\n", "- **Structured Outputs**: Results are formatted as JSON with:\n", " - **Statement**: The claim being analyzed.\n", " - **Status**: `confirmed`, `refuted`, `unverifiable`, or `vague`.\n", " - **Explanation**: A rationale for the status.\n", " - **Keywords**: Suggested for further investigation.\n", " - Implemented using the `with_structured_output` method for consistency.\n", "\n", "- **Search Integration**:\n", " - DuckDuckGo’s API retrieves relevant search results.\n", " - Tools like `WebBaseLoader` and `BeautifulSoupTransformer` extract and clean web content\n", "\n", "- **Tailored Prompting**: A custom prompt guides the AI to analyze claims, flag inaccuracies, and suggest further research paths.\n", "\n", "### Workflow\n", "\n", "1. Extract claims from the text.\n", "2. Fetch evidence using web search.\n", "3. Analyze and categorize claims with supporting explanations.\n", "4. Output findings in a structured, clear format.\n", "\n", "### Benefits\n", "\n", "This approach combines AI precision with real-time data to deliver:\n", "- **Transparency**: Claims are backed by evidence.\n", "- **Efficiency**: Automated workflows save time.\n", "- **Consistency**: Standardized outputs ensure reliability.\n", "\n", "A seamless tool to enhance content credibility and journalistic integrity.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The models used for the structured output" ] }, { "cell_type": "code", "execution_count": 75, "metadata": {}, "outputs": [], "source": [ "class FactCheckStatement(TypedDict):\n", " \"\"\"\n", " Represents a single fact-check statement structure.\n", " \"\"\"\n", " statement: str\n", " status: str\n", " explanation: str\n", " suggested_keywords: List[str]\n", "\n", "\n", "class FactCheckResult(TypedDict):\n", " \"\"\"\n", " Represents the result of a fact-checking process.\n", " \"\"\"\n", " result: List[FactCheckStatement]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Define the internet search helper functions" ] }, { "cell_type": "code", "execution_count": 76, "metadata": {}, "outputs": [], "source": [ "# The search_ddg function is cached to avoid repeated searches for the same keywords\n", "# This was useful to test the search and summarization pipeline without waiting for the search results each time\n", "@lru_cache\n", "def search_ddg(keywords: str, max_results: int = 1):\n", " \"\"\"\n", " This function searches DuckDuckGo for the given keywords and returns the top max_results results.\n", " \"\"\"\n", " # in case of timeout wait retry after 5 seconds\n", " try:\n", " text_results = ddgs.text(keywords=keywords, max_results=max_results)\n", " except Exception as e:\n", " print(\"Keywords\", keywords)\n", " print(\"Error: \", str(e))\n", " time.sleep(5)\n", " try:\n", " text_results = ddgs.text(keywords=keywords, max_results=max_results)\n", " except Exception as e:\n", " print(\"Error: \", str(e))\n", " return [{}]\n", " \n", " return text_results\n", "\n", "\n", "\n", "def search_and_summarize(keywords: str, max_results: int = 1):\n", " \"\"\"\n", " Search for the given keywords using DuckDuckGo and summarize the content of the top search results.\n", " \"\"\"\n", " text_results = search_ddg(keywords, max_results)\n", "\n", " results = []\n", " for result in text_results:\n", " loader = WebBaseLoader([str(result['href'])])\n", " html_content = str(loader.scrape())\n", " bs_transformer = BeautifulSoupTransformer()\n", " html_transform = (\n", " bs_transformer.remove_unwanted_tags(html_content, [\"script\", \"style\", \"noscript\"])\n", " )\n", " \n", " # Based on various attempt I oberserved that the content is mostly in
tags,\n", " # so I am extracting only
tags, but is not the best approach for all the websites\n",
" html_transform = bs_transformer.extract_tags(html_transform, [\"p\"], remove_comments=True)\n",
" html_transform = bs_transformer.remove_unnecessary_lines(html_transform)\n",
"\n",
" # summarize the content using the previously defined summarization pipeline\n",
" summary_result = summarize_article(page_content)\n",
"\n",
" results.append({\n",
" \"title\": result['title'],\n",
" \"url\": result['href'],\n",
" \"summary\": summary_result\n",
" })\n",
"\n",
" return results"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Prepare the fact-checking pipeline"
]
},
{
"cell_type": "code",
"execution_count": 77,
"metadata": {},
"outputs": [],
"source": [
"# Define the fact-checking prompt\n",
"fact_checking_prompt = PromptTemplate(\n",
" input_variables=[\"text\"],\n",
" template=(\n",
" \"Fact-check the texts provided. For each statement, identify any factual inaccuracies, misleading information, \"\n",
" \"unsupported claims, or vague language lacking specific details. Confirm accuracy for each claim where possible, \"\n",
" \"or provide suggestions for further searches. Flag statements as 'vague' if they are overly broad or lacking \"\n",
" \"critical specifics (e.g., missing names, dates, or descriptions of technologies).\"\n",
" \"Suggest keyword if you can't confirm or refute the statement.\\n\\n\"\n",
" \"{text}\\n\\n\"\n",
" \"Return the results in this JSON format:\\n\"\n",
" \"{{\\n\"\n",
" \" \\\"results\\\": [\\n\"\n",
" \" {{\\n\"\n",
" \" \\\"statement\\\": \\\"