langsmith-mcp-server

Call this tool when you need to understand how to create and push prompts to LangSmith. This is a documentation-only tool that explains how to: - Create prompts using LangChain's prompt templates - Push prompts to LangSmith for version control and management - Handle prompt creation vs. version updates Use the LangSmith Client's push_prompt() method. See LangSmith documentation for details.

Get a specific prompt by its exact name. Args: prompt_name (str): The exact name of the prompt to retrieve Returns: Dict containing the prompt details and template, or an error message

Retrieve one page of message history for a specific conversation thread. Uses char-based pagination: pages are built by character budget (max_chars_per_page). Long strings are truncated to preview_chars. Supply page_number (1-based) on every call; use the returned total_pages to request further pages. Args: thread_id (str): The unique ID of the thread to fetch history for project_name (str): The name of the project containing the thread page_number (int): 1-based page index (required) max_chars_per_page (int): Max character count per page, capped at 30000 (default: 25000) preview_chars (int): Truncate long strings to this length (default: 150) Returns: Dict with result (list of messages), page_number, total_pages, etc.

List LangSmith experiment projects (reference projects) with mandatory dataset filtering. Requires either reference_dataset_id or reference_dataset_name. Args: reference_dataset_id (str, optional): Dataset ID to filter experiments by reference_dataset_name (str, optional): Dataset name to filter experiments by limit (int): Maximum number of experiments to return (default: 5) project_name (str, optional): Filter by name (partial match)

Read a specific example from LangSmith. Args: example_id (str): Example ID to retrieve as_of (str, optional): Dataset version tag or ISO timestamp

Fetch prompts from LangSmith with optional filtering. Args: is_public (str): Filter by prompt visibility - "true" for public prompts, "false" for private prompts (default: "false") limit (int): Maximum number of prompts to return (default: 20) Returns: Dict with prompts and metadata

Fetch LangSmith runs from one or more projects with flexible filters and automatic pagination. All results are paginated by character budget to keep responses manageable. Use page_number and total_pages from the response to iterate through multiple pages. Args: project_name (str): The project name. For multiple projects, use JSON array string. limit (int): Max runs to fetch from LangSmith API (capped at 100). These runs are then paginated by character budget into pages. page_number (int): 1-based page index. Use with total_pages from response to iterate through pages. trace_id (str, optional): Return only runs that belong to this trace. run_type (str, optional): Filter by type: "llm", "chain", "tool", "retriever". error (str, optional): "true" for errored runs, "false" for successful. is_root (str, optional): "true" for only top-level traces. filter (str, optional): Filter Query Language (FQL) expression. trace_filter (str, optional): Filter applied to the root run. tree_filter (str, optional): Filter applied to any run in the trace tree. order_by (str, optional): Sort field; prefix with "-" for descending. Default "-start_time". reference_example_id (str, optional): Filter runs by reference example ID. max_chars_per_page (int): Max chars per page, capped at 30000. Default 25000. preview_chars (int): Truncate long strings to this length. Default 150.

Fetch LangSmith datasets. If no arguments are provided, all datasets will be returned. Args: dataset_ids (str, optional): Dataset IDs as JSON array string or single ID data_type (str, optional): Filter by dataset data type (e.g., 'chat', 'kv') dataset_name (str, optional): Filter by exact dataset name dataset_name_contains (str, optional): Filter by substring in dataset name metadata (str, optional): Filter by metadata as JSON object string limit (int): Max number of datasets to return (default: 20)

Call this tool when you need to understand how to create datasets in LangSmith. This is a documentation-only tool.

List LangSmith projects with optional filtering and detail level control. Args: limit (int): Maximum number of projects to return (default: 5) project_name (str, optional): Filter projects by name (partial match) more_info (str): "true" for full details, "false" for simplified (default: "false") reference_dataset_id (str, optional): Filter by reference dataset ID reference_dataset_name (str, optional): Filter by reference dataset name

Fetch examples from a LangSmith dataset with advanced filtering options. Either dataset_id, dataset_name, or example_ids must be provided. Args: dataset_id (str, optional): Dataset ID to retrieve examples from dataset_name (str, optional): Dataset name to retrieve examples from example_ids (str, optional): Example IDs as JSON array string or single ID filter (str, optional): Filter string using LangSmith query syntax metadata (str, optional): Metadata filter as JSON object string splits (str, optional): Dataset splits as JSON array string or single split inline_s3_urls (str, optional): "true" or "false" include_attachments (str, optional): "true" or "false" as_of (str, optional): Dataset version tag or ISO timestamp limit (int): Max examples to return (default: 10) offset (str, optional): Number of examples to skip

Call this tool when you need to understand how to update dataset examples in LangSmith. This is a documentation-only tool.

Fetch organization billing usage (trace counts) with workspace names inline. Args: starting_on (str): Start of date range (ISO 8601) ending_before (str): End of date range (ISO 8601) workspace (str, optional): Optional workspace UUID or display name to filter on_current_plan (str): "true" to include only usage on current plan (default: "true")

Read a specific dataset from LangSmith. Either dataset_id or dataset_name must be provided. Args: dataset_id (str, optional): Dataset ID to retrieve dataset_name (str, optional): Dataset name to retrieve

Call this tool when you need to understand how to run experiments and evaluations in LangSmith. This is a documentation-only tool.