# LangSmith Troubleshooting Guide ## Installation Issues ### Package Not Found **Error**: `ModuleNotFoundError: No module named 'langsmith'` **Fix**: ```bash pip install langsmith # Verify installation python -c "import langsmith; print(langsmith.__version__)" ``` ### Version Conflicts **Error**: `ImportError: cannot import name 'traceable' from 'langsmith'` **Fix**: ```bash # Upgrade to latest version pip install -U langsmith # Check for conflicts pip check # If conflicts exist, create clean environment python -m venv venv source venv/bin/activate pip install langsmith ``` ## Authentication Issues ### API Key Not Found **Error**: `LangSmithAuthError: Authentication failed` **Solutions**: 1. **Set environment variable**: ```bash export LANGSMITH_API_KEY="your-api-key" # Or in .env file LANGSMITH_API_KEY=your-api-key ``` 2. **Pass directly to client**: ```python from langsmith import Client client = Client(api_key="your-api-key") ``` 3. **Verify key is set**: ```python import os print(os.environ.get("LANGSMITH_API_KEY", "NOT SET")) ``` ### Invalid API Key **Error**: `LangSmithAuthError: 401 Unauthorized` **Fix**: ```bash # Verify key at https://smith.langchain.com/settings # Test connection python -c "from langsmith import Client; c = Client(); print(list(c.list_projects()))" ``` ### Wrong Endpoint **Error**: `LangSmithConnectionError: Connection refused` **Fix**: ```python import os # Default endpoint os.environ["LANGSMITH_ENDPOINT"] = "https://api.smith.langchain.com" # Or for self-hosted os.environ["LANGSMITH_ENDPOINT"] = "https://your-langsmith-instance.com" ``` ## Tracing Issues ### Traces Not Appearing **Problem**: Traced functions don't appear in LangSmith. **Solutions**: 1. **Enable tracing**: ```python import os os.environ["LANGSMITH_TRACING"] = "true" # Verify print(os.environ.get("LANGSMITH_TRACING")) ``` 2. **Check project name**: ```python import os os.environ["LANGSMITH_PROJECT"] = "my-project" # Or in decorator from langsmith import traceable @traceable(project_name="my-project") def my_function(): pass ``` 3. **Flush pending traces**: ```python from langsmith import Client client = Client() client.flush() # Wait for all pending traces to be sent ``` 4. **Verify connection**: ```python from langsmith import Client client = Client() try: projects = list(client.list_projects()) print(f"Connected! Found {len(projects)} projects") except Exception as e: print(f"Connection failed: {e}") ``` ### Missing Child Runs **Problem**: Nested function calls don't appear as child runs. **Fix**: ```python from langsmith import traceable # All nested functions must be decorated @traceable def parent_function(): child_function() # This will be a child run @traceable def child_function(): pass # Or use tracing context from langsmith import trace with trace("parent", run_type="chain") as parent: with trace("child", run_type="tool") as child: # Child automatically nested under parent pass ``` ### Async Tracing Issues **Problem**: Async functions not traced correctly. **Fix**: ```python from langsmith import traceable import asyncio # Decorator works with async functions @traceable async def async_function(): await asyncio.sleep(1) return "done" # For async context from langsmith import AsyncClient async def main(): client = AsyncClient() async for run in client.list_runs(project_name="my-project"): print(run.name) asyncio.run(main()) ``` ## Evaluation Issues ### Dataset Not Found **Error**: `LangSmithNotFoundError: Dataset 'xyz' not found` **Fix**: ```python from langsmith import Client client = Client() # List available datasets for dataset in client.list_datasets(): print(f"Dataset: {dataset.name}, ID: {dataset.id}") # Use correct name or ID results = evaluate( my_model, data="correct-dataset-name", # Or use dataset ID evaluators=[my_evaluator] ) ``` ### Evaluator Errors **Problem**: Custom evaluator fails silently. **Fix**: ```python def safe_evaluator(run, example): try: prediction = run.outputs.get("answer", "") reference = example.outputs.get("answer", "") if not prediction or not reference: return {"key": "accuracy", "score": 0.0, "comment": "Missing data"} score = compute_score(prediction, reference) return {"key": "accuracy", "score": score} except Exception as e: # Return error as comment instead of crashing return { "key": "accuracy", "score": 0.0, "comment": f"Evaluator error: {str(e)}" } ``` ### Evaluation Timeout **Problem**: Evaluation hangs or times out. **Fix**: ```python from langsmith import evaluate import asyncio # Use async evaluation with timeout async def run_with_timeout(): try: results = await asyncio.wait_for( aevaluate(my_model, data="test-set", evaluators=[my_evaluator]), timeout=300 # 5 minutes ) return results except asyncio.TimeoutError: print("Evaluation timed out") return None # Or reduce concurrency results = evaluate( my_model, data="test-set", evaluators=[my_evaluator], max_concurrency=5 # Reduce from default ) ``` ## Performance Issues ### High Latency from Tracing **Problem**: Tracing adds significant latency. **Solutions**: 1. **Enable background batching** (default): ```python from langsmith import Client client = Client(auto_batch_tracing=True) ``` 2. **Use sampling**: ```python import os os.environ["LANGSMITH_TRACING_SAMPLING_RATE"] = "0.1" # 10% of traces ``` 3. **Reduce payload size**: ```python from langsmith import traceable def truncate_inputs(inputs): return {k: str(v)[:1000] for k, v in inputs.items()} @traceable(process_inputs=truncate_inputs) def my_function(large_input): pass ``` ### Memory Issues **Problem**: High memory usage during evaluation. **Fix**: ```python from langsmith import evaluate # Process in smaller batches def evaluate_in_batches(model, dataset_name, batch_size=100): from langsmith import Client client = Client() examples = list(client.list_examples(dataset_name=dataset_name)) all_results = [] for i in range(0, len(examples), batch_size): batch = examples[i:i + batch_size] results = evaluate( model, data=batch, evaluators=[my_evaluator] ) all_results.extend(results) # Clear memory import gc gc.collect() return all_results ``` ### Rate Limiting **Error**: `LangSmithRateLimitError: 429 Too Many Requests` **Fix**: ```python import time from langsmith import Client client = Client() def retry_with_backoff(func, max_retries=5): for attempt in range(max_retries): try: return func() except Exception as e: if "429" in str(e): wait_time = 2 ** attempt print(f"Rate limited, waiting {wait_time}s...") time.sleep(wait_time) else: raise raise Exception("Max retries exceeded") # Use with operations retry_with_backoff(lambda: client.create_run(...)) ``` ## Data Issues ### Large Payload Errors **Error**: `PayloadTooLarge: Request payload exceeds maximum size` **Fix**: ```python from langsmith import traceable def limit_size(data, max_chars=10000): if isinstance(data, str): return data[:max_chars] elif isinstance(data, dict): return {k: limit_size(v, max_chars // len(data)) for k, v in data.items()} elif isinstance(data, list): return [limit_size(item, max_chars // len(data)) for item in data[:100]] return data @traceable( process_inputs=limit_size, process_outputs=limit_size ) def process_large_data(data): return large_result ``` ### Serialization Errors **Error**: `TypeError: Object of type X is not JSON serializable` **Fix**: ```python import json from datetime import datetime import numpy as np def serialize_value(obj): if isinstance(obj, datetime): return obj.isoformat() elif isinstance(obj, np.ndarray): return obj.tolist() elif hasattr(obj, "__dict__"): return obj.__dict__ return str(obj) def safe_serialize(data): return json.loads(json.dumps(data, default=serialize_value)) @traceable( process_inputs=safe_serialize, process_outputs=safe_serialize ) def my_function(complex_input): return complex_output ``` ## Network Issues ### Connection Timeout **Error**: `LangSmithRequestTimeout: Connection timed out` **Fix**: ```python from langsmith import Client # Increase timeout client = Client(timeout_ms=60000) # 60 seconds # Or set via environment import os os.environ["LANGSMITH_TIMEOUT_MS"] = "60000" ``` ### SSL Certificate Errors **Error**: `SSLCertVerificationError` **Fix**: ```python # For self-signed certificates (not recommended for production) import os os.environ["LANGSMITH_VERIFY_SSL"] = "false" # Better: Add certificate to trusted store # Or use proper CA-signed certificates ``` ### Proxy Configuration **Problem**: Behind corporate proxy. **Fix**: ```python import os # Set proxy environment variables os.environ["HTTP_PROXY"] = "http://proxy.company.com:8080" os.environ["HTTPS_PROXY"] = "http://proxy.company.com:8080" # Then use client normally from langsmith import Client client = Client() ``` ## Debugging Tips ### Enable Debug Logging ```python import logging logging.basicConfig(level=logging.DEBUG) logging.getLogger("langsmith").setLevel(logging.DEBUG) ``` ### Verify Configuration ```python from langsmith import Client import os print("Configuration:") print(f" API Key: {'SET' if os.environ.get('LANGSMITH_API_KEY') else 'NOT SET'}") print(f" Endpoint: {os.environ.get('LANGSMITH_ENDPOINT', 'default')}") print(f" Project: {os.environ.get('LANGSMITH_PROJECT', 'default')}") print(f" Tracing: {os.environ.get('LANGSMITH_TRACING', 'not set')}") # Test connection client = Client() try: info = client.info print(f" Connected: Yes") print(f" Version: {info}") except Exception as e: print(f" Connected: No ({e})") ``` ### Test Simple Trace ```python from langsmith import traceable import os os.environ["LANGSMITH_TRACING"] = "true" @traceable def test_trace(): return "Hello, LangSmith!" # Run and check LangSmith UI result = test_trace() print(f"Result: {result}") print("Check LangSmith UI for trace") ``` ## Getting Help 1. **Documentation**: https://docs.smith.langchain.com 2. **GitHub Issues**: https://github.com/langchain-ai/langsmith-sdk/issues 3. **Discord**: https://discord.gg/langchain 4. **Stack Overflow**: Tag `langsmith` ### Reporting Issues Include: - LangSmith SDK version: `pip show langsmith` - Python version: `python --version` - Full error traceback - Minimal reproducible code - Environment (local, cloud, etc.)